“This Page Can’t Be Displayed” When Accessing /’protocol’ /healthcheck.htm in Exchange 2013

An issue presented itself when attempting to load balance Exchange 2013 servers with a Citrix NetScaler VPX (virtual appliance); the configured virtual servers and service groups in the NetScaler showed portions of one of two Exchange servers being down.

NetScaler Services Down

The monitors configured are checking for the presence of the healthcheck.htm file. Microsoft has given us a way to check each virtual directory (vdir) in Exchange 2013 for availability and health with Health Probe Checking. Healthcheck.htm is available for each vdir when they are healthy and not when “not healthy”.

Each of the monitors configured in the NetScaler rely on /healthcheck.htm to determine whether a Exchange service area is available or not based on the state of a component associated with a specific protocol. It is important to note that healthcheck.htm exists only in memory when the component state is active for its associated protocol.

Below are the Exchange 2013 vdirs and the associated component protocol URLs available to perform cursory health checks of Exchange.

Exchange 2013 Vdirs in IIS

NOTE: Replace “IPADDRESS” with the internal IP address or the NetBIOS name or internal FQDN of an Exchange 2013 server. In a load balanced scenario, the internal IP address or NetBIOS name will bypass any DNS records or VIPs that may be configured that directs traffic through the load balancer.

https://IPADDRESS/Autodiscover/healthcheck.htm
https://IPADDRESS/ecp/healthcheck.htm
https://IPADDRESS/EWS/healthcheck.htm
https://IPADDRESS/mapi/healthcheck.htm
https://IPADDRESS/Microsoft-Server-ActiveSync/healthcheck.htm
https://IPADDRESS/OAB/healthcheck.htm
https://IPADDRESS/owa/healthcheck.htm
https://IPADDRESS/PowerShell/healthcheck.htm
https://IPADDRESS/Rpc/healthcheck.htm

 

When checking each of those URLs, we should receive a “200 OK” message with the internal FQDN of the server we are targeting when the health of the vdir is good…

Exchange 2013 Vdirs Health Good

…And if the health of the vdir is bad, the result will show “This page can’t be displayed”.

Exchange 2013 Vdirs Health Bad

 

NOTE: At the time this issue presented itself, the Exchange 2013 servers were running CU7. Additionally, there are 2 multi-role servers which are members of a database availability group (DAG) hosting 1 active database each.

The “problem” server was investigated for health issues using Get-HealthReport “SERVERNAME” which identified the AlertValue of several HealthSets as being “unhealthy”. The databases were even moved back and forth in an attempt to “shake something loose”. Further troubleshooting proved fruitless and the server was ultimately restarted. However, a restart did nothing to resolve the issue either; as running Get-HelthReport provided the same results. At this point, the server was determined to be working normally as users could still access mailboxes via Outlook, OWA and smart devices. Therefore, it was decided to schedule another day to dig into the issue.

Along comes CU8… On Tue 17 Mar 2015, with the release of CU8 for Exchange 2013, I was all over updating the “problem” server as a possible resolution. The server was updated and restarted. Sure enough the NetScaler was happy again and reported all target services up.

However, that was very short-lived. Apparently, the “problem” server showed healthy from the NetScaler’s perspective for just a brief moment and then showed the same services on the same server being inaccessible.

I really was at a loss and didn’t know what to search for. As we all do, I made a request to the community on Twitter and then went to Bing searching for anything that referenced “healthcheck.htm”. Findings were miniscule and hope was all but lost when a crumb was found in the TechNet Exchange Server Forums that provided some welcomed guidance.

As was suggested, Get-ServerComponentState “SERVERNAME” was run against the “problem” server that showed 5 of the 7 components that the NetScaler was monitoring were in an “inactive” state. Ah ha! And when the same command was run on the other Exchange server, the state of all the components showed active.

Exchange 2013 ServerComponents Inactive

 

Though a restart of the server did not address the issue previously, I went ahead and restarted the ‘Microsoft Exchange Health Manager’ service. That did not help either. I went as far as attempting to force the components into an active state with the following command. Still, no joy.

Set-ServerComponentState -Identity "SERVERNAME" -Component "COMPONENTNAME" -State Active -Requester Maintenance

 

Next, Looking to ADUC in the ‘Microsoft Exchange System Objects/Monitoring Mailboxes’ OU, twenty-six (26) “HealthMailboxes” were found. This seemed like a really high number of mailboxes given what was stated in the forum post. However, with an article posted by the Exchange Team on Fri 20 Mar 2015, it appeared that the number of monitoring mailboxes was about right.

NOTE: Based on the Exchange Team article as I understand it, my environment (with 2 multi-role servers and 2 mailbox databases [with DAG copies]) should have 24 monitoring mailboxes. But there were 26.

We can also use EMS to get a list of the monitoring mailboxes we have using one of these two commands…

Get-Mailbox -Monitoring
Get-Mailbox -Monitoring | ft -auto

Exchange 2013 Monitoring Mailboxes (condensed)

Exchange 2013 Monitoring Mailboxes (full)

…or get a count of the number of monitoring mailboxes we have with this command…

(Get-Mailbox -Monitoring).count

Exchange 2013 Monitoring Mailboxes (count)

 

Moving on…

I deleted all 26 of the monitoring mailboxes and, as expected, new monitoring mailboxes were created once the ‘Microsoft Exchange Health Manager’ service was restarted. For good measure, the “problem” server was restarted. After the restart, interestingly enough, only 24 monitoring mailboxes were created. That leads me to believe there must have been a few corrupted monitoring mailboxes. I also restarted the other DAG member (the non-problematic Exchange server).

Exchange 2013 Monitoring Mailboxes (ADUC)

 

Now, for the moment of truth…

{Drum roll}

Get-ServerComponentState was run against the “problem” server again. This time, all components that previously were in an inactive state now showed active. Huzzah!!!

Exchange 2013 ServerComponents (active)

 

The NetScaler? … Currently, it’s as happy as I am.

And Get-HealthReport? … Every ‘HealthSet’ showed the ‘AlertValue’ of healthy.

 

Conclusion

This was a relatively easy fix for what was a couple days of research and frustration. Ultimately, we could not track down the source of why the problem began in the first place; but considering everything was fine the day before, I chalk it up to some database failover testing we were performing at the time the issue presented itself. I can safely say that many environments have this issue present but Exchange admins are unaware of it. You don’t need a load balancer to find out there is a problem. And it doesn’t only present itself when a DAG is configured.

Again, good luck and have fun! Happy troubleshooting!

 

Reference(s):

Advertisements

10 thoughts on ““This Page Can’t Be Displayed” When Accessing /’protocol’ /healthcheck.htm in Exchange 2013

  1. Hi

    thanks for your valuable information just curious do you have Exchange Server config document on Citrix LB? not very clear citrix official document. ssadoglu at ereteam dot com
    Thanks

  2. Brilliant article! Fixed all my issues, thanks so much for taking the time to post a very clear and easy to follow article.

  3. Pingback: Microsoft Exchange Monitorering

  4. I am having same problems. I have two servers in DAG, both MSX2013 CU13. Both servers show identical unhealthy health sets. I tried with deleting monitoring mailboxes but to no avail. Could be there an error with my split DNS configuration or perhaps with bindings in IIS? I am running out of options 😦

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s