Today I had an issue escalated to me where the following Exchange 2003 services would not start due to problems speaking with Active Directory:
- Microsoft Exchange Information Store
- Microsoft Exchange MTA Stacks
- Microsoft Exchange System Attendant
While diagnosing this issue I had the following diagnostic logging turned to medium for all sub components of the following:
- MSExchangeDSAccess
- MSExchangeSRS
- MSExchangeSA
Diagnostic logging is turned on in the Exchange 2003 server properties under the diagnostic logging tab.
![](https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiAPITxOMV5sY6ikeoL5QnsQyxXDh6uPQCPedDHbHKAJgNXi_w4NBY1uFfIJZ4NgexKGJ0470A_Lv3BdzyzMRwFaRS3UrvYiYYHfpJ-fYMPkxft3-GD-0QYqHlF-xfr0U3xX4G06DsShIU/s400/screenshot5.PNG)
There were two problems causing this issue - I will go through how I fixed both of them.
Problem 1 - The Local DC was not listening on TCP 3268
After looking at the issue initially I found out that the domain controller in the same active directory site as Exchange 2003 server was not listening on TCP 3268 (Global Catalog). The server was marked as being a global catalog server. During this time the following errors were being generated.
Event Type: Error
Event Source: Service Control Manager
Event Category: None
Event ID: 7024
Date: 01/11/2010
Time: 4:03:31 PM
User: N/A
Computer: MELB-EXCH-31
Description:
The Microsoft Exchange Information Store service terminated with service-specific error 2147500037 (0x80004005).
![](https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgyRzm1NL98Kw9pK9XXK6LgDB8ahRkPXf9H4TTnJWF6uLnjB7cyATF9NJ7hldc-2HWMM3NkkyYVKTok7jz2263_6akIMBbdr_RpOd-9aCfTBg8ozgFX1oKXiJxs1U8fvlpcF-ol8-Fqogw/s400/screenshot4.PNG)
Event Type: Error
Event Source: MSExchangeMTA
Event Category: Operating System
Event ID: 2248
Date: 01/11/2010
Time: 5:18:22 PM
User: N/A
Computer: MELB-EXCH-31
Description:
A fatal error occurred. Directory operation (ds_initialize) failed with problem RD Server. [MAIN BASE 1 1 %5] (16)
![](https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEisC-ZjKUOMClfQLoUa_6nR0Dxawju4zONiZdcmWaZZsJbGVlUs3d6th5XMZykPTk6JKggJz-CQpSO9Wi_F7mTSUvRNAIftb7LUBUrI0Vklnqoi83L2QJLcXml2n2ZfQauTmqzk3owdia8/s400/screenshot8.PNG)
Event Type: Error
Event Source: MSExchangeIS
Event Category: General
Event ID: 1121
Date: 01/11/2010
Time: 6:14:38 PM
User: N/A
Computer: MELB-EXCH-31
Description:
Error 0xfaf connecting to the Microsoft Active Directory.
![](https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgymqtBAA0-ca7ZtcTvSbeBG7WxYVMMaqD_a6xBk7pzoV7PN7hhN4UOc1nTKgVQyUwW-jzeuei5k9pOcg2uWvbFIxQP572v4xG1gLvc5hPf30rZ7MSJ7LoWH5P-C3gwXwcdiAtbxuEwR0U/s400/screenshot7.PNG)
This last error lead me down the right track, "0xfaf". I found a handy forum post that lets you understand all the different codes in relation to the information store connecting to Active Directory:
- Error: 0x96e, Service: Microsoft Active Directory - This problem occurs because the domain controller and other Exchange-Server-dependent services do not start completely when Exchange 2007 tries to start. See 940845 for details.
- Error: 0x8004010f, Service: Microsoft Active Directory - This behavior may occur if the organization name that you select during setup contains the forward slash mark (/) character. See 329599 for more information on this problem.
- Error: 4015, Service: Microsoft Exchange Information Server Directory - The Directory service will not start if the system date is later than January 17, 2038. See 154595 for details about this problem.
- Error: 0x80004005, Service: Microsoft Exchange Server Directory - This behavior may occur because the information store database is not initializing properly. See 322315 to solve this problem.
- Error: 0x80004005, Service: Microsoft Active Directory - You may not be able to mount Exchange 2000 information store databases and this event is logged. See 314294 and 822579 for details.
- Error: 0xfb5, Service: Microsoft Exchange Server Directory - This issue can occur if the Information Store service has been configured to log on as a system account or to use the Local System account. See 288952 to solve this problem.
- Error: 0xfaf, Service: Microsoft Active Directory - This behavior can occur because you do not have a global catalog in the forest or the connection to the global catalog is lost. See 303186 and 823163 to find out how to fix this problem.
- Error: 0xfaf, Service: Microsoft Exchange Server Directory - This issue can occur if the server is renamed after Exchange Server installation; some values in the MSExchangeIS key contain the name of the test server. See 248124 to solve this problem.
- Error: 0xfaf, Service: Microsoft Active Directory - This behavior can occur because you do not have a global catalog in the forest or the connection to the global catalog is lost. See 303186 and 823163 to find out how to fix this problem.
- Error: 0xfaf, Service: Microsoft Exchange Server Directory - This issue can occur if the server is renamed after Exchange Server installation; some values in the MSExchangeIS key contain the name of the test server. See 248124 to solve this problem.
As per the post above - I went to Microsoft Knowledgebase article 823163:
http://support.microsoft.com/kb/823163
This article stated:
This issue may occur if one or both of the following conditions are true:
- There is no global catalog in the forest.
- There is a problem with the connection to the global catalog.
There was only a single domain controller in the same Active Directory site as the Exchange 2003 server. This domain controller was marked as being a global catalog server however performing a "netstat -ant" revealed that it was not listening on the global catalog port TCP 3268.
Initially I tried removing the global catalog role of the server rebooting, re-adding the global catalog role then rebooting again. This did not resolve the problem.
To resolve this problem I demoted the server from being a domain controller and re-promoted it. After re-promoting the DC it then came up as a global catalog server again. There were no errors in the event logs about not being a global catalog server.
Problem 2 - The Exchange 2003 Server could not find the Active Directory Site
After re-promoting the domain controller I ran into another problem, the Exchange 2003 server could not find which Active Directory Site it was in.
Exchange was producing the following error:
c007077f no site available
The following errors were experienced in the event logs:
Event Type: Error
Event Source: MSExchangeDSAccess
Event Category: Topology
Event ID: 2114
Date: 01/11/2010
Time: 9:55:46 PM
User: N/A
Computer: MELB-EXCH-31
Description:
Process MAD.EXE (PID=1936). Topology Discovery failed, error 0x80040952.
![](https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjzwL38uUXMM4b5LOPwWnSbtAyMwUgoC8kH2_QMr7roNDEYsgGhspfCHAkWZPDZSayh4lZYNO6-d_Jcy8Ez2m7g2YDQG787-1ZukkX5ig4wS6ATpuitjrb6hFKbOjBsJ1X2BwwF7YBtb-4/s400/screenshot6.PNG)
I ran NLTEST /DSGETSITE to see if windows server itself could find its active directory site. This command threw out the following error:
1919 0x77f ERROR_NO_SITENAME
![](https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhXLSde66hTaXOnd90jGiaM6bNtlQ82xFTJJmFDkP0nTeea4bL1QLiNK7_GsfGpgZ_noKu22ic_Kp7kV8n9a_L7FM2i8dzyFZYXY9O3wL6FMiW93dz-0zsflT2Fa0cPgY22RvsDL8dtQvQ/s400/screenshot1.PNG)
The exchange server was speaking to random domain controllers in other Active Directory sites. I determined this by following this article:
http://clintboessen.blogspot.com/2010/05/how-to-find-out-which-domain-controller.html
All other workstations and servers in the same Active Directory site could determine its site location. The problem was isolated to the Exchange 2003 server. To resolve this problem I found a workaround with a registry key where I hard coded the Exchange 2003 server's Active Directory site.
HKLM\SYSTEM\CurrentControlSet\Services\Netlogon\Parameters
"SiteName" as a String Value
Enter the active directory site name as it appears in AD Sites and Services.
![](https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiuQ80Ovz4xnULQYhDNZJ5FlDrPuqk0VZ4RiFNtudbnMaz6ZpgyHYXjw9eWX3ywl7UvCnmSmq9S_8FF_j9XHrwUvOKzwO8WAHmrZFKJnnp-6puem3eJ-VSKHMMfbWY6rZYMXv5STtzoZf4/s400/screenshot2.png)
Then restart the Netlogon service. After this the Exchange Server 2003 was behaving properly and the services were able to be started.
![](https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgQ3MtXXJkPBbl2arirQWcE7EI4ZeWwkgee9jPVBeHKXWhqk4jVcz6OcEdjbxQZbcPRVMdb5_NKrV5ehHpHImc41Y4GJVimSdQNxj0BZAUGDfrbKRthpQtdVzeeBzT_pqbxTwKhu2ONPeM/s400/screenshot3.PNG)
- Microsoft Exchange Information Store
- Microsoft Exchange MTA Stacks
- Microsoft Exchange System Attendant
While diagnosing this issue I had the following diagnostic logging turned to medium for all sub components of the following:
- MSExchangeDSAccess
- MSExchangeSRS
- MSExchangeSA
Diagnostic logging is turned on in the Exchange 2003 server properties under the diagnostic logging tab.
There were two problems causing this issue - I will go through how I fixed both of them.
Problem 1 - The Local DC was not listening on TCP 3268
After looking at the issue initially I found out that the domain controller in the same active directory site as Exchange 2003 server was not listening on TCP 3268 (Global Catalog). The server was marked as being a global catalog server. During this time the following errors were being generated.
Event Type: Error
Event Source: Service Control Manager
Event Category: None
Event ID: 7024
Date: 01/11/2010
Time: 4:03:31 PM
User: N/A
Computer: MELB-EXCH-31
Description:
The Microsoft Exchange Information Store service terminated with service-specific error 2147500037 (0x80004005).
Event Type: Error
Event Source: MSExchangeMTA
Event Category: Operating System
Event ID: 2248
Date: 01/11/2010
Time: 5:18:22 PM
User: N/A
Computer: MELB-EXCH-31
Description:
A fatal error occurred. Directory operation (ds_initialize) failed with problem RD Server. [MAIN BASE 1 1 %5] (16)
Event Type: Error
Event Source: MSExchangeIS
Event Category: General
Event ID: 1121
Date: 01/11/2010
Time: 6:14:38 PM
User: N/A
Computer: MELB-EXCH-31
Description:
Error 0xfaf connecting to the Microsoft Active Directory.
This last error lead me down the right track, "0xfaf". I found a handy forum post that lets you understand all the different codes in relation to the information store connecting to Active Directory:
- Error: 0x96e, Service: Microsoft Active Directory - This problem occurs because the domain controller and other Exchange-Server-dependent services do not start completely when Exchange 2007 tries to start. See 940845 for details.
- Error: 0x8004010f, Service: Microsoft Active Directory - This behavior may occur if the organization name that you select during setup contains the forward slash mark (/) character. See 329599 for more information on this problem.
- Error: 4015, Service: Microsoft Exchange Information Server Directory - The Directory service will not start if the system date is later than January 17, 2038. See 154595 for details about this problem.
- Error: 0x80004005, Service: Microsoft Exchange Server Directory - This behavior may occur because the information store database is not initializing properly. See 322315 to solve this problem.
- Error: 0x80004005, Service: Microsoft Active Directory - You may not be able to mount Exchange 2000 information store databases and this event is logged. See 314294 and 822579 for details.
- Error: 0xfb5, Service: Microsoft Exchange Server Directory - This issue can occur if the Information Store service has been configured to log on as a system account or to use the Local System account. See 288952 to solve this problem.
- Error: 0xfaf, Service: Microsoft Active Directory - This behavior can occur because you do not have a global catalog in the forest or the connection to the global catalog is lost. See 303186 and 823163 to find out how to fix this problem.
- Error: 0xfaf, Service: Microsoft Exchange Server Directory - This issue can occur if the server is renamed after Exchange Server installation; some values in the MSExchangeIS key contain the name of the test server. See 248124 to solve this problem.
- Error: 0xfaf, Service: Microsoft Active Directory - This behavior can occur because you do not have a global catalog in the forest or the connection to the global catalog is lost. See 303186 and 823163 to find out how to fix this problem.
- Error: 0xfaf, Service: Microsoft Exchange Server Directory - This issue can occur if the server is renamed after Exchange Server installation; some values in the MSExchangeIS key contain the name of the test server. See 248124 to solve this problem.
As per the post above - I went to Microsoft Knowledgebase article 823163:
http://support.microsoft.com/kb/823163
This article stated:
This issue may occur if one or both of the following conditions are true:
- There is no global catalog in the forest.
- There is a problem with the connection to the global catalog.
There was only a single domain controller in the same Active Directory site as the Exchange 2003 server. This domain controller was marked as being a global catalog server however performing a "netstat -ant" revealed that it was not listening on the global catalog port TCP 3268.
Initially I tried removing the global catalog role of the server rebooting, re-adding the global catalog role then rebooting again. This did not resolve the problem.
To resolve this problem I demoted the server from being a domain controller and re-promoted it. After re-promoting the DC it then came up as a global catalog server again. There were no errors in the event logs about not being a global catalog server.
Problem 2 - The Exchange 2003 Server could not find the Active Directory Site
After re-promoting the domain controller I ran into another problem, the Exchange 2003 server could not find which Active Directory Site it was in.
Exchange was producing the following error:
c007077f no site available
The following errors were experienced in the event logs:
Event Type: Error
Event Source: MSExchangeDSAccess
Event Category: Topology
Event ID: 2114
Date: 01/11/2010
Time: 9:55:46 PM
User: N/A
Computer: MELB-EXCH-31
Description:
Process MAD.EXE (PID=1936). Topology Discovery failed, error 0x80040952.
I ran NLTEST /DSGETSITE to see if windows server itself could find its active directory site. This command threw out the following error:
1919 0x77f ERROR_NO_SITENAME
The exchange server was speaking to random domain controllers in other Active Directory sites. I determined this by following this article:
http://clintboessen.blogspot.com/2010/05/how-to-find-out-which-domain-controller.html
All other workstations and servers in the same Active Directory site could determine its site location. The problem was isolated to the Exchange 2003 server. To resolve this problem I found a workaround with a registry key where I hard coded the Exchange 2003 server's Active Directory site.
HKLM\SYSTEM\CurrentControlSet\Services\Netlogon\Parameters
"SiteName" as a String Value
Enter the active directory site name as it appears in AD Sites and Services.
![](https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiuQ80Ovz4xnULQYhDNZJ5FlDrPuqk0VZ4RiFNtudbnMaz6ZpgyHYXjw9eWX3ywl7UvCnmSmq9S_8FF_j9XHrwUvOKzwO8WAHmrZFKJnnp-6puem3eJ-VSKHMMfbWY6rZYMXv5STtzoZf4/s400/screenshot2.png)
Then restart the Netlogon service. After this the Exchange Server 2003 was behaving properly and the services were able to be started.
No comments:
Post a Comment