ERROR_REPLICA_SYNC_FAILED_THE TARGET PRINCIPAL NAME IS INCORRECT

I had an issue with a client's Active Directory environment where Replication was only working one way. They have two sites that run inter site replication. Replication was only working one way:



Users were able to login and access network resources in the site that contained the Orion2 domain controller but users were unable to work in the site that contained OrionCH domain controller.

Below I'm going to go through all the errors received to show you all symptoms that caused this problem so you can relate it to your environment if your having the same issue. I will then provide a fix.

Error recieved in Replmon:



Below are trouble shooting commands I ran and what server I ran them on.

repadmin /showrepl on ORIONCH

repadmin running command /showrepl against server localhost

CastleHill\ORIONCH
DC Options: (none)
Site Options: (none)
DC object GUID: 8e92a542-61f2-4e4a-a06c-b19932d0412e
DC invocationID: 2e3ea760-e835-4050-8386-95c4e29d66bd

==== INBOUND NEIGHBORS =====================================

DC=orion,DC=net,DC=au
Balcatta\ORION2 via RPC
DC object GUID: 1d83042b-1be4-41d1-9e5d-117829335117
Last attempt @ 2010-02-23 17:02:45 was successful.

CN=Configuration,DC=orion,DC=net,DC=au
Balcatta\ORION2 via RPC
DC object GUID: 1d83042b-1be4-41d1-9e5d-117829335117
Last attempt @ 2010-02-23 17:02:45 was successful.

CN=Schema,CN=Configuration,DC=orion,DC=net,DC=au
Balcatta\ORION2 via RPC
DC object GUID: 1d83042b-1be4-41d1-9e5d-117829335117
Last attempt @ 2010-02-23 17:02:46 was successful.

DC=DomainDnsZones,DC=orion,DC=net,DC=au
Balcatta\ORION2 via RPC
DC object GUID: 1d83042b-1be4-41d1-9e5d-117829335117
Last attempt @ 2010-02-23 17:02:46 was successful.

DC=ForestDnsZones,DC=orion,DC=net,DC=au
Balcatta\ORION2 via RPC
DC object GUID: 1d83042b-1be4-41d1-9e5d-117829335117
Last attempt @ 2010-02-23 17:02:46 was successful.

repadmin /showrepl on ORION2

repadmin running command /showrepl against server localhost

Balcatta\ORION2
DC Options: IS_GC
Site Options: (none)
DC object GUID: 1d83042b-1be4-41d1-9e5d-117829335117
DC invocationID: d0e38ce4-9011-4db7-b7e4-efc2738b9074

==== INBOUND NEIGHBORS ======================================

DC=orion,DC=net,DC=au
CastleHill\ORIONCH via RPC
DC object GUID: 8e92a542-61f2-4e4a-a06c-b19932d0412e
Last attempt @ 2010-02-23 13:54:02 failed, result -2146893022 (0x80090322):
Can't retrieve message string -2146893022 (0x80090322), error 1815.
93 consecutive failure(s).
Last success @ 2010-02-22 15:22:01.

CN=Configuration,DC=orion,DC=net,DC=au
CastleHill\ORIONCH via RPC
DC object GUID: 8e92a542-61f2-4e4a-a06c-b19932d0412e
Last attempt @ 2010-02-23 13:54:03 failed, result -2146893022 (0x80090322):
Can't retrieve message string -2146893022 (0x80090322), error 1815.
93 consecutive failure(s).
Last success @ 2010-02-22 15:22:01.

CN=Schema,CN=Configuration,DC=orion,DC=net,DC=au
CastleHill\ORIONCH via RPC
DC object GUID: 8e92a542-61f2-4e4a-a06c-b19932d0412e
Last attempt @ 2010-02-23 13:54:03 failed, result -2146893022 (0x80090322):
Can't retrieve message string -2146893022 (0x80090322), error 1815.
93 consecutive failure(s).
Last success @ 2010-02-22 15:22:01.

DC=DomainDnsZones,DC=orion,DC=net,DC=au
CastleHill\ORIONCH via RPC
DC object GUID: 8e92a542-61f2-4e4a-a06c-b19932d0412e
Last attempt @ 2010-02-23 13:54:04 failed, result -2146893022 (0x80090322):
Can't retrieve message string -2146893022 (0x80090322), error 1815.
93 consecutive failure(s).
Last success @ 2010-02-22 15:22:02.

DC=ForestDnsZones,DC=orion,DC=net,DC=au
CastleHill\ORIONCH via RPC
DC object GUID: 8e92a542-61f2-4e4a-a06c-b19932d0412e
Last attempt @ 2010-02-23 13:54:04 failed, result -2146893022 (0x80090322):
Can't retrieve message string -2146893022 (0x80090322), error 1815.
93 consecutive failure(s).
Last success @ 2010-02-22 15:22:02.

Source: CastleHill\ORIONCH
******* 93 CONSECUTIVE FAILURES since 2010-02-22 15:22:02
Last error: -2146893022 (0x80090322):
Can't retrieve message string -2146893022 (0x80090322), error 1815.

repadmin /replsummary on ORIONCH

Replication Summary Start Time: 2010-02-23 17:12:30

Beginning data collection for replication summary, this may take awhile:
.....

Source DC largest delta fails/total %% error
ORION2 09m:45s 0 / 5 0
ORIONCH 22h:50m:29s 5 / 5 100 (2148074274) Can't retrieve message string -2146893022...

Destination DC largest delta fails/total %% error
ORION2 22h:50m:52s 5 / 5 100 (2148074274) Can't retrieve message string -2146893022...
ORIONCH 09m:46s 0 / 5 0

repadmin /replsummary on ORION2

Replication Summary Start Time: 2010-02-23 14:10:53

Beginning data collection for replication summary, this may take awhile:
.....

Source DC largest delta fails/total %% error
ORIONCH 22h:48m:52s 5 / 5 100 (2148074274) Can't retrieve message string -2146893022 (0x800903...

Destination DC largest delta fails/total %% error
ORION2 22h:48m:52s 5 / 5 100 (2148074274) Can't retrieve message string -2146893022 (0x800903...

Experienced the following operational errors trying to retrieve replication information:
8341 - ORIONCH.orion.net.au

Example of Replication Failing

repadmin /replicate orionch orion2 /force
DsReplicaSync() failed with status 87 (0x57):
Can't retrieve message string 87 (0x57), error 1815.

dcdiag /v on ORIONCH

Domain Controller Diagnosis

Performing initial setup:
* Verifying that the local machine ORIONCH, is a DC.
* Connecting to directory service on server ORIONCH.
* Collecting site info.
* Identifying all servers.
* Identifying all NC cross-refs.
* Found 2 DC(s). Testing 1 of them.
Done gathering initial info.

Doing initial required tests

Testing server: CastleHill\ORIONCH
Starting test: Connectivity
* Active Directory LDAP Services Check
* Active Directory RPC Services Check
......................... ORIONCH passed test Connectivity

Doing primary tests

Testing server: CastleHill\ORIONCH
Starting test: Replications
* Replications Check
* Replication Latency Check
DC=ForestDnsZones,DC=orion,DC=net,DC=au
Latency information for 4 entries in the vector were ignored.
4 were retired Invocations. 0 were either: read-only replicas and are not verifiably latent, or dc's no longer replicating this nc. 0 had no latency information (Win2K DC).
DC=DomainDnsZones,DC=orion,DC=net,DC=au
Latency information for 4 entries in the vector were ignored.
4 were retired Invocations. 0 were either: read-only replicas and are not verifiably latent, or dc's no longer replicating this nc. 0 had no latency information (Win2K DC).
CN=Schema,CN=Configuration,DC=orion,DC=net,DC=au
Latency information for 5 entries in the vector were ignored.
5 were retired Invocations. 0 were either: read-only replicas and are not verifiably latent, or dc's no longer replicating this nc. 0 had no latency information (Win2K DC).
CN=Configuration,DC=orion,DC=net,DC=au
Latency information for 5 entries in the vector were ignored.
5 were retired Invocations. 0 were either: read-only replicas and are not verifiably latent, or dc's no longer replicating this nc. 0 had no latency information (Win2K DC).
DC=orion,DC=net,DC=au
Latency information for 5 entries in the vector were ignored.
5 were retired Invocations. 0 were either: read-only replicas and are not verifiably latent, or dc's no longer replicating this nc. 0 had no latency information (Win2K DC).
* Replication Site Latency Check
......................... ORIONCH passed test Replications
Test omitted by user request: Topology
Test omitted by user request: CutoffServers
Starting test: NCSecDesc
* Security Permissions check for all NC's on DC ORIONCH.
* Security Permissions Check for
DC=ForestDnsZones,DC=orion,DC=net,DC=au
(NDNC,Version 2)
* Security Permissions Check for
DC=DomainDnsZones,DC=orion,DC=net,DC=au
(NDNC,Version 2)
* Security Permissions Check for
CN=Schema,CN=Configuration,DC=orion,DC=net,DC=au
(Schema,Version 2)
* Security Permissions Check for
CN=Configuration,DC=orion,DC=net,DC=au
(Configuration,Version 2)
* Security Permissions Check for
DC=orion,DC=net,DC=au
(Domain,Version 2)
......................... ORIONCH passed test NCSecDesc
Starting test: NetLogons
* Network Logons Privileges Check
Unable to connect to the NETLOGON share! (\\ORIONCH\netlogon)
[ORIONCH] An net use or LsaPolicy operation failed with error 1203, Win32 Error 1203.
......................... ORIONCH failed test NetLogons
Starting test: Advertising
Warning: DsGetDcName returned information for \\orion2.orion.net.au, when we were trying to reach ORIONCH.
Server is not responding or is not considered suitable.
The DC ORIONCH is advertising itself as a DC and having a DS.
The DC ORIONCH is advertising as an LDAP server
The DC ORIONCH is advertising as having a writeable directory
The DC ORIONCH is advertising as a Key Distribution Center
The DC ORIONCH is advertising as a time server
......................... ORIONCH failed test Advertising
Starting test: KnowsOfRoleHolders
Role Schema Owner = CN=NTDS Settings,CN=ORION2,CN=Servers,CN=Balcatta,CN=Sites,CN=Configuration,DC=orion,DC=net,DC=au
Role Domain Owner = CN=NTDS Settings,CN=ORION2,CN=Servers,CN=Balcatta,CN=Sites,CN=Configuration,DC=orion,DC=net,DC=au
Role PDC Owner = CN=NTDS Settings,CN=ORION2,CN=Servers,CN=Balcatta,CN=Sites,CN=Configuration,DC=orion,DC=net,DC=au
Role Rid Owner = CN=NTDS Settings,CN=ORION2,CN=Servers,CN=Balcatta,CN=Sites,CN=Configuration,DC=orion,DC=net,DC=au
Role Infrastructure Update Owner = CN=NTDS Settings,CN=ORION2,CN=Servers,CN=Balcatta,CN=Sites,CN=Configuration,DC=orion,DC=net,DC=au
......................... ORIONCH passed test KnowsOfRoleHolders
Starting test: RidManager
* Available RID Pool for the Domain is 4804 to 1073741823
* orion2.orion.net.au is the RID Master
* DsBind with RID Master was successful
* rIDAllocationPool is 4304 to 4803
* rIDPreviousAllocationPool is 4304 to 4803
* rIDNextRID: 4305
......................... ORIONCH passed test RidManager
Starting test: MachineAccount
Checking machine account for DC ORIONCH on DC ORIONCH.
* SPN found :LDAP/ORIONCH.orion.net.au/orion.net.au
* SPN found :LDAP/ORIONCH.orion.net.au
* SPN found :LDAP/ORIONCH
* SPN found :LDAP/ORIONCH.orion.net.au/ORION
* SPN found :LDAP/8e92a542-61f2-4e4a-a06c-b19932d0412e._msdcs.orion.net.au
* SPN found :E3514235-4B06-11D1-AB04-00C04FC2DCD2/8e92a542-61f2-4e4a-a06c-b19932d0412e/orion.net.au
* SPN found :HOST/ORIONCH.orion.net.au/orion.net.au
* SPN found :HOST/ORIONCH.orion.net.au
* SPN found :HOST/ORIONCH
* SPN found :HOST/ORIONCH.orion.net.au/ORION
* SPN found :GC/ORIONCH.orion.net.au/orion.net.au
......................... ORIONCH passed test MachineAccount
Starting test: Services
* Checking Service: Dnscache
* Checking Service: NtFrs
* Checking Service: IsmServ
* Checking Service: kdc
* Checking Service: SamSs
* Checking Service: LanmanServer
* Checking Service: LanmanWorkstation
* Checking Service: RpcSs
* Checking Service: w32time
* Checking Service: NETLOGON
......................... ORIONCH passed test Services
Test omitted by user request: OutboundSecureChannels
Starting test: ObjectsReplicated
ORIONCH is in domain DC=orion,DC=net,DC=au
Checking for CN=ORIONCH,OU=Domain Controllers,DC=orion,DC=net,DC=au in domain DC=orion,DC=net,DC=au on 1 servers
Object is up-to-date on all servers.
Checking for CN=NTDS Settings,CN=ORIONCH,CN=Servers,CN=CastleHill,CN=Sites,CN=Configuration,DC=orion,DC=net,DC=au in domain CN=Configuration,DC=orion,DC=net,DC=au on 1 servers
Object is up-to-date on all servers.
......................... ORIONCH passed test ObjectsReplicated
Starting test: frssysvol
* The File Replication Service SYSVOL ready test
The registry lookup failed to determine the state of the SYSVOL. The
error returned was 0 (Win32 Error 0). Check the FRS event log to see
if the SYSVOL has successfully been shared.
......................... ORIONCH passed test frssysvol
Starting test: frsevent
* The File Replication Service Event log test
There are warning or error events within the last 24 hours after the
SYSVOL has been shared. Failing SYSVOL replication problems may cause
Group Policy problems.
An Warning Event occured. EventID: 0x800034FD
Time Generated: 02/23/2010 17:34:40
(Event String could not be retrieved)
An Warning Event occured. EventID: 0x800034C4
Time Generated: 02/23/2010 17:37:19
(Event String could not be retrieved)
......................... ORIONCH failed test frsevent
Starting test: kccevent
* The KCC Event log test
Found no KCC errors in Directory Service Event log in the last 15 minutes.
......................... ORIONCH passed test kccevent
Starting test: systemlog
* The System Event log test
An Error Event occured. EventID: 0xC25A001D
Time Generated: 02/23/2010 16:48:08
(Event String could not be retrieved)
An Error Event occured. EventID: 0xC25A001D
Time Generated: 02/23/2010 17:06:53
(Event String could not be retrieved)
An Error Event occured. EventID: 0xC25A001D
Time Generated: 02/23/2010 17:25:38
(Event String could not be retrieved)
An Error Event occured. EventID: 0x00000C8A
Time Generated: 02/23/2010 17:34:25
Event String: This computer could not authenticate with
\\orion2.orion.net.au, a Windows domain
controller for domain ORION, and therefore this
computer might deny logon requests. This
inability to authenticate might be caused by
another computer on the same network using the
same name or the password for this computer
account is not recognized. If this message
appears again, contact your system administrator.

......................... ORIONCH failed test systemlog
Test omitted by user request: VerifyReplicas
Starting test: VerifyReferences
The system object reference (serverReference)
CN=ORIONCH,OU=Domain Controllers,DC=orion,DC=net,DC=au and backlink on
CN=ORIONCH,CN=Servers,CN=CastleHill,CN=Sites,CN=Configuration,DC=orion,DC=net,DC=au
are correct.
The system object reference (frsComputerReferenceBL)
CN=ORIONCH,CN=Domain System Volume (SYSVOL share),CN=File Replication Service,CN=System,DC=orion,DC=net,DC=au
and backlink on CN=ORIONCH,OU=Domain Controllers,DC=orion,DC=net,DC=au
are correct.
The system object reference (serverReferenceBL)
CN=ORIONCH,CN=Domain System Volume (SYSVOL share),CN=File Replication Service,CN=System,DC=orion,DC=net,DC=au
and backlink on
CN=NTDS Settings,CN=ORIONCH,CN=Servers,CN=CastleHill,CN=Sites,CN=Configuration,DC=orion,DC=net,DC=au
are correct.
......................... ORIONCH passed test VerifyReferences
Test omitted by user request: VerifyEnterpriseReferences
Test omitted by user request: CheckSecurityError

Running partition tests on : ForestDnsZones
Starting test: CrossRefValidation
......................... ForestDnsZones passed test CrossRefValidation
Starting test: CheckSDRefDom
......................... ForestDnsZones passed test CheckSDRefDom

Running partition tests on : DomainDnsZones
Starting test: CrossRefValidation
......................... DomainDnsZones passed test CrossRefValidation
Starting test: CheckSDRefDom
......................... DomainDnsZones passed test CheckSDRefDom

Running partition tests on : Schema
Starting test: CrossRefValidation
......................... Schema passed test CrossRefValidation
Starting test: CheckSDRefDom
......................... Schema passed test CheckSDRefDom

Running partition tests on : Configuration
Starting test: CrossRefValidation
......................... Configuration passed test CrossRefValidation
Starting test: CheckSDRefDom
......................... Configuration passed test CheckSDRefDom

Running partition tests on : orion
Starting test: CrossRefValidation
......................... orion passed test CrossRefValidation
Starting test: CheckSDRefDom
......................... orion passed test CheckSDRefDom

Running enterprise tests on : orion.net.au
Starting test: Intersite
Skipping site CastleHill, this site is outside the scope provided by
the command line arguments provided.
Skipping site Balcatta, this site is outside the scope provided by the
command line arguments provided.
......................... orion.net.au passed test Intersite
Starting test: FsmoCheck
GC Name: \\orion2.orion.net.au
Locator Flags: 0xe00001fd
PDC Name: \\orion2.orion.net.au
Locator Flags: 0xe000017d
Time Server Name: \\orion2.orion.net.au
Locator Flags: 0xe000017d
Preferred Time Server Name: \\orion2.orion.net.au
Locator Flags: 0xe000017d
KDC Name: \\orion2.orion.net.au
Locator Flags: 0xe000017d
......................... orion.net.au passed test FsmoCheck
Test omitted by user request: DNS
Test omitted by user request: DNS

dcdiag /v on ORION2

Domain Controller Diagnosis

Performing initial setup:
* Verifying that the local machine orion2, is a DC.
* Connecting to directory service on server orion2.
* Collecting site info.
* Identifying all servers.
* Identifying all NC cross-refs.
* Found 2 DC(s). Testing 1 of them.
Done gathering initial info.

Doing initial required tests

Testing server: Balcatta\ORION2
Starting test: Connectivity
* Active Directory LDAP Services Check
* Active Directory RPC Services Check
......................... ORION2 passed test Connectivity

Doing primary tests

Testing server: Balcatta\ORION2
Starting test: Replications
* Replications Check
[Replications Check,ORION2] A recent replication attempt failed:
From ORIONCH to ORION2
Naming Context: DC=ForestDnsZones,DC=orion,DC=net,DC=au
The replication generated an error (1256):
Win32 Error 1256
The failure occurred at 2010-02-23 14:38:16.
The last success occurred at 2010-02-22 15:22:02.
96 failures have occurred since the last success.
[Replications Check,ORION2] A recent replication attempt failed:
From ORIONCH to ORION2
Naming Context: DC=DomainDnsZones,DC=orion,DC=net,DC=au
The replication generated an error (1256):
Win32 Error 1256
The failure occurred at 2010-02-23 14:38:16.
The last success occurred at 2010-02-22 15:22:02.
96 failures have occurred since the last success.
[Replications Check,ORION2] A recent replication attempt failed:
From ORIONCH to ORION2
Naming Context: CN=Schema,CN=Configuration,DC=orion,DC=net,DC=au
The replication generated an error (-2146893022):
Win32 Error -2146893022
The failure occurred at 2010-02-23 14:38:17.
The last success occurred at 2010-02-22 15:22:01.
96 failures have occurred since the last success.
[Replications Check,ORION2] A recent replication attempt failed:
From ORIONCH to ORION2
Naming Context: CN=Configuration,DC=orion,DC=net,DC=au
The replication generated an error (-2146893022):
Win32 Error -2146893022
The failure occurred at 2010-02-23 14:38:17.
The last success occurred at 2010-02-22 15:22:01.
96 failures have occurred since the last success.
[Replications Check,ORION2] A recent replication attempt failed:
From ORIONCH to ORION2
Naming Context: DC=orion,DC=net,DC=au
The replication generated an error (-2146893022):
Win32 Error -2146893022
The failure occurred at 2010-02-23 14:38:16.
The last success occurred at 2010-02-22 15:22:01.
96 failures have occurred since the last success.
* Replication Latency Check
REPLICATION-RECEIVED LATENCY WARNING
ORION2: Current time is 2010-02-23 14:40:10.
DC=ForestDnsZones,DC=orion,DC=net,DC=au
Last replication recieved from ORIONCH at 2010-02-22 15:22:02.
Latency information for 3 entries in the vector were ignored.
3 were retired Invocations. 0 were either: read-only replicas and are not verifiably latent, or dc's no longer replicating this nc. 0 had no latency information (Win2K DC).
DC=DomainDnsZones,DC=orion,DC=net,DC=au
Last replication recieved from ORIONCH at 2010-02-22 15:22:01.
Latency information for 3 entries in the vector were ignored.
3 were retired Invocations. 0 were either: read-only replicas and are not verifiably latent, or dc's no longer replicating this nc. 0 had no latency information (Win2K DC).
CN=Schema,CN=Configuration,DC=orion,DC=net,DC=au
Last replication recieved from ORIONCH at 2010-02-22 15:22:01.
Latency information for 4 entries in the vector were ignored.
4 were retired Invocations. 0 were either: read-only replicas and are not verifiably latent, or dc's no longer replicating this nc. 0 had no latency information (Win2K DC).
CN=Configuration,DC=orion,DC=net,DC=au
Last replication recieved from ORIONCH at 2010-02-22 15:22:01.
Latency information for 4 entries in the vector were ignored.
4 were retired Invocations. 0 were either: read-only replicas and are not verifiably latent, or dc's no longer replicating this nc. 0 had no latency information (Win2K DC).
DC=orion,DC=net,DC=au
Last replication recieved from ORIONCH at 2010-02-22 15:22:01.
Latency information for 4 entries in the vector were ignored.
4 were retired Invocations. 0 were either: read-only replicas and are not verifiably latent, or dc's no longer replicating this nc. 0 had no latency information (Win2K DC).
* Replication Site Latency Check
......................... ORION2 passed test Replications
Test omitted by user request: Topology
Test omitted by user request: CutoffServers
Starting test: NCSecDesc
* Security Permissions check for all NC's on DC ORION2.
* Security Permissions Check for
DC=ForestDnsZones,DC=orion,DC=net,DC=au
(NDNC,Version 2)
* Security Permissions Check for
DC=DomainDnsZones,DC=orion,DC=net,DC=au
(NDNC,Version 2)
* Security Permissions Check for
CN=Schema,CN=Configuration,DC=orion,DC=net,DC=au
(Schema,Version 2)
* Security Permissions Check for
CN=Configuration,DC=orion,DC=net,DC=au
(Configuration,Version 2)
* Security Permissions Check for
DC=orion,DC=net,DC=au
(Domain,Version 2)
......................... ORION2 passed test NCSecDesc
Starting test: NetLogons
* Network Logons Privileges Check
Verified share \\ORION2\netlogon
Verified share \\ORION2\sysvol
......................... ORION2 passed test NetLogons
Starting test: Advertising
The DC ORION2 is advertising itself as a DC and having a DS.
The DC ORION2 is advertising as an LDAP server
The DC ORION2 is advertising as having a writeable directory
The DC ORION2 is advertising as a Key Distribution Center
The DC ORION2 is advertising as a time server
The DS ORION2 is advertising as a GC.
......................... ORION2 passed test Advertising
Starting test: KnowsOfRoleHolders
Role Schema Owner = CN=NTDS Settings,CN=ORION2,CN=Servers,CN=Balcatta,CN=Sites,CN=Configuration,DC=orion,DC=net,DC=au
Role Domain Owner = CN=NTDS Settings,CN=ORION2,CN=Servers,CN=Balcatta,CN=Sites,CN=Configuration,DC=orion,DC=net,DC=au
Role PDC Owner = CN=NTDS Settings,CN=ORION2,CN=Servers,CN=Balcatta,CN=Sites,CN=Configuration,DC=orion,DC=net,DC=au
Role Rid Owner = CN=NTDS Settings,CN=ORION2,CN=Servers,CN=Balcatta,CN=Sites,CN=Configuration,DC=orion,DC=net,DC=au
Role Infrastructure Update Owner = CN=NTDS Settings,CN=ORION2,CN=Servers,CN=Balcatta,CN=Sites,CN=Configuration,DC=orion,DC=net,DC=au
......................... ORION2 passed test KnowsOfRoleHolders
Starting test: RidManager
* Available RID Pool for the Domain is 4804 to 1073741823
* orion2.orion.net.au is the RID Master
* DsBind with RID Master was successful
* rIDAllocationPool is 2804 to 3303
* rIDPreviousAllocationPool is 2304 to 2803
* rIDNextRID: 2759
* Warning :There is less than 9% available RIDs in the current pool
......................... ORION2 passed test RidManager
Starting test: MachineAccount
Checking machine account for DC ORION2 on DC ORION2.
* SPN found :LDAP/orion2.orion.net.au/orion.net.au
* SPN found :LDAP/orion2.orion.net.au
* SPN found :LDAP/ORION2
* SPN found :LDAP/orion2.orion.net.au/ORION
* SPN found :LDAP/1d83042b-1be4-41d1-9e5d-117829335117._msdcs.orion.net.au
* SPN found :E3514235-4B06-11D1-AB04-00C04FC2DCD2/1d83042b-1be4-41d1-9e5d-117829335117/orion.net.au
* SPN found :HOST/orion2.orion.net.au/orion.net.au
* SPN found :HOST/orion2.orion.net.au
* SPN found :HOST/ORION2
* SPN found :HOST/orion2.orion.net.au/ORION
* SPN found :GC/orion2.orion.net.au/orion.net.au
......................... ORION2 passed test MachineAccount
Starting test: Services
* Checking Service: Dnscache
* Checking Service: NtFrs
* Checking Service: IsmServ
* Checking Service: kdc
* Checking Service: SamSs
* Checking Service: LanmanServer
* Checking Service: LanmanWorkstation
* Checking Service: RpcSs
* Checking Service: w32time
* Checking Service: NETLOGON
......................... ORION2 passed test Services
Test omitted by user request: OutboundSecureChannels
Starting test: ObjectsReplicated
ORION2 is in domain DC=orion,DC=net,DC=au
Checking for CN=ORION2,OU=Domain Controllers,DC=orion,DC=net,DC=au in domain DC=orion,DC=net,DC=au on 1 servers
Object is up-to-date on all servers.
Checking for CN=NTDS Settings,CN=ORION2,CN=Servers,CN=Balcatta,CN=Sites,CN=Configuration,DC=orion,DC=net,DC=au in domain CN=Configuration,DC=orion,DC=net,DC=au on 1 servers
Object is up-to-date on all servers.
......................... ORION2 passed test ObjectsReplicated
Starting test: frssysvol
* The File Replication Service SYSVOL ready test
File Replication Service's SYSVOL is ready
......................... ORION2 passed test frssysvol
Starting test: frsevent
* The File Replication Service Event log test
There are warning or error events within the last 24 hours after the
SYSVOL has been shared. Failing SYSVOL replication problems may cause
Group Policy problems.
An Warning Event occured. EventID: 0x800034C4
Time Generated: 02/22/2010 19:32:13
(Event String could not be retrieved)
......................... ORION2 failed test frsevent
Starting test: kccevent
* The KCC Event log test
An Warning Event occured. EventID: 0x8000061E
Time Generated: 02/23/2010 14:28:17
Event String: All domain controllers in the following site thatcan replicate the directory partition over thistransport are currently unavailable.
Site: CN=CastleHill,CN=Sites,CN=Configuration,DC=orion,DC=net,DC=au
Directory partition: DC=orion,DC=net,DC=au
Transport: CN=IP,CN=Inter-Site Transports,CN=Sites,CN=Configuration,DC=orion,DC=net,DC=au
An Error Event occured. EventID: 0xC000051F
Time Generated: 02/23/2010 14:28:17
Event String: The Knowledge Consistency Checker (KCC) hasdetected problems with the following directorypartition. Directory partition:
DC=orion,DC=net,DC=au
There is insufficient site connectivity information in Active Directory Sites and Services for the KCC to create a spanning tree replication topology. Or, one or more domain controllers with this directory partition are unable to replicate the directory partition information. This is probably due to inaccessible domain controllers.
User Action
Use Active Directory Sites and Services to perform one of the following actions:

- Publish sufficient site connectivity information so that the KCC can determine a route by which this directory partition can reach this site. This is the preferred option.
- Add a Connection object to a domain controller that contains the directory partition in this site from a domain controller that contains the same directory partition in another site.
If neither of the Active Directory Sites and Services tasks correct this condition, see previous events logged by the KCC that identify the inaccessible domain controllers.

An Warning Event occured. EventID: 0x80000749
Time Generated: 02/23/2010 14:28:17
Event String: The Knowledge Consistency Checker (KCC) was unable to form a complete spanning tree network topology. As a result, the following list of sites cannot be reached from the local site.
Sites:
CN=CastleHill,CN=Sites,CN=Configuration,DC=orion,DC=net,DC=au
An Warning Event occured. EventID: 0x8000061E
Time Generated: 02/23/2010 14:28:17
Event String: All domain controllers in the following site that can replicate the directory partition over this transport are currently unavailable.

Site:
CN=CastleHill,CN=Sites,CN=Configuration,DC=orion,DC=net,DC=au
Directory partition: DC=ForestDnsZones,DC=orion,DC=net,DC=au
Transport: CN=IP,CN=Inter-Site Transports,CN=Sites,CN=Configuration,DC=orion,DC=net,DC=au
An Error Event occured. EventID: 0xC000051F
Time Generated: 02/23/2010 14:28:17
Event String: The Knowledge Consistency Checker (KCC) has detected problems with the following directory partition.
Directory partition: DC=ForestDnsZones,DC=orion,DC=net,DC=au
There is insufficient site connectivity information in Active Directory Sites and Services for the KCC to create a spanning tree replication topology. Or, one or more domain controllers with this directory partition are unable to replicate the directory partition information. This is probably due to inaccessible domain controllers.
User Action
Use Active Directory Sites and Services to perform one of the following actions:
- Publish sufficient site connectivity information so that the KCC can determine a route by which this directory partition can reach this site. This is the preferred option.
- Add a Connection object to a domain controller that contains the directory partition in this site from a domain controller that contains the same directory partition in another site. If neither of the Active Directory Sites and Services tasks correct this condition, see previous events logged by the KCC that identify the inaccessible domain controllers.

An Warning Event occured. EventID: 0x80000749
Time Generated: 02/23/2010 14:28:17
Event String: The Knowledge Consistency Checker (KCC) was unable to form a complete spanning tree network topology. As a result, the following list of sites cannot be reached from the local site.
Sites:
CN=CastleHill,CN=Sites,CN=Configuration,DC=orion,DC=net,DC=au
An Warning Event occured. EventID: 0x8000061E
Time Generated: 02/23/2010 14:28:17
Event String: All domain controllers in the following site thatcan replicate the directory partition over this transport are currently unavailable.

Site: CN=CastleHill,CN=Sites,CN=Configuration,DC=orion,DC=net,DC=au
Directory partition: DC=DomainDnsZones,DC=orion,DC=net,DC=au
Transport:
CN=IP,CN=Inter-Site Transports,CN=Sites,CN=Configuration,DC=orion,DC=net,DC=au
An Error Event occured. EventID: 0xC000051F
Time Generated: 02/23/2010 14:28:17
Event String: The Knowledge Consistency Checker (KCC) has detected problems with the following directory partition.
Directory partition: DC=DomainDnsZones,DC=orion,DC=net,DC=au
There is insufficient site connectivity information in Active Directory Sites and Services for the KCC to create a spanning tree replication topology. Or, one or more domain controllers with this directory partition are unable to replicate the directory partition information. This is probably due to inaccessible domain controllers.
User Action
Use Active Directory Sites and Services to perform one of the following actions:
- Publish sufficient site connectivity information so that the KCC can determine a route by which this directory partition can reach this site. This is the preferred option.
- Add a Connection object to a domain controller that contains the directory partition in this site from a domain controller that contains the same directory partition in another site.
If neither of the Active Directory Sites and Services tasks correct this condition, see previous events logged by the KCC that identify the inaccessible domain controllers.

An Warning Event occured. EventID: 0x80000749
Time Generated: 02/23/2010 14:28:17
Event String: The Knowledge Consistency Checker (KCC) was unable to form a complete spanning tree network topology. As a result, the following list of sites cannot be reached from the local site.
Sites:
CN=CastleHill,CN=Sites,CN=Configuration,DC=orion,DC=net,DC=au
An Warning Event occured. EventID: 0x8000061E
Time Generated: 02/23/2010 14:28:17
Event String: All domain controllers in the following site that can replicate the directory partition over this transport are currently unavailable.
Site: CN=CastleHill,CN=Sites,CN=Configuration,DC=orion,DC=net,DC=au
Directory partition: CN=Configuration,DC=orion,DC=net,DC=au
Transport: CN=IP,CN=Inter-Site Transports,CN=Sites,CN=Configuration,DC=orion,DC=net,DC=au
An Error Event occured. EventID: 0xC000051F
Time Generated: 02/23/2010 14:28:17
Event String: The Knowledge Consistency Checker (KCC) has detected problems with the following directory partition.
Directory partition: CN=Configuration,DC=orion,DC=net,DC=au
There is insufficient site connectivity information in Active Directory Sites and Services for the KCC to create a spanning tree replication topology. Or, one or more domain controllers with this directory partition are unable to replicate the directory partition information. This is probably due to inaccessible domain controllers.
User Action Use Active Directory Sites and Services to perform one of the following actions:
- Publish sufficient site connectivity information so that the KCC can determine a route by which this directory partition can reach this site. This is the preferred option.
- Add a Connection object to a domain controller that contains the directory partition in this site from a domain controller that contains the same directory partition in another site. If neither of the Active Directory Sites and Services tasks correct this condition, see previous events logged by the KCC that identify the inaccessible domain controllers.

Starting test: systemlog
* The System Event log test
An Error Event occured. EventID: 0x40000004
Time Generated: 02/23/2010 14:10:54
Event String: The kerberos client received a KRB_AP_ERR_MODIFIED error from the server host/orionch.orion.net.au. The target name used was ldap/ORIONCH.orion.net.au. This indicates that the password used to encrypt the Kerberos service ticket is different than that on the target server. Commonly, this is due to identically named machine accounts in the target realm (ORION.NET.AU), and the client realm.
......................... ORION2 failed test systemlog
Test omitted by user request: VerifyReplicas
Starting test: VerifyReferences
The system object reference (serverReference)
CN=ORION2,OU=Domain Controllers,DC=orion,DC=net,DC=au and backlink on
CN=ORION2,CN=Servers,CN=Balcatta,CN=Sites,CN=Configuration,DC=orion,DC=net,DC=au
are correct.
The system object reference (frsComputerReferenceBL)
CN=ORION2,CN=Domain System Volume (SYSVOL share),CN=File Replication Service,CN=System,DC=orion,DC=net,DC=au
and backlink on CN=ORION2,OU=Domain Controllers,DC=orion,DC=net,DC=au
are correct.
The system object reference (serverReferenceBL)
CN=ORION2,CN=Domain System Volume (SYSVOL share),CN=File Replication Service,CN=System,DC=orion,DC=net,DC=au
and backlink on
CN=NTDS Settings,CN=ORION2,CN=Servers,CN=Balcatta,CN=Sites,CN=Configuration,DC=orion,DC=net,DC=au
are correct.
......................... ORION2 passed test VerifyReferences
Test omitted by user request: VerifyEnterpriseReferences
Test omitted by user request: CheckSecurityError

Running partition tests on : ForestDnsZones
Starting test: CrossRefValidation
......................... ForestDnsZones passed test CrossRefValidation
Starting test: CheckSDRefDom
......................... ForestDnsZones passed test CheckSDRefDom

Running partition tests on : DomainDnsZones
Starting test: CrossRefValidation
......................... DomainDnsZones passed test CrossRefValidation
Starting test: CheckSDRefDom
......................... DomainDnsZones passed test CheckSDRefDom

Running partition tests on : Schema
Starting test: CrossRefValidation
......................... Schema passed test CrossRefValidation
Starting test: CheckSDRefDom
......................... Schema passed test CheckSDRefDom

Running partition tests on : Configuration
Starting test: CrossRefValidation
......................... Configuration passed test CrossRefValidation
Starting test: CheckSDRefDom
......................... Configuration passed test CheckSDRefDom

Running partition tests on : orion
Starting test: CrossRefValidation
......................... orion passed test CrossRefValidation
Starting test: CheckSDRefDom
......................... orion passed test CheckSDRefDom

Running enterprise tests on : orion.net.au
Starting test: Intersite
Skipping site Balcatta, this site is outside the scope provided by the
command line arguments provided.
Skipping site CastleHill, this site is outside the scope provided by
the command line arguments provided.
......................... orion.net.au passed test Intersite
Starting test: FsmoCheck
GC Name: \\orion2.orion.net.au
Locator Flags: 0xe00001fd
PDC Name: \\orion2.orion.net.au
Locator Flags: 0xe00001fd
Time Server Name: \\orion2.orion.net.au
Locator Flags: 0xe00001fd
Preferred Time Server Name: \\orion2.orion.net.au
Locator Flags: 0xe00001fd
KDC Name: \\orion2.orion.net.au
Locator Flags: 0xe00001fd
......................... orion.net.au passed test FsmoCheck
Test omitted by user request: DNS
Test omitted by user request: DNS

The Resolution


Perform this resolution only if you are experiancing the above symptoms.

1.On the DC that is broken (the one that when using replmon reports the error above) set the Kerberos Key Distribution Center Service to manual and stop the service.

2.From a command prompt on the broken DC enter the following:
netdom resetpwd /s:name_of_working_DC /ud:domain\user /pd:*
where domain\user is an administrator of the domain in the domain_name\user_name format. You will be prompted to enter your password. This is the DC where users cannot get to network resources (in our case ORIONCH).



3.Upon pressing Enter, if the command fails then restart the broken DC and repeat the above command (this restart clears the Kerberos ticket cache and so clears the broken credential attempts that it has stored).

4.Upon successful completion of the command in step 2 restart the broken DC. You must do this even if done already in step 3.

5.Check that replication is working, and if so restart the Kerberos Key Distribution Center Service and set the service back to automatic.

Once done your domain will now be working correctly, use replmon to verify replication:



For more information about this Netdom command view:

http://support.microsoft.com/kb/325850