Wednesday, December 24, 2014

Exchange 2013 FAST Search Technology Failed

I had a customer where Exchange 2013 FAST Search Technology failed and rebuilding the search indexes for the databases did not resolve the problem on a single Exchange 2013 SP1 server.

In this blog post I'm going to share with you the symptoms of my issue along with the resolution.

Symptoms of Issues

Users on the network were unable to search their mailbox from Outlook Web App or using Microsoft Outlook in online mode.

In addition to not being able to search in Outlook Web App or Microsoft Outlook, the Test-ExchangeSearch cmdlet failed with "Time out for test thread".


Despite this issue, the ContentIndexState on the databases remained in a Healthy report status with the Get-MailboxDatabaseCopyStatus cmdlet as shown below:


In the Server Application logs the following error was logged numerous times:

Log Name:      Application
Source:        MSExchangeIS
Date:          4/12/2014 1:54:06 PM
Event ID:      1012
Task Category: General
Level:         Error
Keywords:      Classic
User:          N/A
Computer:      ExchangeServer.domain.local
Description:
Exchange Server Information Store has encountered an error while executing a full-text index query ("eDiscovery search query execution on database a0976351-948a-4625-8840-f649f8b98e0e failed."). Error information: System.ServiceModel.EndpointNotFoundException: Could not connect to net.tcp://localhost:3847/. The connection attempt lasted for a time span of 00:00:02.0821592. TCP error code 10061: No connection could be made because the target machine actively refused it 127.0.0.1:3847.  ---> System.Net.Sockets.SocketException: No connection could be made because the target machine actively refused it 127.0.0.1:3847


Checking the size of the catalog folder was only around 16MB when the database size was 150GB.  From my experience a FAST index for a 150GB database is generally between 5GB and 10GB in size depending on how small the items are inside the database.

Issue Resolution

To resolve the issue the following steps were taken in order.

1. Stop the Microsoft Exchange Search and Microsoft Exchange Search Host Controller service.
 
2. Remove all the files under “Fsis” file:

C:\Program Files\Microsoft\Exchange Server\V15\Bin\Search\Ceres\HostController\Data\Nodes\Fsis
We moved these files to another folder for backup purposes.

3. Open EMS and “Run as Administrator"

4. Navigate to directory C:\Program Files\Microsoft\Exchange Server\V15\Bin\Search\Ceres\Installer

5. Launch the script and run the command below:


.\installconfig.ps1 -action I -datafolder "c:\Program Files\Microsoft\Exchange Server\V15\Bin\Search\Ceres\HostController\Data"
This script will recreate the files under "C:\Program Files\Microsoft\Exchange Server\V15\Bin\Search\Ceres\HostController\Data\Nodes\Fsis" that we backed up/removed previously.

6. Once the script installs the data, you need to uninstall and reinstall again (don't ask, it did not work for me without doing this):

.\installconfig.ps1 -action U -datafolder "c:\Program Files\Microsoft\Exchange Server\V15\Bin\Search\Ceres\HostController\Data"

Then again installed Search component with below command :

.\installconfig.ps1 -action I -datafolder "c:\Program Files\Microsoft\Exchange Server\V15\Bin\Search\Ceres\HostController\Data"


7. Force Active Directory Replication from a domain controller in the same AD Site as the Exchange 2013 server with repadmin /syncall /APeD.

8. Check on the Exchange Server with Get-MailboxDatabaseCopyStatus and review the ContentIndexState.


When the Content Index process has finished crawling we can see the index size of the folder is now 8GB and searching is now working as normal.

 

Tuesday, December 23, 2014

Filtering Exchange MessageTrackingLog Output from Get-MessageTrackingLog Cmdlet

In todays blog I am going to show you how to filter Message Tracking Log output from the Get-MessageTrackingLog command.  I had a customer from an organisation example.com who wanted to see all external recipients who received an email with the message subject "Staffing Update".

Without using some PowerShell filtering, you can view all tracking log related entries for this against all Transport Servers in your organisation by running:

Get-TransportService | Get-MessageTrackingLog -MessageSubject "Staffing Update"

Note: If your running Exchange 2007 or 2010 replace Get-TransportService with Get-TransportServer.

This command will return all messages relayed in the last 30 days by default which is the amount of time the message tracking logs hang around for by default.

However if we want to meet my customers requirement, we want the output to not contain any recipients for "*@example.com" so that we can focus on emails leaving the company.  When I say external users or "leaving the company", I mean users who do not have an email address for "*@example.com", you may have an Exchange Organisation where you have multiple accepted domains setup for multi-tenancy!

To do this we need to do some filtering with PowerShell.  This can be achieved using the where{ } command as follows:

Get-TransportService | Get-MessageTrackingLog -MessageSubject "Staffing Update" | where{$_.recipients -notlike "*@example.com"}

But say we only want to see which users received this message with a @example.com email address.  Easy this can be done by reverting the -notlike to a -like.

Get-TransportService | Get-MessageTrackingLog -MessageSubject "Staffing Update" | where{$_.recipients -like "*@example.com"}

You can also easily filter senders by replacing the $_.recipients line with $_.senders like shown below:

where{$_.sender -like "*@example.com"}

Happy Filtering!

Wednesday, December 10, 2014

Microsoft PST Capture 2.0 Not Supported on Exchange 2013

The PST Capture 2.0 tool is not supported with Exchange 2013 despite being documented as supported under the following Microsoft Exchange Product Team blog post:

http://blogs.technet.com/b/exchange/archive/2013/02/22/time-to-go-pst-hunting-with-the-new-pst-capture-2-0.aspx

I have used this tool successfully with previous versions of Exchange Server however with Exchange Server 2013 this tool does not work.  I have tested PST Capture 2.0 with Exchange Server 2013 SP1 and Update Rollup 6, both unsuccessful.  Testing has been performed both in my lab environment and at a customer site.

I logged this issue with Microsoft under a professional services support case in which Microsoft were able to reproduce the issue in their lab environment.  This issue was then escalated within Microsoft and an issue identified with the tool with no fix provided.  Microsoft explained they are under no obligation to develop a hotfix for this tool as the End User Licensing Agreement (EULA) states that the software is provided "as is" and they are not under obligation to support it should issues or bugs be flagged with the software.  This is shown in my screenshot below.


When attempting to import a mailbox with PST Capture 2.0, the following error message is flagged in the tool:

Import error: Error opening mailbox


This issue is generally caused by one of the following issues as documented on the following website:

http://www.sysadminsblog.com/microsoft/pst-capture-errors/

However with Exchange 2013, none of these resolve the problem.

The PST Capture tool has a more advanced logging available for errors which can be found under the following location:

"C:\ProgramData\Microsoft\Exchange\PST Capture\Logs\MapiComServer 15"


The technical error which is generated when attempting to import to Exchange 2013 is as follows:

Failed to open default store - HR 8004011d


Hopefully Microsoft does get around to developing a fix/work around for this product as it will be a shame to purchase this tool from Red Gate Software and then not maintain it hence making it redundant.

Wednesday, December 3, 2014

How to Change Network Profile in Registry for 2008 Server

In a previous post I wrote about how to change the network location or "firewall profile" in Windows Server 2012 using PowerShell as you could no longer do this from the GUI like you could in previous versions of Windows Server. This article can be found under the following URL:

http://clintboessen.blogspot.com.au/2013/11/how-to-change-your-network-profile-in.html

However yesterday I had a problem at a customer running Windows Server 2008 where I was unable to change the network location from the GUI using Network and Sharing Centre and as a result I had to revert to the registry.  Tracking down the registry key responsible for the network profile was a difficult task so I decided to blog this one.

All network profiles are located under the following registry key:

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\NetworkList\Profiles


Each network card will be represented below in a sub key with a GUID.  The Network Location is controlled by a DWORD value called Category.

Public = 0
Home = 1
Domain = 2

As you see my category is set to 2 which means this network adapter is using the Domain Firewall profile.

Tuesday, December 2, 2014

Remove Power Shell and Server Manager Pinned Icons from Start Menu Server 2012

In Windows Server 2012 and Windows Server 2012 R2 by default all users have a Power Shell icon and a Server Manager icon pinned to their start menu.  When setting up a Remote Desktop Session Host with Windows Server 2012 Remote Desktop Services you may not want users to have the Server Manager icon or Power Shell icon especially when desktop access is enabled.

 
In previous versions of Windows Server there was a Group Policy setting called "Remove pinned programs list from the Start Menu" which administrators could use to remove these pinned applications from the start menu from Remote Desktop Session Hosts.
 
 
This policy no longer works on Windows Server 2012 and there is no well documented method available for doing this on the Internet.
 
In Windows Server 2012, the three default pinned applications on the start menu are located under the following location:
 
C:\Users\username\AppData\Roaming\Microsoft\Internet Explorer\Quick Launch\User Pinned\TaskBar
 
 
The User Pinned\Taskbar  folder in the Quick Launch directory does not exist in the Default profile in Windows Server 2012 as shown below.


So how do these icons get created in the users profile and where do they come from?

This process appears to be hardcoded into Windows Server 2012 and executed upon creating a new user profile.  The user profile creation process creates Quick Launch\User Pinned\TaskBar folder as part of the login then copies a bunch of "lnk" shortcut files from the "All Users Profile".

Server Manager Pinned Icon

The "Server Manager.lnk" file gets copied by the user profile creation From:

"C:\ProgramData\Microsoft\Windows\Start Menu\Programs\Administrative Tools\Server Manager.lnk"

To:

"C:\Users\username\AppData\Roaming\Microsoft\Internet Explorer\Quick Launch\User Pinned\TaskBar\Server Manager.lnk"

Windows PowerShell Pinned Icon

The "Windows PowerShell.lnk" file gets copied by the user profile creation From:

"C:\ProgramData\Microsoft\Windows\Start Menu\Programs\System Tools\Windows PowerShell.lnk"

To:

"C:\Users\username\AppData\Roaming\Microsoft\Internet Explorer\Quick Launch\User Pinned\TaskBar\Windows PowerShell.lnk"

How to Prevent the icons from getting Populated to User Profiles

To stop the "Windows PowerShell.lnk" and "Server Manager.lnk" shortcuts from getting pinned to the start menu on all users, simply delete these icons from the All Users profile under "C:\ProgramData" from the following locations:

"C:\ProgramData\Microsoft\Windows\Start Menu\Programs\Administrative Tools\Server Manager.lnk"

and

"C:\ProgramData\Microsoft\Windows\Start Menu\Programs\System Tools\Windows PowerShell.lnk"

 
Now when new users login for the first time, the User Profile creation process is unable to copy the "Server Manager.lnk" and "Windows PowerShell.lnk" shortcuts as they no longer exist as shown in the following screenshot.
 
 

Sunday, November 30, 2014

Windows Server 2012 R2 Print Managmenet Console - Displays No Printers

There is a bug in Windows Server 2012 R2 which causes Print Management Console to display no printers as shown in the screenshot below.  This makes it impossible to deploy printers with Group Policy using the local server.


Print Management Console does however show drivers for any printers installed.


If you connect to the print server remotely from another print server such as a 2008 R2 server, the printers are displayed in the Print Management console as shown below.

 
This bug is triggered when Microsoft Hotfix KB2995388 is installed on a 2012 R2 print server which is the case in our scenario as shown in the "systeminfo" extract:
 
 
 To resolve the issue, uninstall KB2995388 on the 2012 R2 Print Server and reboot.

Tuesday, November 18, 2014

Distributed File Server Replication on Windows Server 2012 R2 Bug 2951262

I came across a bug with Windows Server 2012 R2 where a spoke server randomly failed to replicate to the hub server.  In total there were three DFSR servers partaking in replication within a single replication group.

The following diagram shows an overview of the topology and the server experiencing issues:

Randomly SPOKE2 stopped replicating to the hub server HUBSERVER and is no longer responding to DFS Health Reports resulting in the health report generation process hanging forever.


When restarting the DFS-R service on SPOKE2, SPOKE2 gets reported to be in an Indeterminate State for approximately 2-3 hours.


 After being in an Indeterminate State for 2-3 hours, the status change to “Auto Recovery” for approximately 6 more hours.  During this time the DFS-R service generates a large amount of disk activity as it goes through and checks all the files.  This can be observed using Windows Resource Monitor.

The Auto Recovery process never completes successfully, nor are there any errors in the event log on SPOKE2.  Rebooting SPOKE2 or restarting the DFS-R service results in the server going back to an indeterminate state for another 2-3 hours then starting the Auto Recovery process again.

Resolution

This issue is caused by a bug with Windows Server 2012 R2 documented on Microsoft KB 2951262.

http://support.microsoft.com/kb/2951262

You must manually request the Hotfix which Microsoft will email to you and install it on the effected servers.  After installing the hotfix the server will revert to an Indeterminate state then start Auto Recovery again however this time after the Auto Recovery process, it will resume replication as normal.

Public Folder Migration Request with StatusDetail of FailedOther

I was in the process of migrating to Exchange 2013 "Modern Public Folders" for a customer of mine here in Western Australia from an Exchange 2010 SP3 server to Exchange 2013 SP1.  After commencing the migration with the New-PublicFolderMigrationRequest cmdlet the migration request shortly after failed.

Looking at the Migration Request Statistics with the following cmdlet it came up with StatusDetail as FailedOther:

Get-PublicFolderMigrationRequest | Get-PublicFolderMigrationRequestStatistics

 
 When looking into the error in more detail with

"Get-PublicFolderMigrationRequest | Get-PublicFolderMigrationRequestStatistics | fl"

 The following was logged regarding the error message:

FailureCode: -2146233088
FailureType: DataValidationException
Message: Error: Property expression "Outlook Security Settings" isn't valid.  Valid values are: Strings formed with characters from A to Z (uppercase and lowercase), digits from 0 to 9, !, #, $, &, ', *, +, -, /, =, ?, ^, _, `, {, |, } or ~. One or more periods may be embedded in the alias, but each period should be preceded and followed by at least one of the other characters.


In the public folder structure at my customer, there was folder named "Outlook Security Settings" as per the error message.

 
This folder name or any of the sub items did not appear to breach any of the invalid characters as per the stated error message.  Luckily this folder was not required by the business and I simply remove the problematic public folder from the Exchange 2010 server and repeated the move request.  After preforming this I was able to successfully complete the public folder migration request.

Monday, November 10, 2014

Problems When Upgrading your domain and forest functional level from 2003 to 2008 R2

A customer of mine upgraded their Domain Functional Level (DFL) and Forest Functional Level (FFL) to Windows Server 2008 R2 yesterday.  Today when employees started work, they experienced lengthy login times, did not receive their network drive mapping to the file server and were unable to connect to Exchange Server 2010 with Microsoft Outlook 2010.

The first thing I did was have a look at the Active Directory replication after the functional level upgrade using the following command "repadmin /showrepl" on one of the Active Directory domain controllers.  This showed the following error:

Last error: -2146892990 (0x80090342)
The encryption type requested is not supported by the KDC

 
In 2003 functional level the Kerberos Key Distribution Centre (KDC) used either RC4-HMAC 128-bit or DES-CBC-MD5 56-bit for Kerberos Encryption however when moving to 2008 Domain Functional Level (or higher) you upgrade the Key Distribution Centre (KDC) to use Advanced Kerberos Encryption which uses AES 128 and AES 256 encryption.
 
Generally this transition is smooth and does not cause problems however in this instance the KDC did not detect the functional level change and continued to operate using the legacy 2003 functional level encryption technology.  As a result the error "The encryption type required is not supported by the KDC".
 
To resolve this problem was very simple, we simply restarted the Kerberos Key Distribution Centre on all of the Active Directory domain controllers in the domain.
 
 
Within 5 minutes of the service restart, things were back to normal.

Tuesday, November 4, 2014

Failed to submit the request because public folder migration has already been successfully completed previously

When migrating to Exchange 2013 modern public folders, you may want to repeat the migration especially if you need to tweak the Public Folder-to-Mailbox Mapping File and perform the migration again.

To repeat the migration progress you need to perform the following steps:
  1. Removing all public folders from the newly created public folder mailbox with Get-PublicFolder –Recurse | Remove-PublicFolder (make sure you run this on Exchange 2013 Management Shell so you don't delete the public folders on your 2007/2010 environment).
  2. Removing the default public folder hierarchy mailbox with Get-Mailbox –PublicFolder | Remove-Mailbox -PublicFolder
  3. Clearing the MsExchDefaultPublicFolderMailbox attribute on the Exchange organisation with ADSIEdit as per Exchange MVP Satheshwaran Manoharan's article http://careexchange.in/how-to-recreate-public-folder-master-hierarchy-in-exchange-2013/
  4. Remove the Exchange 2013 public folder move request using Get-PublicFolderMigrationRequest | Remove-PublicFolderMoveRequest

At this point you would think you should be ready to migrate the public folders to Exchange 2013 again however there is a final step which needs to be performed.  This final step has caught out many Exchange Admins and is not documented at all online.  Without the final step, the following error is experienced:

"Failed to submit the request because public folder migration has already been successfully completed previously"


This error occurs if the PublicFolderMigrationComplete attribute is set to $true on the Exchange Organisation Configuration as shown in the following screenshot.

Simply use the following PowerShell command to set the PublicFolderMigrationComplete attribute to $false.

Set-OrganizationConfig -PublicFolderMigrationComplete $false


After this final step you can go ahead and remigrate your public folders to Exchange 2013 from scratch.

Tuesday, October 14, 2014

Exchange 2010 User Sends Two Out of Office Messages

One of my customers had a mailbox user which generated two Out of Office messages whenever her Out of Office automated replies were enabled.  The first Out of Office message is the message the user set on her mailbox as the automated reply for both internal and external Out of Office.  The second Out of Office message is a different message for a user which no longer exists on the network and the contents of the Out of Office message does not display in Outlook Web App or Outlook.

The legitimate Out of Office is provided with a message subject of:

Automatic reply:

The second unwanted Out of Office is provided with a message subject of:

Out of Office

I created a test user account and send the problematic user a test email when her Out of Office was enable.  The following screenshot shows the extra Out of Office message sent:



First thing I checked was the MailboxAutoReplyConfiguration set on the mailbox which shows the Out of Office message set on the Mailbox for both Internal and External recipients.  This did not reveal the second Out of Office message causing the problem for this user account.


This means the second problematic Out of Office response must be stored elsewhere in the users mailbox.  Before going gown the path of downloading MFCMAPI and manually browsing through the problematic mailbox for the second Out of Office rule, I decided to run the "outlook.exe /cleanrules" Outlook 2010 command.  This command cleans and removes both server side and client side rules including any Out of Office rules which may exist in the mailbox.

Simply close Outlook on the users PC and then execute the following command:

 
After cleaning the rules and retesting the users Out of Office from my test mailbox, I only received the single Out of Office message which is the one we configured on the users mailbox as shown in the following screenshot.
 
 
Summary

Whilst a resolution to the issue was found, I am not sure what caused this issue.  The problematic mailbox is a member of a Database Availability Group cluster and was migrated cross-forest from an Exchange 2007 environment.  This should not cause problems as Exchange 2007 utilises a similar architecture for Out of Office messages as Exchange 2010 supporting the concept of both Internal and External Out of Office responses.  I believe this mailbox has been around for a long time and this legacy rule may have been brought across from Exchange 2003 as this particular customer uses role based mailboxes and when new users come on-board they inherit the last users mailbox keeping the same email address.

Please leave your comments if you also experience this issue.

Monday, October 13, 2014

Error Moving Mailbox Exchange 2007 to Exchange 2010

Today I had an issue moving a mailbox from Exchange 2007 to Exchange 2010 where the following error was generated in Exchange Management Shell.

Active Directory operation failed on domaincontroller.domain.local. This error is not retriable. Additional information: Insufficient access rights to perform the operation.
Active directory response: 00002098: SecErr: DSID-03150BB9, problem 4003 (INSUFF_ACCESS_RIGHTS), data 0
    + CategoryInfo          : NotSpecified: (0:Int32) [New-MoveRequest], ADOperationException
    + FullyQualifiedErrorId : 7189915F,Microsoft.Exchange.Management.RecipientTasks.NewMoveRequest


 
The error occurs if the account being moved does not have correct Exchange 2010 permissions set on the user account in Active Directory.  On the user account in Active Directory Users and Computers on the Security Tab under Advanced, "Include inheritable permissions from this object's parent" was not selected as shown below:
 
Note: In order to see the Security Tab on the user account in Active Directory Users and Computers you must enable "Advanced Features" which can be found under the view menu.
 
Simply select Include inheritable permissions from this object's parent.
 
 
 After making this change the move worked as expected.

 

Thursday, October 9, 2014

Windows XP clients must match the Certificate Common Name in Outlook for RPC over HTTP

Back in 2009 I wrote an article called "Configuring Outlook Anywhere Settings via Autodiscover" which can be found at the following URL:

http://clintboessen.blogspot.com.au/2009/06/configuring-outlook-anywhere-settings.html

Also in 2009 I wrote an article called "Outlook Anywhere keeps prompting for Password" which can be found at the following URL:

http://clintboessen.blogspot.com.au/2009/06/outlook-anywhere-keeps-prompting-for.html

In this blog post I am going to touch on these two topics a bit further as this issue will impact many organisations moving to Exchange 2013 for those still running Windows XP (despite the fact it is no longer supported) seeming Exchange 2013 only supports RPC over HTTP or MAPI over HTTP (out of scope for this article).  To recap what I wrote about in the previous articles 5 years ago, for RPC over HTTP(s) aka Outlook Anywhere to work on any version of the Outlook Client on the XP, the MSSTD value must match the "Common Name" on the certificate.  What am I talking about?  Well let me show you...

The MSSTD is specified under "Only connect to proxy servers that have this principal name in their certificate" which can be found within Outlook under Account Settings, Open the Account, More Settings, Connection Tab and finally Exchange Proxy Settings.

 
This value must match the "Common Name" of the certificate which in this example is mail.example.com as shown below next to "Issued to:"
 
 
From Windows Vista onwards the MSSTD can match any name in the Certificate which includes the Certificate Common Name as shown above and any Subject Alternative Names (SAN) which may exist on the certificate.  These are often used and can be easily located on the details tab of a certificate in Microsoft Windows as shown below:
 
 
For Windows XP the "Only connect to proxy servers that have this principal name in their certificate" value MUST MATCH the common name on the Certificate.  The symptom for having this not matching is the user continuously being prompted for credentials in an infinite loop as I addressed under the article Outlook Anywhere keeps prompting for Password.
 
 Note: This also applies to Windows Server 2003 for people with legacy Terminal Server / Citrix environments.
 
What about for Wild Card certificates, how do I set the "Only connect to proxy servers that have this principal name in their certificate" MSSTD value for these for legacy Windows XP clients?
 
For Wild Card certificates the MSSTD value must be set to:
 
msstd:*.domain.com
 
You must actually put the astricts in the DNS name instead of the name the client is connecting to in order for Windows XP to be happy so it matches the exact name of the Wildcard certificate.
 
What about Autodiscover?
 
Now that we understand that the MSSTD must match the common name of the certificate for our legacy Windows XP clients, how do we automatically push this out to our clients via Autodiscover?
 
Exchange 2013 no longer directly uses EXPR/EXCH Outlook Providers which we configured in Exchange 2007/2010, it has two different dynamically generated EXHTTP providers. Users with mailboxes on 2013 will get one set of EXHTTP settings for internal usage and one set of EXHTTP settings for external usage.  You will see this returned to the Outlook client in the Autodiscover response via what looks like a new provider, ExHTTP.
 
However ExHTTP isn’t an actual provider, it is a calculated set of values from the EXCH (internal Outlook Anywhere) and EXPR (External Outlook Anywhere) settings.  The concept of internal Outlook Anywhere and External Outlook Anywhere is new in Exchange 2013 as we
got rid of direct RPC to the Exchange server.  We want the ability to set NTLM authentication for internal and Basic authentication for External as well as different connection URLs.
 
To change the internal MSSTD returned value from Autodiscover use the following command:
 
Internal Outlook Anywhere:
 
Get-OutlookProvider "EXCH" | Set-OutlookProvider -CertPrincipalName "msstd:mail.domain.com"
 
External Outlook Anywhere:
 
Get-OutlookProvider "EXPR" | Set-OutlookProvider -CertPrincipalName "msstd:mail.domain.com"
 
If you have a private internal namespace such as .local, .lan or .internal you may also need to setup split horizon DNS so that you can make the Certificate Common Name match the MSSTD!
 
Hope this article has been helpful.

Monday, October 6, 2014

Setup Wizard for Update Rollup 7 for Exchange 2010 Service Pack 3 (KB2961522) ended prematurely

When installing Update Rollup 7 on Exchange Server 2010 Service Pack 3 I received the following error message:

Setup Wizard for Update Rollup 7 for Exchange 2010 Service Pack 3 (KB2961522) ended prematurely.

Setup Wizard for Update Rollup 7 for Exchange 2010 Service Pack 3 (KB2961522) ended prematurely because of an error.  Your system has not been modified.  To install this program at a later time, please run the installation again.


This error was encountered after running the Exchange2010-KB2961522-x64-en.msp file downloaded from the Microsoft Website.

In the ServiceControl.txt file located under C:\ExchangeSetupLogs the following errors were present:

[12:58:25] Service 'IIS Admin Service (IISAdmin)' cannot be configured due to the following error: Access is denied
[12:58:25] Enabling service 'MSExchangeAB'.
[12:58:25] Service 'Microsoft Exchange Address Book (MSExchangeAB)' cannot be configured due to the following error: Access is denied
[12:58:25] Enabling service 'MSExchangeADTopology'.
[12:58:25] Service 'Microsoft Exchange Active Directory Topology (MSExchangeADTopology)' cannot be configured due to the following error: Access is denied
[12:58:25] Enabling service 'MSExchangeAntispamUpdate'.
[12:58:25] Service 'Microsoft Exchange Anti-spam Update (MSExchangeAntispamUpdate)' cannot be configured due to the following error: Access is denied
[12:58:25] Enabling service 'MSExchangeEdgeSync'.
[12:58:25] Service 'Microsoft Exchange EdgeSync (MSExchangeEdgeSync)' cannot be configured due to the following error: Access is denied
[12:58:25] Enabling service 'MSExchangeFBA'.
[12:58:25] Service 'Microsoft Exchange Forms-Based Authentication service (MSExchangeFBA)' cannot be configured due to the following error: Access is denied
[12:58:25] Enabling service 'MSExchangeFDS'.
[12:58:25] Service 'Microsoft Exchange File Distribution (MSExchangeFDS)' cannot be configured due to the following error: Access is denied
[12:58:25] Enabling service 'MSExchangeIMAP4'.
[12:58:25] Service 'Microsoft Exchange IMAP4 (MSExchangeIMAP4)' cannot be configured due to the following error: Access is denied
[12:58:25] Enabling service 'MSExchangeIS'.
[12:58:25] Service 'Microsoft Exchange Information Store (MSExchangeIS)' cannot be configured due to the following error: Access is denied
[12:58:25] Enabling service 'MSExchangeMailboxAssistants'.
[12:58:25] Service 'Microsoft Exchange Mailbox Assistants (MSExchangeMailboxAssistants)' cannot be configured due to the following error: Access is denied
[12:58:25] Enabling service 'MSExchangeMailboxReplication'.
[12:58:25] Service 'Microsoft Exchange Mailbox Replication (MSExchangeMailboxReplication)' cannot be configured due to the following error: Access is denied
[12:58:25] Enabling service 'MSExchangeMailSubmission'.
[12:58:25] Service 'Microsoft Exchange Mail Submission (MSExchangeMailSubmission)' cannot be configured due to the following error: Access is denied
[12:58:25] Enabling service 'MSExchangeMonitoring'.


and so on..

Generally when you run a .msp update file it automatically provides an User Account Control (UAC) exception however on this customers network it did not.  As a result UAC prevented the update from performing the required tasks.

To resolve the problem I simply needed to run the update from an elevated command prompt window as follows:

 

Cannot bind argument to parameter 'Identity' because it is null

When attempting to install Exchange 2010 SP3 Service Pack on an Exchange 2010 SP1 Server I received the following on the Mailbox Server Role.

Error:
The following error was generated when "$error.Clear();
          if ($RoleCreatePublicFolderDatabase)
          {
            $publicDB = get-PublicFolderDatabase -Server:$RoleFqdnOrName -ErrorAction SilentlyContinue;
            $DB = get-MailboxDatabase -Server:$RoleFqdnOrName -ErrorAction SilentlyContinue;
            if ($publicDB -and $DB)
            {
                set-mailboxdatabase `
                  -Identity:$DB.Identity `
                  -publicFolderDatabase:$publicDB.Identity `
                  -DomainController $RoleDomainController
            }                 
          }
        " was run: "Cannot bind argument to parameter 'Identity' because it is null.".

Cannot bind argument to parameter 'Identity' because it is null.


If you received this error you most likely also received:

http://clintboessen.blogspot.com.au/2014/10/this-server-role-cant-be-installed.html

This error occurs if the original Exchange 2010 SP1 or SP2 installation did not finish "Finalizing Setup" which is the final stage in the installation process.  Failure means important registry keys are not set correctly.  In my case the Finalizing Setup did not complete because of the following error:
 
 
This error is generated if the MailboxRole is missing a ConfiguredVersion string value under the following registry key:
 
HKLM\SOFTWARE\Microsoft\ExchangeServer\v14\MailboxRole
 
As you see in the following screenshot the ConfiguredVersion string value is missing:
 
 
Simply create the string value called ConfiguredVersion and give it the same value as UnpackedVersion as shown in the following screenshot:
 
 
After this I deleted the Exchange 2010 SP3 installation watermark (both the “Action” and “Watermark” entries in the registry) and re-ran setup.  Google for this, lots of information already available on the Internet

This server role can't be installed because the following roles aren't current: AdminToolsRole

When attempting to install Exchange 2010 SP3 service pack on an Exchange 2010 SP1 server I received the following error:

This server role can't be installed because the following roles aren't current: AdminToolsRole

 
This error occurs if the original Exchange 2010 SP1 or SP2 installation did not finish "Finalizing Setup" which is the final stage in the installation process.  Failure means important registry keys are not set correctly.  In my case the Finalizing Setup did not complete because of the following error:
 
 
The above error is triggered if the ConfiguredVersion does not match the UnpackedVersion registry key.  These registry keys can be found under the following location:
 
HKLM\SOFTWARE\Microsoft\ExchangeServer\v14\AdminTools
 
To fix this I made sure the ConfiguredVersion key matches the UnpackedVersion key as shown in the following screenshots:
 
Before:
 
 
After:
 
 
After this I deleted the Exchange 2010 SP3 installation watermark (both the “Action” and “Watermark” entries in the registry) and re-ran setup.  Google for this, lots of information already available on the Internet.
 
Note: If you experienced this error you may also experience the following:
 

Couldn't resolve the user or group "domain.local/Microsoft Exchange Security Groups/Discovery Management." If the user or group is a foreign forest principal, you must have either a two-way trust or an outgoing trust.

I ran into a problem when running the Exchange 2010 SP1 setup in an environment with a single Exchange 2007 SP3 server.  The setup failed on the Mailbox Server role with the following error:

Error:
The following error was generated when "$error.Clear();
          $name = [Microsoft.Exchange.Management.RecipientTasks.EnableMailbox]::DiscoveryMailboxUniqueName;
          $dispname = [Microsoft.Exchange.Management.RecipientTasks.EnableMailbox]::DiscoveryMailboxDisplayName;
          $dismbx = get-mailbox -Filter {name -eq $name} -IgnoreDefaultScope -resultSize 1;
          if( $dismbx -ne $null)
          {
            $srvname = $dismbx.ServerName;
            if( $dismbx.Database -ne $null -and $RoleFqdnOrName -like "$srvname.*" )
            {
              Write-ExchangeSetupLog -info "Setup DiscoverySearchMailbox Permission.";
              $mountedMdb = get-mailboxdatabase $dismbx.Database -status | where { $_.Mounted -eq $true };
              if( $mountedMdb -eq $null )
              {
                Write-ExchangeSetupLog -info "Mounting database before stamp DiscoverySearchMailbox Permission...";
                mount-database $dismbx.Database;
              }

              $mountedMdb = get-mailboxdatabase $dismbx.Database -status | where { $_.Mounted -eq $true };
              if( $mountedMdb -ne $null )
              {
                $dmRoleGroupGuid = [Microsoft.Exchange.Data.Directory.Management.RoleGroup]::DiscoveryManagementWkGuid;
                $dmRoleGroup = Get-RoleGroup -Identity $dmRoleGroupGuid -DomainController $RoleDomainController -ErrorAction:SilentlyContinue;
                if( $dmRoleGroup -ne $null )
                {
                  Add-MailboxPermission $dismbx -User $dmRoleGroup.Identity -AccessRights FullAccess -DomainController $RoleDomainController -WarningAction SilentlyContinue;
                }
              }
            }
          }
        " was run: "Couldn't resolve the user or group "domain.local/Microsoft Exchange Security Groups/Discovery Management." If the user or group is a foreign forest principal, you must have either a two-way trust or an outgoing trust.".

Couldn't resolve the user or group "domain.local/Microsoft Exchange Security Groups/Discovery Management." If the user or group is a foreign forest principal, you must have either a two-way trust or an outgoing trust.
The trust relationship between the primary domain and the trusted domain failed.


The mailbox role is the last role which is installed as part of the Exchange 2010 SP1 setup process.  Despite this error occurring, the Exchange 2010 SP1 installation did complete with all Exchange 2010 services present on the server, started and in a functional state.  The Exchange 2010 SP1 management tools were also installed and working.

As a result I pushed on and began installing the Exchange 2010 SP3 upgrade installation package.  The Exchange 2010 SP3 upgrade installation package also did not complete the installation correctly and failed with the same error:

Error:
The following error was generated when "$error.Clear();
          $name = [Microsoft.Exchange.Management.RecipientTasks.EnableMailbox]::DiscoveryMailboxUniqueName;
          $dispname = [Microsoft.Exchange.Management.RecipientTasks.EnableMailbox]::DiscoveryMailboxDisplayName;
          $dismbx = get-mailbox -Filter {name -eq $name} -IgnoreDefaultScope -resultSize 1;
          if( $dismbx -ne $null)
          {
            $srvname = $dismbx.ServerName;
            if( $dismbx.Database -ne $null -and $RoleFqdnOrName -like "$srvname.*" )
            {
              Write-ExchangeSetupLog -info "Setup DiscoverySearchMailbox Permission.";
              $mountedMdb = get-mailboxdatabase $dismbx.Database -status | where { $_.Mounted -eq $true };
              if( $mountedMdb -eq $null )
              {
                Write-ExchangeSetupLog -info "Mounting database before stamp DiscoverySearchMailbox Permission...";
                mount-database $dismbx.Database;
              }

              $mountedMdb = get-mailboxdatabase $dismbx.Database -status | where { $_.Mounted -eq $true };
              if( $mountedMdb -ne $null )
              {
                $dmRoleGroupGuid = [Microsoft.Exchange.Data.Directory.Management.RoleGroup]::DiscoveryManagementWkGuid;
                $dmRoleGroup = Get-RoleGroup -Identity $dmRoleGroupGuid -DomainController $RoleDomainController -ErrorAction:SilentlyContinue;
                if( $dmRoleGroup -ne $null )
                {
                  Add-MailboxPermission $dismbx -User $dmRoleGroup.Identity -AccessRights FullAccess -DomainController $RoleDomainController -WarningAction SilentlyContinue;
                }
              }
            }
          }
        " was run: "Couldn't resolve the user or group "domain.local/Microsoft Exchange Security Groups/Discovery Management." If the user or group is a foreign forest principal, you must have either a two-way trust or an outgoing trust.".

Couldn't resolve the user or group "domain.local/Microsoft Exchange Security Groups/Discovery Management." If the user or group is a foreign forest principal, you must have either a two-way trust or an outgoing trust.
The trust relationship between the primary domain and the trusted domain failed.


In the Exchange 2010 with SP1 installation package, the Management Tools were installed before the Hub Transport, Client Access and Mailbox Server roles.  When running Exchange 2010 SP3 upgrade package, the Management Tools are installed after the Hub Transport, Client Access and Mailbox Server roles and as a result were not upgraded as part of the Service Pack.  When the Service Pack failed on the Mailbox role it skipped updating the Management Tools - as a result this problem needed to be addressed.

The error is stating it could not resolve user or groups for Discovery Management - there is only one discovery user created by default with Exchange 2010 called:

DiscoverySearchMailbox {D919BA05-46A6-415f-80AD-7E09334BB852}

As a result I decided to recreate this mailbox by deleting it and recreating it.  This was done with the following commands:

Disable-Mailbox "DiscoverySearchMailbox {D919BA05-46A6-415f-80AD-7E09334BB852}"

Enable-Mailbox "DiscoverySearchMailbox {D919BA05-46A6-415f-80AD-7E09334BB852}" -Arbitration


Next I assigned the Discovery Management user permissions to the new mailbox with the following command:

Add-MailboxPermission -Identity:"cosp.internal/Users/DiscoverySearchMailbox {D919BA05-46A6-415f-80AD-7E09334BB852}" -User:"Discovery Management" -AccessRights:"FullAccess"


After this I deleted the Exchange 2010 SP3 installation watermark (both the “Action” and “Watermark” entries in the registry) and re-ran setup.  Google for this, lots of information already available on the Internet.

After making these changes I was able to successfully patch the Exchange 2010 SP1 server with Exchange 2010 SP3.