Unable to relay outbound email - DNS query failed

One of my clients this morning had an issue sending outbound email from their Exchange server. The exchange server was relaying email between tenants however external emails were building up in the SMTP queue.

MSExchangeTransport was logging the following error in the event logs.

MSExchangeTransport - EventID 16025

The DNS servers could not be retrieved from network adapter GUID.  Check if the computer is connected to a network and Get-NetworkConnectionInfo returns any results.

Messagesin the transport queues were providing the following error message:

451 4.4.0 DNS query failed

After diagnosing the issue we noticed the network adapter to the server had changed and hence a different GUID.  The network adapter GUID being displayed in the event logs no longer matched a network interface on the server.  To view a list of all network interface GUID's on the server look at the following registry key:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters\Interfaces

To restore outbound email to the Internet, we needed to update the GUID on the TransportServer to match the GUID of the new network adapter.  We updated Transport Server with the new GUID obtained from the registry key above using the Set-TransportServer powershell command.  This resolved the issue.

Also please look at this post, a related problem for the 451 4.4.0 DNS query failed:

http://clintboessen.blogspot.com.au/2010/12/451-440-dns-query-failed.html

Compare the Difference between two Group Policy Objects

There’s no built-in functionality in Windows for comparing two GPOs to see how their settings differ.  Back in 2005 a company named DesktopStandard Corp made a tool called GPOVault which extended the functionality of GPMC providing administrators the ability to compare two group policy objects.

DesktopStandard Corp however was brought out by Microsoft and navigating to their website http://www.desktopstandard.com/ simply redirects to the Microsoft Group Policy webpage.
GPOVault does not only provide the ability to compare group policy objects within Group Policy Management Console but also provides other much needed functionality such as change control, notification, approval, rollback, offline editing, templates, and difference reporting directly into the GPMC.

Since the acquisition of DesktopStandard Corp, finding GPOVault on the Internet has become a difficult task.  As a result I have uploaded a copy of GPOVault.msi to this blog which can be downloaded from the following link:

https://sites.google.com/site/cbblogspotfiles/gpovault.msi

To compare two group policy objects perform the following steps:

1. Expand the Forest - Domains and the domain and select the "Change Control" leaf.

2. In the details pane, select the Contents tab, which will display a list of GPOs. At this point, you probably need to select the "Uncontrolled" child tab to see your normal GPOs.

3. Select the GPOs for which you want to see the differences (hold down the Ctrl key as you select each GPO), then right-click and select Differences - HTML Report. (You can also select to output to XML format as the figure shows.)

4. A report will display the differences between the selected GPOs, as the figureshows.

Microsoft SCM - Export Only Critical Severity

Microsoft SCM is a great tool for quickly deploying security baselines to your organisation.  Microsoft SCM version 2 and higher provides security policies which fall into 4 severity categories:

• Critical
• Important
• Optional
• None

For the average enterprise organisation, they want to implement all policies stipulated under the Critical severity category as most settings in the critical category are in alignment with the former Enterprise Client (EC) baseline.

When you export a Microsoft baseline however to a GPO, it exports policies in all categories - something you want to be very caucious doing as it will significantely reduce functionality.  The "Important" category aligns with the older Specialized Security-Limited Functionality (SSLF), a model that "looks at disabling everything then allowing what is required". 

So how do you go about exporting just policies in the Critical category?

To export all policies in the Critical category, perform the following steps:

Create a custom baseline, for example select one of the Microsoft baselines and then click Duplicate in the Actions pane. Now select your custom baseline, then change to "simple view" by clicking the button to the left　 of Advanced View above the middle pane, click the drop-down menu labeled Group View and select Simple View. Now sort the settings by severity by clicking the top of column labeled Severity. Next, you can use shift-click to click multiple settings, then click Delete in the Actions pane to remove them from the baseline. Use this method to remove the settings with a severity other than critical. Now you can export the baseline with only the desired severity.

Call-ClusterExe: cluster.exe did not succeed, but 5058 was not a retry-able error code

You attempt to stop maintenance mode on an Exchange 2010 server using the StopDagServerMaintenance.ps1 script however you receive the following error.

WARNING: [02:57:24.640 UTC] Call-ClusterExe: cluster.exe did not succeed, but 5058 was not a retry-able error code. Not attempting any other servers. This may be an expected error by the caller.

The problem?

UAC.  Run Exchange managemnet console as Administrator.

Exchange 2007 Uninstall hangs on Remove Exchange Files

Today I was decomissioning an Exchange 2007 server for a customer.  The uninstall process got down to "Remove Exchange Files" where it just hanged and did not proceed.  The environment was running Exchange 2007 SP2 on Windows Server 2008 Standard SP2.

After a quick search I found that I ran into a known uninstall issue. The hanging in removing the Exchange files is caused by a final Powershell script that won't run so instead of ending in failure it just hangs and gives the appearance that it is still processing the removal.

All you need to do is open task manager and kill powershell.exe.  Doing this will automatically spawn a new Powershell and allow the removal to complete.  In my situation, the removal completed within 10 seconds of the new Powershell launching.

Caution with Remove-PublicFolder cmdlet

Today I had ran into some trouble at a client with the Remove-PublicFolder cmdlet in Exchange 2010.  The Remove-PublicFolder cmdlet by default removes a public folder from all public folder databases within an Exchange organisation.

This example removes a public folder called "My Public Folder" from all public folder databases in an Exchange environment.

 Remove-PublicFolder -Identity "\My Public Folder"

If you want to remove "My Public Folder" on a specific Exchange server, you can do this with the following command:

Remove-PublicFolder -Identity "\My Public Folder" -Server Server01

If I want to list all public folders on a particular server I can do this with the following command:

Get-PublicFolder -Server Server01 -Recurse

Easy stuff right?  The problem I found however was with the following command:

Get-PublicFolder -Server Server01 -Recurse | Remove-PublicFolder

Here I piped the output of the Get-PublicFolder command to the Remove-PublicFolder command.

Get-PublicFolder -Server Server01 -Recurse displays a list of all public folders on Server01.  When piped into the Remove-PublicFolder command the Remove-PublicFolder command removes the public folders on all servers even though I only specified Server01.  The pipe between the commands does not have the logic to pipe through the server I selected in the Get command, something which other Exchange cmdlets do.  Definately a gotcha and something to watch out for.

To ensure this problem does not happen, you must specify which server you are removing public folders for on the Remove-PublicFolder cmdlet.  For example:

Remove-PublicFolder -Identity "\My Public Folder" -Server "My Server"

Hopefully this will avoid you restoring from backup as I found myself doing today.

Unable to Delete Email from Outlook Web App 2010

A customer of mine cannot delete emails from Outlook Web App (OWA) 2010.  The customer is running Exchange 2010 SP2 with UR1.  When deleting an email the following error is experianced in OWA:

An unexpected error occured and your request couldn't be handled.

Also whenever the error occurs in Outlook Web App the following errors are logged to the Application Log of the server.

Outlook Web App couldn't connect Exchange Web Services due to a configuration error. Response code = "500".

WebHost failed to process a request.
 Sender Information: System.ServiceModel.ServiceHostingEnvironment+HostingManager/20974680
 Exception: System.ServiceModel.ServiceActivationException: The service '/EWS/exchange.asmx' cannot be activated due to an exception during compilation.  The exception message is: A binding instance has already been associated to listen URI 'http://tst-dc1-cas-1.tsthosting.local/EWS/Exchange.asmx'. If two endpoints want to share the same ListenUri, they must also share the same binding object instance. The two conflicting endpoints were either specified in AddServiceEndpoint() calls, in a config file, or a combination of AddServiceEndpoint() and config. . ---> System.InvalidOperationException: A binding instance has already been associated to listen URI 'http://tst-dc1-cas-1.tsthosting.local/EWS/Exchange.asmx'. If two endpoints want to share the same ListenUri, they must also share the same binding object instance. The two conflicting endpoints were either specified in AddServiceEndpoint() calls, in a config file, or a combination of AddServiceEndpoint() and config.
   at System.ServiceModel.Description.DispatcherBuilder.InitializeServiceHost(ServiceDescription description, ServiceHostBase serviceHost)
   at System.ServiceModel.ServiceHostBase.InitializeRuntime()
   at System.ServiceModel.ServiceHostBase.OnOpen(TimeSpan timeout)
   at System.ServiceModel.Channels.CommunicationObject.Open(TimeSpan timeout)
   at System.ServiceModel.ServiceHostingEnvironment.HostingManager.ActivateService(String normalizedVirtualPath)
   at System.ServiceModel.ServiceHostingEnvironment.HostingManager.EnsureServiceAvailable(String normalizedVirtualPath)
   --- End of inner exception stack trace ---
   at System.ServiceModel.ServiceHostingEnvironment.HostingManager.EnsureServiceAvailable(String normalizedVirtualPath)
   at System.ServiceModel.ServiceHostingEnvironment.EnsureServiceAvailableFast(String relativeVirtualPath)
 Process Name: w3wp
 Process ID: 4516

After investigating I found the problem was with Exchange Web Services in the web.config file located under:

C:\Program Files\Microsoft\Exchange Server\V14\ClientAccess\exchweb\ews

This particular client had an F5 Load Balancer and required that SSL Offloading was configured in the Exchange environment.  We had followed Microsoft's instructions for configuring SSL offloading on Exchange 2010 which are documented here:

http://social.technet.microsoft.com/wiki/contents/articles/1267.how-to-configure-ssl-offloading-in-exchange-2010-en-us.aspx

This article instructs customers to do a find and replace on the EWS web.config file to replace all instances of "httpsTransport" with "httpTransport".  However as of Exchange 2010 SP1 the web.config file contains both the httpTransport and httpsTransport settings in the XML configuration file so this is no longer required.

I went back and changed the httpTransport occurances to httpsTransport to ensure the config file accomodated both HTTP and HTTPS.

Postfix and Exchange Intergration

A client of mine decided to use Postfix as a SmartHost for Exchange 2010.  This Exchange environment is to be setup as a multi-tenant Exchange environment and as a result the Postfix servers need to understand which SMTP domain namespaces are authoritative within the Exchange environment.  As a result we needed to configure the Linux Postfix server to query Active Directory to determine a list of "Accepted Domains" within Exchange 2010 which are located under the following location within the Active Directory schema.

CN=Accepted Domains,CN=Transport Settings,CN=TSTHOSTING,CN=Microsoft Exchange,CN=Services,CN=Configuration,DC=domain,DC=local

The linux distribution used for this config is Ubuntu.

Software packages required:

• postfix
• postfix-ldap
• ldap-utils
• libldap-2.4-2

apt-get install postfix-ldap will automatically install all required packages.

A seperate configuration file was put together called ad_relay_domains.cf which we placed under /etc/postfix/ad_relay_domains.cf

server_host = 192.168.1.2
server_port = 389
version = 3
bind = yes
start_tls = no
bind_dn = Administrator
bind_pw = MyPassw0rd (__mailto:MyPassw0rd)
search_base = cn=Accepted Domains,cn=Transport Settings,cn=TSTHOSTING,cn=Microsoft Exchange,cn=Services,cn=Configuration,dc=domain,dc=local
scope = sub
query_filter = (msExchAcceptedDomainName=%s)
result_attribute= msExchAcceptedDomainName

• Server_Host enter the IP address of an AD Domain Controller
• Server_Port needs to be the standard port for LDAP
• version must be 3
• Bind, yes we are creating an LDAP bind.
• Start_TLS, we are not using a digital certificate on Active Directory (pretty standard).  TLS is more common with AD LDS/ADAM Instances as they are not protected using Kerberos v5.
• bind_dn enter the user account you wish to bind to Active Directory with.
• bind_pw is the password that goes with the user account specified under bind_dn.  You must enter it in "password (__mailto:password)" format.
• search_base is the location in Active Directory where Exchange 2010 stores its accepted domains.
• scope specifies all sub objects
• query_filter is the attribute on the accepted domain objects we wish to query.
• result_attribute is the attribute we are interested in the query result.

This file that contains the above config /etc/postfix/ad_relay_domains.cf must be called from the main Postfix configuration file main.cf.  To call the config we simply need to enter the following symtax into the main.cf file:

relay_domains = _ldap:/etc/postfix/ad_relay_domains.cf

Credits for this solution go to Daniel Borg.

Where are Accepted Domains stored in Active Directory?

Today we needed to make OpenLDAP query a list of all Accepted Domains in Active Directory to ensure the Postfix understands which SMTP Namespaces Exchange is authoratative for.  After digging through the Active Directory configuration partition we found that Accepted Domains are stored under the following location:

CN=Accepted Domains,CN=Transport Settings,CN=TSTHOSTING,CN=Microsoft Exchange,CN=Services,CN=Configuration,DC=domain,DC=local

Exchange POP Your e-mail Server Rejected Your User Name

By default Microsoft Exchange 2010 requires you to use TLS Encryption "Secure POP" for authentication to work.  If your organisation does not require Secure POP then by default your users will not be able to login.  They will receive the following error when attempting to access POP3 services through Outlook.

Log onto incoming mail server (POP3): Your e-mail server rejected your user name.  Verify your user name for this account in Account Settings.  The server responded: -ERR Command is not valid in this state.

To allow POP3 to authenticate using standard POP3 you must change teh authentication level as per:

http://technet.microsoft.com/en-us/library/bb124498.aspx

The default setting is "SecureLogon".  Below are the various settings:

This example allows Basic authentication on an unsecured port if you won't be using TLS encryption.

Set-PopSettings -LoginType PlainTextLogin
This example restricts Basic authentication to use only secured ports if you won't be using TLS encryption.

Set-PopSettings -LoginType PlainTextAuthentication
This example allows authentication after you use TLS encryption.

Set-PopSettings -LoginType SecureLogin

If you configure use PlainTextLogin users can send their Active Directory password over the internet unencrypted.

If you plan on not using TLS, I always recommend using PlainTextAuthentication which allows users to send the password over the internet without using TLS.  This however does require your users configuring "Require logon using Secure Password Authentication (SPA)" under Outlook.

After you change your Pop settings your users will now be able to authenticate over TCP110.

Important: If you have a user account that is an Administrator in your Exchange environment they will NOT be able to login.  This is due to security, POP3 authentication is not deemed as Secure by Microsoft and as a result Microsoft have prevented Administrative users from authenticating over the internet using POP to avoid administrative credentials leeking.

Important: You must restart the POP3 service for configuration changes to take effect.

Windows 7 Slow Access to Network Shares

One of my clients recently deployed Windows 7 across their network and are experiancing performance issues when accessing network resources involving MS-RPC calls such as network shares through the SMB protocol.  Windows Vista/2008/7 has a new TCP network stack to Windows XP/2003 and has a number of new technologies designed to increase network performance.

Windows XP/2003 clients are not receiving performance problems.

Issue

When a Windows 7 PC attempts to accesss file shares on any Windows XP/2003 servers, browsing performance of mapped network shares is aporling.  It feels simlar to accessing the file server over a VPN connection with poor bandwidth and high latency, however in this scenario we are accessing the file server over LAN.  We tested a Windows 7 client on the same 24bit subnet as a number of Windows 2003 file servers on a 1gbps network.  Slow access to network shares also occured over the same subnet.

What I did notice however was slow performance did not occur when a Windows Vista/2008/7 computer accesses file shares on another Windows Vista/2008/7.  The issue only occured when Windows Vista/2008/7 accessed Windows XP/2003.

This issue is not driver related, I ensured to test workstations of different driver architecture including Intel and Broadcom network drivers.

Workaround

After researching into this issue we came identified two Windows components contributing to the performance issues.

• TCP Auto-Tuning
• Remote Differential Compression

Do not disable these components unless you understand what they are and their purpose.

TCP Auto-Tuning

As mentioned above, a new TCP stack was implemented as of Windows Vista that aims to take full advantage of hardware advances such as gigabit networking. Among the new feature in the new TCP stack is Receive Window Auto-Tuning Level for TCP connections. TCP AutoTuning enables TCP window scaling by default and automatically tunes the TCP receive window size for each individual connection based on the bandwidth delay product (BDP) and the rate at which the application reads data from the connection, and no longer need to manually change TcpWindowSize registry key value which applies to all connection. Theoretically, with TCP auto-tuning, network connection throughput in Windows Vista should be improved for best performance and efficiency, without registry tweak or hack. However, this is not always the case, and may cause some performance related issues such as in our case.

The default auto-tuning level is "normal", and the possible settings for the above command are:

disabled: uses a fixed value for the tcp receive window. Limits it to 64KB (limited at 65535).
highlyrestricted: allows the receive window to grow beyond its default value, very conservatively
restricted: somewhat restricted growth of the tcp receive window beyond its default value
normal: default value, allows the receive window to grow to accommodate most conditions
experimental: allows the receive window to grow to accommodate extreme scenarios (not recommended, it can degrade performance in common scenarios, only intended for research purposes. It enables RWIN values of over 16 MB)

In my environment changing this from the default value of normal to restricted resolved my performance issues.

netsh int tcp set global autotuninglevel=restricted

Remote Differential Compression

Remote Differential Compression (RDC) is a client–server synchronization algorithm that allows the contents of two files to be synchronized by communicating only the differences between them. It was introduced with Windows Server 2003 R2 and is included with later Windows client and server operating systems.  As of Vista onwards Remote Differential Compression is enabled by default.

Microsoft does not recommend ever disabling Remote Differential Compression (RDC) unless there is an absolute need to.  Diane from Microsoft wrote a blog post entitled "Debunking Myths about Remote Differential Compression and System Performance" where she states:

A number of third-party blogs are telling people they can speed up Windows Update downloads, and file copy operations, by turning off the Remote Differential Compression (RDC) feature on Windows Vista. This is 100% false. Neither Windows Update or file copy operations use RDC at all.

The RDC feature is simply a DLL that does not consume any system resources, except when you run an application that uses RDC specifically. If you disable RDC, any application that uses it will either not be able to take advantage of RDC or will simply fail. For more information on RDC see this link http://msdn.microsoft.com/en-us/library/aa373254(VS.85).aspx.

As a result I do not recommend disabling RDC unless you have an absolute need to.  In my environemnt I tested my Windows 7 PC's with RDC turned on and RDC turned off.  When turned on I noticed my performance to Windows 2003/XP file shares diminished.  Please disable with care.

You can disable RDC through Windows Features in control panel:

You can also disable RDC through command line using the following command:


ocsetup MSRDC-Infrastructure /uninstall


Deploying the Workaround

I wrote a batch script to deploy this workaround to all Windows 7 computers on my network.  The batch script first identifies IS the computer running Windows 7, if so THEN set TCP Autotuning to Restricted and disable Remote Differential Compression.

Here is a copy of my script:

@echo off
systeminfo | find "OS Name" > %TEMP%\osname.txt
set /p vers=<%TEMP%\osname.txt

echo %vers% | find "Windows 7" > nul
if %ERRORLEVEL% == 0 goto ver_7

goto warnthenexit

:ver_7
ocsetup MSRDC-Infrastructure /uninstall
netsh interface tcp set global autotuning=restricted
goto exit

:warnthenexit
echo Machine undetermined.

:exit


Deploy this script to your workstations through a Group Policy Startup Script.

Note: Due to the Windows 7 performance issues I also needed to configure "Always wait for the network at computer startup and logon" under "Computer -- Administrative Templates -- System -- Logon"

Why isn't my Outlook spam filter working?

The Outlook Spam filter or "Junk E-mail Filter DLL" works only against messages in the local store, i.e. the offline OST file.  For Outlook to filter emails on a workstation using the Outlook Anti-Spam engine you must have "Cached Exchange Mode" enabled.

You configure this on the users Outlook Profile.

DAG Node lost access to File Witness Share

Today on my own companies network 4Logic IT Solutions we had an issue when trying to activate a database copy inside a database availability group (DAG) cluster.

When attempting to move the active mailbox database to another node using Exchange Management Console (EMC) or Exchange Management Shell (EMS) the following error was experianced.

--------------------------------------------------------
Microsoft Exchange Error
--------------------------------------------------------
Cannot activate database copy 'Activate Database Copy...'.

Activate Database Copy...
Failed
Error:
An Active Manager operation failed. Error The database action failed. Error: An error occurred while trying to validate the specified database copy for possible activation. Error: Server 'QV1-EXC1.4logic.lan' is not up according to the Windows Failover Cluster service.. [Database: QV1-EXC1-Database-01, Server: OP-SRV1.4logic.lan]

An Active Manager operation failed. Error An error occurred while trying to validate the specified database copy for possible activation. Error: Server 'QV1-EXC1.4logic.lan' is not up according to the Windows Failover Cluster service.



The error stated that Windows cluster services reported the node as being down. So I fired up Failover Cluster Management MMC snapin to investigate further. The following errors were being generated:

File share witness resource 'File Share Witness (\\qv1-utl1.4logic.lan\DAG01.4logic.lan)' failed to arbitrate for the file share '\\qv1-utl1.4logic.lan\DAG01.4logic.lan'. Please ensure that file share '\\qv1-utl1.4logic.lan\DAG01.4logic.lan' exists and is accessible by the cluster.



Node 'QV1-EXC1' failed to form a cluster. This was because the witness was not accessible. Please ensure that the witness resource is online and available.



I went over to my server hosting the File Witness Share QV1-UTL1.4logic.lan. DAG01$ was the only trusted object added to the Share permissions. I added Exchange Trusted Subsystem and my Administrator account (for testing purposes) to the Share Permissions.



I then restarted the Cluster Services service on QV1-EXC1. After the permission change the cluster service came online again.

Unable to update Mailbox SD in the DS. Mailbox Guid: 33fcbcfe-5c36-4204-91f1-6108fc5b86f6. Error Code 0x8004010f

I had a client who was receiving Application Error 9554 from MSExchangeIS.

Unable to update Mailbox SD in the DS. Mailbox Guid: 33fcbcfe-5c36-4204-91f1-6108fc5b86f6. Error Code 0x8004010f



First thing I did was attempted to search for the user in Active Directory by converting the 32bit GUID to a searchable format using KB322308. I was unable to find the user account matching the mailbox guid.



Using the ADFind utility created by MVP Joe Richards also did not find any results.

adfind -gc -b "" -binenc -f " msExchMailboxGUID={{GUID:33fcbcfe-5c36-4204-91f1-6108fc5b86f6}}" -dn

After further investigation I found that the GUID on the Exchange Mailbox did not match the msExchMailboxGUID on the AD User Account. I determined this by running the following powershell command:

Get-MailboxStatistics -Server troy-exch-2010 fl mailboxguid,displayname > output.txt

MailboxGuid : 33fcbcfe-5c36-4204-91f1-6108fc5b86f6
DisplayName : User Name

The user account in AD had a different GUID not matching the GUID of our Exchange mailbox.



In this instance the user no longer worked at the company so we simply deleted both the user account and the Exchange mailbox.

How to Configure SSL Offloading in Exchange 2010

The following article explains in detail how to configure SSL Offloading in Exchange 2010 RTM, SP1 and SP2. If you require SSL Offloading for any reason on an Exchange deployment, please refer the following article:

http://social.technet.microsoft.com/wiki/contents/articles/1267.how-to-configure-ssl-offloading-in-exchange-2010-en-us.aspx

The HTTP authentication test failed - Connection was closed unexpectedly.

Today I had a problem with Outlook Anywhere at a customer site - everything was configured correctly however Outlook Anywhere would still not work! The server was Exchange 2007 SP2 running on Windows Server 2003 R2 Standard Edition x64.

When using the Exchange Remote Connectivity Analyzer the following error was experianced.

Testing HTTP Authentication Methods for URL https://webmail.example.com/rpc/rpcproxy.dll.
The HTTP authentication test failed.
Additional Details
Exception details:
Message: The underlying connection was closed: The connection was closed unexpectedly.
Type: System.Net.WebException
Stack trace:
at System.Net.HttpWebRequest.GetResponse()
at Microsoft.Exchange.Tools.ExRca.Extensions.RcaHttpRequest.GetResponse()

1. Disable outlook anywhere via EMC

2. Remove RPC proxy component via PowerShell, Server Manager or Add and Remove Programs (Windows Components) depending on your version of windows. You can also use this command: servermanagercmd -r rpc-over-http-proxy

3. Reboot the server

4. Install RPC proxy component via PowerShell

5. Command: servermanagercmd -i rpc-over-http-proxy

6. Enable outlook anywhere

7. Restart Microsoft active directory Topology service


This should resolve your problem, did for me.

Autodiscover settings for Outlook Anywhere are being validated

If you run the Exchange 2010 Remote Connectivity Analyzer (ExRCA) and you get the following message it means the Server attribute is set on the EXPR provider incorrectly.

Autodiscover settings for Outlook Anywhere are being validated.
ExRCA wasn't able to validate Outlook Anywhere Autodiscover settings.
The AuthPackage wasn't specified in the EXPR section of the Autodiscover response.



Simply remove the Server attribute and set it to Null.

Set-OutlookProvider EXPR -Server $null



This will resolve your problem.

IISReset loosing settings in IIS6

People do not know but the IISReset command can cause some settings to be lost.

The IISReset command line tool waits for a normal shutdown of the services before it starts them again. Because of the number of services that are dependent on the IISAdmin service, the shutdown may not occur in a timely manner. When this happens, IISReset forces the shutdown of the services. This can result in metabase changes that are not saved properly.

Today at a customer site when I ran the Remote Conectivity Analyzer (ExRCA) Exchange 2007 server it was complaining about Annonymous authentication being enabled.

The Initial Anonymous HTTPS request didn't fail, but Anonymous isn't a supported authentication method for this scenario.



To fix this we simply need to disable annonymous authentication on the RPC web application in IIS.



When disabling annonymouse authentication and running the IISRESET command it simply re-enables annonymous authentication.

To commit changes while still restarting IIS6 run the following commands:

net stop iisadmin /y
net start iisadmin /y

If you notice now, the changes has been permanently committed to the IIS6 metabase.

For more information about this problem please see the following knowledge base article:

http://support.microsoft.com/kb/286196

Error installing Hub Transport Server "Failed to read directory object with distinguished"

Today I had an issue installing Exchange 2010 with SP1 in a new Active Directory forest. The installation was failing during installation of the Hub Transport role. The following error was experienced:

Hub Transport Role
Failed

Error:
The following error was generated when "$error.Clear();
Add-DelegatedGroupsToLocalAdminGroup -DomainController

$RoleDomainController

" was run: "Failed to read directory object with distinguished name TST-DC2-EXC-1-4.".

Failed to read directory object with distinguished name TST-DC2-EXC-1-4.



After looking at the error "Failed to read directory object with distinguished name" I first checked the SPN for the computer object TST-DC2-EXC-1-4. This checked out fine.

I then removed the computer TST-DC2-EXC-1-4 from the Active Directory domain, deleted the computer object then rejoined it. When rejoining the domain I experienced the following error:

Changing the Primary Domain DNS name of this computer to "" failed. The name will retain "tsthosting.local". The error was:

This specified server cannot perform the requested operation.



I removed the PC from the domain setting it back to workgroup. Before joining it to the domain again I manually set the primary DNS suffix for the computer account by clicking More in the domain settings window.



I then unticked change primary DNS suffix when domain membership changes and manually typed in my domains DNS suffix.



This resolved my problem allowing my setup to complete successfully.



While this procedure fixed the issue, it is not the root cause. I did not have enough time to diagnose and determine the root cause.

Blank Screen in VMware vCloud

I kept getting a black screen when trying to console a virutal machine in a vCloud environment. It would just hang saying Connecting...



To fix it I simply added the vcloud HTTPS address as a trusted site in Internet Options.

Remove Office 365 AD/FS 2.0

When setting up a Hybrid Exchange deployment, AD FS 2.0 is required for single signon (SSO) to ensure the on-premises Active Directory credentials can be used on both Office 365 and on-premises Exchange. The AD FS 2.0 deployment required for Office 365 Hybrid deployments is different to the AD FS 2.0 build that comes with Windows Server 2008 R2.

As a result, after installing AD FS 2.0 it does not appear as enabled in Server Manager roles in Windows Server 2008 R2.



Also it does not appear in remove programs under control panel.



So how do you remove it?

Under programs and features click view installed updates.

It will now appear int he list for you to remove it.



After you uninstall it, you also need to clean up the IIS Web Server role. Ensure IIS is not doing anything else before uninstalling it from Server Manager.

Lastly you have some additional cleanup tasks to perform as documented under http://support.microsoft.com/kb/982813

How to make RDC / MSTSC open on the secondary monitor when it connects

An annoying problem you may have is getting the Remote Desktop Client or mstsc.exe opening on a secondary monitor when it connects. We have validated the following procedure works:

1. Run the client and get it set up the way you prefer, or Open your saved RDP file
2. On the Display tab, choose a non-fullscreen resolution.
3. On the General tab, click "delete these credentials", if possible.
4. Click Connect .
5. You will immediately get a local login dialog. Leave it blank and click OK .
6. When you are connected and the remote login dialog appears, drag the window to the desired monitor.
7. On the remote login dialog, click Cancel . The settings dialog should reappear locally.
8. On the Display tab, you can now choose fullscreen.
You can now save your session with the Save button on the General tab, as well as save your login credentials.

Changing the RPCClientAccessServer - How Outlook Behaves

When an Outlook client goes to connect to an Exchange 2010 database, it looks at an attribute associated with the mailbox database called RPCClientAccess to determine which client access server/client access server array to use for connectivity.

There may be a time in where you need to change which RPCClientAccess server your clients use for connectivity on the Exchange mailbox database. The problem is, if you change the RPCClientAccess on a mailbox database to a different Exchange 2010 client access server/client access server array without "moving" the mailbox, Outlook 2007 and Outlook 2010 clients do not pickup this change automatically.

Further more, if you perform an Outlook Profile repair process, it will update with the new RPC endpoint for the users mailbox database defined under the RPCClientAccess attribute. But guess what, Outlook reverts to a Working Offline mode. Creating a new outlook profile will resolve the issue, however if you remove the Host A record for the old client access server in DNS, Outlook will resume working as normal again.

Outlook only updates to a new RPCClientAccess value smoothly when moving users to the mailbox database. If you want to transition users smoothly to another client access server or client access server array, you can create another database, set the RPCClientAccess as desired, then move users to the database. This will allow the outlook profiles to update without issues.

Other then moving mailboxes there is one other thing you can do to get Outlook to update to the new RPCClientAccess value. If you remove the "Host A" record from the old client access server in DNS, this will cause Outlook to forcefully repair itself and update its profile.

Lets hope Microsoft makes it easier in the next release of Microsoft Outlook.

For additional reading about this problem, please look at the following links:

http://www.outlookforums.com/threads/84315-Outlook-goes-offline-after-changing-RPCClientAccessServer-parameter-on-mailbox-database

http://www.shudnow.net/2010/04/18/creating-databases-and-the-rpcclientaccessserver-database-parameter/

http://www.scottfeltmann.com/blog/2010/06/28/outlook-profile-not-updating-after-creating-cas-array/

http://technet.microsoft.com/en-us/magazine/ff626260.aspx

Add Driver Packages to Boot Image Wizard Not Responding

Hi had an issue today with a WDS Server on Windows Server 2008 R2 SP1. Whenever I went to add a driver package to a boot image the "Add Driver Packages to Boot Image Wizard" would freeze when trying to mount the image and come up with not responding.

When running the wizard again the following error would be received:

Error Occurred while trying to execute this command.
Error Code: 0xc1420127

In which I needed to browse to "HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\WIMMount\Mounted Images" and delete any keys below this to run the wizard again.

To resolve the problem I ran the following command to Uninitialize the server.

wdsutil /Uninitialize-Server



Opening the WDS Console allowed me to run the configuration wizard again. After running the configuration wizard and re-configuring the server I then ran the "Add Driver Packages to Boot Image Wizard" which completed successfully.

Outlook 2007 Connection Issues over Outlook Anywhere

I have seen a problem with Outlook 2007 numerous times where Outlook 2007 cannot connect to Exchange when setting up a new profile. This problem is client related, not server related. For example if the same user experiencing the issue tries to connect to Exchange remotely via Outlook Anywhere from a different PC using the same version of Outlook, it works fine.

The issue only occurs when creating a new Outlook Profile remotely via Outlook Anywhere (RPC over HTTPS). If the profile is already created and the user has used the profile in the past inside the network over direct MAPI you are reading the wrong blog post.

I am still unaware what causes this problem however I do have two working resolution. If I find the root cause in the future, I am sure to come back and update this post for everyone :)

Symptoms

A user tries to create a new Outlook Profile remotely through Outlook Anywhere but experiences the following error:

The connection to Microsoft Exchange is unavailable. Outlook must be online or connected to complete this action.



Note: This error is very generic and can also be caused when there is server side problems with Outlook Anywhere in which all users are effected. Only read this post if you are receiving this error only for an external user trying to create their outlook profile over Outlook Anywhere.

Resolution 1

If the user connects to the internal network either by bringing their PC/Laptop into the office or initiating a VPN connection the Outlook client connects to exchange using direct RPC MAPI and sets up the Outlook Profile as normal. If the user then brings the PC/Laptop outside the network either by disconnecting the VPN or moving the PC to another external network and connects using Outlook Anywhere (RPC over HTTP), Outlook Anywhere now works fine. The profile was created inside the network hence the issue of creating the profile over Outlook Anywhere never occured.

Resolution 2

In the event you have a user in a remote site, perhaps another country and you have no VPN connectivity setup the task of creating the Outlook Profile on the internal network using RPC MAPI calls is made difficult. However I have a work around here...

What I did was create the users profile on another PC running Outlook 2007. I used a virtual machine with Outlook 2007 installed and created the users profile using Outlook Anywhere as I did not have access to the internal network. When the profile is created and working correctly, export the following registry key to a .reg file.

HKEY_CURRENT_USER\Software\Microsoft\Windows NT\CurrentVersion\Windows Messaging Subsystem



Next go to the PC experiencing the Outlook 2007 issue and import the .reg file you have created by double clicking the file. This will import the Outlook Profile created on the remote workstation. As Outlook is no longer trying to create the Outlook profile over Outlook Anywhere, it will simply load the profile as normal allowing the user to work remotely over Outlook Anywhere.

Don't worry about Operating Systems - Performing this registry import/export from another PC with different operating system is fine. I exported my registry content from a Windows XP PC with Outlook 2007 installed and imported it on a Windows 7 PC with Outlook 2007 installed.

Additional Information

I have troubleshooted this issue numerous times and have looked at many things such as debug logging with Outlook, launching Outlook in safe mode etc, nothing provides detailed information useful to diagnosing the root cause.

This issue is not related to the Windows user profile or User Account. For example if the user recreates his profile or another user account on the PC experiencing the issue, the problem reoccurs. We can immediately rule out the users local user hive, it is definitely with the Outlook installation on the PC... so we should be looking at areas such as the local machine registry (HKLM) and the Outlook installation, are all the files correct?

Reinstalling Office 2007 on a PC experiencing the issue does not resolve the problem. This means it is either a setting in Windows or a desktop application which triggers this issue. I have seen this issue in different companies, so if it is another application causing problems, it is not a custom line of business application. It may be a common application that multiple company's use.

I have seen this issue occur on all versions of Outlook 2007 ranging from Outlook 2007 RTM, SP1 and SP2. Upgrading Outlook 2007 to the latest service pack will not resolve this issue.

I have not seen this issue on Outlook 2003 or Outlook 2010 however this is not to say the issue does not occur on these versions. I have just never seen this on these particular versions of Outlook in my professional experience.

Last thing I would like to note is, third party non-Microsoft firewall software installed on a Laptop or PC can cause this issue. If you open up Windows Firewall and Windows Firewall displays the notice something similar to "this PC is being managed by a third party vendor firewall", try uninstalling the third party firewall software as this may resolve the issue.

Assign user rights to modify distribution group membership

A colleague of mine here at 4Logic today needed to perform a simple task in an Exchange 2010 environment today to allow designated users to administer distribution groups in Exchange 2010. Most Microsoft documentation suggesting adding the RBAC Administration role “Distribution Groups” to the user. This created security implementations as any user which was a member of this group, could manage all distribution groups within the organisation.

My colleague eventually worked this out by discovering RBAC user role of "MyDisturbtionGroups" allows users to manage distribution groups that they are an owner of (via Outlook and Exchange Control Panel).

Note: The group owner is controlled by the "Managed By" property.

He enabled this role by using the following PowerShell command:

New-ManagementRoleAssignment -Role MyDistributionGroups -Policy "Default Role Assignment Policy"

The role group however also allows owners of a distribution group to delete the distribution group.

If you want to allow users to only modify group membership a custom RBAC management role is required. The following commands create a custom management role which only allows a user in the "Managed By" field of a group to modify a groups membership.

New-ManagementRole -Name UpdateAddressLists -Parent MyDistributionGroups

Remove-ManagementRoleEntry UpdateAddressLists\New-DistributionGroup -Confirm:$false

Remove-ManagementRoleEntry UpdateAddressLists\Remove-DistributionGroup -Confirm:$false

New-ManagementRoleAssignment -Role UpdateAddressLists -Policy "Default Role Assignment Policy"

Command to find out how many items in each users Sent Items

Today I put together a quick command to explore how many items are in each users sent items which I would like to share with you. This command ignores any users mailbox who does not have any items in the sent items folder.

Get-Mailbox | Get-MailboxFolderStatistics -FolderScope SentItems | Where {$_.ItemsInFolder -gt 0} | Sort-Object -Property ItemsInFolder -Descending | select-object Identity,ItemsInFolder | export-csv c:\test.txt

SACL Watcher servicelet found that the SeSecurityPrivilege privilege is removed from account

In this post I will share with you the resolution to a problem I had on one of my clients Exchange environments. The following error was experienced in the event logs of my Exchange 2010 servers.

Log Name: Application
Source: MSExchange SACL Watcher
Date: 28/12/2011 11:00:43 AM
Event ID: 6006
Task Category: General
Level: Warning
Keywords: Classic
User: N/A
Computer: exchangeserver.domain.local
Description:
SACL Watcher servicelet found that the SeSecurityPrivilege privilege is removed from account S-1-5-21-54938807-350570593-2036031536-21088.



Next I used LDP.exe to translate the SID from the error message into something readable.





After investigating the problem I found out that "SeSecurityPrivilege privilege" translates to "Manage audit and security log" under user rights assignment in group policy. Exchange setup automatically adds "DOMAIN\Exchange Enterprise Servers" and "DOMAIN\Exchange Servers" to the "Manage audit and security log" user rights assignment on the Default Domain Controllers Policy.



My client had unlinked the Default Domain Controllers Policy from the Domain Controllers OU and created their own custom policy - NOT RECOMMENDED. Restoring this policy resolved the problem.

451 4.4.0 Primary target IP address responded with: "451 5.7.3 Cannot achieve Exchange Server authentication."

In this post we will look at what this error means:

451 4.4.0 Primary target IP address responded with: "451 5.7.3 Cannot achieve Exchange Server authentication." Attempted failover to alternate host, but that did not succeed. Either there are no alternate hosts, or delivery failed to all alternate hosts.



Lets have a look in the SMTP Receive protocol logs:

C:\Program Files\Microsoft\Exchange Server\V14\TransportRoles\Logs\ProtocolLog\SmtpReceive

- We can see connections trying to be established from 10.4.8.1 (in a remote AD site) and 10.4.2.8 (in the same AD site)
- We can see the connections are hitting the receive connector SVR01\External
- The receive connector is terminating the connection with "Service closing transmission channel"

2011-12-29T06:10:25.668Z,SVR01\External,08CE9265B5AAB0D9,0,10.4.2.10:25,10.4.8.1:49547,+,,
2011-12-29T06:10:25.668Z,SVR01\External,08CE9265B5AAB0D9,1,10.4.2.10:25,10.4.8.1:49547,*,SMTPSubmit SMTPAcceptAnySender SMTPAcceptAuthoritativeDomainSender AcceptRoutingHeaders,Set Session Permissions
2011-12-29T06:10:25.668Z,SVR01\External,08CE9265B5AAB0D9,2,10.4.2.10:25,10.4.8.1:49547,>,"220 mail.4logic.com.au Microsoft ESMTP MAIL Service ready at Thu, 29 Dec 2011 14:10:24 +0800",
2011-12-29T06:10:25.699Z,SVR01\External,08CE9265B5AAB0D9,3,10.4.2.10:25,10.4.8.1:49547,<,EHLO OP-SRV1.4logic.lan, 2011-12-29T06:10:25.699Z,SVR01\External,08CE9265B5AAB0D9,4,10.4.2.10:25,10.4.8.1:49547,>,250-mail.4logic.com.au Hello [10.4.8.1],
2011-12-29T06:10:25.699Z,SVR01\External,08CE9265B5AAB0D9,5,10.4.2.10:25,10.4.8.1:49547,>,250-SIZE 31457280,
2011-12-29T06:10:25.699Z,SVR01\External,08CE9265B5AAB0D9,6,10.4.2.10:25,10.4.8.1:49547,>,250-PIPELINING,
2011-12-29T06:10:25.699Z,SVR01\External,08CE9265B5AAB0D9,7,10.4.2.10:25,10.4.8.1:49547,>,250-DSN,
2011-12-29T06:10:25.699Z,SVR01\External,08CE9265B5AAB0D9,8,10.4.2.10:25,10.4.8.1:49547,>,250-ENHANCEDSTATUSCODES,
2011-12-29T06:10:25.699Z,SVR01\External,08CE9265B5AAB0D9,9,10.4.2.10:25,10.4.8.1:49547,>,250-STARTTLS,
2011-12-29T06:10:25.699Z,SVR01\External,08CE9265B5AAB0D9,10,10.4.2.10:25,10.4.8.1:49547,>,250-AUTH,
2011-12-29T06:10:25.699Z,SVR01\External,08CE9265B5AAB0D9,11,10.4.2.10:25,10.4.8.1:49547,>,250-8BITMIME,
2011-12-29T06:10:25.699Z,SVR01\External,08CE9265B5AAB0D9,12,10.4.2.10:25,10.4.8.1:49547,>,250-BINARYMIME,
2011-12-29T06:10:25.699Z,SVR01\External,08CE9265B5AAB0D9,13,10.4.2.10:25,10.4.8.1:49547,>,250 CHUNKING,
2011-12-29T06:10:25.746Z,SVR01\External,08CE9265B5AAB0D9,14,10.4.2.10:25,10.4.8.1:49547,<,QUIT, 2011-12-29T06:10:25.746Z,SVR01\External,08CE9265B5AAB0D9,15,10.4.2.10:25,10.4.8.1:49547,>,221 2.0.0 Service closing transmission channel,
2011-12-29T06:10:25.746Z,SVR01\External,08CE9265B5AAB0D9,16,10.4.2.10:25,10.4.8.1:49547,-,,Local
2011-12-29T06:10:32.761Z,SVR01\External,08CE9265B5AAB0DA,0,10.4.2.10:25,10.4.2.8:65117,+,,
2011-12-29T06:10:32.761Z,SVR01\External,08CE9265B5AAB0DA,1,10.4.2.10:25,10.4.2.8:65117,*,SMTPSubmit SMTPAcceptAnySender SMTPAcceptAuthoritativeDomainSender AcceptRoutingHeaders,Set Session Permissions
2011-12-29T06:10:32.761Z,SVR01\External,08CE9265B5AAB0DA,2,10.4.2.10:25,10.4.2.8:65117,>,"220 mail.4logic.com.au Microsoft ESMTP MAIL Service ready at Thu, 29 Dec 2011 14:10:31 +0800",
2011-12-29T06:10:32.761Z,SVR01\External,08CE9265B5AAB0DA,3,10.4.2.10:25,10.4.2.8:65117,<,EHLO QV1-EXC1.4logic.lan, 2011-12-29T06:10:32.761Z,SVR01\External,08CE9265B5AAB0DA,4,10.4.2.10:25,10.4.2.8:65117,>,250-mail.4logic.com.au Hello [10.4.2.8],
2011-12-29T06:10:32.761Z,SVR01\External,08CE9265B5AAB0DA,5,10.4.2.10:25,10.4.2.8:65117,>,250-SIZE 31457280,
2011-12-29T06:10:32.761Z,SVR01\External,08CE9265B5AAB0DA,6,10.4.2.10:25,10.4.2.8:65117,>,250-PIPELINING,
2011-12-29T06:10:32.761Z,SVR01\External,08CE9265B5AAB0DA,7,10.4.2.10:25,10.4.2.8:65117,>,250-DSN,
2011-12-29T06:10:32.761Z,SVR01\External,08CE9265B5AAB0DA,8,10.4.2.10:25,10.4.2.8:65117,>,250-ENHANCEDSTATUSCODES,
2011-12-29T06:10:32.761Z,SVR01\External,08CE9265B5AAB0DA,9,10.4.2.10:25,10.4.2.8:65117,>,250-STARTTLS,
2011-12-29T06:10:32.761Z,SVR01\External,08CE9265B5AAB0DA,10,10.4.2.10:25,10.4.2.8:65117,>,250-AUTH,
2011-12-29T06:10:32.761Z,SVR01\External,08CE9265B5AAB0DA,11,10.4.2.10:25,10.4.2.8:65117,>,250-8BITMIME,
2011-12-29T06:10:32.761Z,SVR01\External,08CE9265B5AAB0DA,12,10.4.2.10:25,10.4.2.8:65117,>,250-BINARYMIME,
2011-12-29T06:10:32.777Z,SVR01\External,08CE9265B5AAB0DA,13,10.4.2.10:25,10.4.2.8:65117,>,250 CHUNKING,
2011-12-29T06:10:32.777Z,SVR01\External,08CE9265B5AAB0DA,14,10.4.2.10:25,10.4.2.8:65117,<,QUIT, 2011-12-29T06:10:32.777Z,SVR01\External,08CE9265B5AAB0DA,15,10.4.2.10:25,10.4.2.8:65117,>,221 2.0.0 Service closing transmission channel,
2011-12-29T06:10:32.777Z,SVR01\External,08CE9265B5AAB0DA,16,10.4.2.10:25,10.4.2.8:65117,-,,Local

If we look at the receive connector "External" on SVR01 we notice that Exchange Server authentication is not enabled meaning SVR01 is not able to receive mail relay from other hub transport servers in the Exchange organisation.

If we tick the box we will resolve the problem.

Hybrid Office 365 Deployment with Threat Management Gateway

After working with a Hybrid Office 365 deployment with Threat Management Gateway performing SSL offloading to an Exchange 2010 SP2 hybrid server for one of my customers I experienced a number of gotcha's which are not documented.

MRSProxy with SSL Offloading

The first issue was with MRSProxy. I found out the hard way that MRSProxy "/EWS/mrsproxy.svc" does not support SSL offloading. The MRSProxy connection must hit the Exchange 2010 Client Access Server on TCP443 using a secure SSL connection. When using SSL offloading with Forefront Threat Management Gateway as soon as unsecure HTTP port 80 connections were passed to MRSProxy on the Exchange 2010 Hybrid server we were receiving was (404) Not Found in the IIS logs.

We created a separate TMG firewall rule with a path rule of "/EWS/mrsproxy.svc" which is processed before our Hybrid firewall rule. This path rule re-encrypts and forwards to the Hybird server on TCP443. All other connections come through on TCP80.

Free/Busy with Office 365

The second issue was with free/busy. Users on-premises were able to view Free/Busy for users in Office 365, however users in Office 365 were unable to view Free/Busy for users on-premises. This was caused by the Web Listener being configured for Basic or Windows Integrated authentication. Free/Busy over an Organization Relationship uses Microsoft.Web.Services.SoapContext to authenticate free/busy... in other words the authentication is handled by the .NET framework. Threat Management Gateway must pass through authentication requests to /EWS/*. Setting the TMG listener to No Authentication achieves this.