Wednesday, November 13, 2013

Office 365 Migration Solutions for Email Services - Must Read

Despite large amounts of effort from Microsoft in online resources and education over the past year, many organisations are still having difficulty with migrating their email infrastructure to the cloud and are confused as to the various migration options available.  This blog post is intended to assist organisations gain an understanding as to the various options available to them for migration to Office 365 and pick the right migration solution to meet their needs.

Microsoft provides organisations four methods for migrating user mailboxes from an on-premises Exchange environment to the Microsoft Office 365 cloud based on an organisations requirements.  These migration solutions include:
  • Exchange Cutover Migration
  • Exchange Staged Migration
  • Hybrid Exchange Deployment
  • IMAP-based Email Migrations
Each of these migration solutions will be covered individually below.
 
Exchange Cutover Migration
 
An Exchange cutover migration involves cutting over all domains from an on-premises infrastructure to the cloud in one hit, usually over a weekend.  The advantage of Cutover Migrations is that no additional infrastructure needs to be provisioned on premises such as the Microsoft Online Services Directory Synchronisation tool, Hybrid Exchange Servers or Active Directory Federation servers.  The disadvantage with Exchange Cutover Migrations is it provides no co-existence between the on-premises Exchange environment and Office 365.  Cutover migrations migrate the user’s entire on-premises mailbox to the cloud and are generally used when there is a small number of on-premises users which need migrating to Office 365.
 
Exchange cutover migrations work by the Office 365 servers connecting to your on-premises Exchange servers through RPC over HTTPS (also known as Outlook Anywhere) and downloading all mailboxes into Office 365.  To ensure Office 365 is kept up to date with new emails/items and calendar information or content deleted by the end user, Office 365 will automatically synchronize with the on-premises Exchange environment once every 24 hours.
 
In a cutover migration, once all on-premises mailboxes are synchronised in Office 365, administrators can plan a date in which disruption can occur (usually a weekend) to cutover the mail MX records to the cloud.  This means all external mail servers on the Internet relaying mail to the organisation will now send mail directly to the cloud.
 
It is recommended prior to performing the MX record update, the time to live (TTL) on the MX record be reduced to a smaller value such as 30 minutes to ensure mail servers around the world release the old record and request the updated record.  If the MX record has a large TTL value such as one week, it could mean that for some mail servers, it may take up to a week before mail is sent to the new Office 365 address.  Directly after the MX records are changed, it is recommended a final on-demand synchronisation be performed within the Office 365 interface.
 
Office 365 will automatically generate new passwords for all user accounts in the cloud.  On Monday morning administrators will need to run around to all computers and recreate the users a new Outlook profile to ensure users are setup correctly.  It is important to note that because the users Outlook profiles need to be recreated, the user will lose their local cached mailbox or OST file.  In an Exchange cutover migration every user will need to re-download their cached mailbox which can present problems especially when there are a large number of users accessing Office 365 behind the same internet link.
 
It is also important to note that not all permissions are preserved when mailboxes are moved to Office 365 using a cutover migration.  For example Send As permissions on mailboxes will be lost and administrators will need to reconfigure this once users are moved across into the cloud.
 
Exchange cutover migrations support Exchange 2003, Exchange 2007, Exchange 2010 and Exchange 2013.
 
Exchange Staged Migration
 
Unlike an Exchange cutover migration, staged migrations allow users to be moved across to Office 365 in small controlled batches.  Staged migrations provide simple co-existence with Office 365 in which mail flow is maintained between the on-premises Exchange organisation and Office 365.  This allows for users to exist both on-premises and in the cloud during migration.  It is important to note that staged migrations only provide simple co-existence meaning only mail flow will be available between the on-premises and environment and Office 365.  Exchange rich collaboration features such as delegation, calendar sharing, free-busy and meeting rooms will not function between Office 365 and on-premises users in a staged migration deployment.

Staged migrations also require additional infrastructure be deployed on premises.  The Microsoft Online Services Directory Synchronisation tool (also known as Forefront Identity Manager or DirSync) needs to be deployed on-premises on a server which is not running as an Active Directory domain controller.  This tool ensures that all recipient objects on premises are synchronised to Office 365 to ensure the Global Address List is fully populated both on-premises and in the Office 365 cloud.  This allows users both on-premises and in the cloud to view the entire global address list - a process which is often referred to in online documentation as GALSync.  It is also important to note that Office 365 tenants must have an M or an E type licensing plan available in their Office 365 subscription in order to use the Microsoft Online Services Directory Synchronisation tool.

Exchange staged migrations also work by the Office 365 servers connecting the on-premises Exchange servers through RPC over HTTPS (also known as Outlook Anywhere) and downloading all mailboxes into Office 365.  As a result the on-premises Exchange environment must be configured with Outlook Anywhere and a valid digital certificate for the migration process.

Unlike cutover migrations however, staged migrations do not support incremental synchronisation of mailbox information.  This is because in a sense, staged migrations do not need incremental synchronisation to due to simple coexistence.  When the Office 365 migration wizard starts migration on a selected batch of on-premises users, the wizard automatically configures the TargetAddress property of the user’s on-premises mailbox using administrative credentials of an on-premises administrator supplied to Office 365 to an Office 365 specific email address.  This ensures for every email the on-premises mailbox receives, the on-premises Exchange server also forward a copy of the email onto Office 365.  This process is shown in the following diagram.
 
 
Note: If the user creates changes in the user’s mailbox such as deleting content or performing tasks which do not involve SMTP transport, these changes will not be synchronised to the cloud during simple co-existence as these activities cannot be sent via an email message.
 
When the Microsoft Online Services Directory Synchronisation tool runs it creates mail-enabled users for every mailbox user on-premises within Office 365.  As soon as the first migration batch is run in a staged migration, Office 365 will convert the mail-enabled user in Office 365 to a mailbox user.  As a result users can begin using the cloud based mailbox instantly – with this in mind however the users mailbox will be completely empty as the migration batch has just been kicked off and has not synchronised yet.  Companies can provide users instant access to their Office 365 cloud mailbox and advise the users their mail “is still coming” and reinsure the users they will not lose any mail.  Alternatively administrators can wait until a synchronisation batch has completely finished and then provide users access to their Office 365 cloud mailboxes.

After every mailbox in the on-premises organisation has been migrated to Office 365, companies can then point the MX records for any on-premises “accepted domains” to the mail gateway addresses provided by Microsoft for Office 365.  Like with cutover migrations, all user profiles will require recreation and users will get a new OST file meaning they need to re-download their entire mailbox from Office 365.  Staged migrations do have an advantage over cutover migrations however - companies can minimize the performance hit of re-downloading the users OST files by allowing users to recreate their profile in batches.  Also as with cutover migrations, all users will receive a new password from Microsoft which is to be used for accessing their Office 365 mailbox.

It is important to note that staged migrations only support Exchange Server 2003 and 2007, not Exchange Server 2010.  In Exchange 2010 and Exchange 2013, the TargetAddress property can't be modified. This is the reason that staged Exchange migration doesn't support migrating Exchange 2010 and Exchange 2013 mailboxes to Exchange Online.
 
Hybrid Exchange Deployment
 
Microsoft Office 365 hybrid migrations provide rich coexistence between an on-premises Exchange environment and Office 365 such as mail routing, calendar and free/busy information and mailbox delegation.  It provides organisations the ability to slowly move individual mailboxes to the Office 365 whilst keeping the rich collaboration experience provided by Microsoft Exchange between on-premises and cloud users.

Unlike cutover and staged migrations, Exchange hybrid migrations do not rely on RPC over HTTPS (Outlook Anywhere) to transfer the user mailboxes from an on-premises Exchange to Office 365.  In fact Exchange hybrid migrations use the mailbox replication proxy (MRSProxy) to perform mailbox moves to the cloud - a concept known as "cross-forest" mailbox moves.  MRSProxy is the same technology built into Exchange for performing things such as on-premises mailbox moves between mailbox databases or between Exchange servers in a local Exchange organisation.  The mailbox moves between an on-premises Exchange server and Office 365 are "online mailbox moves" which means the user is able to work and send/receive email while their mailbox is being uploaded to Office 365.

Out of all the Office 365 migration methods, Exchange hybrid migrations require the most amount of on-premises infrastructure be deployed and is the most complex migration method in terms of integration work.  The complexity built into the solution is what gives users a completely seamless experience irrespective of what environment they are in whether it be cloud or on-premises.

Exchange hybrid migrations require a minimum of three servers be deployed on-premises with more required in the event high availability be implemented.  These servers include:
  • The Microsoft Online Services Directory Synchronisation tool (also known as Forefront Identity Manager or DirSync)
  • Active Directory Federation Services (ADFS)
  • At least one Exchange 2010/2013 server on the on-premises Exchange environment
Note: The term Exchange hybrid server is used many places on the Internet.  This is simply an Exchange 2010 server with at minimum the Client Access and Hub Transport roles installed in an existing Exchange 2003 or Exchange 2007 on-premises organisation.  Exchange hybrid servers are not a different version of Exchange 2010 nor do they have different installation media.  It is the same version of Exchange, just configured to provide rich coexistence through federation.
 
Below is an overview of how hybrid configurations with Office 365 fit together taken from the Microsoft Exchange Deployment Assistant.
 
 
 The DirSync server as with staged migrations is responsible for ensuring the global address list in both on-premises and Office 365 environments remains fully populated with all mail-enabled users, groups and contacts in the Exchange environment.  This process is often referred to as GALSync in online documentation.

The Active Directory Federation Services (ADFS) server is a mission critical server in hybrid configurations and as a result it is often recommended clustering be implemented through Microsoft Network Load Balancing (NLB) or a third party load balancer technology.  When Office 365 is configured in a hybrid deployment with an on-premises environment, the "source of authority" is configured to point to the on-premises Active Directory environment.  What this means is all authentication requests for user which happens within a tenants Office 365 environment is sent back to the on-premises environment through ADFS to authenticate against an on-premises domain controller.  In the event the on-premises ADFS deployment become unavailable, users will not be able to authenticate with Office 365.

The Exchange 2010 / 2013 server is required to setup a federation trust between the on-premises Exchange organization and Microsoft Federation Gateway.  This is what enables the rich co-existence to occur between the on-premises Exchange environment and Office 365 to allow features such as free/busy and delegation to work between the two environments.

A hybrid configuration between Office 365 and an on-premises Exchange environment is the best approach for organisations who wish to setup rich co-existence and migrate users slowly in controlled batches.  Unlike other migration methods such as cutover or staged, hybrid configurations do not require users to obtain a new password.  Through the use of ADFS they can continue using their on-premises username and password for logging in and accessing cloud resources a process known as single sign-on.

Another advantage of performing a hybrid migration to Office 365 is users do not need to recreate their Outlook profile, a major disadvantage of cutover and staged migrations.  This means users do not need to re-re-download the entire Outlook OST file or “cached exchange mode” cache.

Think of a hybrid configuration as making your on-premises Exchange and Office 365 environment one and the same.  Some organisations setup a hybrid configuration for other purposes other than just migration to Office 365.  For example, if you want to put a user’s secondary archive mailbox in the cloud a hybrid configuration is needed.  Hybrid configuration also allows you to migrate users through mailbox moves from the cloud back to an on-premises environment providing the utmost flexibility.

IMAP-based Email Migrations

An Office 365 IMAP migration provides the ability to migrate an IMAP based mailboxes to Office 365 in which Office 365 downloads the user’s mailbox by connecting to the IMAP server.  IMAP migrations are intended to provide a migration path for non-Microsoft email systems to Office 365.

An IMAP migration works by creating a CSV file with all the account names and passwords for each of the IMAP users which are to be migrated to Office 365.  This process means administrators need to gather each user’s password to their IMAP mailbox and manually input the password into the CSV file.  This CSV file is uploaded to Office 365 in the web interface.  Office 365 will then connect to each mailbox and download all email for each user into Office 365.

As with cutover migrations, Office 365 is able to perform incremental syncs every 24 hours for IMAP mailboxes to ensure any new emails which arrive on-premises are also synchronised to the cloud.

When all users have been migrated to Office 365, administrators simply need to cut over the MX records for any domain names associated with the mailboxes and distribute the new Office 365 username and passwords to the end users.


I still need help!

In the event you need assistance with planning or performing your Office 365 email migration, I am more then happy to assist.  Please get in contact with me by emailing me on clint.boessen@avantgardetechnologies.com.au or leave a comment below.

I hope this blog post has been helpful.

21 comments:

  1. Thanks for your detail explanation of Office 365 email migration. It's a great article

    ReplyDelete
  2. Excellent Article.. Thanks for the clear explanation... :)

    ReplyDelete
  3. Thanks Clint for the post, I am sure it will prove to be useful to many online users. I like the simplicity with which you have explained things about email services.

    Email Service Provider

    ReplyDelete
  4. Hi Clint,

    I noticed an issue in this post in the staged migration section. Having recently performed a ~250 user staged migration, while DirSync is active AD passwords are synced to 365 every 3hrs unless forced earlier. Users will only have different passwords if DirSync is disabled after the migration is completed and their password expires according to the policy set within the 365 portal.

    ReplyDelete
  5. This is correct. Whenever DirSync is activated, the source of (AD) authority is the on-premise AD server. If DirSync is disabled - the Source of Authority automaticallly becomes Azure.

    ReplyDelete
  6. It's really an informative and well described post. I appreciate your topic for blogging. Thanks for sharing such a useful post.

    ReplyDelete
  7. Hi Clint,

    I have a 50 mailboxes cut over migration that has only one pending to finish and it is taking for ever. Reporting more bits migrated than the mailbox size is.
    Is there any way to finish the batch "normally" and start the synching process for the rest of the malboxes? I could deal withe the crazy mailbox manually?
    What happen if I stop the batch and delete it and then re create it?

    ReplyDelete
  8. What happens if you export the mailbox to PST in Exchange Shell out of curiosity?? How big is the PST file?

    ReplyDelete
  9. Clint,

    Doing all the imports manually was always my plan B, but I wanted to use cut over migration as it offer the convenience of not having a abrupt change for the users. I can have the mailboxes replicated to O-365 and I can choose when doing the MX change with no rush .

    I finally found the answers to my questions by mean of experience, so for the record let me briefly provide the answers in case somebody need them.

    1) The batch can be terminated with a Power Shell command but I wouldn't call it "normally" as it would cancel the incremental synchronization which in my opinion defeat the whole idea of the cut over migration. I would have to do the MX record change right after.

    2) Regarding stopping the batch or canceling it, I learned that if you stop and re-start it the mailboxes resynchronize again, so no problem with that but it didn't solve the issue of my problematic mailbox. I also deleted the batch and re-created it and had the same result.

    So the solution was to configure the mailbox as hidden from the Global catalog on the on-premisses Exchange Server, and created a new batch and this time the batch did not include the mailbox. The new batch picked up the already migrated data so it ended quickly and I had to export the missed mailbox to pst and import it via Outlook client to Office 365.

    So in conclusion if I want to enjoy the convenience of the incremental synchronization until the installation is ready to move to O-365 I need to have a cut over process ended normally to the status called "Synched".

    Despite of the issue I had I find that cut over migration work pretty good. The only recommendation I learned from MS Support is make sure there are not attachments bigger that 25 MB on any mailbox as that is a limit imposed by the migration process.

    Thanks

    ReplyDelete
  10. Enable Archive and Legal Hold, its part of 365...you just need to get your partner to setup. Our have a great tech who posted this website office 365 backup solution

    ReplyDelete
  11. Office 365 is latest way of converting data and info files from previous office.Office 365 is more securely to share files and documents with customers , co-workers and partners. These are documents you can easily accessible from anywhere.More info http://skynovate.com/

    ReplyDelete
  12. Nice article, thanks for the wonderful information related to migrate exchange server 2003 to office 365, I found this exchange server recovery tool from http://www.lepide.com/exchange-manager/ that provides the facilitate to migrate from exchange server to office 365 mailboxes without loss of any data in the process and helps to recover corrupt on inaccessible exchange server database and get rid of all sorts of exchange corruption issues with out any problem. This tool allows to move mailboxes from un-mounted edb files to different exchange Server on the network and migrates multiple mailboxes data between two Live Exchange Servers, migrate user mailboxes from offline edb or live Exchange to Office 365.

    ReplyDelete
  13. Hi Clint,

    Really nice article you have clearly defined each point in a very interesting manner. Thanks for sharing such useful post

    Email Marketing Solutions

    ReplyDelete
  14. Hi Clint;
    cannot sent mail to user than don't sync with cloud by dirsync from user on cloud. But it is ok for sync user. is it normally ?
    Thank you for your post..

    ReplyDelete
  15. Your blog is sharing really great infomarmation , However if you are looking to migrate from Exchange 2003, 2007,2010 & 2013 to office 365 environment , this blog is sharing some manual as well some professional methods explaining Exchange to office 365 migration. I hope this will help
    For more information to Migrate Exchange to Office 365 visit : http://exchange-migration-tips.blogspot.in/2015/05/exchange-to-office365-migration.html

    ReplyDelete
  16. Great Article.. Thanks for the clear explanation on 365 Migration... Very few are aware about this who are providing email marketing service in India, which you have explained things about email services.

    ReplyDelete
  17. We are backed by an experienced team of workforce, which is expert in their field and offers accurate cutting of the profiles. The services are rendered by the experts keeping in mind the requirement of the clients in mind. Further, we provide training to the professionals on the new machines in order to develop an impeccable range.

    MS Profile Cutting in Ahmedabad

    ReplyDelete
  18. Recommend you guys a good site to get cheap and genuine product keys for office:www.cdekey.com
    , all versions of office keys can be found there.

    ReplyDelete
  19. I have a 50 mailboxes cut over migration that has only one pending to finish and it is taking for ever. Reporting more bits migrated than the mailbox size is.
    Is there any way to finish the batch "normally" and start the synching process for the rest of the malboxes? I could deal withe the crazy mailbox manually?
    What happen if I stop the batch and delete it and then re create it? Web Application Development

    ReplyDelete
  20. Hi, Very nice description about StudentsOffice 365 Migration
    I like your web blog.Because whenever i come into your web blog
    then i always get the new interesting and important information in your web blog.

    Thank You

    Office 365 Migration

    ReplyDelete
  21. Did you know that you can create short urls with Shortest and make money for every visitor to your short urls.

    ReplyDelete