Windows Azure Active Directory Connector part 3: immutable ID

This post is part three of a series of posts focussing on the Azure Active Directory connector.  In part one I introduced the connector, discussing what it is, where and when to use it.  In part two I described a typical multi-forest deployment and provided some recommendations that should be considered when approaching such a deployment.  In this post I want to focus specifically on the immutable ID problem.

When deploying a multi-forest directory synchronisation solution for Azure Active Directory (AAD) the immutable ID used by the default, single-forest topology is generally not appropriate for the solution.  The decision as to what to use as the immutable ID is not a FIM decision but rather a key decision for the overall AAD integration project, generally owned by the AAD identity management solution.

In a typical Office 365 (O365) engagement the DirSync appliance is deployed along with Active Directory Federation Services (AD FS).  The AAD and AD FS Relying Party (RP) trust is configured using the Windows Azure Active Directory Module for Windows PowerShell.  In this typical pattern the immutable ID is the on-premises Active Directory Domain Services (AD DS) objectGUID attribute.  The objectGUID attribute is of the type Object(Replica-Link), which basically means a Byte array.  As a SAML token is XML the immutable ID claim is the base64 encoded format of the value.  DirSync therefore synchronises the objectGUID value as a base64 encoded string.  Here’s an example:

m3baazCHi02P5c3VEEHceg==

In a multi-forest O365 deployment the objectGUID is often not appropriate.  The name of the claim and the corresponding AAD attribute is quite literal and adequately describes why the default value might not be appropriate:

im·mu·ta·ble [ih-myoo-tuh-buhl]
adjective
not mutable; unchangeable; changeless.
Not subject or susceptible to change.

If objects move between forests we have a delete and create, which means a new GUID, thus the value is mutable.  That being said it’s not quite so clear cut in single-forest topologies either, as it’s important that the immutable ID is immutable across the lifetime of the identity object regardless of where that object exists:

  • If a contractor is hired as an employee, is the account reused or a new one created?
  • If an employee or contractor is terminated and rehired in a timeline that is shorter than the organisational identity management tombstone and deletion process, is the original account reanimated or a new one created?
  • If an employee or contractor moves between business units, what actually happens to the AD DS user object?  Typically an intra-forest move is just that, and the GUID remains unchanged, however this is not true of all establishments.  Some multi-domain forests delete and recreate when moving between domains; some single-domain customers delete and recreate when moving between containers (a crazy facet of detailed, granular delegation that security types applaud).
  • If a contractor is extended very late in the contract, i.e. on the contract’s last day, or within a few days of the termination date, what happens to the user object?  I’ve seen a new object created because the IdM process deleted the object as soon as the contract expired before the request for extension reached the IdM process.

As you can see, there are multiple reasons why the AD DS objectGUID might not be a suitable candidate for the immutable ID.  We’re implementing FIM with the AAD connector for single-forest topologies because of an inability to utilise the default immutable ID.  In fact, the lack of flexibility in selecting an immutable ID is a primary cause of why multi-forest DirSync using the DirSync appliance wasn’t used and was eventually canned.

Another option, often muted, for immutable ID is employeeID.  But run through the bulleted list above using employeeID and ask yourself is it really immutable (I’ve never see it be immutable)?  And don’t forget about operational practices, e.g. copying users or linking non-user mailboxes via a unique identifier (e.g. employee ID).

What happens if it changes?

Before going further we should look at why it is important that this value be unchangeable.  What happens if the chosen value changes for one user?  For 25% of your identities annually?  What is the impact of changing this attribute value?

The answer to this question very much depends on what online services are being used by both the specific identity that has changed and your user base in general.  A change can be a minor inconvenience for your admins (although if it’s frequent this becomes a significant operational challenge fast) or it can be a true problem that results in a Customer Support Services (CSS) service request being escalated back to engineering (which means several days of downtime for effected users).

Lets start by looking at what the synchronisation process does.  We’ll use an on-premises user object as our context.  There are two core scenarios here:

  • The on-premises user object’s immutable ID changes but the account is otherwise unchanged, i.e. the account is not deleted, e.g. you’re using employeeID and it changes for whatever reason.
  • The on-premises user object is deleted from the perspective of the synchronisation solution and reappears with a new immutable ID, e.g. delete/create or move out of in-scope container, modify, move back.

Let’s start with the first bullet as this is the quickest answer and only applies to FIM implementations.  The synchronisation solution only flows immutable ID when the metaverse (MV) sourceAnchor (immutable ID) attribute is null and there is no export attribute flow (EAF) defined on the AAD connector (source anchor is written to immutable ID during provisioning code only).  Changes are not flowed (or they shouldn’t be as they will be rejected by AAD).  SSO fails because of a mismatch between the on-premises value being sent in the immutable ID claim and the actual value in AAD.  The solution here is painless from a technical perspective – change the on-premises value and everything will work again.  If that solution doesn’t work for you then you need to read on as you’re going to move into the scope of the second bullet.

The synchronisation solution deletes the current object from AAD (soft-delete) and attempts to create a new object.  The create operation fails with an invalid-soft-match error from Azure Active Directory Web Services (AWS) because the deleted object, with all its attributes, is in the “recycle bin” and the immutable ID doesn’t match (and you can’t overwrite that attribute value).  You delete the object in question from the recycle bin (hard-delete) using PowerShell (or your own solution that uses the AAD Graph) and the next synchronisation schedule will successfully export the creation of the object to AAD.  The object looks almost identical, except the immutable ID is different.  SSO will work again, but the linkage with online services is severed, e.g. you have no licenses (until you reassign), you’ve lost your mailbox, your Lync profile and your SkyDrive.

Note

The only way you can write the immutable ID is by turning off Directory Synchronization, allowing the users to be converted (automatic process) to managed (cloud) identities, performing the modification, and then re-enabling directory synchronisation.  This process is not fast and will cause significant downtime if users are accessing the service.  The conversion from federated identity to managed identity sets an expired password for each user (i.e. you must change your password on first logon).  In the past, on tenants from previous waves, rumour has it that you could convert specific federated users to managed (F2M); delete the immutable ID, i.e. set the value to null or nothing; and then let synchronisation soft-match, however this is not possible today as changes to the immutable ID attribute are rejected.

Now we can look at some specific cases.

What happens if you use SkyDrive Pro?

Of all the issues I’ve seen related to immutable ID change this is the worst.  Inside of SharePoint Online (SPO) there is a linkage between the SPO user profile and the actual backend identity that has permissions within the file system where files physically reside.  The linkage between the SPO user profile and the  backend where files are written is maintained using, in SPO parlance, the PUID, or personal unique ID.  When an object is reprovisioned in AAD it can happily link up with the SPO user profile assuming the profile, in its orphaned state, hasn’t been purged (which happens) however the PUID is different, so while the identity will still see their MY site and SkyDrive they won’t actually have any rights to modify it.  Unfortunately the fix is a service ticket with CSS that gets escalated back to MSO engineering for a manual fix.

What happens if you use Exchange Online?

This is nearly as bad.  In the hybrid Exchange organisation or fully migrated Exchange Online organisation you cannot re-attach a disabled/deleted mailbox to a new on-premises user object, only a cloud (managed) identity.  This means if FIM reprovisions the AAD identity you cannot attach the disconnected mailbox.  What you need to do instead is create a new cloud ID immediately after emptying the original ID from the AAD recycle bin and then reattach the mailbox.  After doing this you must wait for synchronisation and soft-match, i.e. FIM will submit a create request to AWS which will match the data in the create request to the existing cloud ID and update attributes of the cloud ID and set the immutable ID thus marking it synchronised.

Note here that if you don’t maintain the original cloudLegacyExchangeDN as an X500 proxyAddresses value on-premises you’re likely to hit Outlook recipient cache and NDR issues too.

What happens if you use Lync Online?

I have no idea of the server-side implications and will surreptitiously update this post when I have more of an idea.

At the client side your profile is lost and has to be recreated.

Designing your immutable ID

Now that I’ve hopefully painted a bleak picture around why the value must be immutable (i.e. unchangeable) for the lifetime of the identity lets spend a few minutes thinking about what can and should be used for an immutable ID.

As already stated, introduce this discussion early on in your engagement/design discussions.  Leave it too late and the wrong decision will be made and you’ll have both an unhappy customer and a painful experience moving to the right decision technically.  Start by asking the following questions:

  • Does the customer have an immutable ID?  Organisations with a more mature set of identity management processes will have an existing attribute that fulfils this purpose, generally managed by the existing identity management process.  If it exists, use it.  If it exists, but doesn’t exist in AD push hard to get it synchronised into AD and used.
  • Is an immutable ID something that is going to be implemented in the near future?  Is there an existing project to establish an IdM process or infrastructure that will provide this value?  Is such a project planned?  Is it catering for a value of this type?  If so, can this be a dependency on your project, i.e. can you wait?

If there isn’t a clearly defined unique identifier throughout the lifetime of an identity you need to consider creating one or introducing some processes to ensure you have a viable attribute value.  For the purpose of Azure AD SSO (directory synchronisation and AD FS) it only really needs to exist in the Active Directory (or the identity provider defined in your single-sign on considerations if not using the AD FS and AD DS pair) and the typical scope of a directory synchronisation project won’t include the design and implementation of an immutable value, so consider the following:

  • When evaluating candidate values run each through all known identity management processes, paying particular attention to the JML (joiner, mover and leaver) process.
  • When evaluating candidate values run each through all known operational account management processes, again paying particular attention to out-of-band joiner/mover/leaver requests, cloning/copying of objects, linking objects to other objects via string or integer based data.
  • Remember that the value *is* case sensitive, i.e. the string value that is sent as the immutable ID claim is case-sensitive.
  • Devise a strategy for persistence, i.e. consider attribute recall and think about the effect of recall.  I say this because the immutable ID is obviously quality join data, and you should consider disaster recovery-based re-joins and data quality remediation issues in the on-premises identity source, so flow it into the MV, flow it out to other on-premises connectors and flow it back in from AAD again so that repopulation results in the MV always having the immutable ID.

Also, don’t forget that the immutable ID is more important and applicable for users who sign-on to the service, or objects (here I include non-user objects, e.g. room and equipment mailboxes) that either have a license or perform a function in the Office 365 service.  I say this because some candidate values, e.g. employee ID or even IdM-managed immutable ID, are only applicable to users, so you might well end up with a situation whereby you utilise two different sets of immutable ID – one for users and one for non-users.

Note

If you actually find yourself with multiple sets of immutable ID where there’s more than one set of authenticating users you’ll need to handle sending the right claim/assertion values in the SAML token, which means updating your relying party (RP) trust issuance transform rules if you’re using ADFS.  I discuss an approach here.

Some examples/ideas

I’ll wrap up this post with some general examples or ideas.  I’ll keep them short as this is a blog post, not a whitepaper, and it already feels lengthy.

  • Do not EAF sourceAnchor on the AAD connector.  Write it during provisioning only.  IAF sourceAnchor from AAD as well as AD and ensure AD is precedent (this flow is for MV repopulation).
  • Consider EAF sourceAnchor to AD.

My friend and colleague Nilesh Ghodekar came up with a great idea that we’re considering standardising on when we don’t have a sound immutable ID (here in UK MCS, not throughout Microsoft): use the GUID and persist it back into the source directory (we’ll use the mS-DS-ConsistencyGuid as it’s of the same type and unused and not copied when a user is copied).

Here’s an example process flow:

  • If the sourceAnchor attribute (in the metaverse) is null, flow either the object GUID or the mS-DS-ConsistencyGuid as the source anchor, i.e. flow mS-DS-ConsistencyGuid if it is present, otherwise flow objectGUID.
  • Write the sourceAnchor value back to the on-premises mS-DS-ConsistencyGuid.
  • Modify the AD FS issuance transform rule(s) to always send the mS-DS-ConsistencyGuid instead of the objectGUID.

The other option is to build an immutable ID process.  This is not part of a typical O365 engagement, but can often be part of a wider IdM project, e.g. deploying full FIM.  This is not really something I want to recommend as part of a blog post.  To do this properly you need to track what’s issued and support rollback, i.e. if you recover your backup you need to inflate the seed, etc.  All of this is reasonably painless but requires a real design document to do it properly.

I hope this has been helpful.  This might be the last of my AAD connector posts.  Or there may be one more.  I’ll see how I get on when I implement my new solution after following my own advice…

Advertisements

About Paul Williams

IT consultant working for Microsoft specialising in Identity Management and Directory Services.
This entry was posted in Azure Active Directory, DirSync, FIM, FIM 2010 R2, Office 365 and tagged , , , , , , , , . Bookmark the permalink.

15 Responses to Windows Azure Active Directory Connector part 3: immutable ID

  1. Pingback: (2014-03-21) GALSync, DIRSync And SSO With Office 365 Blog Posts From MSResource.NET « Jorge's Quest For Knowledge!

  2. Benoit Boudeville says:

    Funny how we work on same topics but never had any interaction with each others. i’ve done this one year ago (for the setup) and we came to the same idea, more or less. We create a new GUID on the fly, first time the MV object is created and then this becomes the ImmutableID. The GUID is written back to the UID attribute (Text Form), and ms-Ds-ConsistencyGuid (Binary Form) in the source AD.

    We did this for multiple reasons:

    1. L1 Admins tend to delete accounts and reacreating them thinking ‘just because they recreated identitcally they should be the same, ne?’ ==> duh 🙂 – so we have a proper way to relink the user / recover the service with minimum impact.

    2. mutations (cross-forest moves), and we also implemented a join rules on sidHistory just for fun when they can use ADMT as a safety measure in case they forgot to transport the UID / ImmutableID.

    You did not mention to reconfigure ADFS Claim Rules to use the consistency Guid in that case, in place of the ObjectGUID. Mandatory if you had to get through replacing a user, hence changing ObjectGUID but could recover the ms-DS-ConsistencyGuid from the ImmutableId (just a convert, that’s easy…).

    Also don’t forget DirSync has a mecanism to overload the ImmutableId join rule: I think they made it on extensionAttribute4 or something like that. If you copy the Base64 String ImmutableId to this AD attribute, a regular DirSync would join.

    Cheers,
    Ben.

    • Swapnil says:

      As we have single Forest and multiple Sub domain and we are planning to migrate sub domain users (900 users) to one of the sub domain using Quest MMAD tool, it create new ObjecGUID in the target sub domain and Office 365 Mailbox get disconnected. How we can connect disconnected mailbox to target subdomain Ad account?

      • Why is it creating a new GUID? That is wrong. An intRA-forest move doesn’t delete/create from the perspective of the GUID attribute. Check your QMMAD configuration…

  3. Benoit Boudeville says:

    Duh, I already realized I explained that two posts ago in this thread of articles, lol. Anyway, very well explained !

  4. Pingback: (2014-10-01) TroubleShooting Federation/SSO To Windows Azure AD And Office 365 « Jorge's Quest For Knowledge!

  5. Pingback: Office 365 – Why You Need to Understand ImmutableID | Microsoft Enterprise Technologies

  6. Pingback: Revisiting the Microsoft Online immutable ID design decision | Yet another identity management blog

  7. Pingback: Understand and Modify Office 365 users ImmutableID

  8. Prashant says:

    Hi Paul,

    I have an AD account and Azure account created.

    I would like to configure or create an Immutable ID for the User Object.

    Could you help me to know the steps to create/Configure the Immutable ID in the Windows Azure Account.

    Thanks,
    Prashant

    • Without DirSync in the mix you create a user in AAD using New-MsolUser, then you read the objectSID from AD DS using Get-ADUser and base64-encode that value using [Convert]::ToBase64String and write the base64-encoded string into the MsolUser immutable ID using Set-Msoluser.

  9. Pingback: Active Directory Security Id | My Home Security Tech

  10. Pingback: Understand and Modify Office 365 users ImmutableID | Microsoft Expert Talk

  11. Bharani George says:

    Hi, Can you please help me understand how we can sync users from non-AD (ADLDS) to AzureAD?

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s