Forefront Identity Manager 2010 R2 build 4.1.3451.0 released

Posted on June 10, 2013 by

June 9th saw Microsoft release a new hotfix rollup package (build 4.1.3451.0) for Forefront Identity Manager (FIM) 2010 R2.  The official documentation for this build can be found on the Microsoft support website under KB article kb2849119.  Download link is here.  This build supersedes 4.1.3441.0.

Looks like two FIM SYNC fixes, one FIM REPORTING fix and a new feature for FIM CM.  (Quick details (duplicated from the official KB):

FIM Synchronization Service

Issue 1

Password management operations fail because the path for the cached version of the extension .dll file is too long. This problem also affects the WebService connector that is included in Forefront Identity Manager 2010 R2.

Issue 2

In certain cases in which the Synchronization Service processes ancestors, memory leaks occur.

FIM Reporting

If you have Microsoft System Center Service Manager 2012 Service Pack 1 (SP1) installed, and you try to run a change-mode installation for FIM Service and Portal, the installation fails.  If this happens you must re-enable reporting:

  1. In the FIM Identity Management Portal, follow these steps:1.In the Administration menu click All Resources.
  2. Under All Resources, click System Configuration Settings.
  3. Click the System Configuration Settings object and open the properties of this object.
  4. Click Extended Attributes, and then select the Reporting Logging Enabled check box.
  5. Click OK.
  6. Click Submit to save the change.

FIM Certificate Management

This update adds the ability in the SubjectAltName policy to specify the RegisteredID alternate name in the Subject Alternate Name entry when a certificate request is issued.

Posted in FIM, FIM 2010 R2, Hotfix | Tagged , , , , , , , | Leave a comment

Editing the FIM Portal web.config in a farm topology

Posted on May 16, 2013 by

Hit an interesting issue this week.  One of the projects I’m working on was patching pre-production with 4.1.3419.0 and at the end of the process two of the four portals were down.  Properly down.  HTTP 500 error.  The issue was in the web.config.  I won’t bore you with the detail, as there’s too much background, suffice to say that during backup and restore of the web.config the order of some custom elements got messed up which caused IIS to fail to load the web application.

Ultimately this isn’t of much relevance to the readers of this blog but it made me notice something else.  Each time we patch we loose the requireKerberos property of the resourceManagementClient element, i.e. before the patch:

<resourceManagementClient resourceManagementServiceBaseAddress=
http://idmgmt.contoso.com:5725
 timeoutInMilliseconds=”60000″ requireKerberos=”true” />

After the patch:

<resourceManagementClient resourceManagementServiceBaseAddress=
http://idmgmt.contoso.com:5725
 timeoutInMilliseconds=”60000″ />

Chatting with one of the project team who knows a lot more about SharePoint than me and I realise that there’s more to the web.config in a farm than I’d previously considered.  It’s stored in the SharePoint configuration database for one.  Each farm member updates the local file with the persisted object data in the database.  Which means there’s a programmatic process managing that file, so what happens to manually added changes?  Right, they can get lost.  Or maybe they don’t get consumed by all nodes.  In fact, there’s an health analyser rule that checks this for you:

But who manages FIM’s SharePoint infrastructure?  Are you on top of all the alerts?

The FIM Portal patch process triggers a backup of the web.config.  A programmatic backup.  The requireKerberos property was added out-of-band, i.e. via NOTEPAD.  So that isn’t in the database and is lost.  Which brings us to the real question – how do we edit the file then?  How do we define requireKerberos in an SP farm?

PowerShell

Here’s an example.

## modify the $fIMPortalURL variable with the FIM portal URL
[String]$fIMPortalURL = "https://idweb.contoso.com/identitymanagement";
## do not modify anything else

Write-Host "`nUpdateFimPortalFarmWebConfig.ps1 v1.0 Paul Williams May 2013`n";

[System.Reflection.Assembly]$loaded = [System.Reflection.Assembly]::LoadWithPartialName("Microsoft.SharePoint");
if(!$loaded)
{
    Write-Error "Couldn't load required assembly.";
    Exit;
}

[String]$sPPortalURL = $fIMPortalURL.Substring(0, $fIMPortalURL.IndexOf("/", 9));

# get the site
[Microsoft.SharePoint.SPSite]$sPSite = `
    New-Object Microsoft.SharePoint.SPSite($sPPortalURL);

# get the web app from the site
[Microsoft.SharePoint.Administration.SPWebApplication]$sPWebApp = $sPSite.WebApplication;

# get the web service from the web app
[Microsoft.SharePoint.Administration.SPWebService]$sPFarmService = `
    $sPWebApp.Farm.Services | ? {
        $_.TypeName -eq 'Microsoft SharePoint Foundation Web Application'
    }

# get the web.config rmclient element from the web app
[Microsoft.SharePoint.Administration.SPWebConfigModification]$rMClientConfigElement = `
    $sPWebApp.WebConfigModifications | ? {
        $_.Owner -eq '7c43ce5b-a59b-44f5-9e8a-50bd1b696145' -and
        $_.Name -eq 'resourceManagementClient'
    }

if($rMClientConfigElement)
{ # the element exists
    [Xml]$element = $rMClientConfigElement.Value;
    [String]$requireKerberos = $element.resourceManagementClient.requireKerberos;

    Write-Host "Current value:";
    Write-Host "$($rMClientConfigElement.Value)`n" -ForegroundColor Yellow;

    if($requireKerberos)
    { # we have a value that might need to be modified
        if($requireKerberos.ToLower().Equals("true"))
        { # nothing to do
            Write-Host "Require Kerberos property already set to TRUE.";
            Write-Host "No changes required.  Script complete.`n`n";
            $sPSite.Dispose();
            return;
        }
        else
        {
            Write-Host "Require Kerberos property not set to TRUE.  Setting to TRUE.";
            $rMClientConfigElement.Value = `
                $rMClientConfigElement.Value.Replace('requireKerberos="false"', 'requireKerberos="true"');
        }
    }
    else
    { # we need to add it
        Write-Host "Require Kerberos property not defined (default value is FALSE).  Setting to TRUE.";
        [String]$val = $rMClientConfigElement.Value;
        [String]$newVal = $val.Replace("/>", 'requireKerberos="true" />');
        $rMClientConfigElement.Value = $newVal;
    }
}

Write-Host "New value:";
Write-Host "$($rMClientConfigElement.Value)`n" -ForegroundColor Yellow;
Write-Host "Committing update.";
$sPWebApp.Update();
$sPFarmService.ApplyWebConfigModifications();
$sPSite.Dispose();

Write-Host "Script complete.`n`n";

The above script is quite simple and can undoubtedly be improved.  It does provide a supported mechanism to configure this setting in a SharePoint farm however.

Some more reading:

And the guys who provided what I needed to write the above script:

I hope that this has been interesting and helpful.

I alluded to this previously.  And I’ll be complaining about the complexity or pain of a SharePoint farm again when I get a better understanding of what patching does to FIM Reporting too…

Posted in FIM, Scripting, SharePoint | Tagged , , , , , , | 1 Comment

FIM Portal in a SharePoint farm–why you should not do this

Posted on May 15, 2013 by

That is correct.  You read the title of this post correctly.  I am, officially, on a mission to save you from making one of the biggest FIM Portal deployment mistakes you can make.  Do not, I repeat, do not use a SharePoint farm!  Even if you have multiple nodes.  Use multiple standalone SharePoint instances and load balance the requests using your preferred mechanism.

Why?

Well ask yourself this – what does a SharePoint farm actually give you?  Other than a serious headache when patching and more operational maintenance in the form of SQL databases and more complexity around backup and restore (or rebuild) and patching SharePoint itself (and modifying web.config).

I have deployed two multiple node farms – once on WSS 3 and once on SharePoint 2010.  I regret it.  Why did I do it?  Because I didn’t have a compelling argument either way and was talked into it by the customers.  One argument – WID isn’t for enterprises; the other – the simplification of deploying customisations.  I bet the former complicated their AD FS 2.0 deployment using the same argument too.  But I digress.  I’m not bitter.  I’m just wiser; a little older, a little more experienced and now, thanks to this topology, showing the first signs of grey hair!  Smile

I now have a somewhat compelling argument (although until you’ve sat in hours of meetings regarding patching you can’t really appreciate some of my sentiments):

I have to patch my four nodes sequentially, and the process takes quite a bit longer than a standalone node.  The reason?  Each time we patch we retract the solution pack and deploy the new one.  The benefit of using a farm is that you deploy the solution pack once.  Right.  But the FIM installer isn’t optimised for a farm – there isn’t a portal installer and a service installer, there’s one patch for both.  So you have to patch the portal for each node even though you don’t have to.  And this causes downtime.  Each time you retract and deploy a solution pack in SharePoint all application pools related to SharePoint are recycled.  Each time you patch FIM the service must be stopped.  Is it easy to run with fewer nodes and patch sequentially – no, you need to get all nodes on the same build ASAP and the more nodes there are the longer the SharePoint retraction is.  And the retraction and redeployment is global (to SharePoint) so that downtime affects all nodes in the farm – you can’t drop one out of the NLB array, patch, add back, drop the next, etc.

I have to patch all four nodes when patching language packs.  This is where it gets really nasty.  The language packs comprise some file system files for the FIM Service and SharePoint solution packs.  Again, one installer.  Nice!  It actually takes us ~6 hours to patch the LP on one node, and I have to sequentially apply four patches.  Yep, that’s a full day or at least two working days if there aren’t shift-staff.  And, all the time the application pools are being recycled.  So you need scheduled downtime.

I also have three or four database in my SQL instance that now require more management than the optimised WID files that are totally expendable.

And again, what benefit am I getting?  It’s a little easier to redeploy my CSS changes.  True, but scripting that copy operation on all nodes is not exactly hard.  We’re certainly not getting the benefit of a deploy once for many node solution pack.  Also, it’s harder to modify the web.config file of the portal – technically you now have to do this via PowerShell (or .NET) otherwise processes that actually manage the web.config can loose your changes or cause indirect breaking changes.

I have been involved in quite a few FIM Service and Portal design and deployment projects now – various levels of involvement, from owning and authoring the design and deployment documentation and synchronisation and WF code including actually doing the installation to writing the design and running for the hills, including being called in to save a failing project and migration/upgrade planning and technical documentation review.  In all of these cases I can’t think of a single compelling argument for a farm.  True, there were arguments on one of my most recent projects that meant I was in agreement at the time but hindsight has only strengthened my position – truly bolstered it actually.

Humbly happy to understand arguments for a farm via comments, forum or group posts or e-mail.

Posted in FIM | Tagged , , , , , , , , | 1 Comment

FIM Reporting Extract, Transform and Load (ETL) process

Posted on May 9, 2013 by

Forefront Identity Manager (FIM) Reporting and System Center Service Manager (SCSM) Data Warehouse Extract, Transform and Load (ETL) process

Or…How data moves from the FIM Service to the data warehouse (otherwise known as what on earth is all that disk activity and why is the Service Manager transaction log that size!!)

FIM Reporting

FIM Reporting adds a number of changes to the FIM Service.  It introduces new tables and schema, and a SQL Server agent job.  It also updates the SCSM configuration with a set of pre-defined management packs which describe FIM schema and reports to the Data Warehouse.

FIM Reporting flows data from the FIM Service to SCSM using one of three ETL processes (arguably they are not ETL processes but rather Extract processes).  Once the data resides in SCSM the System Center Data Warehouse ETL processes archive the data for long term storage and reporting.

FIM Reporting ETL

The FIM Reporting ETL process is processed by the FIM Reporting Service – a thread within the FIM Service.  msidmReportingJob resources created in the FIM Service invoke the ETL.  An msidmReportingJob resource can be one of three types: Initial, Initial-Partial, or Incremental.  Initial and Initial-Partial are deployment processes, whereas Incremental is a continual process that runs throughout the normal operating conditions of the FIM Service.

  • Initial.  The initial process reads data directly from the FIM Service database and stages it to SCSM.  Initial synchronisation is the step performed during deployment after Reporting has been successfully installed.  This process must complete in order for the other processes to be viable.  This process provides SCSM with a baseline and defines the watermark for the subsequent incremental runs.
  • Initial-Partial.  This process moves new schema and bindings into SCSM.  Every time the FIM Service schema is changed and the changes need to be reflected in FIM Reporting (the changes don’t necessarily need to be defined in FIM Reporting so this is optional depending on whether you wish to report on this data or not[1]) this process must be run to add the new schema to the Data Warehouse configuration.
  • Incremental.  This process reads data from the FIM Reporting export log.  The export log maintains all deltas since the last successful run.  The changes are staged to SCSM.  The process runs every eight hours by default, starting at 1200 local time.  The schedule can be modified.  Generally the timeline of eight hours remains, but the start times are modified to better fit with other scheduled processes.

All of the FIM Reporting processes stage data in SCSM via the Data Access Layer – the interface into SCSM (basically the API).  The actual data is written into the ServiceManager database.  Once the data is staged the FIM Reporting work is complete until the next incremental process.  The System Center Data Warehouse (SCDW) jobs perform the rest of the process.

Figure 1 illustrates the flow of data from FIM to SCSM.

image

Figure 1: FIM Reporting ETL process

System Center Data Warehouse

SCDW has an ETL process that results in the FIM Service data staged to SCSM being added to the SCDW for long term archival and reporting.  SCDW has three physical databases, one for each phase of the ETL.

SCDW jobs

The data warehouse deploys with six jobs, two of which are administrative jobs, the other four are ETL jobs.  The jobs are:

Name Type Default schedule
DWMaintenance ADMIN:Maintenance Hourly
Extract_DW_FIMReporting ETL: Extract 5 minutes
Extract_FIMReporting ETL: Extract 5 minutes
Load.Common ETL: Load Hourly
MPSyncJob ADMIN: Synchronisation Hourly
Transform.Common ETL:Transform 30 minutes

Table 1: SCDW jobs and their schedules

The administrative jobs should be left in their default state.  The jobs are vital to the successful functioning of the DW.  The Extract_DW_FIMReporting extract job can be disabled.  This job is only required for FIM Reporting topologies that integrate with an existing SCSM deployment.  If SCSM is deployed purely for FIM Reporting this job is irrelevant and should be disabled.  The remaining jobs Extract_FIMReporting, Transform.Common and Load.Common are the ETL process.

Extract, Transform and Load

The ETL process first processes the extract job.  The relevant data (all changes since last run) is extracted from the SCSM (ServiceManager) database and written to the SCDW staging (DWStagingAndConfig) database.  The basic format is the same in the staging database.

Next the Transform.Common job runs.  This job takes the raw data from the staging (DWStagingAndConfig) database and performs the necessary aggregation, cleansing, and reformatting required for reporting and writes the data to the transformed (DWRepository) database.

Finally the Load.Common job runs.  The transformed data (DWRepository) is written to the reporting (DWDatamart) database.

When the Load.Common job completes new data is available to reporting.  SQL Server Reporting Services (SSRS) targets the reporting (DWDatamart) database.

Figure 2 illustrates the process (this is a re-drawing of the image in the Service Manager blog post in the references section).  Green arrows are extract; orange transform; blue load.

image

Figure 2: SCDW ETL process

Viewing reports

Once the data is loaded into the DWDatamart database it is available to any permitted consumer.  In the case of FIM Reporting this is SQL Server Reporting Services.  Illustrated simply, Figure 3 depicts this.

image

Figure 3: SSRS data access

Technically SQL Server Analysis Services (SSAS) surfaces the data via an online analytical processing (OLAP) cube, however I don’t think that is wholly relevant to this discussion and is managed by the SCDW.

Summary

Figure 4 depicts the overall FIM Reporting and SCDW ETL architecture.

image

Figure 4: Overall ETL process

References

In addition to nosing at configuration and sending a couple of e-mails the principal sources of information for this post were:

Any mistakes are my own, and I’m grateful for comments and feedback.

I hope this is useful!

—

[1] You only define new schema in the data warehouse if you wish to report on that data.  This does not include historical reporting.  By default the request report handles modifications to both the FIM Service schema and attribute changes in FIM, regardless of whether or not you have pushed this schema into Reporting.  The reason you define new schema and bindings is to report on current values, e.g. instead of trawling through the request history and ascertaining current values you extend the schema and move current values into the data warehouse.  This is an important point, hopefully I have explained it properly.

Posted in FIM, FIM 2010 R2 | Tagged , , , , | 5 Comments

How can I logon to my ADAM or AD LDS Management Agent (MA)?

Posted on April 25, 2013 by

What credentials can I use for the Active Directory Application Mode (ADAM) or Active Directory Lightweight Directory Services (AD LDS) Management Agent (MA) in Forefront Identity Manager (FIM) 2010 or R2?

Bit basic this post but I had to install FIM Synchronization Service and an AD LDS instance during a meeting to get this answer so figured I might as well post the information here for posterity.  Smile

Binding to ADAM/AD LDS

For the rest of this post I’m going to use the term ADAM however everything I say applies to AD LDS (in fact my testing was using AD LDS on Windows Server 2008 R2).  And when I say bind I mean logon. 

OK, with that out of the way.  You can bind to ADAM using an ADAM principal (some user-defined object type in the ADAM instance that “implements” the msDS-BindableObject class, i.e. an ADAM user); a Windows principal (a local user object on the Windows computer running ADAM, i.e. a user in the Local Security Authority or LSA, or in Active Directory Domain Services or AD DS, e.g. a user or inetOrgPerson object); a user proxy (a special type of user object in the ADAM instance that is linked, via objectSid, to an AD DS user); and lastly anonymous binds.

ADAM only authenticates “ADAM principals”.  “Windows Principals” and user proxy objects are authenticated by Windows, i.e. the local LSA or AD DS.

ADAM supports the following credential formats:

  • Windows principals:
    • User Principal Name (UPN).  Domain-based Windows Principals only, e.g. paulw@msresource.net.
    • Account Name (sAMAccountName).  The account name, e.g. computer\paulw or domain\paulw, e.g. msresource\paulw.
    • Distinguished Name (DN).  Domain-based Windows Principals only, e.g. CN=Paul Williams,OU=People,DC=msresource,DC=net.
  • ADAM principals:
    • Display Name.
    • Distinguished Name.
    • UPN.

There’s some obvious caveats for ADAM principals.  UPN or displayName must be unique across all objects (irrespective of type, so if you have OU=paulw,O=msresource,c=GB you cannot logon with a displayName or UPN of paulw).

I heard you can logon with canonical name too, but limited testing did not prove that theory.

Specifying credentials for the ADAM MA

With enough background on the subject of binding to ADAM let’s look at what we can do with the ADAM MA.

By default, the MA is configured to support a Windows Principal, i.e. a Simple Authentication and Security Layer (SASL) authentication – this means SPNEGO a.k.a. Integrated Windows Authentication (IWA).

This means, by default, you are limited to NETBIOS_DOMAIN_NAME\username (or COMPUTER_NAME\username) and username@some-domain-name.tld.  Although when I say DOMAIN\username you don’t actually write it like that – there’s an input box for username, password and domain.  When you introduce an @ into the username the domain box is no longer required although not greyed out.

Fine.  Good.

Now, stick a DN in and you’ll get an error:

image

That picture says:

The user name cannot contain any of the following characters,

‘<’, ‘>’, ‘&’, ‘”’, ‘/’, ‘\’, ‘[‘, ‘]’, ‘:’, ‘;’, ‘|’, ‘=’, ‘,’, ‘+’, ‘*’, ‘?’

Try a basic username and password (no domain, e.g. displayname and password) and you’ll get a validation error:

image

That picture says:

Please complete the credentials.

The reason for this is because we’re in SPNEGO mode.  We need to change to SIMPLE BIND to unlock the ability to specify a DN or a username and password (could be UPN or displayName) with no DOMAIN/COMPUTER requirement.  That’s done via the options button (adjacent to “Configure Connection Security”):

clip_image002

So the default is to not only bind using SASL but use SASL to sign and seal (encrypt) all communications thereafter.  When you “Enable Simple Bind” you can choose to use SSL with or without Certificate Revocation List (CRL) Checking or (dev lab only please people) no SSL, i.e. clear text credentials.

After turning on simple bind you can use a DN to bind (as many LDAP folk expect):

clip_image002[7]

Hopefully someone will find this helpful.  Smile

Posted in FIM | Tagged , , , , , , , , | Leave a comment

Forefront Identity Manager 2010 R2 build 4.1.3441.0 released

Posted on April 22, 2013 by

April 22nd saw Microsoft release a new hotfix rollup package (build 4.1.3441.0) for Forefront Identity Manager (FIM) 2010 R2.  The official documentation for this build can be found on the Microsoft support website under KB article kb2832389.  Download link is here.

There’s quite a few issues fixed in this build, that I won’t reiterate here (although the change in behaviour to how failures in Update-Recipient were handled has reverted back).  There are also some important new features in FIM Synchronization Service – namely ECMA 2.2:

  1. MetadirectoryServicesEx.dll is no longer dependent on FIM Synchronization Service, therefore you can load an ECMA2 connector independently of the FIM Synchronization Service which makes various testing scenarios much easier.  This is a big deal and is here in advance of the PowerShell connector now available on connect.microsoft.com being generally available – it makes writing and debugging the PS scripts manageable.
  2. New capabilities page makes it possible to ask the user for information and connect to the target directory and use that information for the Connector’s capabilities.
  3. No longer need to provide the object type for update and delete operations in delta import.
  4. Support is added for DN as an anchor for LDAP based directories.
Posted in FIM, FIM 2010 R2, Hotfix | Tagged , , , , , , , | 1 Comment

FIM, System.DirectoryServices and a memory leak

Posted on April 19, 2013 by

It all started with a slow synchronisation.  Further inspection yielded several elements to the solution that strayed from recommended practice.  One aspect that was definitely playing a part in the slow synchronisation was several uniqueness checks against Active Directory Domain Services (AD DS) within the rules extension(s).  The check was required – to call-out or not to call-out is an argument for another time – but some of the locations were wrong.  My initial recommendation was to pull the uniqueness attributes, e.g. the sAMAccountName attribute, of all users into a Dictionary<String,String> within IMVSynchronization.Initialize and then use this local dictionary object for uniqueness checks (adding unique values as they are consumed).  There were some other recommendations as well but I won’t bore you with those.

Jump forward two or three months and we’ve got a critical situation – a severity A Premier Support case.  Over the course of a full synchronisation the FIM Synchronization Service exhausts available RAM and (gracefully, I might add) exits.  The host in question is a VM; 8GB RAM, 60GB HDD, 12GB page file.  Customer takes some traces and dumps and provides them to CSS and David quickly identifies that wldap32 is using all the RAM – 7GB in the data we’re looking at (while everything is still running).  The data shows ~65,000 125KB allocations.  Of 8.12GB of outstanding allocations 7.54GB from this stack alone!  FIM is doing this.  We prove it’s the rules extension and not FIM itself by running it in it’s own process and now mmsscrpt.exe (the process that runs rules extension code marked as “run out of process”) is using the RAM and not miiserver.exe (FIM Synchronization Service).

wldap32 is the Windows LDAP API.  Active Directory Services Interfaces (ADSI) sits on top of this.  And System.DirectoryServices (S.DS) sits on top of ADSI.  What is the rules extension using?  Right.  We have a memory leak coming from S.DS.

If we whip out the bible the issue is obvious.  Several of the classes within S.DS have to be explicitly disposed.  Let’s look at some documented remarks:

SearchResultCollection Class

Due to implementation restrictions, the SearchResultCollection class cannot release all of its unmanaged resources when it is garbage collected. To prevent a memory leak, you must call the Dispose method when the SearchResultCollection object is no longer needed.

Furthermore, Joe and Ryan have a sidebar in the bible that discusses Close() and Dispose().  Let me quote some of the text:

Both Close and Dispose are intended to be used to clean up the underlying COM object.  The primary difference between the two is that Dispose also suppresses .NET finalization, and Close does not.  Supressing finalization means that the garbage collector will not bother to run the Finalize method on the object because we have signalled that we have already cleaned up the underlying resource that the finalizer needed to take care of.  Objects that need to be finalized are automatically promoted one garbage collection generation, so the tend to hang around in memory longer, which is something we probably want to avoid if possible.

And:

In summary, use Dispose.  It does everything that Close does, and it takes care of the finalization.

Better yet, use the built-in language features to ensure that objects are Disposed properly.  In C# always use the using construct:

Later in the book they call out the SearchResultCollection class too:

This collection of SearchResult instances represents the result set as accumulated by the server for a specific query.  This class internally holds references to unmanaged resources, and as such should always explicitly be disposed using the Dispose method, just like DirectoryEntry and DirectorySearcher.

The crux of the matter here is this: when using S.DS in a process that runs in a loop it is imperative that the objects are disposed.  In the case of FIM the loop is synchronisation, i.e. calling the rules extension code over and over again (per managed object).

The customer quickly fixed the utility class that was using DirectorySearcher to return a SearchResultCollection to another method that was consuming it by properly calling Dispose in a couple of places and deployed the code.  The result was different but the same.  Different allocation sizes but still a leak.  Bring in the heavy artillery and the ninjas identify the actual method, in the customer rules extension, that is making the call(s).  A look at the revised code quickly identifies the issue.  The S.DS.DE and S.DS.DS objects are being disposed but the SearchResultCollection isn’t – probably because of the way it is being used, i.e. the object is returned by a method.  A second flaw is also identified –not the root of the problem but something worth fixing– all attributes are being returned.  We provide some recommendations around rejigging the method and properly disposing and also favouring a using statement in many places.

The next result was…problem solved.  While we see increased memory usage during synchronisation five minutes after synchronisation and it’s freed up (expected, as in-process rules extensions are unloaded after five minutes of inactivity).

Before I finish this post I’ll state some observations.  Many of us know these values but I cannot see that they’re actually publically documented.

  • A rules extension that runs in-process is unloaded after five minutes of inactivity.  You should expect to see all memory used by your rules extension DLLs returned to the OS five minutes after synchronisation completes.
  • A rules extension that runs out-of-process is unloaded after fifteen minutes of inactivity.
  • Unmanaged memory and/or memory that is leaked is not returned to the host after IMVSynchronization.Terminate and/or IMASynchronization.Terminate is called.  The leaked memory is only cleaned up when the hosting or parent process is recycled.  For out-of-process rules extensions this means the memory is returned after fifteen minutes when FIM kills mmsscrpt.exe.  For in-process rules extensions the memory won’t be returned until the FIM Synchronization Service (miiserver.exe) is recycled.

It was an interesting case and I learned some very valuable information.  Hopefully you’ll find this interesting and helpful too.

p.s. Kudos to David, Steve and Rob Lane, without these clever folk we would not have got to the bottom of the problem!

Posted in FIM, Programming, Troubleshooting | Tagged , , , , , , , , | 7 Comments