Unpopulated sync rule “External System Resource Type” drop-down for FIM/MIM Generic LDAP MA

Learnings, MIM Service, Portal

In order to populate the External System Resource Type drop-down on the Create/Edit Synchronization Rule form of the MIM Portal, the client retrieves the ma-data object from the MIM Service using a WCF call. In certain cases (e.g. for General LDAP MAs connected to extensive back-end data sources like Oracle Identity Directories (OIDs)) the object returned can be very large – and if it is in excess of the maximum allowed size of an WCF call (14MB by default) then the call will fail and the External System Resource Type drop-down will be left blank/empty/not populated.

The solution to this is to edit the Portal’s web.config file and increase the limit to something larger (eg. 50MB).

To find the web.config file (usually something like C:\inetpub\wwwroot\wss\VirtualDirectories\80\web.config) run IIS Manager and right-click “Expore” from the MIM Portal site:

Near the end of the file (usually) add maxReceivedmessageSizeInBytes=”52428800″ to the resourceManagementClient element:

An web site or IIS restart is probably be required to active this change!

More info?
https://docs.microsoft.com/en-us/previous-versions/mim/ff800821(v=ws.10)
https://identityunderground.wordpress.com/2014/11/26/new-hotfix-rollup-released-for-fim2010-r2-now-build-4-1-3613-0/

Various MIM, Broker & UNIFYNow conventions explained

Good, Learnings, Other, Ugly, UNIFYNow

This post is a consolidated collection point for notes about MIM, UNIFYBroker and UNIFYNow conventions that should be followed, with explanations why.

UNIFYNow

MIM Run Profile Operation Names

Convention

In UNIFYNow, operations that run MIM profiles should always have the name of the MA and the Run Profile being invoked clearly evident.

Rationale

When MIM run profiles are recreated, or ‘Refresh MA’ is clicked in UNIFYNow (either intentionally or accidentally) the run profile GUIDs on operations can end up being incorrect.  When this happens it may be necessary or expeditious to simply re-select the appropriate MAs and run profiles from the GUI, in which case having the name of the MA and run profile in the operation name will be extremely beneficial.

In Event Broker 4 (an older version of UNIFYNow) there is a known issue where this can happen unexpectedly when a server is rebooted in certain, rare conditions.  The exact trigger for this behaviour is unknown.  It may be related to starting Event Broker on a system where the WMI Service is not running, but this is by no means certain.

UNIFYBroker-based AD Provisioning Model

Bad, Design Patterns, How To, Learnings, Sync Service, UNIFYBroker

Background

The UNIFYBroker-based Active Directory provisioning model uses a Powershell connector in Broker to action “once-off” operations for an AD user account, such as:

  • Account creation
  • Account deletion
  • Mailbox enable/disable

The Broker Powershell connector has the following configuration components:

  • “Import All” – e.g. pre-load all existing users from AD
  • “Import Changes” – e.g. identity and load newly created users from AD, and remove any deleted users from AD
  • “Add Entities” – called when a new user record is created in MIM, to trigger and perform creation of the required AD account
  • “Update Entities” – called when user attributes are updated in MIM, to trigger any required actions (such as enable/disabling an exchange mailbox)
  • “Delete Entities” – called when a user record is deleted in MIM, to trigger and perform deletion of the corresponding AD account

The Powershell connector can also be used for AD objects other than users.

[CUSTOMERX] Solution

A key element of the user lifecycle at [CUSTOMERX] is the “Identity State” of each user, which is derived from various user attributes (e.g. HR contract start date, manual disable flag, user last login timestamp, etc) and drives the values of various other attributes (e.g. AD Distinguished Name, account disable/enable state, exchange mailbox enable/disable, etc).  There are similar attributes (“Group State” and “Contact State”) for AD groups and AD mail contacts.

The following documents the specific actions performed by Broker connectors:

AD User connector

  • Import All: load all user objects from AD (Get-ADUser command)
  • Import Changes: identity new and deleted user objects in AD (Get-ADObject command)
  • Add Entities: create AD user account (New-ADUser command)
  • Update Entities: depending on the Identity State attribute (from MIM), enable or disable the user’s exchange mailbox (enable-mailbox and disable-mailbox commands via a remote PSSession to the Exchange server)
  • Delete Entities: delete AD user account (Remove-ADObject command)

AD Group connector

  • Import All: load all group objects from AD (Get-ADGroup command)
  • Import Changes: identity new and deleted group objects in AD (Get-ADObject command)
  • Add Entities: create AD group object (New-ADGroup command)
  • Update Entities: depending on the Group State attribute (from MIM), hide or show the group’s address in the GAL (Set-DistributionGroup command via a remote PSSession to the Exchange server)
  • Delete Entities: delete AD group object (Remove-ADObject command)

AD Contact connector

  • Import All: load all contact objects from AD (Get-ADObject command)
  • Import Changes: identity new and deleted contact objects in AD (Get-ADObject command)
  • Add Entities: create AD contact object (New-ADObject command)
  • Update Entities: depending on the Contact State attribute (from MIM), mail enable the contact object (Set-MailContact command via a remote PSSession to the Exchange server)
  • Delete Entities: delete AD contact object (Remove-ADObject command)

These three Broker connectors each have a corresponding Broker adapter, but share one “Provisioning” MA within MIM.  Please refer to the Provisioning MA configuration and the Powershell connector implementation for important aspects of the solution (e.g. required minimum attributes that must be exported from MIM for use when creating the various AD objects).

Issues Encountered and Key Learnings

The following important lessons were learned during implementation.

“Import All” cannot be treated as a periodic baseline as per our normal MIM best practices, because when run it resets (nulls) all connector attribute values other than those it sets directly.  This means that the next MIM export operation (after a MIM full import/sync to identify that the attributes have been cleared – and not a delta import/sync as this does not notify MIM of the changes) will have to restore all export attribute values (i.e. Identity/Group/Contact State, in the [CUSTOMERX] case) and this will re-trigger any “Update Entities” code for all objects (i.e. attempt to re-enable or re-disable every user’s exchange mailbox, in the [CUSTOMERX] case).

“Import Changes” functionality must be implemented and cannot be set to “None”.  Following on from the previous paragraph, because regular baselining of the connector is problematical and best avoided an incremental synchronisation method is needed to keep the connector in sync with the source AD data.  If this is omitted then objects that are created directly in AD by other systems will not be visible in the connector (and consequently won’t be created in the MIM “Provisioning” MA, leading to failed provisioning requests from MIM when the attempt is made to create the already-existing AD object).

The default timeout for MIM export profile runs to ECMA2 connectors such as Broker is 60 seconds, and the default batch size is 5,000 objects.  At [CUSTOMERX], Broker was unable to process requests from MIM fast enough, leading to timeouts which manifest in MIM as “cd-error” errors on each exported object with no other useful information logged anywhere.  Therefore it is highly recommended that the timeout be significantly increased.  It may also be effective to decrease the batch size, but this approach was not tested and is therefore unconfirmed.

Errors during export (e.g. problems during the development of the Broker Powershell connector code, or transient errors when executing commands from the Broker Powershell connector code) cause MIM to report errors against exported objects, even though Broker may actually correctly save the updated value of that attribute.  If a subsequent export attempt is made then Broker returns an “Other” error on the second export attempt, with error default “Internal error #9 (Cannot add value)”.  Debugging this situation and identifying what is causing it to occur can be difficult, since it only occurs on a second export after a previous error has been encountered, for a different reason.  Consequently it is recommended that all Broker powershell “Update Entities” code must include error-checking and be wrapped in exceptions handling and use the Failed Operations mechanism (“$components.Failures.Push($entity)”) to correctly record operation failures.  Refer to the Broker Powershell connector documentation for details of this mechanism.  It is arguably a bug in Broker that it fails to correctly handle the second export’s attribute update.

When using PSSessions in the Powershell connector to perform operations remotely, be sure to call Remove-PSSession to free the session as soon as it is no longer required.  Failure to do so may contribute to Broker misbehaviour, although this is unconfirmed.

Quick Guide to Common MIM Export Errors

“cd-error” with no other details – a timeout from the MIM side.  Increase the export profile’s timeout (“Operation Timeout (s)” on the third screen of the Run Profile wizard – NOT the Custom Data “Timeout (seconds)” on the second)

“Other” Internal error #9 (Cannot add value) – a previous export failed and Broker updated the attribute but reported the error back to MIM, which retries the export and confuses Broker which has already saved a value to this attribute.

If you are having other errors during export, remove your powershell code altogether and leave just a empty stub in place that does nothing (not even load a blank external script).  Try running your entire export (full data set) and make sure it works correctly first.  Then try some delta exports (just a few records) and make sure they work.  Once you’re sure you have that mechanism working correctly, but your code back in gradually – add functionality bit by bit, making sure you do both the entire export (full data set) and the the delta export (just a few records) each time.

Changing case/capitalisation on Identity Broker attribute names

Learnings, Ugly, UNIFYBroker

UNIFY’s Identity Broker keeps old attribute names (both connector and adapter), in the ‘CollectionKey’ SQL table.  The attribute’s name (‘Caption’ column) is a unique key in that table.

Even if you delete an attribute from every connector and adapter in Broker, it still keeps that attribute name record.  Even if you delete all data for the attribute!  It’s there for good, with whatever capitalisation you first used.

If you try to use that same attribute name with a different capitalisation (e.g. in an Rename adapter transformation) Broker does a case-sensitive search for that attribute name’s CollectionKey record and doesn’t find it (because the case doesn’t match).  So it tries to create one… but this fails because the SQL unique key for the Caption field is case insensitive!  The error is:

Violation of UNIQUE KEY constraint 'DF_CollectionKey_Caption'.
Cannot insert duplicate key in object 'dbo.CollectionKey'.
The duplicate key value is (xxxx).

Solutions to this predicament are:

  1. Don’t ever change the capitalisation of the attribute name (avoidance)
  2. If you have to change it (e.g. because you got it wrong in the first place) then go into the SQL table and change the Caption to whatever capitalisation you want 🙂

Criteria-based sets and “prior to today” evaluation in the Portal vs the MIM Service

Learnings, Lithnet, MIM Service, Portal, Ugly

So… maybe you have a set like this:

You might wonder exactly what “prior to today” actually means.  Especially when it comes to timezone handling.  And if you’re not wondering that, well you probably should be 🙂

First of all, remember that all your time DateTimes are (or at least should be) stored in UTC inside the MIM Service database.  That’s the sensible approach, and the Portal will format your DateTimes correctly based on the configured timezone if you do.

Anyway, back in the Portal and you’re clicking the “View Member” button.  It’s not hard to work out that “prior to today” seems to actually mean “prior to now” there.  Specifically, the UTC time in your attribute needs to be before the current UTC time.  I’m not specifically sure if that’s the web browser’s UTC time or the server’s UTC time, but in any sensible environment they’ll be the same anyway.

Here’s a quick look at the underlying Filter attribute to get a better understanding of that:

So it’s all about fn:current-dateTime().  Cool.

So far so sensible.  Now we know that temporal sets like this only get re-evaluated when the FIM_MaintainSetsJob runs on the SQL server.  And here’s where the pain sets in – a user with accessExtensionExpiryDate prior to the current time appeared as a member just fine in the Portal, but when I ran the SQL agent job to recalculate my set membership the user object didn’t get added to the set.

After quite a bit of trial and error it appears that the SQL Agent job’s definition of “prior to now” is “before 1AM today in the local server time”.  I can sort-of understand a decision to do the evaluation in local time – at some time someone decided that the job should just do the sensible thing, since “prior to today” inherently needs to know what “today” means, and using local server time is probably what MIM developers want, rather than “today” in the UTC sense.  But the 1am threshold?  That makes basically no sense to me.  I can only guess it’s because FIM_MaintainSetsJob is configured to run at 1am by default.  If that’s the case then hard-coding the definition of “today” to the time when a SQL job runs by default isn’t my idea of sensible programming 🙁

No wonder temporal sets cause headaches!

Addendum: if you really want to see what’s in your Set, LithnetRMA is your friend:

(Search-Resources -XPath "/Set[DisplayName='My Lovely Set']" -AttributesToGet @("ComputedMember")).ComputedMember

Skipped-Not-Precedent only cares about your rules, not your data

How To, Learnings, MIM Service, Sync Service

The customer wanted to set AD’s gidNumber to the last digit of the user’s objectSid, plus a constant – but only if there wasn’t a value present in the field already (so that they could manually set it in AD directly, in certain cases).  I figured I’d be fine with both an import and an export scoped sync rule, since the export rule would only apply when there was no import flow value for that attribute.  But, it turns out the “skipped-non-precedent” check looks at the configuration, not the (presence or absence of) actual data!

For the solution to this problem please read this blog post: Conditional Attribute Updates in Scoped Sync Rules.

Of course this would be much easier if we just used a Rules Extension EAF, but they’re deprecated 🙁