Criteria-based sets and “prior to today” evaluation in the Portal vs the MIM Service

Learnings, Lithnet, MIM Service, Portal, Ugly

So… maybe you have a set like this:

You might wonder exactly what “prior to today” actually means.  Especially when it comes to timezone handling.  And if you’re not wondering that, well you probably should be 🙂

First of all, remember that all your time DateTimes are (or at least should be) stored in UTC inside the MIM Service database.  That’s the sensible approach, and the Portal will format your DateTimes correctly based on the configured timezone if you do.

Anyway, back in the Portal and you’re clicking the “View Member” button.  It’s not hard to work out that “prior to today” seems to actually mean “prior to now” there.  Specifically, the UTC time in your attribute needs to be before the current UTC time.  I’m not specifically sure if that’s the web browser’s UTC time or the server’s UTC time, but in any sensible environment they’ll be the same anyway.

Here’s a quick look at the underlying Filter attribute to get a better understanding of that:

So it’s all about fn:current-dateTime().  Cool.

So far so sensible.  Now we know that temporal sets like this only get re-evaluated when the FIM_MaintainSetsJob runs on the SQL server.  And here’s where the pain sets in – a user with accessExtensionExpiryDate prior to the current time appeared as a member just fine in the Portal, but when I ran the SQL agent job to recalculate my set membership the user object didn’t get added to the set.

After quite a bit of trial and error it appears that the SQL Agent job’s definition of “prior to now” is “before 1AM today in the local server time”.  I can sort-of understand a decision to do the evaluation in local time – at some time someone decided that the job should just do the sensible thing, since “prior to today” inherently needs to know what “today” means, and using local server time is probably what MIM developers want, rather than “today” in the UTC sense.  But the 1am threshold?  That makes basically no sense to me.  I can only guess it’s because FIM_MaintainSetsJob is configured to run at 1am by default.  If that’s the case then hard-coding the definition of “today” to the time when a SQL job runs by default isn’t my idea of sensible programming 🙁

No wonder temporal sets cause headaches!

Addendum: if you really want to see what’s in your Set, LithnetRMA is your friend:

(Search-Resources -XPath "/Set[DisplayName='My Lovely Set']" -AttributesToGet @("ComputedMember")).ComputedMember

Hints when performing bulk updates using LithnetRMA

Good, How To, Lithnet, MIM Service

When you have a lot of records to update in the MIM Service, LithnetRMA is your friend.  And I mean your really good friend.

But sometimes things don’t go all your way.  If you have a lot of objects to update, you might see this error after a few thousand have been processed:

I suspect there’s an internal pointer (SQL maybe?) that loses its way after a large block of records have been deleted.  Sadly, even Search-ResourcesPaged has the same problem:

So you will need to put a ring on it.  Well, a loop anyway:

This sort of thing typically gives me a throughput of about 1 request per second, which isn’t stellar.  Fortunately, Powershell ISE has a File -> New Powershell Window menu option, so you can open up a few windows and set yourself up with some hacky parallelisation:

Take particularly note of the XPath here that processes Ident values starting with ‘a’, ‘b’, ‘c’ and ‘d’.  In your second ISE tab, paste in the script and change it to process ‘e’, ‘f’, ‘g’ and ‘h’, and so on.

I find that I get good performance improvement up to around 5-6 windows running in parallel.  Beyond that I don’t see a lot of request speed-up, but your mileage may vary!

Note: in this post’s screen snapshots “Ident” is a custom attribute that was being deprecated in the customer’s solution.  I wanted to clear all the values currently in the MIM Service, in order to then let me remove the binding.  Usually you’d use AccountName to divide objects up for parallel scripting, regardless of what attributes you are updating.