An associate asked, ” How could there be performance problems for something like Lithnet?”

Here’s my response:

MIM has a unsupported .NET library which exposes low level calls to retrieve separate bits of data from the MIM internal data structures.  Lithnet uses these calls, and compiles together complete results for a call like “Get-PendingImports”.  Lithnet has to make many calls to get the information it needs: probably one for the list of connector space object IDs, then for each connector space object one to get metadata (DN, object type etc), one to get the attribute hologram, one to get the pending import metadata, one to get the pending import attribute delta, one to get the pending import hologram, one to get the unapplied (pending) export metadata, one to get the unapplied (pending) export deltas, one to get the unapplied (pending) export hologram, one to get the escrowed (unconfirmed) export metadata, one to get the escrowed (unconfirmed) export delta, one to get the escrowed (unconfirmed) export hologram, etc etc.  Unfortunately calls like “Get-PendingImports” return objects with all of that data (yes, even the pending export data) and that takes time to retrieve, so if there are a lot of pending operations then it can be quite time-consuming.

CSExport.exe isn’t much better, but it is a little bit more controllable than the Lithnet library, so it can be used to improve the performance slightly.  The best approach would be to use the underlying .NET library directly and only make the absolute minimum number of calls to get just the data I actually need.  That’s a road I haven’t gone down… yet.

Audit drop files don’t require any .NET calls to get the update information, but there are three big problems with them.  Firstly, they’re expensive to generate and write out to disk.  Secondly, for a Full Import they include all records and you therefore can’t tell which ones are adds, updates or deletes unless you compare the entire data set to the existing connector space, which is incredibly slow.  Thirdly, even for deltas some MAs just send a drop file with the whole object and “replace” operations (i.e. update if exists or add if does not exist) and so again you therefore don’t know if it’s an update or an add operation.

Here endeth today’s lesson; Thanks be to Microsoft.  AMIM.