{"id":38,"date":"2018-09-05T05:43:38","date_gmt":"2018-09-05T05:43:38","guid":{"rendered":"https:\/\/www.aladan.net\/MIMicry\/?p=38"},"modified":"2018-09-05T05:44:55","modified_gmt":"2018-09-05T05:44:55","slug":"hints-when-performing-bulk-updates-using-lithnetrma","status":"publish","type":"post","link":"https:\/\/www.aladan.net\/MIMicry\/2018\/09\/05\/hints-when-performing-bulk-updates-using-lithnetrma\/","title":{"rendered":"Hints when performing bulk updates using LithnetRMA"},"content":{"rendered":"<p>When you have a lot of records to update in the MIM Service, LithnetRMA is your friend.\u00a0 And I mean your <em>really good <\/em>friend.<\/p>\n<p>But sometimes things don&#8217;t go all your way.\u00a0 If you have a <em>lot<\/em> of objects to update, you might see this error after a few thousand have been processed:<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-39\" src=\"https:\/\/www.aladan.net\/MIMicry\/wp-content\/uploads\/2018\/09\/clear-ident.jpg\" alt=\"\" width=\"1244\" height=\"111\" srcset=\"https:\/\/www.aladan.net\/MIMicry\/wp-content\/uploads\/2018\/09\/clear-ident.jpg 1244w, https:\/\/www.aladan.net\/MIMicry\/wp-content\/uploads\/2018\/09\/clear-ident-300x27.jpg 300w, https:\/\/www.aladan.net\/MIMicry\/wp-content\/uploads\/2018\/09\/clear-ident-768x69.jpg 768w, https:\/\/www.aladan.net\/MIMicry\/wp-content\/uploads\/2018\/09\/clear-ident-1024x91.jpg 1024w\" sizes=\"auto, (max-width: 1244px) 100vw, 1244px\" \/><\/p>\n<p>I suspect there&#8217;s an internal pointer (SQL maybe?) that loses its way after a large block of records have been deleted.\u00a0 Sadly, even Search-ResourcesPaged has the same problem:<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-40\" src=\"https:\/\/www.aladan.net\/MIMicry\/wp-content\/uploads\/2018\/09\/paged.jpg\" alt=\"\" width=\"1043\" height=\"827\" srcset=\"https:\/\/www.aladan.net\/MIMicry\/wp-content\/uploads\/2018\/09\/paged.jpg 1043w, https:\/\/www.aladan.net\/MIMicry\/wp-content\/uploads\/2018\/09\/paged-300x238.jpg 300w, https:\/\/www.aladan.net\/MIMicry\/wp-content\/uploads\/2018\/09\/paged-768x609.jpg 768w, https:\/\/www.aladan.net\/MIMicry\/wp-content\/uploads\/2018\/09\/paged-1024x812.jpg 1024w\" sizes=\"auto, (max-width: 1043px) 100vw, 1043px\" \/><\/p>\n<p>So you will need to put a ring on it.\u00a0 Well, a loop anyway:<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-41\" src=\"https:\/\/www.aladan.net\/MIMicry\/wp-content\/uploads\/2018\/09\/loop-it.jpg\" alt=\"\" width=\"890\" height=\"206\" srcset=\"https:\/\/www.aladan.net\/MIMicry\/wp-content\/uploads\/2018\/09\/loop-it.jpg 890w, https:\/\/www.aladan.net\/MIMicry\/wp-content\/uploads\/2018\/09\/loop-it-300x69.jpg 300w, https:\/\/www.aladan.net\/MIMicry\/wp-content\/uploads\/2018\/09\/loop-it-768x178.jpg 768w\" sizes=\"auto, (max-width: 890px) 100vw, 890px\" \/><\/p>\n<p>This sort of thing typically gives me a throughput of about 1 request per second, which isn&#8217;t stellar.\u00a0 Fortunately, Powershell ISE has a File -&gt; New Powershell Window menu option, so you can open up a few windows and set yourself up with some hacky parallelisation:<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-42\" src=\"https:\/\/www.aladan.net\/MIMicry\/wp-content\/uploads\/2018\/09\/parallel.jpg\" alt=\"\" width=\"1553\" height=\"206\" srcset=\"https:\/\/www.aladan.net\/MIMicry\/wp-content\/uploads\/2018\/09\/parallel.jpg 1553w, https:\/\/www.aladan.net\/MIMicry\/wp-content\/uploads\/2018\/09\/parallel-300x40.jpg 300w, https:\/\/www.aladan.net\/MIMicry\/wp-content\/uploads\/2018\/09\/parallel-768x102.jpg 768w, https:\/\/www.aladan.net\/MIMicry\/wp-content\/uploads\/2018\/09\/parallel-1024x136.jpg 1024w\" sizes=\"auto, (max-width: 1553px) 100vw, 1553px\" \/><\/p>\n<p>Take particularly note of the XPath here that processes Ident values starting with &#8216;a&#8217;, &#8216;b&#8217;, &#8216;c&#8217; and &#8216;d&#8217;.\u00a0 In your second ISE tab, paste in the script and change it to process &#8216;e&#8217;, &#8216;f&#8217;, &#8216;g&#8217; and &#8216;h&#8217;, and so on.<\/p>\n<p>I find that I get good performance improvement up to around 5-6 windows running in parallel.\u00a0 Beyond that I don&#8217;t see a lot of request speed-up, but your mileage may vary!<\/p>\n<p>Note: in this post&#8217;s screen snapshots &#8220;Ident&#8221; is a custom attribute that was being deprecated in the customer&#8217;s solution.\u00a0 I wanted to clear all the values currently in the MIM Service, in order to then let me remove the binding.\u00a0 Usually you&#8217;d use AccountName to divide objects up for parallel scripting, regardless of what attributes you are updating.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>When you have a lot of records to update in the MIM Service, LithnetRMA is your friend.\u00a0 And I mean your really good friend. But sometimes things don&#8217;t go all your way.\u00a0 If you have a lot of objects to update, you might see this error after a few thousand have been processed: I suspect<span class=\"post-excerpt-end\">&hellip;<\/span><\/p>\n<p class=\"more-link\"><a href=\"https:\/\/www.aladan.net\/MIMicry\/2018\/09\/05\/hints-when-performing-bulk-updates-using-lithnetrma\/\" class=\"themebutton\">Read More<\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[3,6,12,8],"tags":[],"class_list":["post-38","post","type-post","status-publish","format-standard","hentry","category-good","category-howto","category-lithnet","category-mimservice"],"_links":{"self":[{"href":"https:\/\/www.aladan.net\/MIMicry\/wp-json\/wp\/v2\/posts\/38","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.aladan.net\/MIMicry\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.aladan.net\/MIMicry\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.aladan.net\/MIMicry\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.aladan.net\/MIMicry\/wp-json\/wp\/v2\/comments?post=38"}],"version-history":[{"count":3,"href":"https:\/\/www.aladan.net\/MIMicry\/wp-json\/wp\/v2\/posts\/38\/revisions"}],"predecessor-version":[{"id":45,"href":"https:\/\/www.aladan.net\/MIMicry\/wp-json\/wp\/v2\/posts\/38\/revisions\/45"}],"wp:attachment":[{"href":"https:\/\/www.aladan.net\/MIMicry\/wp-json\/wp\/v2\/media?parent=38"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.aladan.net\/MIMicry\/wp-json\/wp\/v2\/categories?post=38"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.aladan.net\/MIMicry\/wp-json\/wp\/v2\/tags?post=38"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}