Updating Records in Mass via the SDK

For some time I've been updating (or importing) records in mass via the SDK (when I say mass I mean some where in the hundred's of thousands). However, whenever I have tried to do this I have run into an issue with my processes over taking all of the socket connections. Whether via webpage or windows app, either way the SDK would overrun the connections that IIS would allow. So, I would have to plan out my imports or updates through the SDK accordingly with the most records I could run before the connections get all used. Then once I had run that maximum number I would stop, run an IISRESET and then start again. Needless to say it made these kinds of jobs even longer and more tedious.

So, after quite a bit of searching and testing I have found what I believe to be the best way to mass update records in CRM via the SDK without locking up sockets or connections. I have seen people who like to update the registry by changing the following values:

HKLM\System\CurrentControlSet\Services\Tcpip\Parameters\MaxUserPort to be 65534
And
HKLM\System\CurrentControlSet\Services\Tcpip\Parameters\TCPTimedWaitDelay to be 30

And that works most of the time but I personally don't think that it's 100% reliable. Even after making these changes I have run into the same problem of locking up the sockets and connections. Here's the reference I found on this:

http://blogs.msdn.com/dgorti/archive/2005/09/18/470766.aspx

The best solution I found however is something that is to be implemented in your code. I have to give credit to AndrewN23 who posted his code in the groups. But the code is as follows:


int RETRY_ITERATIONS = 50;
int p = 0;
do
{
try
{
crmService.Update(entity);
p = RETRY_ITERATIONS;
}
catch (Exception e)
{
p++;
if (p >= RETRY_ITERATIONS)
{
throw (e);
//WriteToFile("updateContact error: " + e.Message + " " + e.InnerException);
}
System.Threading.Thread.Sleep(500);
}
} while (p < RETRY_ITERATIONS);

Basically the code goes and catches the connection error and waits for a connection to be freed up. Once a connection is freed up it continues on. I ran this against a group of more than 4,000 records as my first test and then ran it against more than 50,000 records without any issue.

One comment I will make on this however, is that if your update code triggers any other callout or plug-in code that will affect your update code and by using up connections that are already limited. I had to go through and add this block of code to ALL methods of my custom code that was triggered by my update code. When I only applied this code to my mass update code I would get the socket connection error showing up in the event log from my other custom code (callout or plug-in). Once I updated the rest of my custom code everything worked like a dream.

Hope this helps someone else out.

David Fronk
Dynamic Methods Inc.