MSCRM Server Performance Counter Errors

In the last week or two I've gotten bombarded with Performance Counter information. Some was voluntary, some was forced due to errors on client's servers. I am by no means an expert but I do have to say that I have gleaned a fair amount of information that I bet the typical MSCRM admin or consultant doesn't know about. There just isn't much information out there specifically about Performance Counters within MSCRM.

My post last week talked about running the lodctr /R command. I found a post that stated that running just lodctr /R will rebuild all system Performance Counters and thinking this would clean things up I ran it on a server...not smart. At least not smart in the MSCRM world. Everything worked just fine, including CRM, until I ran an IISRESET later in the day and then CRM just died. I found articles on how to manually rebuild Performance Counters (http://support.microsoft.com/kb/300956) and that works just fine for Windows Server Performance Counters just fine. But not MSCRM. There is even a KB article in PartnerSource about Performance Counters but it says that running a repair would work just fine and resolve the problem. Well, I'd tried that before I'd even found the article and about 70% of the way through the repair it would error out with (the all too familiar error to me by now):

The requested Performance Counter is not a custom counter, it has to be initialized as ReadOnly.

Only this was on the server and what I did to fix the client was what caused the problem. So, running it again (which I tried) was zero help.

After many, many hours of deliberation I finally spoke with Adam in Australia from MS Support and he helped clear up everything and demistify Performance Counters for me. There are 4 important registry keys that need to be set up correctly in order for Performance Counters to work properly in MSCRM. Oh, and for whatever reason, if the Performance Counters fail then the whole application fails (as if I didn't realize that by now). But most other applications just allow them to fail and move on. They aren't system critical, they just track the performance of the application. Adam has submitted the request of Performance Counters not being a system requirement in future versions, hopefully they listen to his request and/or read this post.

Anyway, back to the fix. In the registry there are 3 keys that need to exist, all are under the following paths:

My Computer\HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\Services (add the last two hives of each hive listed out below to get to the actual keys)

1. CRM Async Service
2. CRM Authentication\Performance
3. CRM Discovery\Performance
4. CRM LocatorService\Performance
5. CRM OutlookSync\Performance
6. CRM Platform\Performance
7. CRM Server\Performance
8. CRM Sql Governor\Performance
9. CRM Sql Governor Multi Instance

(Basically and hive under the Services hive that has CRM in it is what you need to check.)
Oh and one more for workflow performance

10. Windows Workflow Foundation 3.0.0.0

The keys that must be there and their values are:

Key--------------------------Value
Library----------------------netfxperf.dll
First Counter----------------(any value is fine, it just needs to exist)
Last Coutner----------------(any value is fine, it just needs to exist)

Running a repair on CRM will fix all of the Performance Counters for the CRM hives but it won't fix the Workflow hive. So, if you get the error regarding Performance Counters in the middle of your repair I can almost guarantee that your problem is that you are missing keys in your Workflow Performance hive. That's what I had to do and according to Adam (who has written the majority of the Performance Counter KB articles for the MSCRM Support team) only the Workflow Performance Counters would cause the repair to fail since the repair rebuilds them.

Now that we know what needs fixed, how do we fix it? If its CRM Performance Counters, just run a repair on the server. If you error during the repair with a Performance Counter error then you will need to reload the Library, First and Last Counter keys for Workflow Foundation. In order to do this you must use the lodctr command. DO NOT recreate these manually. I haven't tried it so I can't speak to what bad things could occur if you did I just know that you should rebuild them using the lodctr command.

You need to open a command prompt and go to the following directory:

%SystemRoot%:\WINDOWS\Microsoft.NET\Framework\v3.0\Windows Workflow Foundation

Then run the following command:

lodctr perfcounter.ini

That will rebuild the Workflow Foundation Performance Counters (which MSCRM is dependant on). If you look at the registry you should now see the keys there with correct values. Continue your repair, or try it again and you should be all set. I would recommend a reboot after the repair in this case to get everything reset and you should be back up and running.

For those of you who don't know what Performance Counters are used for, they can be very useful in showing the performance of the server and where the majority of the load is so that you can appropriately manage the server/network and correct the bottleneck of CRM. To view them open a run prompt and type "perfmon". This will open the Performance Monitor. By default it shows Pages/sec (if IIS is installed), Avg. Disk Queue Length, and % Processor Time. You can add other Counters to this list by right clicking in the list of counters at the bottom of the main section of Performance Monitor. Choose "Add Counters..." In the new window that comes up expand the "Performance object" dropdown and pick the application/object you want to monitor. Below that will be all of the counters you can monitor. Take a look at those and you'd be amazed at what you can track for CRM. There are also a lot of really good SQL ones for your SQL server. If you are looking for metrics to show companies the performance increase by going to MSCRM 4.0, or moving to a 64 bit SQL box, this will give you a great comparison and actually show the big wigs that their decision made a big difference.

I have learned a lot about Performance Counters and hopefully my lengthy post has helped someone else glean a little more information as well.

David Fronk
Dynamic Methods Inc.

Outlook client Offline Performance Counter error when attempting to go Offline

When going offline users see the following message:

The requested Performance Counter is not a custom counter, it has to be initialized as ReadOnly.

Event Log Errors:

Event Type: Error
Event Source: LoadPerf
Event Category: None
Event ID: 3013
Date: 7/29/2008
Time: 11:38:09 AM
User: N/A
Computer: LFISKNBXP
Description:
Unable to update the performance counter strings of the 009 language ID. The Win32 status returned by the call is the first DWORD in Data section.

For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.
Data:
0000: 43 02 00 c0 17 0d 00 00 C..À....

Event Type: Error
Event Source: LoadPerf
Event Category: None
Event ID: 3009
Date: 7/29/2008
Time: 11:38:09 AM
User: N/A
Computer: LFISKNBXP
Description:
Installing the performance counter strings for service CRM Client (%2) failed. The Error code is the first DWORD in Data section.

For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.
Data:
0000: 43 02 00 c0 f6 13 00 00 C..Àö...

Event Type: Error
Event Source: MSCRMPerfCounters
Event Category: None
Event ID: 17188
Date: 7/29/2008
Time: 11:38:09 AM
User: N/A
Computer: LFISKNBXP
Description:
The description for Event ID ( 17188 ) in Source ( MSCRMPerfCounters ) cannot be found. The local computer may not have the necessary registry information or message DLL files to display messages from a remote computer. You may be able to use the /AUXSOURCE= flag to retrieve this description; see Help and Support for details. The following information is part of the event: Client.

Event Type: Error
Event Source: MSCRMOfflineSync
Event Category: None
Event ID: 6000
Date: 7/30/2008
Time: 10:30:28 AM
User: N/A
Computer: LFISKNBXP
Description:
An error occurred during Offline Synchronization. Try going offline again, or restart Microsoft Outlook. Exception happened The requested Performance Counter is not a custom counter, it has to be initialized as ReadOnly..
Stack Trace Info:
at System.Diagnostics.PerformanceCounter.Initialize()
at System.Diagnostics.PerformanceCounter..ctor(String categoryName, String counterName, String instanceName, Boolean readOnly)
at System.Diagnostics.PerformanceCounter..ctor(String categoryName, String counterName, Boolean readOnly)
at Microsoft.Crm.CrmPerformanceCounterFactory.LoadCounters(CounterContainer counterContainer, PerformanceCounterLoadSetting settings, String component)
at Microsoft.Crm.CrmPerformanceCounterFactory.LoadCounters(PerformanceCounterLoadSetting settings, String component)
at Microsoft.Crm.Performance.OutlookPerformanceCounterTracker.Initialize()
at Microsoft.Crm.Performance.OutlookPerformanceCounterTracker.Increment(OutlookPerformanceCounterType type)
at Microsoft.Crm.Application.Outlook.OfflineSync.SyncData.SyncInfoArray(String syncInfo)
at Microsoft.Crm.Application.Outlook.OfflineSync.SyncData.MoveData(String syncInfo, ITransferDataProvider transferDataProvider, Int32 progressSteps)
at Microsoft.Crm.Application.Outlook.OfflineSync.SyncData.Sync(ITransferDataProvider transferDataProvider, Int32 callPriority)
at Microsoft.Crm.Application.SMWrappers.OfflineSync.SyncThreadMethod() .

For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.



Fix:

Uninstalled CRM client and then ran “lodctr /R:loadperf” on client machine from a command prompt. Reinstalled CRM client and went offline successfully.


Hope this helps someone else out as well.

David Fronk
Dynamic Methods Inc.

Hiding Fields and Labels on the fly via JScript

This is actually in reponse to a comment/question that was posted by Schnugie but I figured that this would benefit everyone so it became a post instead of a comment response.

So, how do you hide fields and then redisplay them based on the input of another field? Here's a code sample of how to do it:


var oField = crmForm.all.field;
var oLabel1 = crmForm.all.field_c;

switch(oField.SelectedText) {
case "0":
crmForm.all.field.style.visibility = 'hidden';
oLabel1.style.visibility = 'hidden';
break;
case "1":
crmForm.all.field.style.visibility = 'visible';
oLabel1.style.visibility = 'visible';
break;
default:
crmForm.all.field.style.visibility = 'hidden';
oLabel1.style.visibility = 'hidden';
break;
}

If crmForm.all.field is a picklist then the .SelectedText will work. But basically, this code looks at a picklist field, evaluates the value and depending on what the value is determines whether or not a field is displayed or hidden. You will also notice that I have some _c after my field name as a second variable, that is for the label. If you just hide the field it will not hide the label with it. So you will need to show/hide both.

Use this script on the onLoad, onChange, or where ever you see fit.

Happy scripting!

David Fronk
Dynamic Methods Inc.

Context.CallerOrigin

I have spent countless hours fighting infinite loops for numerous projects. Each loop has it's own logic and checks to avoid looping forever. I was writing some code that was updating another object which in turn would update the originating object and I really only needed either code to run once and only once. So, I set up all kinds of logic and was still getting loops. I then came across the CallerOrigin within the Context of the Plugin. The CallerOrigin tells you where the call came from:

1. Application (someone took action on a form)
2. AsyncService (workflow or an asynchronous plugin)
3. WebServiceApi (webpage or synchronous plugin or callout)

I could then check my code and say that unless the call came from the application to not perform any update. That killed my infinite loop real fast.

This also can be very useful for code that you want to fire when someone is taking an action or the system is taking action on something. I can't say that I've come across this scenario all that often but it's nice flexibility.

I will note that my check looked like this:


if (context.CallerOrigin.ToString() == "Microsoft.Crm.Sdk.ApplicationOrigin")

Maybe not as clean as some of us might like but it works. As I just found this I'm looking for more strongly typed means to run my checks but for this one issue this works just fine for me.

Happy coding!

David Fronk
Dynamic Methods Inc.

Email Router Hotfix

I received word from Microsoft that they have an update available to fix an issue that occurs with the email router. For whatever reason the email router will poll for an hour and then lock for an hour. If it unlocks itself after that second hour it will work for another hour and then lock itself again. The hour comes from the default polling time that is set up in the email router config file. So, if you were to modify that polling time then you would extend the life of your email router from locking up but also extend the amount of time to get the email router back up once it is locked.

I used a work around to get by until this hotfix became available by restarting the email router service each hour via a bat script.

However the hotfix you want to check out is KB article 952019. You will need to contact Microsoft via PartnerSource or CustomerSource in order to obtain the hotfix, but you will not be charged for the ticket if you are requesting the hotfix.

Hope this comes in handy for someone else as well.

David Fronk
Dynamic Methods Inc.

Updating Records in Mass via the SDK

For some time I've been updating (or importing) records in mass via the SDK (when I say mass I mean some where in the hundred's of thousands). However, whenever I have tried to do this I have run into an issue with my processes over taking all of the socket connections. Whether via webpage or windows app, either way the SDK would overrun the connections that IIS would allow. So, I would have to plan out my imports or updates through the SDK accordingly with the most records I could run before the connections get all used. Then once I had run that maximum number I would stop, run an IISRESET and then start again. Needless to say it made these kinds of jobs even longer and more tedious.

So, after quite a bit of searching and testing I have found what I believe to be the best way to mass update records in CRM via the SDK without locking up sockets or connections. I have seen people who like to update the registry by changing the following values:

HKLM\System\CurrentControlSet\Services\Tcpip\Parameters\MaxUserPort to be 65534
And
HKLM\System\CurrentControlSet\Services\Tcpip\Parameters\TCPTimedWaitDelay to be 30

And that works most of the time but I personally don't think that it's 100% reliable. Even after making these changes I have run into the same problem of locking up the sockets and connections. Here's the reference I found on this:

http://blogs.msdn.com/dgorti/archive/2005/09/18/470766.aspx

The best solution I found however is something that is to be implemented in your code. I have to give credit to AndrewN23 who posted his code in the groups. But the code is as follows:


int RETRY_ITERATIONS = 50;
int p = 0;
do
{
try
{
crmService.Update(entity);
p = RETRY_ITERATIONS;
}
catch (Exception e)
{
p++;
if (p >= RETRY_ITERATIONS)
{
throw (e);
//WriteToFile("updateContact error: " + e.Message + " " + e.InnerException);
}
System.Threading.Thread.Sleep(500);
}
} while (p < RETRY_ITERATIONS);

Basically the code goes and catches the connection error and waits for a connection to be freed up. Once a connection is freed up it continues on. I ran this against a group of more than 4,000 records as my first test and then ran it against more than 50,000 records without any issue.

One comment I will make on this however, is that if your update code triggers any other callout or plug-in code that will affect your update code and by using up connections that are already limited. I had to go through and add this block of code to ALL methods of my custom code that was triggered by my update code. When I only applied this code to my mass update code I would get the socket connection error showing up in the event log from my other custom code (callout or plug-in). Once I updated the rest of my custom code everything worked like a dream.

Hope this helps someone else out.

David Fronk
Dynamic Methods Inc.

Dynamically changing the name of a tab

This comes from one of my coworkers, great hack. This works not only in the onLoad but also on the onChange of any field on the form. This is a great, simple script that might help users out a bit more.

Code:
var title = crmForm.all.primarycontactid.DataValue;

if(title != null)
{
document.all.tab4Tab.innerText = title[0].name;
}


You will need to change your tab4Tab to whatever tab you have on the form (remember, tabs start at 0 for the first position so make sure you get your numbering correct).

Simple as that. Now your tab will show the text of the name of the Primary Contact. Oh, one stipulation, the size of the tab will not regrow using this code, only the text. So, if your tab is named "Test" and I were to replace it with "David Fronk" then my name would get "wordwrapped" and it could look pretty funny. So, make your initial name something longer. If I come across how to dynamically grow the tab so as to fit the name being placed in it I'll definitely post that up here as well.

Enjoy!

David Fronk
Dynamic Methods Inc.