NotifyChangeDirectory throws INSUFFICIENT RESOURCES

We had an application called Kentico CMS which we deployed on Windows Server 2008 R2 with IIS 7.5.

Looking in the event log for the app, it's getting "File change notification errors" and tracking (causing the restarts) it with ProcMon we see:

NotifyChangeDirectory calls being made - for a while they succeed and then they start throwing INSUFFICIENT RESOURCES in the status.

The following errors were experienced in event viewer under the application logs:

EventID: 9145
EventType: W
EventTime: 2/11/2011 12:25:04 AM
Source: Application_End
EventCode: ENDAPP

EventDescription: Message: File Change Notification Error in \\files\domains$\k\kwinana.wa.gov.au\public_html\app_themes\default\images\cmsmodules\cms_groups
File Change Notification Error in \\files\domains$\k\kwinana.wa.gov.au\public_html\app_themes\default\images\cmsmodules\cms_groups
File Change Notification Error in \\files\domains$\k\kwinana.wa.gov.au\public_html\app_themes\default\images\cmsmodules\cms_medialibrary
File Change Notification Error in \\files\domains$\k\kwinana.wa.gov.au\public_html\app_themes\default\images\cmsmodules\cms_messageboards
File Change Notification Error in \\files\domains$\k\kwinana.wa.gov.au\public_html\app_themes\default\images\cmsmodules\cms_newsletter
File Change Notification Error in \\files\domains$\k\kwinana.wa.gov.au\public_html\app_themes\default\images\cmsmodules\cms_polls
File Change Notification Error in \\files\domains$\k\kwinana.wa.gov.au\public_html\app_themes\default\images\cmsmodules\cms_reporting
File Change Notification Error in \\files\domains$\k\kwinana.wa.gov.au\public_html\app_themes\default\images\cmsmodules\cms_webanalytics
File Change Notification Error in \\files\domains$\k\kwinana.wa.gov.au\public_html\app_themes\default\images\cmsmodules\cms_tools
File Change Notification Error in \\files\domains$\k\kwinana.wa.gov.au\public_html\app_themes\default\images\cmsmodules\cms_groups
File Change Notification Error in \\files\domains$\k\kwinana.wa.gov.au\public_html\app_themes\default\images\cmsmodules\cms_medialibrary
File Change Notification Error in \\files\domains$\k\kwinana.wa.gov.au\public_html\app_themes\default\images\cmsmodules\cms_messageboards
File Change Notification Error in \\files\domains$\k\kwinana.wa.gov.au\public_html\app_themes\default\images\cmsmodules\cms_newsletter
File Change Notification Error in \\files\domains$\k\kwinana.wa.gov.au\public_html\app_themes\default\images\cmsmodules\cms_polls
File Change Notification Error in \\files\domains$\k\kwinana.wa.gov.au\public_html\app_themes\default\images\cmsmodules\cms_reporting
File Change Notification Error in \\files\domains$\k\kwinana.wa.gov.au\public_html\app_themes\default\images\cmsmodules\cms_webanalytics
File Change Notification Error in \\files\domains$\k\kwinana.wa.gov.au\public_html\cmspages
CONFIG change
File Change Notification Error in \\files\domains$\k\kwinana.wa.gov.au\public_html\cmspages
File Change Notification Error in \\files\domains$\k\kwinana.wa.gov.au\public_html\CMSPages
File Change Notification Error in \\files\domains$\k\kwinana.wa.gov.au\public_html\cmspages
HostingEnvironment initiated shutdown
HostingEnvironment caused shutdown<br />
Shutdown stack: at System.Environment.GetStackTrace(Exception e, Boolean needFileInfo)
at System.Environment.get_StackTrace()
at System.Web.Hosting.HostingEnvironment.InitiateShutdownInternal()
at System.Web.Hosting.HostingEnvironment.InitiateShutdown()
at System.Web.HttpRuntime.ShutdownAppDomain(String stackTrace)
at System.Web.Configuration.HttpConfigurationSystem.OnConfigurationChanged(Object sender, InternalConfigEventArgs e)
at System.Configuration.Internal.InternalConfigRoot.OnConfigChanged(InternalConfigEventArgs e)
at System.Configuration.BaseConfigurationRecord.OnStreamChanged(String streamname)
at System.Web.Configuration.WebConfigurationHostFileChange.OnFileChanged(Object sender, FileChangeEvent e)
at System.Web.DirectoryMonitor.FireNotifications()
at System.Web.Util.WorkItem.CallCallbackWithAssert(WorkItemCallback callback)
at System.Web.Util.WorkItem.OnQueueUserWorkItemCompletion(Object state)
at System.Threading._ThreadPoolWaitCallback.WaitCallback_Context(Object state)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
at System.Threading._ThreadPoolWaitCallback.PerformWaitCallbackInternal(_ThreadPoolWaitCallback tpWaitCallBack)
at System.Threading._ThreadPoolWaitCallback.PerformWaitCallback(Object state)<br />
Call stack: at Global.LogApplicationEnd()
at Global.Application_End(Object sender, EventArgs e)
at System.RuntimeMethodHandle._InvokeMethodFast(Object target, Object[] arguments, SignatureStruct& sig, MethodAttributes methodAttributes, RuntimeTypeHandle typeOwner)
at System.RuntimeMethodHandle.InvokeMethodFast(Object target, Object[] arguments, Signature sig, MethodAttributes methodAttributes, RuntimeTypeHandle typeOwner)
at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture, Boolean skipVisibilityChecks)
at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
at System.Web.HttpApplication.ProcessSpecialRequest(HttpContext context, MethodInfo method, Int32 paramCount, Object eventSource, EventArgs eventArgs, HttpSessionState session)
at System.Web.HttpApplicationFactory.FireApplicationOnEnd()
at System.Web.HttpApplicationFactory.Dispose()
at System.Web.HttpRuntime.Dispose()
at System.Web.HttpRuntime.ReleaseResourcesAndUnloadAppDomain(Object state)
at System.Threading._ThreadPoolWaitCallback.WaitCallback_Context(Object state)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
at System.Threading._ThreadPoolWaitCallback.PerformWaitCallbackInternal(_ThreadPoolWaitCallback tpWaitCallBack)
at System.Threading._ThreadPoolWaitCallback.PerformWaitCallback(Object state)

And...

EventMachineName: GIR1
EventID: 289983
EventType: E
EventTime: 2/11/2011 4:24:23 PM
Source: Application_End
EventCode: EXCEPTION
EventDescription: Message: File Change Notification Error in \\files\domains$\o\omsintl.com\public_html
File Change Notification Error in \\files\domains$\o\omsintl.com\public_html
File Change Notification Error in \\files\domains$\o\omsintl.com\public_html
File Change Notification Error in \\files\domains$\o\omsintl.com\public_html
Directory rename change notification for '\\files\domains$\o\omsintl.com\public_html'.
public_html dir change or directory rename
Change Notification for critical directories.
bin dir change or directory rename
Change Notification for critical directories.
App_Browsers dir change or directory rename
CONFIG change
File Change Notification Error in \\files\domains$\o\omsintl.com\public_html
File Change Notification Error in \\files\domains$\o\omsintl.com\public_html
File Change Notification Error in \\files\domains$\o\omsintl.com\public_html
File Change Notification Error in \\files\domains$\o\omsintl.com\public_html\cmsdesk
CONFIG change
File Change Notification Error in \\files\domains$\o\omsintl.com\public_html\cmsdesk
File Change Notification Error in \\files\domains$\o\omsintl.com\public_html\cmspages
CONFIG change
HostingEnvironment initiated shutdown
HostingEnvironment caused shutdown
File Change Notification Error in \\files\domains$\o\omsintl.com\public_html\cmspages<br />
Shutdown stack: at System.Environment.GetStackTrace(Exception e, Boolean needFileInfo)
at System.Environment.get_StackTrace()
at System.Web.Hosting.HostingEnvironment.InitiateShutdownInternal()
at System.Web.Hosting.HostingEnvironment.InitiateShutdown()
at System.Web.HttpRuntime.ShutdownAppDomain(String stackTrace)
at System.Web.Configuration.HttpConfigurationSystem.OnConfigurationChanged(Object sender, InternalConfigEventArgs e)
at System.Configuration.Internal.InternalConfigRoot.OnConfigChanged(InternalConfigEventArgs e)
at System.Configuration.BaseConfigurationRecord.OnStreamChanged(String streamname)
at System.Web.Configuration.WebConfigurationHostFileChange.OnFileChanged(Object sender, FileChangeEvent e)
at System.Web.DirectoryMonitor.FireNotifications()
at System.Web.Util.WorkItem.CallCallbackWithAssert(WorkItemCallback callback)
at System.Web.Util.WorkItem.OnQueueUserWorkItemCompletion(Object state)
at System.Threading._ThreadPoolWaitCallback.WaitCallback_Context(Object state)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
at System.Threading._ThreadPoolWaitCallback.PerformWaitCallbackInternal(_ThreadPoolWaitCallback tpWaitCallBack)
at System.Threading._ThreadPoolWaitCallback.PerformWaitCallback(Object state)<br />
Call stack: at Global.LogApplicationEnd()
at Global.Application_End(Object sender, EventArgs e)
at System.RuntimeMethodHandle._InvokeMethodFast(Object target, Object[] arguments, SignatureStruct& sig, MethodAttributes methodAttributes, RuntimeTypeHandle typeOwner)
at System.RuntimeMethodHandle.InvokeMethodFast(Object target, Object[] arguments, Signature sig, MethodAttributes methodAttributes, RuntimeTypeHandle typeOwner)
at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture, Boolean skipVisibilityChecks)
at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
at System.Web.HttpApplication.ProcessSpecialRequest(HttpContext context, MethodInfo method, Int32 paramCount, Object eventSource, EventArgs eventArgs, HttpSessionState session)
at System.Web.HttpApplicationFactory.FireApplicationOnEnd()
at System.Web.HttpApplicationFactory.Dispose()
at System.Web.HttpRuntime.Dispose()
at System.Web.HttpRuntime.ReleaseResourcesAndUnloadAppDomain(Object state)
at System.Threading._ThreadPoolWaitCallback.WaitCallback_Context(Object state)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
at System.Threading._ThreadPoolWaitCallback.PerformWaitCallbackInternal(_ThreadPoolWaitCallback tpWaitCallBack)
at System.Threading._ThreadPoolWaitCallback.PerformWaitCallback(Object state)

The following INSUFFICIENT RESOURCES errors were experienced in process monitor:



To resolve the problem, there are a few settings that can be configured in IIS and ASP.NET To optimize this behavior.

To resolve the issue we set the DWORD HKLM\Software\Microsoft\ASP.NET\FCNMode is a Decimal value of "2" as per http://support.microsoft.com/kb/911272.

If your running a 32bit process on a 64bit server then set this key instead:

HKLM\SOFTWARE\Wow6432Node\Microsoft\ASP.NET\FCNMode