Thursday, July 21, 2016

Error downloading unsupported File Types with IIS

A customer of mine was complained that when users attempted to download unknown file types, they received a "404 - File or directory not found" error message.

The file type my user was attempting to download was a ".indd" file.


This error is caused when the filetype is not a supported "MIME" an IIS Web Server.

In the event you want to add a new unknown file type under the IIS Website, click MIME Types.


Add the unknown file extension users are attempting to download.


After adding the mime, do an "iisreset".

Users will now be able to download the unknown file type.

Wednesday, July 20, 2016

DFSR Event ID 6404 - The Replicated Folder has an Invalid Local Path

I was attempting to add a new DFSR node to an existing DFSR Replication Group consisting of 17 file servers.  When adding the server the following error was experienced.

DFS Replication cannot replicate the replicated folder REPLICATEDFOLDER because the local path E:\replicationgroup is not the fully qualified path name of an existing, accessible local folder. This replicated folder is not replicating to or from this server. Event ID: 6404


This particular error is generally experienced when people attempt a non NTFS volume such as ReFS to a DFSR replication group as documented on the following forum.

https://social.technet.microsoft.com/Forums/windowsserver/en-US/8860b354-3bf2-43ed-9acf-07fa43931046/dfsr-replication-cant-be-used-on-a-refs-volume?forum=winserver8gen

 In my case, I experienced this error with DFSR setup on NTFS volume on a new branch server.

The customer moved the DFSR node from one office, to another office.  The C:\ "SYSTEM" volume was formatted and reloaded with a fresh copy of 2012 R2 with latest patches, however the E:\ "DATA" volume was not formatted.  As a result it has its legacy DFSR Database, Staging data etc.

To clean up the staging data I created a new directory in C:\ called "empty":

C:\Empty

I then ran the following commands after stopping the DFSR Replication service on the spoke server:

Robocopy "C:\Empty" "E:\System Volume Information\DFSR" /MIR

Followed by

rmdir "E:\System Volume Information\DFSR"

Starting the DFSR replication service resulted in the error above.

After playing around a bit more, I found that in addition to flushing all data in System Volume Information\DFSR you must also remove the DfsrPrivate link which points to the System Volume Information\DFSR sub directory for DfsrPrivate data.


After doing this, starting the DFS Replication service kicks of its normal DFSR Initial Sync shown by a state of 2:

Wmic /namespace:\\root\microsoftdfs path dfsrreplicatedfolderinfo get replicationgroupname,replicatedfoldername,state


State Values (Uninitialized, Initialized, Initial Sync, Auto Recovery, Normal, In Error), ValueMap (0, 1, 2, 3, 4, 5)}

Need help with DFSR In Perth?  Click for IT Support.

Tuesday, July 19, 2016

DFSR Backlog Count Not Reducing - Server 2012 R2

Microsoft recently released an update which causes DFSRS.exe CPU usage to hit 100%.

https://support.microsoft.com/en-us/kb/3156418

I wrote about this problem on the following blog post:

http://clintboessen.blogspot.com.au/2016/07/dfsr-service-100-cpu-usage.html

Since installing this update and removing this update, one of our spoke servers failed to replicate back to the primary hub server.  All data from the spoke server replicated correctly to the hub, however data from the hub was not propagating down to the spoke.

The backlog count continued to increase.
 


DFSR Hotfix https://support.microsoft.com/en-us/kb/2996883 was not installed on the affected hub server.

We knew there was no issue with our hub server as this was replicating successfully to 16 other DFSR spoke servers.

On the affected spoke server, I worked with the Directory Services team from Microsoft who made the following changes to the Network Interface.


C:\Windows\system32>netsh int tcp set global chimney=disabled
Ok.


C:\Windows\system32>netsh int tcp set global rss=disabled
Ok.


C:\Windows\system32>netsh int ip set global taskoffload=disabled
Ok.


C:\Windows\system32>netsh int tcp set global autotuninglevel=disabled
Ok.


C:\Windows\system32>netsh int tcp set global ecncapability=disabled
Ok.


C:\Windows\system32>netsh int tcp set global timestamps=disabled
Ok.


C:\Windows\system32>netsh advf set allp state off
Ok.


Microsoft then disabled the following Offload features on the HP Network Interface card on the spoke server:
  • IPv4 Large Send Offload
  • Large Send Offload V2 (IPv4)
  • Large Send Offload V2 (IPv6)
  • TCP Connection Offload (IPv4)
  • TCP Connection Offload (IPv6)
  • TCP/UDP Checksum Offload (IPv4)
  • TCP/UDP Checksum Offload (IPv6)
 
After disabling these components of the network interface card and restarting the DFS Replication service, the backlog count from the hub server to the spoke server slowly started decreasing.
 
TCP offload engine is a function used in network interface cards (NIC) to offload processing of the entire TCP/IP stack to the network controller. By moving some or all of the processing to dedicated hardware, a TCP offload engine frees the system's main CPU for other tasks

Sunday, July 10, 2016

DFSR Service 100% CPU Usage

I had an enterprise DFSR customer of mine with 17 File Server nodes complaining that CPU utilisation was at 100% on numerous DFSR nodes in the cluster.


In addition to the DFSR.exe process maxing out CPU on the file server, the following error was being generated on a regular basis:

Log Name:      Application
Source:        ESENT
Date:          7/07/2016 2:33:18 PM
Event ID:      623
Task Category: Transaction Manager
Level:         Error
Keywords:      Classic
User:          N/A
Computer:      SERVER.DOMAIN.LOCAL
Description:
DFSRs (1696)
\\.\E:\System Volume Information\DFSR\database_5AAC_EEEA_ACEE_C01D\dfsr.db: The version store for this instance (0) has reached its maximum size of 127Mb. It is likely that a long-running transaction is preventing cleanup of the version store and causing it to build up in size. Updates will be rejected until the long-running transaction has been completely committed or rolled back.
Possible long-running transaction:
 SessionId: 0x00000032FFE94EC0
 Session-context: 0x00000000
 Session-context ThreadId: 0x00000000000012BC
 Cleanup: 0
 Session-trace:
32997@2:32:18 PM


 
After speaking with the Directory Services team at Microsoft, this was due to a bug with KB3156418.  This was documented on the knowledge base website:
 
 
Symptoms
If you installed update rollup 3156418 on Windows Server 2012 R2, the DFSRS.exe process may consume a high percentage CPU processing power (up to 100 percent). This may cause the DFSR service to become unresponsive to the point at which the service cannot be stopped. You must hard-boot affected computers to restart them.

Workaround

To work around this problem, uninstall update rollup 3156418.

Status

Microsoft is aware of this problem and is working on a solution.

Make sure you do not install KB3156418 on any DFSR nodes until this issue has been fixed by Microsoft.
 

Wednesday, July 6, 2016

Bug with Windows 7 and Access Based Enumeration

I encountered an interesting bug with Windows 7 workstations with Access Based Enumeration enabled on a SMB Share and DFS Namespace running on Windows Server 2012 R2.

When a user tries to create a file or folder under a location which they have "Full Modify Rights" in Windows Explorer they receive the following error:

Drive Mapping refers to a location that is unavailable. It could be a hard drive on this computer, or on a network. Check to make sure that the disk is properly inserted, or that you are connected to the Internet on your network, and then try again. If it still cannot be located, the information might have been moved to a different location.


This issue occurs under the following circumstances:
  • Access Based Enumeration is enabled on a Network Share or DFS Namespace
  • If a Mapped Network Drive is created to the Share
  • The user is connecting from a Windows 7 workstation.

The Windows 7 client works under the following circumstances:
  • If the user creates a file from an application such as Microsoft Word (not Windows Explorer) using a mapped network drive to the folder share, it works corrctly.
  • If the user opens the UNC Path of the share \\server\share\folder, not via the mapped network drive it works correctly.
Note: If the user connects from 2008 R2, Windows 8.1 or Windows 10 it connects without issues.

When setting up Access Based Enumeration, the root folder should have:
  • List Folder / Read Data
  • Applies to "This Folder Only"
This ensures that users have the rights to list all folders at the base level folder for Access Based Enumeration, but requires additional rights to all sub folders hence the folders "hidden" as expected.

The root level permissions are shown below.  All sub folders are provided with full modify permissions for the respective security groups.

This works with 2008 R2, Windows 8.1 and Windows 10.

 
With Windows 7 clients they need additional permissions granted at the root level folder including:
  • Transverse folder / execute file
  • List folder / read data
  • Read Attributes
  • Read extended attributes
  • Read permissions
 
These extra permissions at the root level folder are only required if:
  • You run Access Based Enumeration
  • You have Windows 7 Clients on the network