Problem Description
A few weeks a go, our Windows Server 2008 R2 file server got a bluescreen. I immediately opened the Event Viewer and saw the mentioned below System Events. Event ID 1001 reported that a dump was saved in C:\Windows\MEMORY.DMP.
Event Logs
Log Name: System Source: Microsoft-Windows-WER-SystemErrorReporting Date: 6/4/2013 11:56:38 AM Event ID: 1001 Task Category: None Level: Error Keywords: Classic User: N/A Computer: XXXXXXX Description: The computer has rebooted from a bugcheck. The bugcheck was: 0x000000f5 (0x000000000000006d, 0xfffffa801197e650, 0xfffffa801197e5f0, 0x0000000000000000). A dump was saved in: C:\Windows\MEMORY.DMP. Report Id: 060413-49124-01. Log Name: System Source: Microsoft-Windows-Kernel-Power Date: 6/4/2013 11:56:05 AM Event ID: 41 Task Category: (63) Level: Critical Keywords: (2) User: SYSTEM Computer: XXXX.domain.local Description: The system has rebooted without cleanly shutting down first. This error could be caused if the system stopped responding, crashed, or lost power unexpectedly.
Solution
Afterwards I installed the Debugging tools for Windows and analysed the dump and the result is as follows. A filter driver named FSAFilter_x64.sys from Scriptlogic caused the bluescreen.
After a short research in Google we found the software where the driver belongs to. File System Auditor from Scriptlogic. I immediately removed the File System Auditor from the server and the server is now working without any problems.
Dump Analysis Result :
BugCheck F5, {6d, fffffa8012146a20, fffffa80121469c0, 0}
*** ERROR: Module load completed but symbols could not be loaded for FSAFilter_x64.sys
Probably caused by : FSAFilter_x64.sys ( FSAFilter_x64+3962 )
Followup: MachineOwner
---------
6: kd> !analyze -v
*******************************************************************************
* *
* Bugcheck Analysis *
* *
*******************************************************************************
FLTMGR_FILE_SYSTEM (f5)
An unrecoverable failure occured inside the filter manager.
Arguments:
Arg1: 000000000000006d, The reason for the failure
Arg2: fffffa8012146a20
Arg3: fffffa80121469c0
Arg4: 0000000000000000
Debugging Details:
------------------
DEFAULT_BUCKET_ID: WIN7_DRIVER_FAULT
BUGCHECK_STR: 0xF5
PROCESS_NAME: System
CURRENT_IRQL: 0
LAST_CONTROL_TRANSFER: from fffff8800139233d to fffff800018d8c00
Stack Text’e baktığımızda BugCheck gerçekleşmeden önce CPU’daki en son işlem FSAFilter MiniFilter Driver tarafından gerçekleşiyor ve ardından hemen Crash gerçekleşiyor.
STACK_TEXT:
fffff880`02b39b58 fffff880`0139233d : 00000000`000000f5 00000000`0000006d fffffa80`12146a20 fffffa80`121469c0 : nt!KeBugCheckEx
fffff880`02b39b60 fffff880`0c1b6962 : fffffa80`1c2cb278 fffffa80`1c2cb270 fffffa80`1c10c3d0 00000000`00000000 : fltmgr! ?? ::FNODOBFM::`string'+0x1309
fffff880`02b39ba0 fffff880`0c1b7672 : fffffa80`12146a20 00000000`00000000 fffff880`013baeb0 fffff800`018e0100 : FSAFilter_x64+0x3962
fffff880`02b39bd0 fffff880`013baef3 : fffff880`013baeb0 00000000`00000000 fffffa80`1c326c90 00000000`00000000 : FSAFilter_x64+0x4672
fffff880`02b39c70 fffff800`018e2251 : fffff880`013baeb0 fffff800`01a7e280 fffffa80`0c78c660 fffff800`00000000 : fltmgr!FltpProcessGenericWorkItem+0x43
fffff880`02b39cb0 fffff800`01b76ede : 00000000`00000000 fffffa80`0c78c660 00000000`00000080 fffffa80`0c778040 : nt!ExpWorkerThread+0x111
fffff880`02b39d40 fffff800`018c9906 : fffff880`024a4180 fffffa80`0c78c660 fffff880`024af2c0 00000000`00000000 : nt!PspSystemThreadStartup+0x5a
fffff880`02b39d80 00000000`00000000 : fffff880`02b3a000 fffff880`02b34000 fffff880`02b399e0 00000000`00000000 : nt!KiStartSystemThread+0x16
STACK_COMMAND: kb
FOLLOWUP_IP:
FSAFilter_x64+3962
fffff880`0c1b6962 eb00 jmp FSAFilter_x64+0x3964 (fffff880`0c1b6964)
SYMBOL_STACK_INDEX: 2
SYMBOL_NAME: FSAFilter_x64+3962
FOLLOWUP_NAME: MachineOwner
MODULE_NAME: FSAFilter_x64
IMAGE_NAME: FSAFilter_x64.sys
DEBUG_FLR_IMAGE_TIMESTAMP: 4b6c69df
FAILURE_BUCKET_ID: X64_0xF5_FSAFilter_x64+3962 <<<<< ************ BUCKET_ID: X64_0xF5_FSAFilter_x64+3962 Followup: MachineOwner --------- 6: kd> lmvm FSAFilter_x64
start end module name
fffff880`0c1b3000 fffff880`0c1c4000 FSAFilter_x64 (no symbols)
Loaded symbol image file: FSAFilter_x64.sys
Image path: \SystemRoot\system32\DRIVERS\FSAFilter_x64.sys
Image name: FSAFilter_x64.sys
Timestamp: Fri Feb 05 20:56:31 2010 (4B6C69DF)
CheckSum: 0000BB4B
ImageSize: 00011000
Translations: 0000.04b0 0000.04e4 0409.04b0 0409.04e4
Good luck !