Saturday, December 31, 2011

Troubleshooting 0xF4 CRITICAL_OBJECT_TERMINATION

The Debugging Tools for Windows are required to analyze crash dump files. If you do not have the Debugging Tools for Windows installed or dump files are not being generated on system crash, see this post for installation/configuration instructions:

http://mikemstech.blogspot.com/2011/11/windows-crash-dump-analysis.html

0x000000F4 CRITICAL_OBJECT_TERMINATION is a common blue screen error on the Windows platform (Windows XP, Windows Server 2003, Windows Vista, Windows Server 2008, Windows 7, Windows Server 2008 R2, and Windows 8). This error occurs when a critical system process or thread is terminated. The termination is detected and results in a bug check that dumps information on the state of the system when the thread or process is killed. Critical system processes include
  • smss.exe - Session Management Subsystem
  • csrss.exe - Client/Server Runtime Subsystem
  • wininit.exe - Session 0 initialization
  • logonui.exe - Windows logon process
  • lsass.exe - Local Security Authority Subsystem
  • services.exe - Service Control Manager
  • services.exe processes hosting RPC Endpoint Mapper (RPCSS), DCOM Server Process Launcher, and Plug and Play services
To illustrate the mechanics of debugging, I created two crashes, one that shows a critical thread termination and one that shows a critical process termination. The debugging process is fairly straightforward and an example is given for each below. The main difference is identified by parameter 1.

Case 1: Critical Process Termination (Parameter 1 = 3) 

A starting point for debugging a critical process termination dump is to use the !analyze -v debugger command,


kd> !analyze -v
*******************************************************************************
*                                                                             *
*                        Bugcheck Analysis                                    *
*                                                                             *
*******************************************************************************

CRITICAL_OBJECT_TERMINATION (f4)
A process or thread crucial to system operation has unexpectedly exited or been
terminated.
Several processes and threads are necessary for the operation of the
system; when they are terminated (for any reason), the system can no
longer function.
Arguments:
Arg1: 0000000000000003, Process
Arg2: fffffa80022fd060, Terminating object
Arg3: fffffa80022fd340, Process image file name
Arg4: fffff800017d2240, Explanatory message (ascii)

Debugging Details:
------------------


PROCESS_OBJECT: fffffa80022fd060

IMAGE_NAME:  _

DEBUG_FLR_IMAGE_TIMESTAMP:  0

MODULE_NAME: _

FAULTING_MODULE: 0000000000000000 

PROCESS_NAME:  procexp64.exe

BUGCHECK_STR:  0xF4_procexp64.exe

CUSTOMER_CRASH_COUNT:  1

DEFAULT_BUCKET_ID:  DRIVER_FAULT_SERVER_MINIDUMP

CURRENT_IRQL:  0

LAST_CONTROL_TRANSFER:  from fffff80001855142 to fffff800014c9f00

STACK_TEXT:  
... : nt!KeBugCheckEx
... : nt!PspCatchCriticalBreak+0x92
... : nt! ?? ::NNGAKEGL::`string'+0x17a06
... : nt!NtTerminateProcess+0xf4
... : nt!KiSystemServiceCopyEnd+0x13
... : 0x7707017a
... : nt!KiCallUserMode


STACK_COMMAND:  kb

FOLLOWUP_NAME:  MachineOwner

FAILURE_BUCKET_ID:  X64_0xF4_procexp64.exe_IMAGE__

BUCKET_ID:  X64_0xF4_procexp64.exe_IMAGE__

Followup: MachineOwner
---------
 
 
The PROCESS_NAME string should hopefully identify a process that caused the exit. Parameter 2 contains the address for the process object that terminated. This can be viewed using the !process debugger command. The Image lne indicates the name of the process (in this example, this was csrss.exe).

kd> !process fffffa80022fd060
GetPointerFromAddress: unable to read from fffff80001700000
PROCESS fffffa80022fd060
    SessionId: none  Cid: 01b0    Peb: 7fffffd5000  ParentCid: 01a8
    DirBase: 7a7ea000  ObjectTable: fffff8a0010d3a50 
                       HandleCount: 
    Image: csrss.exe
    VadRoot fffffa80023326f0 Vads 75 Clone 0 Private 300. Modified 209. Locked 0.
    DeviceMap fffff8a000008b30
    Token                             fffff8a0010da970
    ReadMemory error: Cannot get nt!KeMaximumIncrement value.
fffff78000000000: Unable to get shared data
    ElapsedTime                       00:00:00.000
    UserTime                          00:00:00.000
    KernelTime                        00:00:00.000
    QuotaPoolUsage[PagedPool]         0
    QuotaPoolUsage[NonPagedPool]      0
    Working Set Sizes (now,min,max)  (1242, 50, 345) (4968KB, 200KB, 1380KB)
    PeakWorkingSetSize                1244
    VirtualSize                       42 Mb
    PeakVirtualSize                   42 Mb
    PageFaultCount                    1596
    MemoryPriority                    BACKGROUND
    BasePriority                      13
    CommitCharge                      439

        *** Error in reading nt!_ETHREAD @ fffffa800231e060
 
 
Parameter 3 contains the process image file name, usually in ASCII format. Use the display memory command to display an ASCII string (da) or a Unicode string (du).

kd> da fffffa80022fd340
fffffa80`022fd340  "csrss.exe" 
 
Parameter 4 contains a pointer to an explanatory message written in ascii, display with the da debugger command
 
kd> da fffff800017d2240
fffff800`017d2240  "Terminating critical process 0x%"
fffff800`017d2260  "p (%s)."  

Case 2: Critical Thread Termination (Parameter 1 = 6)

A starting point for debugging a critical thread termination dump is to use the !analyze -v debugger command,


kd> !analyze -v
*******************************************************************************
*                                                                             *
*                        Bugcheck Analysis                                    *
*                                                                             *
*******************************************************************************

CRITICAL_OBJECT_TERMINATION (f4)
A process or thread crucial to system operation has unexpectedly exited or been
terminated.
Several processes and threads are necessary for the operation of the
system; when they are terminated (for any reason), the system can no
longer function.
Arguments:
Arg1: 0000000000000006, Thread
Arg2: fffffa8001d4e900, Terminating object
Arg3: fffffa8001d65e10, Process image file name
Arg4: fffff8000178c210, Explanatory message (ascii)

Debugging Details:
------------------


CUSTOMER_CRASH_COUNT:  1

DEFAULT_BUCKET_ID:  DRIVER_FAULT_SERVER_MINIDUMP

BUGCHECK_STR:  0xF4

PROCESS_NAME:  procexp64.exe

CURRENT_IRQL:  0

LAST_CONTROL_TRANSFER:  from fffff8000180f142 to fffff80001483f00

STACK_TEXT:  
... : nt!KeBugCheckEx
... : nt!PspCatchCriticalBreak+0x92
... : nt! ?? ::NNGAKEGL::`string'+0x29a68
... : nt! ?? ::NNGAKEGL::`string'+0x3f47d
... : nt!KiSystemServiceCopyEnd+0x13
... : 0x776503ea


STACK_COMMAND:  kb

FOLLOWUP_IP: 
nt!PspCatchCriticalBreak+92
fffff800`0180f142 cc              int     3

SYMBOL_STACK_INDEX:  1

SYMBOL_NAME:  nt!PspCatchCriticalBreak+92

FOLLOWUP_NAME:  MachineOwner

MODULE_NAME: nt

IMAGE_NAME:  ntkrnlmp.exe

DEBUG_FLR_IMAGE_TIMESTAMP:  4a5bc600

FAILURE_BUCKET_ID:  X64_0xF4_nt!PspCatchCriticalBreak+92

BUCKET_ID:  X64_0xF4_nt!PspCatchCriticalBreak+92

Followup: MachineOwner
---------
 
The PROCESS_NAME string should hopefully identify a process that caused the exit. Parameter 2 contains the address for the thread object that terminated. This can be viewed using the !thread debugger command. The Owning Process line indicates the name of the process (in this example, this was smss.exe).

kd> !thread fffffa8001d4e900
GetPointerFromAddress: unable to read from fffff800016ba000
THREAD fffffa8001d4e900  Cid 0130.0134  Teb: 000007fffffdd000 
    Win32Thread: 0000000000000000 WAIT: (UserRequest) UserMode Non-Alertable
    fffffa8001e0d730  NotificationEvent
Not impersonating
GetUlongFromAddress: unable to read from fffff800015f8b74
Owning Process            fffffa8001d65b30       Image:         smss.exe
Attached Process          N/A            Image:         N/A
fffff78000000000: Unable to get shared data
Wait Start TickCount      692          
Context Switch Count      479             
ReadMemory error: Cannot get nt!KeMaximumIncrement value.
UserTime                  00:00:00.000
KernelTime                00:00:00.000
Win32 Start Address 0x0000000048347d9c
Stack Init fffff88002222db0 Current fffff88002221fd0
Base fffff88002223000 Limit fffff8800221d000 Call 0
Priority 12 BasePriority 11 UnusualBoost 0 ForegroundBoost 0 IoPriority 2 
PagePriority 5
Kernel stack not resident.
 
 
Parameter 3 contains the process image file name, usually in ASCII format. Use the display memory command to display an ASCII string (da) or a Unicode string (du).

kd> da fffffa8001d65e10
fffffa80`01d65e10  "smss.exe" 
 
Parameter 4 contains a pointer to an explanatory message written in ascii, display with the da debugger command
 
kd> da fffff8000178c210
fffff800`0178c210  "Terminating critical thread 0x%p"
fffff800`0178c230  " (in %s)." 
 

Further Troubleshooting

Further troubleshooting and potential fixes involve identifying the cause of the termination, this can be anything from a virus to a corrupted image or a problem with the registry. An error in a custom credential provider (Vista and later) or Graphical Identification and Authentication (GINA) DLL (pre-Vista) could also cause this error.

Things to try if the system is unbootable/unusable due to this error,

  • Perform an offline system integrity check
  • Roll back any recent changes (applications, patches, etc) in safe mode
  • Perform a clean boot of the system
  • Run startup repair
  • Examine the registry offline and compare key parts of the registry related to the critical processes and services to a working system
  • If you have a custom provider or GINA extension (most user's don't), remotely debug the system and notify the developers of the issue. Temporarily disable the custom provider or GINA DLL through the offline registry edit process mentioned above.
In the end, it may not be possible to recover the system from this error and the system may need to have Windows reinstalled. To perform a clean install, files from the existing system may need to be saved.

See Also,

Windows Crash Dump Analysis

No comments:

Post a Comment