Summary
The document describes the process of investigating intermittent bluescreens after restoring from sleep/hibernation on a Surface Pro 7+. The author provides information on error logs and analyzes a dump file using Windbg, identifying an issue with a driver blocking an IRP for too long. They then search for nipcibrd, a driver related to National Instruments (NI) software, and attempt to update NI packages and drivers. However, they find that the issue may be related to a known issue with NI-VISA driver package, leading them to remove the driver from their laptop. The document concludes with a follow-up plan and possible resolution of the issue.
DRIVER_POWER_STATE_FAILURE
Some error logging/screenshots will be in Dutch since this is my native language and set as such on my device.
Symptom
Turn on laptop, screen lights up, nothing shows up on the screen and after 30~50 seconds a BSOD appears with the following message: “DRIVER_POWER_STATE_FAILURE”
Information
Windows error log after such an event:
De computer is opnieuw opgestart na een bugcontrole. De bugcontrole is 0x0000009f (0x0000000000000003, 0xffffb6018fc9b360, 0xffffca834928f118, 0xffffb6018fc6a010). Er is een dump opgeslagen in: C:\WINDOWS\MEMORY.DMP. Rapport-id: 21fd4ad4-5661-4273-ac3f-3b5c6b9bcca3.
A quick search on the internet did not provide me with relevant information concerning my specific BSOD
Microsoft does provide some basic information regarding this specific error: https://learn.microsoft.com/en-us/windows-hardware/drivers/debugger/bug-check-0x9f–driver-power-state-failure
Parameter one indicates the type of violation: 0x3 in my case
| Parameter 1 | Parameter 2 | Parameter 3 | Parameter 4 | Cause |
| 0x3 | The physical device object (PDO) of the stack | nt!_TRIAGE_9F_POWER. | The blocked IRP | A device object has been blocking an IRP for too long a time. |
Investigation
A Device object has been blocking an IRP for too long a time, so lets investigate with windbg and review the minidump file generated by the OS during the crash.
Investigation with Windbg and review of minidump file generated by the OS during the crash
I’ve opened the dumpfile and left all settings to default, since the error occurred on my laptop and it is a standard minidump file.
I hope this dump file contains some relevant information regarding the device/driver which is causing this and the relevant stack pointer from the error message.
After loading the dmp file I run the analyze -v command;
For analysis of this file, run !analyze -v
nt!KeBugCheckEx:
fffff80781a28c50 48894c2408 mov qword ptr [rsp+8],rcx ss:0018:ffffca834928f0d0=000000000000009f
Analysis of the dump file
4: kd> !analyze -v
This returned the following:
*******************************************************************************
* *
* Bugcheck Analysis *
* *
*******************************************************************************
DRIVER_POWER_STATE_FAILURE (9f)
A driver has failed to complete a power IRP within a specific time.
Arguments:
Arg1: 0000000000000003, A device object has been blocking an IRP for too long a time
Arg2: ffffb6018fc9b360, Physical Device Object of the stack
Arg3: ffffca834928f118, nt!TRIAGE_9F_POWER on Win7 and higher, otherwise the Functional Device Object of the stack
Arg4: ffffb6018fc6a010, The blocked IRP
Debugging Details:
------------------
KEY_VALUES_STRING: 1
Key : Analysis.CPU.mSec
Value: 936
Key : Analysis.DebugAnalysisManager
Value: Create
Key : Analysis.Elapsed.mSec
Value: 8739
Key : Analysis.IO.Other.Mb
Value: 6
Key : Analysis.IO.Read.Mb
Value: 0
Key : Analysis.IO.Write.Mb
Value: 35
Key : Analysis.Init.CPU.mSec
Value: 875
Key : Analysis.Init.Elapsed.mSec
Value: 321249
Key : Analysis.Memory.CommitPeak.Mb
Value: 105
Key : Bugcheck.Code.DumpHeader
Value: 0x9f
Key : Bugcheck.Code.Register
Value: 0x9f
Key : WER.OS.Branch
Value: ni_release
Key : WER.OS.Timestamp
Value: 2022-05-06T12:50:00Z
Key : WER.OS.Version
Value: 10.0.22621.1
FILE_IN_CAB: 031223-15265-01.dmp
TAG_NOT_DEFINED_202b: *** Unknown TAG in analysis list 202b
BUGCHECK_CODE: 9f
BUGCHECK_P1: 3
BUGCHECK_P2: ffffb6018fc9b360
BUGCHECK_P3: ffffca834928f118
BUGCHECK_P4: ffffb6018fc6a010
DRVPOWERSTATE_SUBCODE: 3
IMAGE_NAME: pci.sys
MODULE_NAME: pci
FAULTING_MODULE: fffff80784d10000 pci
BLACKBOXBSD: 1 (!blackboxbsd)
BLACKBOXNTFS: 1 (!blackboxntfs)
BLACKBOXPNP: 1 (!blackboxpnp)
BLACKBOXWINLOGON: 1
CUSTOMER_CRASH_COUNT: 1
PROCESS_NAME: System
STACK_TEXT:
ffffca83`4928f0c8 fffff807`81b71cd6 : 00000000`0000009f 00000000`00000003 ffffb601`8fc9b360 ffffca83`4928f118 : nt!KeBugCheckEx
ffffca83`4928f0d0 fffff807`81b71bac : 00000000`00000004 ffffa500`d0c51180 ffffb601`ac7990e8 ffffca83`4928f249 : nt!PopIrpWatchdogBugcheck+0x122
ffffca83`4928f150 fffff807`818bb72b : ffffb601`00000003 ffffb601`00000001 ffffca83`00000000 00000000`00000002 : nt!PopIrpWatchdog+0xc
ffffca83`4928f180 fffff807`818bcea6 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`01c70ee9 : nt!KiProcessExpiredTimerList+0x1eb
ffffca83`4928f2b0 fffff807`81a2da4e : ffffa500`d0c51180 ffffa500`d0c51180 ffffb601`8e536040 ffffb601`abbd2080 : nt!KiRetireDpcList+0xed6
ffffca83`4928f540 00000000`00000000 : ffffca83`49290000 ffffca83`49289000 00000000`00000000 00000000`00000000 : nt!KiIdleLoop+0x9e
IMAGE_VERSION: 10.0.22621.1391
STACK_COMMAND: .cxr; .ecxr ; kb
FAILURE_BUCKET_ID: 0x9F_3_POWER_DOWN_nipcibrd_IMAGE_pci.sys
OS_VERSION: 10.0.22621.1
BUILDLAB_STR: ni_release
OSPLATFORM_TYPE: x64
OSNAME: Windows 10
FAILURE_ID_HASH: {57484b5f-651d-2835-5fd3-87d7cd3ed74d}
Followup: MachineOwner
---------
A few things that standout
The IRP message. “A driver has failed to complete a power IRP within a specific time.”
First hit on google regarding “IRP”;
An IRP is the basic I/O manager structure used to communicate with drivers and to allow drivers to communicate with each other. A packet …
Secondly pci.sys is involved.
First hit on google regarding pci.sys;
The file pci.sys is a Windows device driver file located in. %SYSTEMROOT%:\WINDOWS\System3\DriverStore\FileRepository\machine. inf_amd64_neutral_9e6bb86c3b39a3e9\. The pci. sys is used to support your Windows hardware device driver NT Plug and Play PCI Enumerator.
So something is communicating with/through pci.sys and isn’t responding (fast enough).
The following line also contains some interesting information:
FAILURE_BUCKET_ID: 0x9F_3_POWER_DOWN_nipcibrd_IMAGE_pci.sys
Let’s do a quick search on this and try to find out what nipcibrd is.
Search for nipcibrd and its relation to National Instruments (NI) software
Searching for nipcibrd returns results for nipcibrd.sys and a lot of other people mentioning bluescreens of death after installen software from NI (National Instruments).
I have software from “NI” installed on my laptop so this might be a pointer for me to investigate further.
I use the NI (VISA) software to control measurement devices like Spectrum Analysers and Oscilloscopes via Labview and other 3rd party software to automate EMC and EMI measurements and generate standardised reports.
Checking installed NI software versions and nipcibrd.sys on the laptop
Let’s check the software versions I have installed and if I can find nipcibrd.sys on my laptop.
The NI Package manager on my laptop shows the following screen (and that there are updates available)
Installed
Updates:
Nipcibrd.sys on my PC:
It is a quite recent version of the software but perhaps not the latest version.
So before we revert to completely removing the software I’m going to try to see if an update installs al new version of the driver and if the issue is resolved after that.
Attempting to update NI packages and drivers
Step one, install new packages, I’ve clicked the upgrade button….
Let’s see if an update from NI will also update this NI PCI Bridge driver
After the reboot we see the same details for the sys file;
So we need to go another route to see if we can update or need to remove this file.
Checking the device manager and default driver for the PCI-Default host-CPU-bridge
I’ve looked up the device in the Windows device manager;
It seems that the default driver for the PCI-Default host-CPU-bridge is replaced or altered by the Driver from National Instruments (it later turns out it is added by NI to this device)
Since we need this default PCI bridge we can’t remove it. We can try to restore the drivers or update it.
Since the package manager from NI does not take care of this (there was no update for this driver) we have two options.
- There are no updates
- this driver is not managed by the package manager from NI.
Downloading the NI-VISA driver package from the NI website
I suspect it is the latter, and we have to check the NI-VISA driver package
Checking the NI website I found the following: https://www.ni.com/nl-nl/support/downloads/drivers/download.ni-visa.html#460225
This timestamp is suspiciously close to the time stamp from the driver on my laptop
I’ve downloaded ISO from the website and it contains the exact same driver I currently have installed on my PC.
Double checking with the NI package manager showed the same version is incorporated in the NI package manager. Conclusion, there is no update for the NI-VISA driver
Finding known issues in the readme page of the NI-VISA driver package
The readme page from the download location contains a page with known issues;
https://www.ni.com/nl-nl/support/documentation/bugs/22/ni-visa-2022-q3-known-issues.html
An issue which sound similarities to my problem is mentioned on that page;
| 449848 | 145966 | Third-Party USB device communication may timeout after host PC is put into standby A Windows-based machine communicating with a third-party USB device using a driver not developed by the VISA Driver Development Wizard may timeout if the PC is put into standby mode in the middle of a NI-VISA USB transfer. After the PC is woken up, it thinks the USB device should resend the entire data packet and eventually times out because the packet is not resent. Workaround: There is currently no known workaround for this issue. |
There is no workaround according to this document.
The Fix
Sadly this means I will remove the driver from my laptop to see if this resolves the issue.
I won’t be able to use the NI software/drivers for now…
Attempting to remove the driver from the laptop and checking if the issue is resolved
So I tried to use the package manager to remove the drivers;
After a reboot, to my surprise I found that the PCI host bridge driver was not removed, it still contained the sys file from NI
Installed apps also showed NI software.
So I removed those too (I tried..)
When removing the software the “NI Package Manager” shows up again..
I seems I should have removed a checkbox “products only” from the NI Package Manager;
After removing that checkbox 5 other applications showed which I swiftly removed too. Including the VISA runtime, I hope this is the culprit containing the driver/sys file.
After a second reboot both files which where previously showing in the PCI-Standaard host-cpu-bridge have been removed from c:\windows\system32\drivers
Also the device manager entry does not show the specific files anymore in the device window.
Follow-up plan and conclusion
Let’s wait and see if this resolve the issue or another error/bug appears which steers us in a new direction.
Update: 2023-11-10
After removing the NI-VISA driver the problem was fully resolved. I’ve tried re-installing the NI-VISA drivers (latest version) and the problems resurfaced after a few days. I’ve removed the software after the issues reappeared.
#NI #NI-VISA #BSOD #CRASH #DRIVERS #WINDOWS #SURFACE #MICROSOFT