Mid-raid, mid-ranked, doesn’t matter: your screen goes black. Or the whole desktop hard-locks and you have to hold the power button while your squad asks where you went. Or it just reboots, no warning, no crash dialog, nothing. A few seconds later you’re staring at a login prompt, and the one thing you actually want to know, “was that heat,” is nowhere to be found.
Here’s what actually survives that moment on Linux, what it can tell you, and the one thing none of it can. No dev background needed, every command below is copy-paste.
What survives: the kernel log, if you know where to look
dmesg right after a hard lock is often empty or truncated. That’s not a bug, it’s
physics: dmesg’s in-memory ring buffer doesn’t survive a hard reset, and if the crash
happened during the write itself, the message never made it in. The place to actually
look is the persistent systemd journal, and specifically the previous boot’s log:
journalctl -k --since "10 minutes ago"
If the crash caused a reboot, you want the log from before that reboot happened, which
means checking timestamps carefully, or narrowing the window around when you remember
things going wrong. This only works at all if journald is configured for persistent
storage (Storage=persistent in /etc/systemd/journald.conf, the default on most
desktop distros but worth checking if you’ve never seen it survive a reboot before).
No persistent journal, no forensics, full stop.
The four markers that actually mean something
Assuming the log survived, you’re looking for one of a handful of specific patterns. These are the exact markers Thermal Canary Pro’s own crash detector watches for in the kernel log, because they’re the ones that reliably distinguish “the GPU driver actually died” from “something else killed the process”:
| Marker | What it means |
|---|---|
NVRM: Xid |
NVIDIA’s proprietary driver logging a hardware/driver-level fault. The Xid number after it identifies the fault class, but NVIDIA has never fully documented the numbering, so treat the presence of an Xid as the signal, the number as a clue. |
nvidia-drm ... fallen off the bus |
The GPU stopped responding to the PCIe bus entirely. Usually a power delivery or seating issue, not a software crash. |
amdgpu ... ring timeout / GPU reset |
The amdgpu driver’s own hang detection tripped and it reset the GPU. This is the AMD equivalent of an Xid, and unlike NVIDIA, AMD’s driver source is open, so the reset paths are inspectable if you want to go further. |
i915 ... GPU HANG |
Intel’s integrated graphics driver recovering from a hung GPU. |
If you see any of these within a minute or two of when things went wrong, you have real
evidence the GPU driver itself faulted, not just that your desktop session died for some
unrelated reason (an OOM kill, a compositor crash, systemd-oomd, someone hitting the
power button).
What none of it tells you
Read that table again. Every one of those lines tells you the driver died. Not one of
them tells you the GPU’s temperature, clock speed, power draw, or fan speed at the
moment it happened. Kernel drivers log faults. They do not log telemetry. There is no
NVRM: Xid, GPU was at 91C message, because that isn’t what an Xid is for.
This is the actual gap, and it’s not a Thermal Canary Pro talking point, it’s just how kernel-level crash reporting works on any OS: the fault handler’s job is to log that something broke, not to snapshot the sensors on the way down. If your working theory is “the card cooked itself,” the kernel log alone will never confirm or deny it.
What people do instead, and why it falls short
The obvious DIY fix is a background sensor logger: watch -n1 sensors >> gpu.log or
similar, left running in a terminal. It mostly works, until the crash happens, because
the buffered write that would have recorded the last few readings often never flushes
to disk before the machine goes down.
You’re left with a log that ends a few seconds before the moment you actually care about, with no window guarantee and nothing to read afterward except by hand.
How Thermal Canary Pro closes the gap
This is the exact problem Thermal Canary Pro’s Crash Black Box exists to solve: it keeps a 60-second rolling buffer of real sensor samples (CPU/GPU temp, CPU usage, memory, VRAM) in memory, and flushes it to disk atomically (write to a temp file, then rename over the real one, so a crash mid-write can’t corrupt it) every 5 seconds. Worst case, you lose under 5 seconds of history, not the whole session.
An unclean exit alone isn’t enough to call it a crash, though. kill -9, a desktop
environment’s “Force Quit,” and routine suspend/resume paths all end the process
uncleanly too, and none of those are what a “crash” report should mean to a user.
So the dump only gets promoted to a real crash record if there’s actual corroborating evidence: either the rolling buffer shows the GPU at 86C or above (or CPU at 90C or above), or the kernel log shows one of the four markers above around the same time.
Both thresholds sit about 2C under Thermal Canary’s own CRITICAL tier (GPU 88, CPU 92), deliberately not at the WARM tier, since 80C, and hotspot readings routinely 10 to 15C higher, is normal at full gaming load on plenty of consumer cards. Gating on WARM produced false crash reports on ordinary hot sessions; gating a bit under CRITICAL doesn’t.
Put together: the next time the app starts and finds a stale tombstone with real evidence behind it, you get an actual answer. Not “the driver reset,” which the kernel log already told you, but “the driver reset, and here’s what the GPU and CPU were doing in the 60 seconds leading up to it.”
No cloud, no upload
Crash dumps are plain JSON files on your own disk
(~/.local/share/thermalcanary/crashes/). Nothing is uploaded anywhere. There’s no
telemetry, and the only network call this app ever makes is a one-time license
activation. If you’re the kind of person who runs Pi-hole and checks what your software
phones home to, there’s nothing to catch here, because there’s nothing to send.
