Four Weeks With DGX Spark: 20,249 Jobs and Four Recorded Panics
Four weeks after the acceptance trial began, the decision still holds: keep the DGX Spark as an additive compute node. Do not make it the only workstation or the only copy of state.
The application queue recorded 20,249 completed jobs from June 30 through the last recorded job on July 22. It recorded zero jobs in its failed state for that window.
The frozen acceptance record also contained four serving-era watchdog or panic events through July 9.
Useful. Not flawless.
Completion and reliability are different measurements
The 20,249 count comes from the application database. It tells me the queue kept moving work into its completed state.
It does not prove that every model response was correct. It does not prove that the host had no lower-level faults. It does not prove that a retry never hid a transient failure.
That distinction matters because application metrics are designed around application outcomes. A job can finish after infrastructure recovers. A host can panic while a durable queue preserves enough state to continue. High completion and lower-level incidents can exist in the same system.
The mistake would be to publish “zero failed jobs” as “zero failures.” Those are not the same claim.
The four incidents stay in the record
By the July 9 decision cutoff, I had observed four serving-era events involving the NVIDIA driver, out-of-memory pressure, or watchdog recovery. Service returned in roughly 80 to 90 seconds in the recorded cases.
The later observation did not find current crash-storage files. That does not reset the historical count. Reboots and cleanup change what the current filesystem contains. The frozen incident record remains the stronger source for what happened before the cutoff.
This is why operations evidence needs dates. “No crash file exists now” and “four incidents occurred by July 9” can both be true.
I also avoid assigning every reboot to the same cause. The unit moved into a different long-running evaluation workload after the queue window. A reboot timestamp without the corresponding logs is not a diagnosis.
The queue made the hardware acceptable
The Spark is not accepted because it never fails. It is accepted because the work is recoverable.
The operating shape is simple:
- durable job state outside the worker process;
- bounded retries;
- health checks that test service behavior;
- job design that makes retries safe; and
- no unique data stored only in the active process.
That turns a host reset from a lost-work event into a delay. The delay still matters. It becomes measurable and recoverable instead of existential.
An interactive workstation has a different contract. A panic interrupts the person, the display, the editor, and every unsaved decision. The same 80-second recovery can feel acceptable in a background queue and unacceptable at a desk.
That is why the original verdict separated roles. The Spark earned the always-on CUDA and batched-serving work. It did not earn laptop replacement.
The later pause was deliberate
At the later observation point, the regular serving workload was paused while the unit was assigned to a long-running speech evaluation.
That is planned reassignment, not downtime.
Operational reporting often confuses “service is not running” with “service failed.” The difference is intent and evidence. A scheduled workload move should appear in the operating record so a cold reviewer does not invent an outage from a stopped process.
The public operations summary publishes the dates, application outcomes, and incident cutoff without exposing application records or service topology. The operations report states the interpretation boundary beside it.
What I would measure next
The application count needs two companions.
First, direct service-level outcomes: accepted responses, rejected responses, retry count, and latency to successful completion. Second, host-level incident telemetry that survives reboots and records the reason for recovery.
That would let the system answer three separate questions:
- Did the job finish?
- Was the result accepted?
- Did the infrastructure recover along the way?
Combining those into one “reliability” percentage would lose the part operators need.
The month-one verdict
The Spark completed enough real queued work to justify the keep decision. The recorded incidents justify the additive-node boundary.
I do not need the machine to be perfect. I need the failure mode to be visible, bounded, and recoverable. The queue delivered that. The host did not earn permission to become irreplaceable.
That is the difference between trusting a box and operating one.
Get new posts by email — first
The newsletter is in the works — join the waitlist and be first to know when it launches. Everything here stays free to read.
The Solo Stack is written by Matt — building products solo with AI, on his own infrastructure. If a claim isn’t backed by experience or a measurement, it doesn’t ship.
Not sending yet: joining stores your address on the waitlist. One confirmation email at launch — nothing sends unless you confirm.