Jonathan, Erik, Dave:

h1daqnds1, which is the default NDS for workstations and guardian, started reporting a full /run file system at 00:55 PDT Tue 07may2024. This caused daqd and nds processes to stop serving data.

Jonathan restarted the rts-nds service at 08:13 PDT which cleared out the /run/nds/jobs directory, freeing up the disk space.

We noted that nds0 had 1.2GB of nds jobs files, 5% of the 26GB. Jonathan pruned this down to 1% usage.

I have added a check to my hourly cds_report to warn if the nds /run file systems exceed 50% usage.