Per WP 12570 we stopped the cameras on h1digivdeo[45] and updated the pylon and pylon-camera-server software. This was to bring in a fix in the pylon library for leaked file descriptors when a camera dropped and reconnected.
Last week we just restarted the software to close all the leaked file descriptors. This should solve the problem. This was not a feature update of the camera software, just a rebuild against a newer pylon library.
This was done 8:50-9:00am local time (16:00 UTC).
The procedure was to:
1. run apt-get update
2. stop all the camera processes via the management web page
3. run apt-get install pylon pylon-camera-server
4. restart all the camera processes via the management web page
5. spot check a few cameras to make sure things come back.
For posterity, pylon was updated to 8.1.0 and pylon-camera-server was updated to 0.1.18