WP7161: Dan, Dave:
h1fw1 is running again, writing frame files to its new E18. First file was written at 16:18 PDT this afternoon.
The two SATABOYS (originally used as a pair by h1ldasgw1) have been split, one per ldas-gateway machine.
Because both NDS servers are serving h1ldasgw0's framed data (data acquired since 10/18), Dan is building h1ldasgw0's SATABOY first and will populate its raw minute trend area with the latest backup (May-Oct 2017) first. He estimates this file copy will take about 8 hours, so will be available to nds in the morning.
The SATABOYs store two types of files: archived raw minute trend files and MD5 check sum files for GWF framed files (both full and trend) written by the frame writer.
The sataboy will be exported by the ldasgw as /sataboy-0 and /sataboy-1 for the two systems. The directory structure under this mount point is:
/sataboy-n/fchksums/full* (contains directories with 5digit-GPS name, same as /frames/full)
/sataboy-n/fchksums/trend/second (contains directories with 5digit-GPS name, same as /frames/trend/second)
/sataboy-n/fchksums/trend/minute (contains directories with 5digit-GPS name, same as /frames/trend/minute)
/sataboy-n/minute_raw (contains one directory per archived data block, with names minute_raw_10-digit-GPS)
Note: for now we are not creating the /frames/science directory, this was removed when the commissioning/science frame distinction was abandoned.
correction, the path the the archived raw minute trend data is:
/sataboy-0/frames/trend/minute_raw/minute_raw_1192382538
Dan has recovered all the files in this area, I'll add this to h1nds1's configuration.
But first....
we noticed that the archived minute trends were actually too big to fit on the SATABOY uncompressed. We have two options:
1. Keep the SATABOY all QFS and i) manually compress the trends, meaning ii) changing the DAQ code to uncompress files when needed.
2. Dan suggests splitting the SATABOY into two partitions, one small QFS for MD5 files, the second a large ZFS file system with firmware compression built-in.
We mulled this around and decided on Dan's suggestion. He has done some further investigation to determine the best compression algorithm for our application. He also found that only 2TB is needed to store all the archived min trend data taken so far in compressed files, leaving plenty of room left in the 11TB file system for future data.