note that fw2 and tw1 have issued retransmission requests since the last daq restart

Here are the current versions of daqd and nds Jonathan built and installed today:

Note that h1tw0 was upgraded to U12 because it was showing RAID errors, h1tw1 has been kept back with it original Gentoo OS (these machines were the original frame writers)

here is the nds process table (NDS-1 servers run two processes; daqd and nds)

Today, I calculated the volume of piping that feeds the circulation loop for the TCSY laser.  The total volume of water in the piping alone (to and from the laser to the chiller on the mech room mezzanine) to be 36 liters.  Wow, much more than I had assumed!  The chiller reservoir holds 7 liters, for a total of 43L in the system at any given "full" time.

Recall that the system popped a mesh filter and ran the reservoir dry on Sept 28th alog 30041 - at which time only 6L was added to ~fill the reservoir.  At the time, they also noticed that there was air in noisy lines down at the table so there was air pushed through some or all of the piping volume (at least the chiller->laser piping was full of air as they mentioned, so half of the 36L piping would have needed to be refilled).

I've added up the small-ish amounts of water we have been adding daily to keep the chiller reservoir topped off - we have added 10.5L over the last 3 weeks.  With the 6L added Sept 28th, we've added 16.5L to a 43L system so far.  Even assuming there was some water still in the pipes during the Sept 28th "leak", we likley still have a ways to go before we have the system full.

Keep on filling...

(From VE drawings, I estimated ~2880" of piping length round trip chiller-laser-chiller.  The piping ID looks to be 1".)

WP6251 RCG3.2 upgrade

Jonathan, Jim, Dave:

The main work today was in upgrading the front end models and the DAQ systems to RCG3.2. All models were recompiled yesterday evening (see alog 30609). This morning we installed this new code (took 1hr 27mins). New mx code was compiled, new daqd binaries were created for all DAQ systems. The install sequence was:

• stop all daq systems
• kill all models on all front end computers
• start two front end systems (h1psl0 and h1susauxb123), prior to starting the models mx_stream was restarted
• start DAQ h1dc0 data concentrator (bug with EDCU was discovered, initial restart was with old daqd and Jonathan installed the corrected daqd on the second restart)
• start h1tw1 to start minute trending the VAC channels
• start h1nds0 with old daqdrc configuration to keep guardian happy
• start all front end models, again restarting new mx_stream prior to starting  models
• hand front end models over to operator and guardian usage
• start all daq systems except the frame writers and nds1 (at this point they are waiting for single-raw-frame upgrade)

WP6258 DAQ removal of science frame

Jonathan, Greg, Jim, Dave:

All frame writers were configured to no longer write science frames, and rename the old commissioning frames as the new raw frame.

Detailed procedure given in wiki page https://lhocds.ligo-wa.caltech.edu/wiki/MakingTheCommissioningFrameTheNewFullFrame

After this change, there are no C-named frames, only R-named frames. Archived C-Frames are now R-Frames, new frames are what were previously called commissioning frames with R-names in the frames/full directories. What were called science frames are not written, no new frames in frames/science directories.

h1nds0 and h1nds1 were reconfigured to the new R-names and restarted

the wiper scripts on h1ldasgw0 and h1ldasgw1 were changed to not use science frames, and to give the disk these frames used to the full frames instead

WP6237 Remove h1ldasgw2

Jim, Dave

a third QFS/NFS server was install in the MSR during the summer as part of the attempt to fix h1fw0/h1fw1 instability. It offloaded the exporting of the frame directories (in read-only mode) to the NDS servers. It later proved to be a liability when corrupted frame files were served by h1ldasgw2 to both NDS servers.

Today we decommissioned h1ldasgw2 and reconfigured h1ldasgw0, h1ldasgw1 to serve their respective file systems as a read-only export to the NDS machines. h1nds0 and h1nds1 were configured to no longer use h1ldasgw2.

TITLE: 10/17 Eve Shift: 15:00-23:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Lock Aquisition

SHIFT SUMMARY: We restarted the world, now initial alignment, fss, iss and pmc are being fun.

WP 6176
WP 6247

Pulled in new power and network cables for the RGA (next to HAM4). Cables need to be terminated.
Cables at EY were terminated.

Each main turbo pump (MTP) has an adjoining box containing (4) ea. 12v 18Ah sealed lead acid (SLA) batteries connected in series that provide 48VDC for pump rotor levitation during loss of AC power.  These only get charged when the turbo controllers are energized.  As preventative maintenance during the prolonged periods of pump inactivity, we would energize the controllers for a few hours during Tuesday's maintenance period.  We now will keep these battery packs centrally located (hallway between AHU1 and AHU2 in the Corner Station Fan room) and on a battery tender.  The trade off is that we now will need to remember to take a battery pack with us when activating an MTP at an outbuilding.  

This morning, Chandra and I weighed an empty BSC LTS container. After weighing that container, we decided to check the weight on a loaded BSC LTS container. The loaded container never lifted off the floor. The crane capacity is only 10,000 lbs. and another 380 lbs. would not have lifted the container.   The results are in the photos here. 

(Kyle, Gerardo)

Today compressors #3, #4, and #5 were all greased and pressure tested.

All compressors passed their compression test, #3 at 120 psi, #4 at 120 psi, and #5 at 130 psi.
Then all compressors and electrical motors were greased.

All compressor assemblies were run tested after service was performed.

Work performed under WP#6257.

(See WPs #6221 and #6255)

Will need to make daily check each morning.  Plan to run through 10/23 inclusive unless too egregious for commissioners

M. Pirello

This FPGA code ( E1200034 ) disconnects the 1pps signal from the 4 internal LED's and grounds them.  All four timing comparators at LHO were updated to v5.

S1107952 in the CER

S1201224 in the MSR

S1201886 at X end

S1201222 at Y end

Work Permit 6256

Work Permit Number: 6252

The DMT calibration has been updated to gstlal-calibration 1.0.6-2.el7. Because of dependencies GDS was also updated to: gds-2.17.10-2.

John Zweizig has restarted the DMT monitors.
 

The medm SYS_CUST_SHUTTER_SUMMARY.adl shows the ALS shutters, but the names are wrong.  Attached is a map of what each shutter did today.

DCS verified all H1_R frames were copied from the CDS "science" directories into the LDAS archive up to the GPS endTime 1160847040.

LDAS has started copying the new larger H1_R frames from the CDS "full" directories, with the first larger H1_R frame with the "full" channel list starting from 1160854656.

Hiro asked me about LHO BRDF measurements like LLO (https://alog.ligo-la.caltech.edu/aLOG/index.php?callRep=28662).

Using numbers measured by Vinny (https://dcc.ligo.org/T1600085) when the arm power was about 100kW back in January, the corresponding BRDF numbers are:

There's no evidence that this changes with power: alog 28449

We're really struggling to acquire DRMI lock. It looks like our aux length loops suddenly have way more noise than they did before our last long lock stretch.  We're not at all sure why this is, but may want to look into it more, in case it hints at why we have more noise in the aux loops now than we did before we did the grouting / realignment in April. We can lock PRMI, but it is very noisy, and if we try to improve the buildups the noise gets worse.

We had a nice long lock, but then struggled to relock.  Nutsinee did an initial alignment, during which we decided to move IM2 and IM3 closer to their nominal positions, in acknowledgement of the warnings that have been on DiagMain lately.  However, we still couldn't aquire lock, so started to look at the noise.

Sheila and I found a time from earlier today (17Oct2016 16:03:00 UTC) where we were locked on PRMI for a few minutes, and saw that the MICH and PRCL noise was quite good.  In the attached screenshot that time is labeled "10+ hours ago".  When we first caught the PRMI, we had noise that looked like the "really bad" time in the screenshot.  This is more than 2 orders of magnitude worse than the quiet time for MICH between 10Hz-200Hz.  For PRCL the noise is only worse below about 150Hz, but it is still egregious.  By moving the alignment of the recycling cavity mirrors and the input steering mirrors, we could reduce the noise to the third set of traces, at the cost of losing at least a third or so of our sideband buildup. 

Thinking that this could be somehow related to our restoration of the IM2 and IM3 positions, we restored all suspensions (except RMs and OMs) to their witness or oplev values from 14 hours ago.  Patrick then did a careful initial alignment, but unfortunately our noise is still the same.  We can acquire PRMI but not DRMI.  At this point we're going to leave things as-is for now, and look into it again tomorrow after maintenence. 

We wonder though if this is a clue as to how / why we have so much extra noise in our aux length loops since April.

Looking at the coupling of carrier jitter with misaligned RF sidebands in REFL:

Jitter (alog 30237): 10^-6/√Hz (level) to n x 10^-4/√Hz (peaks)
IMC suppression (alog 30124): ~¹⁄200
⇒ at IFO: 5 x 10^-9/√Hz to ⁿ⁄2 x 10^-6/√Hz

Fixed misalignment of RF sidebands: Δα < 0.3
DC power in reflection with unlocked ifo at 50W: REFL_DC^unlocked ~ 300 mW
Error offset in REFL = jitter * REFL_DC^unlocked * Δα
⇒ 5 x 10^-9/√Hz  * 0.3 W * 0.1 ~ 1.5 x 10^-10 W/√Hz (low)
⇒ ⁿ⁄2 x 10^-6/√Hz * 0.3 W * 0.3 ~ ⁿ⁄2 x 10^-7/√Hz (high)

Frequency noise coupling into DARM (alog 29893):
⇒10^-10 m/W at 1kHz (approx. f-slope)

at 1kHz: 10^-20 m to 10^-17 m
at 300 Hz: n x 10^-18 m (high) with periscope peak n ~ 4.