J. Kissel for S. Karki

I've moved the roaming calibration line to its highest frequency we intend to go, and it's also the last super-long duration we need. We may run through the lower frequency points again, given that (a) they need much less data, and (b) those data points were taken at various input powers that will likely confuse/complicate the analysis. Below is the current schedule status.

Current Schedule Status:
Frequency    Planned Amplitude        Planned Duration      Actual Amplitude    Start Time                 Stop Time                    Achieved Duration
(Hz)         (ct)                     (hh:mm)                   (ct)               (UTC)                    (UTC)                         (hh:mm)
---------------------------------------------------------------------------------------------------------------------------------------------------------
1001.3       35k                      02:00                   39322.0           Nov 11 2016 21:37:50 UTC    Nov 12 2016 03:28:21 UTC      ~several hours @ 25 W
1501.3       35k                      02:00                   39322.0           Oct 24 2016 15:26:57 UTC    Oct 31 2016 15:44:29 UTC      ~week @ 25 W
2001.3       35k                      02:00                   39322.0           Oct 17 2016 21:22:03 UTC    Oct 24 2016 15:26:57 UTC      several days (at both 50W and 25 W)
2501.3       35k                      05:00                   39322.0           Oct 12 2016 03:20:41 UTC    Oct 17 2016 21:22:03 UTC      days     @ 50 W
3001.3       35k                      05:00                   39322.0           Oct 06 2016 18:39:26 UTC    Oct 12 2016 03:20:41 UTC      days     @ 50 W
3501.3       35k                      05:00                   39322.0           Jul 06 2016 18:56:13 UTC    Oct 06 2016 18:39:26 UTC      months   @ 50 W
4001.3       40k                      10:00                   39322.0           Nov 12 2016 03:28:21 UTC    Nov 16 2016 22:17:29 UTC      days     @ 30 W (see LHO aLOG 31546 for caveats)
4301.3       40k                      10:00                   39322.0           Nov 16 2016 22:17:29 UTC    Nov 18 2016 17:08:49 UTC      days     @ 30 W          
4501.3       40k                      10:00                   39322.0           Nov 18 2016 17:08:49 UTC    Nov 20 2016 16:54:32 UTC      days     @ 30 W (see LHO aLOG 31610 for caveats)   
4801.3       40k                      10:00                   39222.0           Nov 20 2016 16:54:32 UTC    Nov 22 2016 23:56:06 UTC      days     @ 30 W
5001.3       40k                      10:00                   39222.0           Nov 22 2016 23:56:06 UTC

J. Kissel, C. Vorvick

This morning, during repair of an infrasound microphone in the PEM stay-clear zone the corner station sensor correction STS got an impulse.  Unfortunately sensor correction to the corner-station SEI was not turned off, because it was unclear to those present that these two things would be interrelated. As such, this impulse was sent to the SEI systems and tripped a large fraction of their watchdogs (see LHO aLOG 31735). 

This includes the HAM2 ISI, on which the IMs are suspended. Whenever the HAM2 ISI trips or goes offline, the IMs all move to a new position, regardless of their requested alignment slider values.

This is merely another instance of the on-going issue with the IMs, as reported in
Integration Issue 4698
(note that as of this aLOG I've cleaned up the long and confusing digital paper trail associated with this bug.)

Sadly, we must wait until WHENVENT HAM2 before we can resolve this issue.

Here's how much the IMs moved:

      dP [urad]     dY [urad]
IM1      0.2          0.8             
IM2      31           5
IM3      6            0.5        
IM4      0.5          1
I've attached a 24-hour trend of the position to show this as a function of time. The black vertical line on each plot indicates the time of HAM2 ISI trip. 

Notes:
- From Cheryl's experience, a "large" jump that we need to correct for is 1 [urad] and 5 [urad] for IM1 and IM2/IM3 respectively, thus I've bolded the values in the table that are a large jump. I put "large" in quotes, because to-date, these numbers have just been empirically determined from commissioning experience. Cheryl is working on an analytical quantitative method for determining what is a large move based on metrics of the input faraday like the alignment's impact on the IFI's extinction ratio, etc.

- IM2 and IM3 are those SUS that typically move. IM4 never moves appreciably (so its typically not reported), and indeed that has been a defining bit of evidence in the study as to why IM2/IM3 move.

We've restored their alignment to the most recent out of lock positions as reported by their M1_DAMP_P/Y_IN1_DQ channels (or whatever EPICs versions of those channels you prefer). These must be restored to an out of lock alignment, because IM4 is globally controlled for alignment to follow the IFO when locked, which is decidedly the wrong alignment for lock acquisition.

at 17:35 UTC today, work on the corner station infrasound mic, in the biergarten, close to the ITMY STS caused a corner station trip that included all BSC ISI and HEPI as well as all HAM ISIs (NOT HAM HEPIS). Some suspensions were tripped as well. It is believed that the heavy piece of equipment, replaced onto the ground in the PEM "stay clear" zone is what cause the spike.

Tied the shunt trip main breakers to the thermal protection and E-Stop button in the DAQ room.  Tested that the system trips with either an over temperature condition or pushing the E-Stop Button.

WP 6344

Jim B, Dave B, TJ

Reverted nds2-client software to version 0.12.2 for Ubuntu 12, Ubuntu 14, and Debian8 workstations.

nds2-client-0.13.1 software was causing problems with a python getdata request for 30 seconds of data from 40 seconds ago, and/or multiple channels.  We will characterize this issue a bit more after the holiday to file a bug report.

John Z., Aaron V.

John restarted the primary GDS pipeline at GPS time 1163860820 +- several minutes, and I restarted the redundant pipeline at GPS time 1163887460. The latency was ~5-10 seconds and the CPU usage is ~70%, both usual. This restart picked up gstlal-calibration-1.0.8, which includes these changes:

- Addition of option --kappas-default-to-median to replace rejected kappas with last good median instead of default
- Bug fix to prevent/reduce latencies due to bad dataValid flags
- Addition of option --demodulation-filter-settle-time to allow filter to settle before accepting computed kappas.

A new filters file is also being used. See this aLOG for information on the filters: https://alog.ligo-wa.caltech.edu/aLOG/index.php?callRep=31712

Updated the firmware,  Assigned IP address ending in .170 plugged Ethernet cable into port 5.  Verified that we can see the unit from the corner station machine. 
All units in each building are powered up.

WP 6327 to replace all of the anemometers is complete.  The End Y station was completed today.  All buildings Now have the same orientation Wind from CS to EndS is 0deg.  From Ey to CS is 90.
R

Attached is the approximate plan of our injections. 

We will do corner station early in the week and end stations later in the week.

We may change the tests based on what we find.

6338 CAL model changes

Jeff, Dave:

model changes made to h1calex, h1caley and h1calcs. First two required a DAQ restart.

6315 FW1 upgrade to latest Frame-cpp

Jonathan, Jim, Dave:

fw1 code made the same as fw0 and fw2. Some problems restarting daqd which Jim resolved. Now all three framewriters are writing byte identical files, MEDM was reverted back to its original comparison on frame size (no longer expects a +2 difference)

6336 Add channels to DAQ GDS-Broadcaster

Jess, John Z, Dave:

As per ECR1600350 243 channels were added to the DAQ GDS-Broadcaster. The ECR lists channel H1:GDS-CALIB_STRAIN which does not exist in the DAQ and was ommited.

We removed any channels in H1BROADCAST0.ini which no longer exist in the DAQ (72 channels were removed)

I've attached a text file showing the additions and subtractions.

6340 Removal of fast channels from SUS PI models and OMCPI model

Peter, Terra, Dave:

The obsolete zeroed 64kHz DAQ channels (which were needed in old RCG versions) were removed from h1susitmpi, h1susetmxpi and h1susetmypi. On Peter's recommendation, the H1:OMC-PI_DCPD_64KHZ_BHF_DQ channel was removed from h1omcpi.

6337 Reboot Guardian machine

TJ, Dave:

h1guardian0 was patched and rebooted. The patch upgraded SWIG from version 2 to 3. No problems on restarting the guardian nodes.

h1tw0 disk problems

Jim:

during the offloading of raw minute trends from the SSD-RAID, h1tw0 developed disk problems. For now it is not running as a trend writer.

DAQ RESTART

Jim, Dave:

The DAQ was restarted to support the above changes. Only issue was with h1fw1's restart problems. EDCU is RED due to missing h1tw0 connection.

WP6337 and WP6341 complete

h1guardian0 reboot

Dave restarted the h1guardian0 machine and all of the nodes came back up without issues. The machine updates have not shown any problems so far. Only thing to note is DIAG_MAIN  now has a VERY long loop time that is caused by cdsutil calls to get some data via nds1. The average loop time was ~1sec and is now ~65sec. There have been other reports and issues that I have seen that make it seem as though the nds is feeding some bad data and also taking much longer than it used it. tw0 has been having some issues today, so perhaps it is related to that. I will investigate further when I can.

HPI_HAM testing

I found the easy bug, a comma left at the end of a line. Sadly, this led to a bigger bug. The length of the list of dofs that we had put in created a state that exceeds the 39 character limit for states. After talking with Jamie we decided to try to split up these dofs and see if that will work for now. We will probably try this out next Tuesday if we can.

Gerardo, Chandra

Completed WP 6342

Turns out the annulus ion pump was bad on HAM 11 (replaced in May 2016). R&R and also upgraded the controller to a brand-new, out of package unit. Now both HAM 11 & 12 are reading 4 mA! IPs were valved in w/ turbo aux cart reading 1e-5 Torr pressure at cart turbo (we also used a hung turbo). Turbo cart is OFF, but still connected.

Gerardo updated the PV assignments on HAM 11/12 and 7/8 by swapping wires at Beckhoff racks (they were backward before). Now what you see on MEDM is what you get.

Awesome team work!

(see WP6332)

Also running turbo pump connected to PT180 hardware on dome of BSC8.  Coordinating with PEM folks to see if the IFO commissioning community can live with this noise source for a few days.  If so, we (Vacuum) would then bake the PT180 hardware while being valved-out from the YBM and while being pumped by these temporary pumps.  

Will be leaving these run past the official "Maintenance Day" period so as to see what these look like if/when the IFO is in low-noise.  May need to do some "on/off" tests as per Robert S. 

While at the End Station doing weekly fluid level checks, the HEPI tripped off due to a fluid level trip.  Two Things:

1) I have the fluid level trip point set pretty close to the running level to keep the mess minimal in the event of a leak. 2) The pressure wash task of the roof ladder had the jar wide open with an air temperature, what, mid 30s?

I did just manage to record the fluid level before the trip and it was down 1/16".  I surmise the cold air reduced the volume of the fluid enough to trigger the level switch.

I've lowered the trip point about 1/4" so this should not be an issue for some time, 1/16" was just a little too little head room.  Let's keep the door as closed as possible in extreme weather.

The system recovered just fine.

I have increased the chilled water set points at both end stations to 43 degrees F. This is an increase of 4 degrees.

The /trend file system (a SSD RAID) became corrupted yesterday evening at 22:59 PST.  This prevented raw trend data from being written and caused the daqd process to repeatedly restart all night.  

The file system has been repaired and the trend writer started again.

The old raw minute trend data was being copied to h1fw0 at the time, and was close to half done.  The copy operation will be cleaned up and resumed.

J. Oberling, P. King

We turned off the PSL Diagnostic Breadboard this morning, per LHO WP 6317.  We turned off the control signals for the DBB PZTs in MEDM, then shut off power to the DBB.  It will remain in this state for the duration of O2.  This closes WP 6317.

I also took the opportunity to perform the weekly PSL power watchdog reset.

C. Cahillane

ER10 Actuation uncertainty budget for LHO based on the N/ct transfer function measurements from November 8, 9, 11, 12, and 18th.
To be clear, these measurements have the suspension models already divided out.  Additionally, we only take measurement points for UIM up to 30 Hz, and PUM up to 100 Hz, since these are the only regions they have any actuation authority.

We perform 4th order linear fits in the log(f) domain of the three actuation stages, UIM, PUM, and TST:

Fit = ( a_0 + a_1 * log(f+1) + a_2 * log(f+1)**2 + a_3 * log(f+1)**3 + a_4 * log(f+1)**4 ) * exp(i * 2*pi*f*a_5 + i * a_6)

The coefficients [a_0, ..., a_5] are the linear coefficients governing the magnitude of the fit.  
The coefficient a_5 is the time delay term, and a_6 is some phase offset.

I fit the magnitude in the log(f+1) domain because the data is taken with logarithmic frequency spacing, and this gives the MCMC more freedom to fit the more-uncertain low frequency data points while constraining itself in the bucket.  The plus one is so log(0 + 1) = 0 and not log(0) = -infinity, so at DC we don't explode our MCMC predicted results.
I fit the phase in the linear f domain because exp(i* 2*pi*f*time_delay) is the typical expression for a time delay, and this seemed to be a sufficient fit given our data.

There are two types of plots shown below: the corner plot and the measurement + fit + uncertainty results.  There is one such plot for each stage.

Plots 1, 2, and 3 are the UIM, PUM, and TST measurements + fit + uncertainty results.  The shapes represent the ER10 measurement sweeps, the green line is the MCMC MAP fit, and each individual grey line is a potential fit according to the MCMC.  The 1000 grey lines together form our uncertainty budget.

Plots 4, 5, and 6 are the UIM, PUM, and TST fit corner plots.  These show each parameter of the MCMC, its MAP value, and its 68% confidence interval in the title.  The PDF of each parameter is shown along the diagonal, while the bulk shows the covariance of each parameter with one another.  
The names of each parameter is slightly different from my equation above:

a_0 = H_{Stage}
a_1 = alpha
a_2 = beta
a_3 = gamma
a_4 = delta
a_5 = tau   (time delay)
a_6 = phi


In comparison with LHO aLOG 31344, the DC coefficient values found were:

H_UIM = 1.2 (+ 0.8 or - 0.7) * 10^-7
H_PUM = 6.8 (+ 1.5 or - 1.5) * 10^-10
H_TST = 4.0 (+ 0.2 or - 0.2) * 10^-12


However, 4th order fits may not be the best method of finding the DC coefficient, since even a small change in value on the 4th order coefficient can have a huge effect on the 0th order coefficient.  But for characterizing uncertainty in our actuation plant, this is the correct way since we don't in principle know the function we're fitting for.

Overall Uncertainty:

UIM Stage: < 3% magnitude and < 1 degree phase uncertainty
PUM Stage: < 2% magnitude and < 1 degree phase uncertainty 
TST Stage: < 2% magnitude and < 1 degree phase uncertainty

(All stages have their maximum uncertainty at low frequency.)

This level of uncertainty is exceptionally tiny, even with 4th order polynomial fits.  The great deal of measurements taken constrains the fit, lowering uncertainty drastically, especially in the bucket.  Once we can account for time dependence, we may be close to achieving 5% and 5 degrees, depending on the sensing function uncertainty, or becoming kappa-limited.