The seismic computer at EX froze (see attached picture).  It was rebooted
along with the AI chassis and IO chassis.

    Dave Barker was notified.





Richard / Peter

At 13:10utc (6:10amPDT):

•  Received Verbal Alarms about HEPI/ISI/SEI ETMX Watchdog Trips.  
• CDS Overview WHITE for H1IOPSEIEX, H1HPIETMX, H1ISIETMX
• All through this, ETMx SEI systems remain damped and H1 is still in Nominal Low Noise, but am OUT OF OBSERVING.
• Flagging this as CORRECTIVE MAINTENANCE
• 13:30 Richard is on-site and he will be taking a drive to EX to investigate.
• Holding off calling Dave since it is early
• Called LLO Operator (Gary T) to notify them of our status.  
• Will be down for atleast 2hrs (addressing frontends, re-locking H1).

(attached is a screenshot of all the WHITE screens we have for EX.

Random lockloss (no obvious seismic to blame).

• Had to FIND IR by hand, but no big deal.
• Spent a couple of minutes dealing with an SDF Diff for H1:ASC-CHARD_P_GAIN:  Was -0.14, but I walked it down to the setpoint of -0.10---->Was -0.14 related to the ASC work from Wednesday Commissioning?
• PI MODE27 was damped

Out of OBSERVING for 33min.

Steady-running for H1 with an 18hr lock (15hrs in Observing).  

The Lock Clock was restarted & does NOT show the correct time, so until the next lockloss, Add 14.5hrs To The Lock Clock time (i.e. now it is ~3.5 + ~14.5 = ~18hrs).

useism is slightly higher than last night but still happily down near 50th percentile.  Here's to more steady running.

TITLE: 04/27 Owl Shift: 07:00-15:00 UTC (00:00-08:00 PST), all times posted in UTC
STATE of H1: Observing at 67Mpc
OUTGOING OPERATOR: Nutsinee
CURRENT ENVIRONMENT:
    Wind: 6mph
    Primary useism: 0.02 μm/s
    Secondary useism: 0.15 μm/s (at 50 percentile)
QUICK SUMMARY:

H1 has been at NLN since 17:32utc (~14.5hrs).  I need to get the Lock Timer going on nuc1 & also Inspiral Integrand on video0.

TITLE: 04/27 Eve Shift: 23:00-07:00 UTC (16:00-00:00 PST), all times posted in UTC

STATE of H1: Observing at 62Mpc

INCOMING OPERATOR: Corey

SHIFT SUMMARY: Been observing the whole shift. No issue to report.

Been observing since Jim handed over. Wind speed has been fluctuating. Not much else going on.

FAMIS 6895

I do not see any that are particularly elevated.

I added two band stops to the CHARD loops, which have reduced the CHARD drives from 15-25 Hz by about a factor of 10.  ASC noise should no longer be limiting our DARM sensitivity above 15Hz, the noise is only slightly better from 15-20 Hz.  

The first attachment shows the slight improvement in the DARM noise, the difference in the drives (the cut offs I added were 2nd order elliptic bandstops), and the reduction in the coherence between the ASC drives and DARM.  For CHARD Y the first screenshot shows the loop measurement before I added the bandstop, the bandstop should have reduced the phase at the upper ugf (2.5 Hz) by about 5 degrees.  For CHARD P I reduced the gain by about 3dB, the third screenshot shows the before measurement in blue, a measurement after I reduced the gain but before I added the cut off in red.  For CHARD the cut off only reduced the phase at the upper ugf of 3 Hz by 6 degrees, we are left with almost 50 degrees of phase margin.

I also re ran the noise budget injections for CHARD, DHARD, MICH, SRCL, PRCL and IMC PZT jitter.  There only real change is that the ASC noise is lower, and there is a larger gap between the sum and the measured noise at 25 Hz.  I am not able to download GDS strain from this afternoon, so I will post a noise budget when I can get data. 

Adding to the list of vacuum monitoring striptools being ran on the virtual machine, I've included the vacuum pressures striptool (called press.stp). Reminder, these images are available on the web page:

vacuum_striptools

(accessed from the CDS home page by clicking the Vac (screen shots) link)

Starting CP3 fill. LLCV enabled. LLCV set to manual control. LLCV set to 50% open. Fill completed in 2460 seconds. TC B did not register fill. LLCV set back to 16.0% open.
Starting CP4 fill. LLCV enabled. LLCV set to manual control. LLCV set to 70% open. Fill completed in 654 seconds. TC A did not register fill. LLCV set back to 41.0% open.

The new FMCS IOC (BacNET interface) was down for about 30 minutes while we investigated a routing issue. It will be quickly restarted later this morning to run it within a screen session.

WP 6577

Dave B., Carlos P., Bubba G., John W., Patrick T.

I have migrated a subset of the EPICS channels provided by the FMCS IOC on h0epics to an IOC I created on fmcs-epics-cds. The IOC on fmcs-epics-cds connects to the BACNet server that Apollo has installed as part of the FMCS upgrade. The channels that I migrated have been taken over by this upgrade and can no longer be read out by the server that the IOC on h0epics reads from. The fmcs-epics-cds computer connects to the slow controls network (10.105.0.1) on eth0 and the BACNet network (10.2.0.1) on eth1. It is running Debian 8.

The IOC on h0epics is started from the target directory /ligo/lho/h0/target/h0fmcs (https://cdswiki.ligo-wa.caltech.edu/wiki/h0fmcs). I commented out the appropriate channels from the fmcs.db and chiller.db files in the db directory of this path and restarted this IOC. I made no changes to the files in svn.

The IOC on fmcs-epics-cds uses code from SNS: http://ics-web.sns.ornl.gov/webb/BACnet/ and resides in /home/cdsadmin/BACnet_R0-8. This is a local directory on fmcs-epics-cds. This IOC is started as cdsadmin:

> ssh cdsadmin@10.105.0.112
cdsadmin@fmcs-epics-cds: screen
Hit Enter
cdsadmin@fmcs-epics-cds: cd /home/cdsadmin/BACnet_R0-8/iocBoot/e2b-ioc/
cdsadmin@fmcs-epics-cds: ../../bin/linux-x86_64/epics2bacnet st.cmd
Hit CTRL-a then 'd'

Issues:

I came to realize during this migration that the logic behind the binary input channels is different in BACNet. In BACNet a value of 0 corresponds to 'inactive' and a value of 1 corresponds to 'active'. In the server being migrated from a value of 0 corresponds to 'invalid'. This was verified for the reverse osmosis alarm: H0:FMC-CS_WS_RO_ALARM. In the BACNet server it reads as 0 or 'inactive' when not in alarm. When John W. forced it into alarm it read as 1 or 'active'. I believe Dave has updated his cell phone alarm notifier to match this.

A similar situation exists for the state of the chiller pumps. In the server being migrated from a value of 1 appears to correspond to 'OFF' and a value of 2 appears to correspond to 'ON'. It has not been verified, but I believe in the BACNet server a value of 0 corresponds to 'OFF' and a value of 1 corresponds to 'ON'. The pump status for each building is calculated by looking at the state of the pumps. The calculation in the database for the IOC being migrated from appears to be such that as long as one pump is running the status is 'OPERATIONAL'. If no pump is running the status is 'FAILED'. I need to double check this with John or Bubba. I updated the corresponding calc records in the database for the BACNet IOC to match this.

In the server being migrated from, channels that are read by BACNet as binary inputs and binary outputs are read as analog inputs. I changed these in the database for the BACNet IOC to binary inputs and set the ONAM to 'active' and the ZNAM to 'inactive'. The alarm levels also need to be updated. They are currently set through the autoburt snapshot files that contain a separate channel for each alarm field. The autoburt request file has to be updated for the binary channels to have channels for .ZSV and .OSV instead of .HSV, .LSV, etc. So currently there is no control room alarm set for the binary channels, including the reverse osmosis alarm.

I also need to update the medm screens to take account of this change.

Also, there is an invalid alarm on the control room alarm station computer for the mid X air handler reheat temperature. Looking on the BACNet FMCS server this channel actually does appear to be genuinely invalid.

It should be noted that this BACNet IOC is a temporary install until an OPC server is installed on the BACNet server.

I would like to leave the permit for this work open until the FMCS upgrade is complete and all the channels have been migrated to the BACNet IOC.

We've also reserved three hours for calibration on Thu. Apr. 27, 1600-1900 UTC (0900-1200 Pacific) in coincidence with LLO.

Sheila, Jason, Evan G, Krishna

Mode hoping of the ETMY oplev has been showing up in hveto since April 19th, although the oplev damping is off.  The glitches that show up are definitely glitches in the sum, and the oplev is well centered, so the issue is not that the optic is moving. There is a population of DARM glitches around 30 Hz that is present on days when the oplev is glitching but not on other days.  We are curious about the coupling mechanism for these glitches and wonder if this coupling could be causing problems even when the oplev is not glitching loudly . 

Evan, Jason and I connected the monitor on the oplev laser diode power to one of the PEM ADC channels used to monitor sus rack power (we used H1:PEM-EY_ADC0_14_OUT_DQ, which was monitoring the +24V power and is channel 7 on the field rack patch pannel.  Jason can make the laser glitch by tapping it, with this test we saw clear glitches in the sum but no sign of anything in the monitor so this monitor might not be very useful.  Plugging this in means that the lid of the cooler is slightly open.

We also unplugged the fiber, so that for the time being there is no light going into the chamber from the oplev. If these glitches are coupling to DARM electromagnetically, we expect to keep seeing them in DARM.  If they were somehow coupling through the light (radiation pressure, something else), we would expect them to go away now.  One glitch that we looked at is about a 75 uW drop in the laser power on the optic.  (A=2P/(c*m*omega^2)= 3e-19 meters if all the power were at 20 Hz).  We don't really know how centered the beam is on the optic, or what the reflectivity is for the oplev laser, but it seems like radiation pressure could be at the right level to explain this.   

Using an ASD of the oplev sum during a time when the oplev is quiet, this noise is more than 3 orders of magnitude below DARM at 30 Hz.

[Heather Fong, Sheila Dwyer]

Over the last few months, Sheila and I have been trying to find correlations between auxiliary channels and LHO's sensitivity range. In order to do so, we first made changes to the OAF BLRMS range channels by adding notch filters such that they can track changes to the SENSMON range (see alog 33437). After we made these changes, the summed OAF BLRMS range contributions now have a linear relationship with the SENSMON range, with their units roughly calibrated to be Mpc.

I then wrote a Python script that does the following:

- Loads in desired auxiliary channel data (using NDS and GWpy) for a specified period of time (we analyze minute trends)
- Calculates the Pearson correlation coefficient (PCC) between auxiliary channels and the OAF BLRMS range channels in order to determine how linearly correlated the channels are
- Plots and saves the channels with the highest PCC (aux channels vs. OAF BLRMS range and aux channels vs. time)

Attached to this entry are examples of the aux channels vs. OAF BLRMS range for the time period of Feb 24 2017 to Apr 10 2017 (45 days, H1 observing mode only). The complete list of channels that were analyzed are attached under the file name 'BLRMS_channel_list.txt'. With the exception of the OAF channels, both mean and RMS channels were analyzed. For ~360 channels over a time range of 45 days, this script takes ~2 hours to complete, where the data retrieval from NDS is the bottleneck. The Python script has been uploaded to the LIGO git repository and can be found here:

git clone https://heather-fong@git.ligo.org/heather-fong/BLRMS-channels-correlations.git

where BLRMS_channels_analysis.py is the Python script, and BLRMS_channel_list.txt is an example of channels that can be analyzed.

We found the channels with the highest absolute PCC values (and are therefore most correlated with the range) to be the following (plots are attached):
H1:ASC_AS_B_RF36_Q_PIT_OUT16
H1:ASC-AS-B-RF36_Q_YAW_OUT16
H1:SUS-ITMY_R0_DAMP_Y_INMON

Other channels we analyzed that appear to be correlated with the range include:
H1:SUS-ITMX_M0_DAMP_R_INMON (max PCC = -0.5 for 38-60Hz range)
H1:SUS-ITMY_R0_DAMP_L_INMON (max PCC = 0.5 for 38.60 Hz range)

The results of this analysis gives us hints as to which parts of the interferometer are affecting the sensitivity range. In particular, the results suggest that there are problems with the ITMY reaction mass that are not seen in the ITMX reaction mass, and we can, for example, try putting different offsets in ITMY to confirm this.