Entry by Kyle

LLCV % open for CP2 increased by 20% of its traditional output ~ 10 days ago for some reason -> today's delivery resulted in slow response (shouldn't be related) -> xfer line likely will warm up tonight before PID settles -> delivery rate wasn't abnormally short -> uncertain why response is different -> will investigate

(Doug)
I realigned the ETMx pointing needing only a few micro-radians of adjustment since it was upgraded. Several other OPLVR lasers will need to be realigned in the near future but I am in hopes that we can do them as we replace them with the new installation components.

The ETMy OPLVR laser over heated when its cooling fan quit. I pulled it and Jason and I repaired the unit and will run it over night on the bench. We tweaked the alignment, temperature settings as well as the current to get the best stabilization from it and will look at it in the AM to see if continues to be stable. Given Control Room permission we will reinstall it in the AM. We had added 1 too many diodes to the fan to slow down the RPM to quiet the acoustic noise which caused too large of a current draw causing the fan to over heat and stop.
Doug and Jason

07:00 Jeff K begins RCG upgrade; on-table External went closed just after; ISS having some trouble. Sigg took control of the FE and is working on the ISS issue

08:15 ISS fixed and shutter reset

08:20 Hgh to LVEA to check HEPI accumulators. (pressure and charging)

08:23 Elli & Nutsi out to HWS table by HAM4

08:25 Started un-tripping all suspensions except for BS pending the M2 stage coil driver swap.

08:27 Fil swapping the BS M2 coil driver

08:32 Corey to MY

08:40 Jim Batch restarting DAQ

08:43 Fil reports coil driver swapped. Also, seismometer in beer garden is connected.

08:48 Rick, Jason and Kiwamu headed towards PSL to fix periscope PZT

08:50 Kyle into LVEA and then to Y VEA

08:51 Elli and Nutsi out of LVEA

09:00 brought end stations to offline and safe. (sei/sus)

09:11 Jim and Corey to EX to restart damper on BRS

09:13 Betsy out to LVEA to look for stuff

09:25 McCarthy out to LVEA

09:31 Greg to TCSY table

09:40 ISS FE model re-started

09:44 McCarthy out of LVEA

09:45 Corey out to LVEA/Jim back from EX

09:53 Hugh out of LVEA

09:54 Hugh out to Ends to check HEPI actuators

09:55 Corey out of LVEA

09:56 RCG upgrade is COMPLETE! .....

10:00 Elli and Nutsi to end stations to put plates on cameras

10:12 Doug to EX to check opLev - did a tiny bit of alignment

10:29 Cris and Karen out of LVEA. Karen to EY

10:32 Elli and Nutsi back from ends

10:38 Betsy out to LVEA

10:41 Dick out to LVEA to ISC racks 1/2/4

10:46 Kyle back from end stations

10:53 Gerardo making deliveries to end stations

10:54 2nd LN₂ delivery arrived. I don't remember the first one getting here but the alarms tell the tale.

11:01 Cris to EX

11:15 Port-O-Let maintenance on site

11:15 Hugh back from EY

11:20 original BS M2 coil driver returned to SUS C5.

11:36 Karen leaving EY

12:04 Jason/Rick/Kiwamu out of PSL

12:07 Greg, Elli & Nutsi out of LVEA

12:21 Dick out of LVEA

12:24 Fil and Hugh out to LVEA to press the centering button on the new seismometer

13:23 FIl and Andres @ MY

13:33 Gerardo out to LVEA to retreive a PC by the X manifold.

13:35 Hugh out of LVEA

13:50 Gerardo out of LVEA

14:01 Kyle to Mid Stations

14:37 Hugh into CER

15:14 Fil and co. back from Mid station

15:15 Kyle back from Mid Stations

While clearing out many settings diffs on the SDF after the RCG 2.9.1 upgrade today, Kissel and I discovered a few issues, the worse offender is the first one:

-  OPTICALIGN values are not being monitored on SDF since they constantly change.  However after reboots, these values are read in via a burt from the SAFE.SNAP file.  Often these files are older than 1 day (in many cases weeks old) so therefore restore an old IFO alignment.  We need a better way to save local IFO alignments in the SAFE.SNAP now that we have rerouted how we transition suspensions from aligned to misaligned.

-  Precision of the SDF writing feature is different than the precision in the reading feature, therefore they do not clear from the diff list when you attempt to ACCEPT them.

-  Useage of CONFIRM and LOAD TABLE buttons is still confusing

-  Settings on the DIFFS screen represent the switch settings via reading SWSTAT however the MON list still shows the SW1 and SW2 settings.  This means the TOTAL DIFFs and NOT MONITORED counts never add up to the same number.  1 SW line item on the DIFF screen turns into 2 line items when pushed to the NOT MONITORED screen.

-  The ramp matrix (specifically looking at LSC) is constantly reporting changes.  The time stamp keeps updating even though the value is not actually changing.  Haven't looked at other ramp matrices yet.

The 50W laser is now running on the TCS Y table. Unfortunately the issues with the HWS kept us from being able to start trying to align everything. The projection beam is currently being dumped. Also took the opportunity to install the DC power switch. Currently the Y arm has the switch installed while the X arm does NOT have one installed. While setting up the IR fault kept triggering; if the laser suddenly shuts off that would be my first suspect.

J. Kissel, D. Sigg, D. Barker, J. Batch, B. Weaver, E. Merilh

We've upgraded all front-end models to RCG 2.9.1. Recovery of the IFO is on-going, given that we had our usual chaotic array of activities, but all signs point towards success. The upgrade was a little bit harder for some models / front-ends than others (details below), but we now have a completely green CDS overview screen (except for the saturation indicators that are always present on certain ISC models, and the currently-railing STS2 plugged into STS B causing all corner station seismic models to saturate -- more on that later). Details of the hiccups below.

Details
---------
Problem children:
- PSL ISS front-end model
This guy was the nastiest. The model some how did not have a safe.snap file or softlink in the 
/opt/rtcds/lho/h1/target/h1psliss/h1pslissepics/burt/
directory. We didn't notice this until much later, but at first this caused the model to simply not start because it couldn't find (what used to be) "the burt button" EPICs record that gave to OK status to get started. Daniel and I tried blindly restarting the model, recompiling-reinstalling-and-restarting the model, with no success. Finally, Daniel figured out that if he burt restored *while* the front end was coming up, he could get the model to start. Later, Betsy was making the effort to turn on the PSL ISS model's SDF system, created a table, but could not load it. After about 30 [sec] of trying, the front end model core dumped, seg faulted, and just died (not only this user model crashed, every other model on the front end survived). It was only after investigating this, that we found out about this missing target-area-safe.snap. The userapps repo had a safe.snap, so we softlinked the target-area to the userapps repo, and restarted the front-end model. 
All has been well since. 
We don't understand how a front end model can exist with out *something* in the target area called "safe.snap".

- The PCAL X and Y front-end model
Because it appears to be related (both in symptom and potentially in responsible parties) I mention this here. Last night, when I was capturing all settings in prep for today's model restarts, I captured new safe.snaps for the pcal front end models, because they were not yet under SDF control. In doing so immediately noticed that these models *also* didn't have safe.snap files in the target area. I didn't think much of it at the time, because I know the history of the pcal models, BUT now that I see a similar problem with the psliss model, I worry that the problem is systematic. Will investigate further.

- The end-station ISC, SEI, and SUS models
We had only planned to restart the end station's ISC and PEM front ends, because the SUS and SEI had been upgraded last week. However, when we restarted the ISC models, we found lots of continuous IPC errors between the ISC models and SEI and SUS. We think we traced this down to the clearing of the entire IFO's IPC file yesterday. Dave and Jim had thought they'd recompiled and reinstalled the SEI and SUS, which should have populated the IPC file *without* restarting the models, but this didn't appear to be successful. So, we ended up recompiling, reinstalling, and restarting the SEI and SUS models in addition to every other model at the end station (again). All errors are clear now, as mentioned above.

Also:
As mentioned by Daniel (LHO aLOG 17969), we had forgotten to update the some ODC library parts before getting started with the recompiling yesterday, so only some models received the upgrade to their ODC. There also seems to be a few bugs with the updated version, from what we can see. Will follow up with the ODC team. However, since we've run out of time, we'll include the remainder of these in the model recompiling already planned for next Tuesday. 
Models that need restarting to receive the ODC update:
- All corner station SEI and SUS models
- Corner station TCS model
- Corner station PEM
- Corner LSC, ASC, and OMC models
- Corner Station CAL model
(Note, all SUS AUX models don't have any ODC in them, so they do not need the update).

Elli. Greg. Aidan. Nutsinee.

The HWSY SLED has ended it's useful lifespan. The power has decayed by about 90% as shown in the attached plot of the last 6 months.

This is due to running it consistently at a high current. At present, there is a hardware maximum current limit on the SLED driver (https://dcc.ligo.org/LIGO-T1000662) which has been set to the maximum limit specified by the SLED supplier (it's also unqiue for every diode). I think we need to set this limit lower to increase the lifetime of these SLEDs - we always wanted to run them at roughly half the output power, but this has control has been hanlded in software. 

I shutdown h1conlog-old and removed its disk. It was hosting data from a previous version of Conlog. I copied SQL dumps of the databases to /ligo/lho/data/conlog/h1/backups/h1conlog-old. This computer will be transitioned to backup storage for the new version of Conlog.

This completes WP 5163.

JasonO, KiwamuI, RickS (and RobertS, in spirit)

Today, we moved the PZT-controlled mirror from the top of the IO periscope down to the surface of the optical table and swapped it with with a turning mirror that was on the table.  I.e. IO_MB_M6 (top of periscope) swapped with IO_MB_M4 (turning mirror immediately downstream of thin-film polarizers).  Note that the PZT blocks the (weak) beam transmitted through the mirror, so we removed beam dump IO_MB_BD6.

We first installed an iris at the top mirror ouput, using a C-clamp to attach a temporary plate to the top of the periscope. 

We removed the top mirror mounting plate, installed a remporary Ameristat skirt using cleanroom tape, and used a single-edge razor blade to remove some of the periscope damping material. This allowed the top plate to drop down to the required position.

We swapped the pitch actuator on the upper mirror mount to use a non-lockable actuator that doesn't interfere wtih the mounting plate.

We then used existing irises on the table in the path transmitted by the bottom  periscope mirror and the iris we installed plus the spot on the outside of the PSL enclosure that reflects from the HAM1 input port to align the two mirrors we swapped.

We were able to re-install the protective shield for the vertical path up the periscope in it's original orientation.

We expect that RobertS will assess whether or not this reduced noise induced by the PZT mirror by not having it at the top of the periscope where the noise is amplified by the periscope resonance.

A few images are attached below.

After today's upgrade to RCG 2.9.1 we looked to see if the DAC AUTOCAL was successful using these procedural notes.  The last check was done on the BSCs only a few weeks ago (alog 17597).  During today's check, we found errors on h1susb123 again and also on h1sush56.  Both show that the AUTOCAL failed for 1 of their DACs.  As well, Kissel logged into LLO and found all AUTOCALs reported SUCCESS on all SUS front ends since their last computer restart.

The LHO errors were as follows:

controls@h1sush56 ~ 0$ dmesg | grep AUTOCAL
[   60.359217] h1iopsush56: DAC AUTOCAL SUCCESS in 5134 milliseconds
[   65.510812] h1iopsush56: DAC AUTOCAL SUCCESS in 5134 milliseconds
[   70.661410] h1iopsush56: DAC AUTOCAL SUCCESS in 5134 milliseconds
[   75.813017] h1iopsush56: DAC AUTOCAL FAILED in 5134 milliseconds
[   80.963620] h1iopsush56: DAC AUTOCAL SUCCESS in 5134 milliseconds
[8443363.521348] h1iopsush56: DAC AUTOCAL SUCCESS in 5134 milliseconds
[8443368.669944] h1iopsush56: DAC AUTOCAL SUCCESS in 5133 milliseconds
[8443373.818544] h1iopsush56: DAC AUTOCAL SUCCESS in 5133 milliseconds
[8443378.967145] h1iopsush56: DAC AUTOCAL FAILED in 5133 milliseconds
[8443384.115739] h1iopsush56: DAC AUTOCAL SUCCESS in 5133 milliseconds

The first of the 2 above calibrations was the reboot 97 days ago, on Jan 14, 2015 when we upgraded to RCG 2.9.

________________________________________________________________

controls@h1susb123 ~ 0$ dmesg | grep AUTOCAL
[   61.101850] h1iopsusb123: DAC AUTOCAL SUCCESS in 5134 milliseconds
[   66.252460] h1iopsusb123: DAC AUTOCAL SUCCESS in 5134 milliseconds
[   71.833569] h1iopsusb123: DAC AUTOCAL SUCCESS in 5133 milliseconds
[   77.416848] h1iopsusb123: DAC AUTOCAL SUCCESS in 5134 milliseconds
[   82.567454] h1iopsusb123: DAC AUTOCAL SUCCESS in 5134 milliseconds
[   87.718046] h1iopsusb123: DAC AUTOCAL SUCCESS in 5134 milliseconds
[   92.869668] h1iopsusb123: DAC AUTOCAL SUCCESS in 5134 milliseconds
[   98.021279] h1iopsusb123: DAC AUTOCAL FAILED in 5134 milliseconds
[6643697.827654] h1iopsusb123: DAC AUTOCAL SUCCESS in 5134 milliseconds
[6643702.976255] h1iopsusb123: DAC AUTOCAL SUCCESS in 5133 milliseconds
[6643708.553386] h1iopsusb123: DAC AUTOCAL SUCCESS in 5134 milliseconds
[6643714.130498] h1iopsusb123: DAC AUTOCAL SUCCESS in 5134 milliseconds
[6643719.278911] h1iopsusb123: DAC AUTOCAL SUCCESS in 5133 milliseconds
[6643724.427687] h1iopsusb123: DAC AUTOCAL SUCCESS in 5134 milliseconds
[6643729.576295] h1iopsusb123: DAC AUTOCAL SUCCESS in 5133 milliseconds
[6643734.724703] h1iopsusb123: DAC AUTOCAL FAILED in 5133 milliseconds
[8443632.920341] h1iopsusb123: DAC AUTOCAL SUCCESS in 5133 milliseconds
[8443638.069031] h1iopsusb123: DAC AUTOCAL SUCCESS in 5133 milliseconds
[8443643.646215] h1iopsusb123: DAC AUTOCAL SUCCESS in 5134 milliseconds
[8443649.223336] h1iopsusb123: DAC AUTOCAL SUCCESS in 5134 milliseconds
[8443654.371885] h1iopsusb123: DAC AUTOCAL SUCCESS in 5133 milliseconds
[8443659.520452] h1iopsusb123: DAC AUTOCAL SUCCESS in 5134 milliseconds
[8443664.669135] h1iopsusb123: DAC AUTOCAL SUCCESS in 5133 milliseconds
[8443669.817631] h1iopsusb123: DAC AUTOCAL FAILED in 5133 milliseconds

The first of the 3 above calibrations was the reboot 97 days ago, on Jan 14, 2015 when we upgraded to RCG 2.9, then a restart/calibration by Kissel April 1 (mentioned above), then today's RCG 2.9.1 upgrade.

A couple of weeks ago Jeff and I turned off the damper on the BRS because high winds were activating the damper and sending impulses into the BRS. This morning I took Corey and Patrick down to show them how to stop/start the software and reposition the masses. The masses were out of position ~45 degrees, so I recentered them. We then went into the rack room and restarted the software and re-enabled the damper. Seems to be running fine now.

Detchar would like to request that the HWS cameras in the center building be turned off for a few minutes at a known time. We're trying to track down some glitches in PEM and ISI sensors that happen every second, and Robert suspects the HWS. Just a few minutes with them off, and then on again, would be fine; we don't need the IFO to be in any particular state, as long as the ISIs are running fine. We would need the precise times (UTC or GPS preferred), as the channels that record the camera state don't seem trustworthy (alog).

Betsy, Jamie

summary

We just tested a new guardian core version (r1438, with cdsutils r474) on SUS_SRM, to see if it gets rid of the "SPM DIFF flickering" issue that we've been seeing.  The new version does appear to fix the issue.  The test is now complete and the SUS_SRM guardian has been reverted back to the current nominal guardian core installation version (r1390, cdsutils r443).

backstory

We've been seeing a minor guardian issue since the last guardian upgrade with guardian nodes that have set point monitoring (SPM) enabled (ca_monitor = True).

When SPM is enabled, and set point differences are detected, the GRD SPM_CHANGED channel indicates the number of changed set points.  The current problem is that in the presence of set point differences, the node SPM is seemingly "flickering" between the "set point changes detected" state and the default "all clear" state.  This causes the SPM indicators on the GRD screens to flicker.

I was not able to reproduce this issue locally, but I reworked and cleaned up the SPM code path in the guardian core in the hopes that it would make the issue go away.  The issue was not see in the new version recently deployed at LLO, so the hope was that the issue was resolved.  This test was to see if the new release fixes the issue at LHO, which it does appear to do.

I will put in a change request for an upgrade to this latest version for H1.

Andy, Duncan

When looking for times when the HWS camera was on or off, I found that the minute trends indicated that it was off on Apr 18 6:30 UTC for ~27 minutes. But the second trends indicate that it was turned off 20 minutes later than that (and back on at the same time). The raw data (sampled at 16 Hz) indicates that the camera was never turned off.

This was originally found using data over NDS2, but Duncan has confirmed by using lalframe to read the frames directly. I've attached a plot below. The channels are H1:TCS-ITM{X,Y}_HWS_DALSACAMERASWITCH.

Added a feature to the IMC ASC model, so we can test a fast PZT servo to stabilize the input pointing. A new column or row has been added to the input and output matrices, respectively---with the output connected back to the input using a unit delay. The idea is that the WFS DC is feed through DOF4 to the PZTs with a high bandwidth servo. The current PZT output is fed to the new output matrix row, fed back to the new input matrix column and added to the fast PZT servo as an input offset. medm screens have not been updated.