Control room crew including Keita Patrick Sheila Jenne Robert

As Betsy pointed out last night, the top mass osems in the corner station show that things have been sagging recently. The first screenshot attached is a 300 day trend of the same top mass osems that Betsy plotted.  

Patrick noticed that twice this morning when we lost lock, ITMY had a large alignment shift, although none of the control signals going to the suspension changed. (PDF attached) This probably explains why operators had to keep repeating initial alingment yesterday. 

Keita suggested that we look at the top mass osem spectra with no damping on, and indeed the spectra of ITMY are different from the other quads, which can be seen in the third attachment. We then added an offset to the top mass vertical osem, which moved it up by about 20 um (so that the top mass osem is back near 50um where it was in December).  The after the move the undamped spectra are much more similar to the other quads (third attachment shows the comparison to ETMY of ITMY with and without the offset, for V only, 4th png shows all DOFs)

We are now proceeding with inital alingment one more time, and will attempt to lock.  

The times are: all 4 quads undamped at 21:24:04 UTC Jan 16th 2017 to 21:43:00.  Offset added to ITMY vertical at 21:47:15 UTC.  

I don't think this is a serious problem at this time, but the 3rd knob (the one closest to the fibers) on the ALS fiber polarization controller box in the MSR, or the motor associated with it, seems to be a little loose, at least when controlling the Xarm. 

I didn't check with the Yarm since it's polarization didn't need adjusting, but when spinning the 3rd knob, the degrees read back on the front panel weren't changing as expected.  I was monotonically turning the knob in one direction, but the readback degrees were bouncing between 2 or 3 values.  When I pulled gently on the knob while turning to give it some more friction, the readback degrees seemed more likely to change as expected - at least I was able to change them at all, rather than just bouncing between 2 values. 

Anyhow, I'm not sure that action is required at this time, but did want to document that this is happening, so that if others see it or it gets worse, we know that it's been happening.  If we have a spare, we may consider swapping it out, but it is not currently an urgent problem.

Starting CP3 fill. LLCV enabled. LLCV set to manual control. LLCV set to 50% open. Fill completed in 420 seconds. LLCV set back to 14.0% open.
Starting CP4 fill. LLCV enabled. LLCV set to manual control. LLCV set to 70% open. Fill completed in 60 seconds. TC A did not register fill. LLCV set back to 34.0% open.

Patrick almost has us initially aligned and ready to start the acquisition sequence, so we have not yet started to look into the problem at LownoiseASC.  But, we have found and resolved the a problem that operators have been having with initial alignment over the last day. 

The TMS and ITM ditherAlign script had been run yesterday for the Yarm, to help find the green beam.  I don't know at this time why that was necessary, but it left some incorrect offsets in ITMY that made initial alignment harder. 

We use the same M0_TEST offset for the dither align script as we do for the MISALIGN suspension guardian state.  The dither align script needs smaller values in there than the usual misaligned location, so it writes the values it needs.  However, it wasn't putting the usual offset values back after the script finished, so when trying to lock the Xarm-only in IR during initial alignment, ITMY was only partially misaligned and there were fringes at the AS port causing confusion and poor error signals. 

I have reverted the M0_TEST offsets to the values in SDF, and have modified the ditherAlign script such that it reads the original offset value, and puts it back upon completion of the script so that this doesn't happen again.

ISC_LOCK at DOWN and observatory mode in corrective maintenance upon arrival. The HAM6 ISI is tripped. I will attempt to lock but given the alogs from last night it sounds like I may need help from a commissioner.

Running 'ops_auto_alog -t Day' reported an error:

patrick.thomas@operator0:~$ ops_auto_alog.py -t Day
Traceback (most recent call last):
  File "/opt/rtcds/userapps/release/cds/h1/scripts/ops_auto_alog.py", line 300, in 
    alog.main(Transition,shift)
  File "/opt/rtcds/userapps/release/cds/h1/scripts/ops_auto_alog.py", line 218, in main
    operator = self.get_oper_w_date('{day}-{month_name}'.format(day=lday, month_name=self.all_months[lmonth]), 'Owl')
  File "/opt/rtcds/userapps/release/cds/h1/scripts/ops_auto_alog.py", line 130, in get_oper_w_date
    date_ln = self.get_date_linum(date) - 1
TypeError: unsupported operand type(s) for -: 'NoneType' and 'int'

TITLE: 01/16 Eve Shift: 00:00-08:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Corrective Maintenance (this is the best fit I could find for this state in the Observatory Mode Wiki)
INCOMING OPERATOR: Patrick
SHIFT SUMMARY: Nutsinee stayed to see through the IA that she started at the end of her shift.  As has been the case for the past several shifts, all seemed well until LOWNOISE_ASC when we noticed that the ASC-MICH_Y and ASC-DHARD_Y error signals seemed to make a jump and start oscillating.  The beam spot on ASAIR could also be seen ringing up in yaw until the lock broke.  It seems that this could be indicative of an ASC tuning issue.
LOG:  I am abandoning ship and leaving the IFO in DOWN for the remainder of Owl shift.  Keita has given an email OK (with instructions NOT to call anyone tonight) and  Corey has been CC'ed.  Consider this a Mayday call to Commissioners.

NOTE: According to the Observatory Mode Wiki, the Corrective Maintenance state that I am leaving the IFO in is supposed to be accompanied by an FRS ticket.  Since we won't know the cause of the issue until it is fixed, I am asking that Commissioners working on the problem file and close out an appropriate FRS ticket to account for the downtime.  Please let me know if Operators are simply supposed to enter an FRS ticket saying something vague like "IFO won't lock" and we can do that in the future, but for now that doesn't seem very useful to me.  Correct me if I'm wrong.

TITLE: 01/15 Eve Shift: 00:00-08:00 UTC (16:00-00:00 PST), all times posted in UTC

STATE of H1: Lock Acquisition

INCOMING OPERATOR: Travis

SHIFT SUMMARY: See alog 33317 and comments therein. After two locklosses at Lownoise ASC I started an initial alignment from scratch (as suggested by Sheila) by putting all the optics except RMs and OMs back to where they were last night (according to oplevs) when we had somewhat good range. My undertainty was less than 0.1 um yet the green flashes were terrible. I spent quite some time maximizing the green arm but after repeated failure when WFS came on I realized that PR3 was also terrible. I adjusted PR3, reworked the arms, then wrapped up the initial alignment and handed the IFO over to Travis.

I also played whack-a-mole with second harmonics violin modes tonight. But not sure if they will come back high after several locklosses.

I happen to be here with the ops tonight for a bit and answered a call from Dan M. since they were busy wollowing in alignment of the IFO:  the LDAS room is overheating again, reaching 94 deg F, although no breakers have tripped.  He has remotely turned off unneccessary computers and instructed me to open the doors at each end of the room in hopes that circulation from the main VPW area (64deg F) would help cool the room.  Indeed, the room was warm - the temp controller on the wall reported being 94 deg F with the request at 67 deg F.  No errors, blue light on at the bottom...

We'll leave the doors open until instructed otherwise - maybe tomorrow.

Temperature sensors for the HVAC system tend to be on the walls. So when it is really cold outside, I think the temperature at the chambers tends to be higher than usual because the system is trying to maintain temperature near the walls, which, because of the higher gradient, tend to be colder relative to the room than usual. Figures 1 and 2 show a moderately good correlation between range/inability to lock and BSC3 temperature (BSC1 is not that different). The trouble really seems to start when BSC3 warms up above 9810 counts. It seems like a long shot, but it might be worth turning the zone 1A temperature set point down, and perhaps others as well, when it is cold, so that the temperature is not so hot away from the walls. Figure 3 shows that BSC3 tends to be warm when the outside temperature is cold. Eventually it might be good to control the HVAC with sensors near the chambers and not on the walls.

Quick Summary: ALSY spot have fringes. ALSX can't be maximized. COMM beatnote wouldn't go beyond 5.6. TMS optics have been moved a lot before I got here. I'm putting the optics back to where they were 19 hours ago when we had th ebest range last night. Stay tuned.

While we were out of observing I did a few quick things to try to make our locklosses a little gentler.  (part of WP 6391)

First, I set limits on the TR QPD pit and yaw signals of 2, rather than 100 which were the previous limits.  These shouldn't go above 1 if we are locked, but can become large after a lockloss and send large signals to the suspensions through the soft loops.  I accepted the new values in SDF so we could go back to observe.

I did a little bit of work on the down state of the ISC_LOCK guardian, and loaded some of the changes.  

• One problem was that it was setting TRAMPS, GAIN and then clearing history for the same filter bank in consecutive lines, which we know doesn't necessarily happen in order, so this might set the gain before setting the tramp, and certaintly will clear the history before ramping the gain down.  
• There were some things that were done in both the ISC_LOCK guardian and in ISC_DRMI, (handling of DRMI ASC) which I've removed from ISC_LOCK.  
• moved the things that are importatnt to the begining of the state, so they will happen faster.  

We can see if this makes things better for the next lockloss.