Tried to do some FSS related measurements to see why it was the FSS kept losing lock.
Unfortunately it seems the FSS fieldbox signals of interest - which is just about all
of them as far as transfer function measurements are concerned - are not of any use
in their current state.

    In doing a measurement this morning, the injection locking was broken which in
turn tripped the high power oscillator's power watch dog.  The laser restarted without
any issue.

The missing equipment hunt concluded with limited success. There are still a few items missing. 
Fred, Vern, and Richard thanked all for the efforts. 

PSL – Looking into an instability with the FSS.
CDS – Working on frame writer problem
VAC – Will need to bake out the End-Y RGA before O2. 
All other subsystems report no problem.

There will be no maintenance window on Thursday.
There will be a safety meeting this afternoon at 15:00.

   The ITM-X ISI Stage 1 and Stage 2 WDs tripped at 12:35:25 and 12:35:30 (UTC) respectively. No problem resetting. Was not highlighted on Ops Overview MEDM. 

painful - we need to make the relocking robust

To do next: TCS tuning (diff?) see whether the new noise depends on it.

Updated the guardian script to reach low noise in both 25W and 40W mode.

Attached is a DCPD cross-power spectrum at 40W (references) and at 25W (live data). Red is the PD_A-PD_B cross power. Blue is the DCPD_SUM power spectrum cast in the same units to give a reference of where shot noise lies.

Note: the excess noise we currently see is also present at 25W. It is almost identical to the 40W noise up to 200Hz. However, the notch structure between 200Hz and 700Hz is different at different power levels - no idea why.

Plot 2 shows the MICH coherence with DARM at 25W - the MICH-correction is not fine-tuned for that power level, but it is not the dominant contribution.

Guardian changes:
ISS_ON: based on IMC-PWR_IN_OUT16, set 25W or 40W offset.
        # crude way to make it work at 25W and 40W (no intermediate!)
        if ezca['IMC-PWR_IN_OUT16'] > 30:
            ezca['PSL-ISS_SECONDLOOP_REF_SIGNAL_ANA']= -0.9326934814453125  # 40W
        else:
            ezca['PSL-ISS_SECONDLOOP_REF_SIGNAL_ANA']= -0.588 # 25W
 

LOWNOISE_ASC: The state used to only run >35W. Changed it to always run, except for the 40W compensation filters on CHARD_Y and DHARD_Y (FM6).

Executive summary: 

* Good news - as expected, the 16-Hz comb due to the OMC length dither is gone (at least at this sensitivity level)
* Bad news - low-frequency 1-Hz combs remain, and some new low-frequency combs & lines have appeared 

Some details:

 I initially looked at 15 hours of 30-minute FScan DELTAL_EXTERNAL SFTs (30 SFTs) generated during ER9 and was aghast at how bad the low-frequency spectrum looked, with a pervasive 0.485308-Hz comb ranging up to its 446th harmonic at 216.4 Hz, but when I exclude the three hours of SFTs when the 2-second ALS- glitches were present, things don't look quite so bad (see figure 1 for a sample without removal and figure 4 with removal). I dewhiten according to the new 6-pole / 6-zero, 0.3 / 30 Hz algorithm.
 The infamous 16-Hz comb due to the OMC length dither tracked down and killed here is gone (at least at this sensitivity level and these SFT statistics). As a result the high-frequency band (up to 2 kHz) is remarkably smooth with only violin modes and sporadic isolated artifacts.
 The 1-Hz comb with 0.5-Hz offset remains pervasive, but other prior 1-Hz or near-1-Hz combs with different offsets (e.g., 0.25 Hz) are not strong.
 On the other hand, there is a new near-1-Hz comb (0.996798-Hz spacing) visible to its 204th harmonic at ~203.3 Hz.
 There is also a new near-2-Hz comb (1.999951-Hz spacing) visible on approximate odd-integer-Hz frequencies, starting from ~9 Hz and visible up to ~175 Hz. This is likely the same 2-Hz comb reported in May by Bryn Pearlstone (which Ansel Neunzert kindly reminded me about today). 
 There is a "new" 56.8406-Hz comb visible to its 11th harmonic at ~625.25 Hz, which in hindsight I can see was buried in the O1 spectrum (I overlooked the pattern and indicated the teeth as isolated lines). This time the pattern was strong enough to jump out at me and to jog my memory that this comb was seen in  H2 1-arm data in 2012 in both the arm feedback channel and a quiet sensor-noise-dominated OSEM channel. This seemed to indicate a DAQ system problem at the time. I can see from O1 NoEMi line lists that this comb is pervasive in ISI, SUS and PEM channels at the corner station and both end stations.
 The old "K" comb-on-comb (0.088425-Hz fine comb attached to teeth of a coarse 76.3235-Hz comb) has more 11 more fine teeth visible on the lowest-frequency coarse-tooth comb.
 The old calibration line at 35.9 Hz has been moved to 35.3 Hz. The new excitation lines at 33.7 and 34.7 Hz are easily visible, but not as strong as the primary calibration lines. I found these changes documented here.
 There are sporadic new isolated lines here and there (indicated in line list - see below)
 There is a "crab-killer" broad bump centered at about 58.6 Hz which degrades sensitivity in the Crab Pulsar band of interest (~59.3 Hz). On the other hand, the whole noise floor is elevated w.r.t. O1, anyway. So it may be premature to worry about the bump.


Figure 1 - spectrum for 50-100 Hz when the 3-hour 2-second-glitches stretch is included (~0.5 Hz lines marked with 'h' for 'half')
Figure 2 - 0-2000 Hz (removing 3-hour bad stretch from here on)
Figure 3 - 20-50 Hz sub-band 
Figure 4 - 50-100 Hz sub-band 
Figure 5 - 100-200 Hz sub-band
Figure 6 - 1300-1400 Hz sub-band (illustration of how clean the high-frequency band is)

Also attached are a larger set of zipped sub-band spectra and a lines list (excluding the bad 3-hour stretch). Note that because the statistics here are two orders of magnitude smaller than used for the full O1 run report, I am not yet removing lines seen then that may yet re-emerge with more accumulated post-O1 data. So many of the line labels in these figures are buried in the noise fuzz for now.

Line label codes in figures:

b - Bounce mode (quad suspension)
r - Roll mode (quad suspension)
Q - Quad violin mode and harmonics
B - Beam splitter violin mode and harmonics
C - Calibration lines
M - Power mains (60 HZ)
O - 1-Hz comb (0.5-Hz offset)
o - weaker 1-Hz and near-1-Hz combs (various offsets, including zero)
H - 99.9989-Hz comb
J - 31.4127 and 31.4149-Hz combs
K - 0.088425-Hz comb-on-comb
t - 1.999951-Hz, 2.07412-Hz and 2.07423-Hz combs
D - 56.8406-Hz comb
x - single line (not all singlets in the vicinity of quad violin modes are marked, given the known upconversion)

Here is a DCPD x-power spectrum (including a self-x-power for SUM, calibrated the same way). Note that the x-power has structure around few hunder Hz, including a humpo at 800Hz.

Jenne, Sheila, Stefan

We used two independent methods to verify that the beam spot positions do not move during the power increase:

Jenne and Sheila put dither lines on all optics. While they have not calibrated the output, the result is that during the power increase they did not observe any motion of the beam, but during the soft offset adjustment just afterwards they did see the spots move.

I used Kiwamu's cameras to make time lapse movies of the power-up (the first two gifs). No spot motion is observed. I also atteched two movies (gif 3 and 4) of the motion during the soft offset adjustment. The spots can be seen wiggling.

FInally, I attached a Striptool chart of the beam positions during power-up. The power increase starts at -30 minutes, and finishes at -22minutes. Then the soft loops start moving until about -16 minutes. At -12 minutes Jenne took the soft offsets out again.

Nutsinee, Kiwamu,

WP5990

We have (re-) set up the polarization monitors on the HWS table by HAM4. We have confirmed that they are functional. For those who are interested in the polarization data, here are the channels to look at:

• H1:TCS-IFO_POLZ_S_HWS_OUT
• H1:TCS-IFO_POLZ_P_HWS_OUT

In theory, they should be in unit of watts as measured at the HWS table.

[Installation notes]

This time, we have newly installed a short pass optic (DMSP950L from Thorlabs) to pick off the main interferometer beam without getting too much contamination from either the SLED light (790 nm) or the ALS beam (532 nm). The short pass mirror was inserted between the bottom periscope mirror and the first iris (D1400252-v1). Looking at the green light at the table from the end stations, we learned that the beam size is already pretty small and (visually) small enough for the beams to fit into the PDA50Bs without a lens. So we decided to go without lenses as opposed to the previous setup (24046).

The short pass mirror reflects the interferometer beam toward the left on D1400252. We placed a PBS (CM1-PBS25-1064-HP) on the left side of the short pass and placed the PDA50Bs. The power reflectivity of the newly installedshort pass mirror was measured to be 5% +/-3% for 532 nm. The absolute power (assuming the Nova hand held power meter is accurate) of the reflected green light was measured to be 1 uW.

One thing we leaned today was that the green light is not so trustable to get the optimum alignment. We first aligned the optics with the green light and then noticed that the infrared beams were almost falling off of the PDA80Bs. So we then closed the shutters and aligned them with the actual infrared beam.

The manual gain settings are:

• S_POL: 70 dB
• P_POL: 50 dB

The digital gains were also changed accordingly so that the calibration of these channels should be accurate.

J. Kissel, S. Dwyer, S. Ballmer

We continue to have trouble with the FSS oscillating after a lock loss, in that it'll often either take several minutes to relax, or it requires manual intervention such as briefly reducing the common gain of the FSS loop. As such, Sheila took a look at the IMC PDH loop to look for problems and instabilities there. I looked over her shoulder at her results, and saw some areas for improvement in the loop design. The current loop design has an UGF at 66 [kHz], with a phase margin of 68 [deg]. However the gain margin around ~200 [kHz] is pretty dismal because of what looks to be some icky features in the physical plant. These features have been shown to be directly influenced by the FSS common gain (see second attachment in LHO aLOG 28183).

I figure, given that we've got oodles of phase margin, what harm could be done by just adding a simple 200 [kHz] pole in loop, and reducing the gain by ~2 [dB]? As such I took Sheila's data, which lives here 
/opt/rtcds/userapps/release/isc/common/scripts/netgpib/netgpibdata/TFAG4395A_12-07-2016_163422.txt
(also attached) and added these modifications offline as a design study. 

In the attached plots, I compare the as-measured IMC PDH Open Loop Gain, G, Loop Suppression, (1/1+G), and the Closed Loop Gain, (G/1+G), against one modified as described above (blue is as measured, and green is the modified design study). 

The results are encouraging: a still-substantial UGF of 47 [kHz], and a very-healthy phase margin of 58 [deg]. However, as can bee seen in the loop suppression and the closed loop gain, there is far less gain-peaking and/or a much great gain margin and we would no longer have to worry about the icky features in the plant that are so sensitive to the FSS common gain.

Where to stick such an analog filter? It's of course dubious to claim that the MEDM screen for such a system is representative of the analog electronics, but assuming it is, one can see that there is the possibility of a switchable daughter board in the FAST path that gets shipped off to the PSL AOM for the FSS. Because it's switchable, we can toss whatever simple filter in there that we like, and then compare and contrast the performance for ~1 week to see if it improves the stability problems we've been having.

What impact would this have on the full IFO's CARM loop? I'll remind you of Evan's loop analysis of the whole frequency stabilization spaghetti monster in LHO aLOG 22188. There he suggests that the CARM UGF is around 17 kHz, so as long as the Closed Loop Gain around there is the same, then this change in the IMC PDH loop should have little impact [[I just made this sentence up based on just a few words from Sheila who asked me to look at the CLG. I'm not confident of its truth. Experts should chime in here]]. Indeed the third .pdf attachment shows that G/(1+G) of the IMC PDH loop, regardless of modification remains unity out to 100 [kHz].

J. Kissel

In the rush and panic of ER9 last week, we neglected to aLOG that we reverted the ETM ESD bias flip that I'd done last Tuesday because -- for some unknown reason -- the sign flip *this* time caused problems for ALS DIFF and subsequently switching to EY later in the acquisition sequence. As such, we didn't gain as much in this week's charge mitigation, as shown by the plots attached below, which includes a new measurement from today. The effective bias voltage is still within the ~10 [V] range, so the reversion did little harm.

Since ER9, we haven't had the time to explore why the flipping failed us. I'll continue to try to push for figuring this out, but we'll see what kind of priority it gets with respect to all other commissioning tasks. 

The mic in the EY electronics bay had been reported by Vinny Roma as having unusually low signal, requiring a calibration factor 10 times larger than the other microphones.  I determined that the fault was with the signal conditioning box - when plugged in to a different channel, the microphone performed normally.  Additionally, I verified that the power supply was delivering the proper voltage to the signal conditioning box.  Using the second channel in the signal conditioning box also did not improve the signal.  We should plan to replace the signal conditioning box next week.

asc filters

The h1asc filters which have been pending install since before ER9 were loaded.

framewriter testing

Jim, Dave

we continued testing of h1fw0 to investigate its instability. The daqd code was rebuilt against a later version of framecpp (2.4.2 instead of 1.19.32). When fw0 was instructed to write all frame types, it was as unstable with the older framecpp. The size of the science frames between the two writers at this point was different, with fw0's frame 2 bytes smaller than fw1's frame. We suspect the framecpp version string is encoded in the header, "2.4.2" is two chars shorter than "1.19.32". To get the frames the same size, we reverted fw0 back to framecpp 1.19.32.

The next test was to install a local 1TB 7200rpm SATA hard disk drive on h1fw0 and build an ext4 file system on it. The daqd process was directed to write to the local disk instead of the NFS mounted SATABOY. The disk access immediately sped up by many seconds. When we got the writer to write all four types of files, very surprisingly the frame writer still crashed, but took longer to do so. Writing all frames to the NFS-QFS crashed in less than 5 minutes, with local ext4 it crashing in 9 to 10 minutes. The crash time is not related to writing second trends, we saw several successful trend file writes over the several tests we ran. Monitoring the threads showed that with local disk the thread did not go into the "D" diskIO wait state. Another surprise was with local disk writing we did see a retransmission request about 40 seconds prior to the slew of requests at the time of crash.

We reconfigured fw0 backj to writing to ldas-h1-frames so they could be used to back-fill fw1's gaps.

As a test we had fw0 write commissioning frames only rather than science frames only. The data loading goes up from 0.9GB to 1.7GB per 64 second time period. fw0 is stable in this configuration. It can handle either 64 second frame, just not both.

We are leaving fw0 writing science frames overnight.

Y sled has been replaced recently and now that X sled is getting very dim, I decided to replace X sled as well. This way it's easier to keep track of when the sleds got replaced (this time both were replaced within one week). The old sled number is 12.05.21 replaced with 07.14.256.

Kiwamu, Stefan

Yesterday Kiwamu realigned the red IR cameras. I reset the centroid setting today. The configuration files are:

ITMX:

[Camera Settings]
Camera Name = H1 ITMX (h1cam21)
maxX = 659
maxY = 494
Exposure = 100000
Analog Gain = 1023
Auto Exposure Minimum = 150
Name Overlay = True
Time Overlay = True
Calculation Overlay = True
Do Calculations = True
Calculation Mask = Circle
Circle Mask X = 373
Circle Mask Y = 150
Circle Mask Radius = 250
Calculation Subtraction File = None
Auto Exposure = False
Calculation Noise Floor = 25
Snapshot Directory Path = /ligo/data/camera
Frame Type = Mono12
Number of Snapshots = 1
Archive Image Minute Interval = 0
Archive Image Directory = /ligo/data/camera/archive/

[No Reload Camera Settings]
Base Channel Name = H1:VID-CAM21
Camera IP = 10.106.0.41
Multicast Group = 239.192.106.41
Multicast Port = 5004
Height = 480
Width = 640
X = 0
Y = 0
 

ITMY:

[Camera Settings]
Camera Name = H1 ITMY (h1cam23)
maxX = 659
maxY = 494
Exposure = 100000
Analog Gain = 1023
Auto Exposure Minimum = 150
Name Overlay = True
Time Overlay = True
Calculation Overlay = True
Do Calculations = True
Calculation Mask = Circle
Circle Mask X = 295
Circle Mask Y = 220
Circle Mask Radius = 250
Calculation Subtraction File = None
Auto Exposure = False
Calculation Noise Floor = 25
Snapshot Directory Path = /ligo/data/camera
Frame Type = Mono12
Number of Snapshots = 1
Archive Image Minute Interval = 0
Archive Image Directory = /ligo/data/camera/archive/

[No Reload Camera Settings]
Base Channel Name = H1:VID-CAM23
Camera IP = 10.106.0.43
Multicast Group = 239.192.106.43
Multicast Port = 5004
Height = 480
Width = 640
X = 0
Y = 0