J. Kissel #SundayInvestigations Rana and Gabriele had the IFO running when I came in, and we meandered through discussing "status of the IFO" topics. After seeing many glitches in the otherwise stable IFO, I realized we haven't turned on any offsets on the ETMX ESD stage that had on the ETMY ESD during O2 that helped avoid zero-crossings of the 18 bit DACs (mostly because the 20-bit DACs were recently on my brain; see ECR E1800306). Thus, while the IFO was locked and in nominal low noise, with a ramp time of 60 seconds, I turned on a +30000 ct offset (~1/4 the range) on each of the UL, LL, UR, and LR quadrants in the ESDOUTF banks (see attached screenshot). I haven't accepted this into the SDF system, because I have no idea if this a source of some of the glitches we're seeing. If we find we like it, we'll then keep it for real. This went in around 2018-10-28 20:00 UTC. The IFO had been locked and at low noise for ~30 minutes or so prior to this with no one doing anything. No one's touching it now (for at least 30 minutes), so this should be a good chunk of time to compare ON vs. OFF. It would be great if @DetChar would use some of their techniques for investigating and finding zero-crossing glitches to see if this was actually a problem, and if these offsets improved things.
I lost lock twice at LOWNOISE_ESD_ETMX with the verbal warnings "PRM; MC2" before losing lock. I turned off these offsets at around Oct 29 2018 02:06:41 UTC, we'll see if this helps relock. If we decide we still want these offsets (which we may not), we should probably switch them on in guardian after switching to the lownoise ESD. EDIT: After turning these off we were able to get to NOMINAL_LOW_NOISE.
Looks like the ligo.org auth system is not allowing logins right now. So we can't login to the control room workstations. The couple of stations that are already logged in continue to work, but we can't work on the interferometer like this.
For DCC/alog access, I'm able to use one of the backup servers. Hopefully, someone will read this and contact the authorities. Is there a way to redirect our control room workstations to use the backup for today?
Jonathan, Dave:
This unfortunate situation appears to have been the result of a series of unrelated problems. The ligo.org main server was down on Sunday (requiring the user to switch to backup servers for web based access) but this is not used for workstation authentication. The first workstation used (zotws11) had a login issue which was verified by Hung Yu this morning and resolved by him booting the machine. The second login attempt on zotws21 appears to have a typo in the login text.
It is still a mystery why zotws11 was refusing console logins over the weekend.
The CDS Overview MEDM on the wall display (video2) is stuck at 13:28 Saturday. Please do not use this to view the CDS status, and if someone could restart MEDM on this computer that would be greatly appreciated.
We've been noticing that the PRCL noise on the calibrated DRMI plot seems off at high frequency. I confirmed today that the cal filters in the OAF CAL DOF screen are all reasonable (ERR filters are flat, CTRL filters have HSTS M3 TF). The ERR filter gain was 3x too low: the CTRL/ERR xover was at 145 Hz instead of the actual UGF of 45 Hz, so I have added a factor of 3 to the filter.
The high frequency slope-up in PRCL is the contribution of frequency noise (above 1 kHz). Its also clear from this plot that the PRCL noise is now much higher than O1 in the 10-100 Hz band.
There is some high coherence with the SRCL length control as well as the PRC1_P WFS signals. These are not so surprising.
There is something that looks like a bandstop filter in PRC at 10-13 Hz; maybe this is a clue.
PRC length loop:
Re-measured the PRC loop shape today. In the LSC-PRC1 filter bank, I replaced the "velocity" filter with a SUS model TF so that we can estimate the loop shape w/ FOTON only (you can export the ascii TF from FOTON and import into DTT as a TF). In the attached Bode plot you can see the comparison; it is pretty good except for the phase. That is to be expected because we don't have all of the Anti-Aliasing filters included.
In the low noise state, the gain was ~3 dB too high, so I edited the ISC_LOCK script to reduce the gain to +11 in this step so the UGF = 45 Hz.
Audio:
The attached MP3 file was generated with ldv-web (https://ldvw.ligo.caltech.edu/ldvw/view). It allows us to easily get specgrams + filtered audio. For PRCL, I took the calibrated channel, bandpassed from 15-80 Hz, and then sped it up by 10x so its easy to hear with laptop speakers. The noise is breathing and popping, but I don't know what it is yet. There is a growl around 200 Hz which I think is really the 20 Hz gain peaking of MICH...
Tonight, some of the lock losses were preceeded by some 20 Hz ringing in the DRMI loops, apparent in the ringing in the excellent RLP2 LSC-BLRMS channel.
Seems like the MICH gain is slowly trending up during the locks such that it gets closer to instability and then finally saturates on of the HSTS (according to the talking box at the Operator station).
For now, I have added a line in LOWNOISE-LENGTH to reduce the gain from 2.8 to 1.7. The loop design just seems unsteady: there is a bounce mode bandstop at 17 Hz, a elliptic low pass at 35 Hz in the BS M3 stage, and a too-fancy low pass (LP80) that is making the gain sort of flat at 20 Hz.
Of course, there are many comments in the ISC_LOCK script about small gain steps. I wonder why the DRMI gains are increasing for the first 30 minutes after power up.
Last night, the microseism went up to ~1 um/s and we had trouble locking. This continued all day today. I redid some DRMI alignment (very small adjustments) and Sheila and I switched the SEI FF state a bit, and now things are locking well. WE have gotten to low noise many times tonight.
I was curious about what counts as 'high useism'. This plot (https://ldas-jobs.ligo.caltech.edu/~detchar/summary/day/20181027/sei/ground_blrms/) compares the BLRMS at the 2 sites. As you can see, LLO has no serious problems with ~1 um/s.
Reading many entries, there are several times in the LHO log over the past few years where people not how the locking is really poor when the useism is at 1-2 um/s. Sometimes this also includes wind.
It would be useful if someone in DetChar could crawl through the locking records over the past few years to find out how touch locking is as a function of seismicity. So that in addition to the word of mouth guidance, we also have a some kind of DHS threat level that incorporates ground, wind, etc.
Rana, TVo, Danny
ITMY HWS is currently not using the HWS power supply chassis but is instead plugged into a nearby outlet via an AC adapter. After the power supply swap there appear to be no peaks in DARM associated with the HWS sync frequency set to 54 Hz and then to 50 Hz.
the cheap AC/DC adapter we have installed is just temporary, but it seems to do the trick. Since this camera pollutes the rack power supplies so badly, we should be careful to not repeat this problem. I recommend getting higher quality AC/DC adapter for each of the Hartmann cameras (with long enough DC cords (>2 m) that the AC/DC block is not inside the optical table, and also ones that can fit in the table cable feedthroughs).
Yesterday we made excitations of the HAM1 HPI loops and saw coupling to DARM.
We saw that the horizontal motion is coupling to DARM more than the horizontal vertical motion. The first attached plot shows projections based on the HPI L4C of the HAM1 motion to PRCL, MICH, SRCL and DARM. Both X + Y directions contribute to the noise of our length degrees of freedom, the L4C motions in X and Y are partially coherent.
The coupling isn't through the CHARD loops, their control signals are not increased increased by the HPI injection (next two screenshots). A few weeks ago Hang made the CHARD blends more aggressive than they were in O2, 44352. That probably explains the difference from what Marie found 38184.
Did you mean to write that horizontal motion couples more than vertical ? Was the elevated DARM noise always coherent with the elevated HAM1 motion? If not (or even if yes), it might be worth repeating these tests, but bandlimiting the HAM1 HEPI excitations to be below 20 Hz, to see if there is any upconversion of HAM1 motion into higher frequencies in DARM. That's of interest because if we replace the HAM1 seismic with a HAM-ISI, it will give much lower platform motion below about 20 Hz, but higher motion above that (less passive filtering at higher frequencies).
Yes, I meant to say horizontal couples more than vertical.
Attached is a plot of coherences between the X L4C during the excitation and different signals. The L4C isn't very coherent with DARM, MICH, PRCL or SRCL, but it is coherent with PR2 ASC signals which are from the REFL WFS now.
TITLE: 10/26 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Commissioning
INCOMING OPERATOR: None
SHIFT SUMMARY: Commissioning continues. Wind gusts up to 30mph.
LOG:
1617 Jonathan to H2
1620 Terry to LVEA TO ISCT6
1630 Vanessa to LVEA
1645 Jonathan out
1712 Vanessa out of LVEA to Ends
1745 Jonathan nd Dave to H2
1846 Terry out
2004 Corey to LVEA to take a picture of a chamber
2007 Corey out
2045 Terry Back to ISCT6
2120 Danny to LVEA to check on Hartmann table grounding
2134 TVo Georgia to LVEA to check on RH rack
2151 TVo, Georgia, Danny, Terry out
2208 Jeff B to Opt Lab and LVEA
2221 Jeff B out
This morning I measured and fit again the SRCL feed-forward. Measurements attached.
The new feed-forward filter is effective in reducing the SRCL coupling (indeed the old feed-forward was amplifying the SRCL noise in DARM between 10 and 20 Hz) (figure 2)
To improve the feed-forward between 10 and 30 Hz I designed a new AC coupling, with less phase rotation at 10 Hz, and with a band-stop between 0.8 and 1.2 Hz. Using this AC coupling I could fit a good filter down to 10 Hz (figure 1).
The band-stop reduced the feed-forward output at around 1 Hz well below the previous version (figure 4).
Unfortunately, the new feed-forward triggered again an instability that was slowly growing in many ASC signals. This time the instability is not at 1.1 Hz (the BS pitch mode) but at 1.2 Hz, which is exactly the corner frequency of the band-stop. So it seems this instability is not due to a too large control signal exciting a mode, but due to a spurious unstable loop. This makes it much harder to understand whether a given feed-forward is stable or not.
Coherence between DARM and ths ISS second loop out-of-loop sensor is at the 1e-2 level (below a few kHz). Similar coherence with the first loop sensors.
The IMC_LOCK guardian was missing a path from ISS_ON or ISS_DC_COUPLED to LOCKED that opened the ISS second loop.
Now it's been added. The ISC_LOCK DOWN state should request IMC_LOCK to got to LOCKED, and thus open the ISS second loop.
Hang, TVo
Hang had most of the tools written up to run the spectrograms using jupyter notebook so i helped tune the notch filtering to get the normalized spectrogram of CAL-DELTAL_EXTERNAL_DQ
The first plot shows broadband DARM, and the second plot shows a zoomed in version from 30 to 200 Hz. By eye, there's not an obvious correlation between the amplitude square wave of DB9 modulation depth to DARM, except there's some extra noise at the 57-58 Hz region which might come from the Hartmann sensor sync frequency. The third plot shows a zoomed-in time version during one of the glitchy times, there's an obvious broadband spike but there is also a long lasting one at the 57-58 Hz that lasts for a few minutes.
I have modified a large quantity of models today, as part of ECR E1800304 / FRS 11676. The goal is to provide a front-end way to shut off ISC-related outputs when we have a lockloss, even if some of the EPICs connections are failing (which causes guardian to not be able to execute the full DOWN reset state).
The trigger of lock or not-lock is generated using a new row of the LSC trigger matrix. That trigger is passed to all of our main IFO suspensions, as well as the ASC and OMC models. Everywhere the trigger is used, it goes through a ramping code written by JoeB some time ago, so that signals can either be ramped to zero or immediately set to zero (by setting the ramp time to 0 seconds, as usual). Each of these trigger blocks also has an Enable switch, so that we can chose to bypass use of the trigger for any particular set of outputs (eg, if we want to be triggering the output of the LSC model but not the ASC model, we'd bypass the trigger on the ASC model).
To enable more flexibility, there are many different locations where the trigger can be used or bypassed. Some of these may seem somewhat redundant, but I wanted to give each site flexibility and also the ability to disable either inputs or outputs of the suspension filter banks. I list the groups here. For each group, there is only one trigger / ramp that controls them all. All of these channels should be initialized properly with their ENABLEs set to 0 by default, which should give no net effect when we first install them, so that we can decide which ones we want to utilize. Each of these also has a monitor channel _IS_RAMPING.
Note in particular the things that I have not given triggers to: the IMC mirrors, or the IMC ASC dof outputs, since we want those to be active separately, and don't want a lockloss to kick the IMC out of lock if it wasn't already going to be. I also did not give triggers to the ADS dithers. I can add these if we think it's important. I also did not include any squeezer-related optics, since that is a pretty independent system. We can give the squeezer suspensions their own trigger if its needed.
I have not yet made any screen modifications (that will be tomorrow, hopefully).
When we are ready to implement this, we will need to restart:
For LLO, we will need to svn-up to get all of the modifications to the suspension library parts, then add the IPC receiver to the top model for each sus. We will also need to hand copy the modifications to the LSC, ASC and OMC models.
EDIT: I have compiled all of the models, but not installed. h1asc has an error, which I will work on debugging in the morning.
Found and fixed the problem with h1asc - a few of my new multiply blocks I had forgotten to connect. Oooops. It compiles nicely now.
Also, I ran make install-h1lsc so that it would generate the lsc trigger matrix for me, but I have not installed any of the other models.
I have now made some screen modifications, enough that we should be able to roll this out on Tuesday.
I've added a row to the LSC trigger matrix, and also from the LSC trigger matrix screen you can access the new lockloss trigger screen.
After consulting with Rana and JeffK, I've moved the triggers in the suspension models to after the drivealign matrices, rather than just after the lock filter banks. The violin triggers remain where they were. No channel names will change as a result.
I have not done a make on any of the models - they should all get one (including LSC, ASC, and OMC) before installing on Tues.
It looks like there is a problem with sending the TRIG_IFO signal to the end stations. We will most probably hold off on the upgrade until 6th November, so for now I've backed out the "make install-h1lsc" to revert the DAQ INI file.
For some unknowable reason, even though the end stations are now on PCIE dolphin, the send / receive parts in the models need to be the old RFM parts. This should be changed, so that the RFM parts have some name that is sensible, like PCIE_ENDS or something.
Anyhow, I have undone all of Dave's temp changes from last week to the LSC model and the SUS common parts. I added a 3rd sender to the LSC model, changed the EY model to use the mis-named RFM block, and and put in a mis-named RFM block into the new susetmx that has the 20 bit DAC.
I realized today that our updates to the inverse sensing FOTON filter would yield slightly incorrect DARM FOM curve in DTT, especially in the ~5 kHz range. This is because the DARM FOM DTT template has a calibration that corrects for "warping" from a correct inverse sensing filter compared to what we are able to install in the front end using an IIR filter. I updated the H1_DARM_FOM.xml file with this new calibration and committed the changes to the SVN. This curve was computed using the following script located in the calibration SVN: aligocalibration/trunk/Runs/O3/H1/Scripts/darm_FOM_calibration.py Attached is a figure showing comparison with the old curve (with slightly incorrect calibration) to the new curve (with correct calibration). Also attached is the current DARM FOM DTT template calibration transfer function that is now being applied to H1:CAL_DELTAL_EXTERNAL_DQ (columns are frequency, magnitude (dB), phase (deg.)).
Upon request, I've removed the effect of analog and digital anti-aliasing filters from the DARM FOM. I have committed this to the SVN repo.
There's a comb visible in recent DARM data at multiples of 0.996795 Hz (last digit uncertain). This is suspiciously similar to a comb seen before O2 (then measured at 0.996798 Hz), which was due to the HWS. Just as before, the comb appears strongly in corner station magnetometer channels.
I've attached an image showing teeth of the comb below 100 Hz (Oct 24, ~50 minutes starting at 9:30 UTC). It is also visible in other lock stretches within the past few weeks. When this comb was seen before, it coupled only intermittently to DARM; its presence seems steadier now.
Relevant past alogs:
Hey Ansel,
The Hartmann camera sync frequency is set to 1Hz for both cameras at the vertex.
Hi Daniel, Right, some of us were wondering in an email thread if that frequency could be set to something different for a period of time, ideally with DARM in low noise, but that's not essential for things to be seen in magnetometers. thanks, Keith
Keith and Ansel,
I adjusted the ITMY and ITMX HWS sync frequency to 55 Hz from gps 1224538388 to 1224539253. Please don't hesitate to let me know if you need anything else related.
Yes, this couples very strongly into DARM. A line at 54.8ish Hz appears with the change (spectrum attached).
Hi Daniel - yes, that definitely had an effect. I checked both DARM, and also the magnetometer channel where the comb was showing up most strongly. The comb disappears in both channels during the time segment you indicated, and reappears afterward.
Indeed this is a known problem with the HWS cameras (FRS 4559). And the problem was solved by placing the cameras on alternate power supplies in 2016. I suspect that following the post-O2 disconnecting and reconnecting of the HWS table and cables, the cameras got plugged into the chassis power supply again. We should do the following for all 8 HWS:
This should be done for the ETMs as well (where the 57Hz is an issue). The original isolation fix does not appear to be robust enough.
Block diagrams (DCC is down so I can't get hyperlinks right now)
Danny, TJ, Georgia, TVo
Big ups to Sheila for running our measurement after ISC was finished with their work for the night.
Yesterday Georgia and Danny put in an iris on ITMX to crop out a prompt reflected beam from an in-chamber lens so that we can try to compare the spherical power on both ITMX and ITMY HWS when injecting 6 Watts of power into the ring heaters (3 top, 3 bottom):


Note that there is still a bit of clipping on ITMX on the top right corner. Using the results of a COMSOL model here, where it quotes

| Model Prediction | ![]() |
| HWS ITMX Measured | ![]() |
| HWS ITMY Measured | ![]() |
From yesterday:
Danny installed an iris on the ITMY HWS path directly before the HWS camera.
This iris blocks a problematic stray beam that appears to be reflected off a surface between the viewport and the SR3 baffle.
Attaching screenshots of the camera image (with hartmann plate removed) before (attachment 1) and after (attachment 2) iris install. We were a bit concerned about the new fringes and any noise they might introduce. Note the two screenshots were taken with different lighting conditions (table door open/closed) so the intensity difference is not a concern.
For reference, here is how the ITMX HWS return beam looked back in 2014 when everything was first installed.
2014 versus 2018
And here's a view with the 2014 and 2018 beams overlapped. Roughly 50% of the HWS probe beam is clipped.

I think I've tracked down the source of the problem with the HWSX probe beam clipping. The issue stems from the fact that the new HWSX STEER M1 optical mount required the base to be moved. This was known and we aimed to keep the optic face in the same location. However, in placing the new optical mount, the wrong face has been kept in the same location - resulting in a displaced front surface.
We aligned the in-vacuum optics assuming the front surface had not moved(aLOG 39053). I'll need to investigate further to trace out the beams but this is almost certainly the cause of the clipping we're seeing.
The attached images show an overlay of two photos of the HWSX STEER M1 optic in 2014 (aLOG 12615) and 2018 (aLOG 39071)
FRS issue (https://services.ligo-la.caltech.edu/FRS/show_bug.cgi?id=11691)

After looking closely at the in-chamber photos, I've tried to estimate what the optical axis is doing. It should move in the -Y direction by ~7-10mm in the Hartmann Scraper Baffle.

If that's the case, then the beam size (at one beam radius) going through the aperture will look something like the following. The red beam is getting close to the edge of the aperture.

TJ took some photos of the in-vacuum optics with his phone and we can see relatively well along the optical axis of the HWS (although not with enough resolution to see the scraper baffle).

We'd like to try to do this with the chamber illuminated and a good SLR camera that is placed in the optical axis and focussed at the same distance as the scraper baffle. We should see a series of concentric circles and ellipses that are the apertures of all the optics and baffles. If, as I suspect, the HWS scraper baffle is now off center relative to the beam, it should be visible as such in this image.