TITLE: 06/06 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Observing at 111Mpc
OUTGOING OPERATOR: Corey
CURRENT ENVIRONMENT:
Wind: 16mph Gusts, 13mph 5min avg
Primary useism: 0.06 μm/s
Secondary useism: 0.12 μm/s
QUICK SUMMARY:
TITLE: 06/06 Owl Shift: 07:00-15:00 UTC (00:00-08:00 PST), all times posted in UTC
STATE of H1: Observing at 109Mpc
INCOMING OPERATOR: Ed
SHIFT SUMMARY:
Busy night with a couple of locklosses & with some odd 37W performance. 48Hz line/bump is observable at 37W & 35W (causing DARM blrms too look noisier). Seems to get rung up after big glitches on DARM.
ALS_YARM has a Stolen by user note on it (which is probably from the computer I am logged into).
Most activity has been alogged earlier.
LOG:
First saw it on the YAW Control strip tool in front of me (1st attachment), but then saw it in the 40min Strip tool as well. It is mostly gone, but can still faintly see it.
W went out of observing very briefly to remove some gain from the ADS yaw loops, who's gain we had increased yesterday. It looked as though this oscillation could be a cross coupling between the ADS and SRC2 loops. Several times yesterday we noted that the SRC2 loops are bringing their error signals to 0 very slowly, so we might need to increase these gains.
Since the gain reduction seems to have gotten rid of the oscillation, I've edited the guardian to keeps the ADS gains lower for now.
The next time the interferometer looses lock, loading ISC_LOCK should make these changes happen.
LOAD was executed at 10:58utc on June 7th.
Noticed a new message (other than Violin Damping) which says: ALS_YARM: Stolen (by: User)
I reckon this must have been me when I had to touch up the Y-arm when I was trying to relock. Not really sure what I could have done differently to steal that node.
"Stolen by USER" will show up on a manager node when one of its subordinates has a new manager. In this case, it is USER. User means that someone had manually clicked on the medm to manage that node. I will change this node to be managed by ISC_LOCK as ALS_XARM is, otherwise we may run into some stalling when there is a lock loss.
I wonder when/how this happened. I don't think I changed the MODE (from ALS_YARM being MANAGED). I did select UNLOCKED for it....would that have been the reason? Hmmmm.
That shouldn't have been the reason. My guess is an accidental clicking of the MANAGE button from the "all states" screen is what happened.
Not sure if this is real (it looked real on DARM spectrum!), but H1 made it up to about 176.7Mpc. And it was in this higher-range state for about 10min. (attached is the last 6hrs of H1 running with first lock on left at 37 W and the last two locks at 35W).
There were multiple ADC overflows in the DARM loop flagged in the database during this period, triggering automated h(t) gates. The DQ times flagged (integer second discreteness) were 1243843959 1243843960 1243843967 1243843969 1243843975 1243843977 1243843981 1243843984 1243843985 1243843988 1243843989 1243843990 1243843991 1243843994 1243843995 1243843996 1243843997 1243843999 1243844000 1243844004 which span 08:12:21 UTC - 08:13:06 UTC.
Andy, TJ, Laura This looks like it was a PI mode ringing up. The broadband monitor (plot 1) sees a nice exponential ringup. The high-bandwidth DCPD channel shows that the line is about 10,430 Hz (plot 2). As Keith posted, the DCPD began to saturate due to the huge line, and plot 3 shows that this looked like a decrease in shot noise on the DCPD which is why the range looked unrealistically high.
From mode identification before the change in test masses https://dcc.ligo.org/LIGO-T1600080 it looks like this could be one of the ~10420Hz mode group. The mechanical eigen-frequency model from this dcc implies this is likely the drumhead mode. No measured mode is within 1Hz of this frequency indicating its probably one of the new test masses. This mode would best be actuated on with the ESD with all four quadrants driven with the same phase. There is also a more complicated mode with a similar frequency, see the 3 and 4th attached figures. And there is also the possibility that this mode is aliased (real frequency 55106) though this is unlikely given there are AMDs.
It looks like this mode also rang up slightly 5th june when the power was increased. It appears that ring heater were not adjusted for this power up. I guess applying some more ring heater will likely avoid this instability.
H1 running OK, but even with 35W the 48Hz noise is still pretty noisy (& our range is a bit all over the place because of it). Currently stand with H1 locked for ~90min.
Oh, and Peter & Karen spotted a pair of deer near the gate...but we couldn't spot them on the camera.
10:25: Lockloss
1) ALS Polarization (see attachment)
Since we were down decided to adjust ALSx polarization (was at 31). Only managed to get to 27. Doing this sent ALSy to la la land, so instead of 1 it shot up over 100. So I reduced it. Only made it to 4. Proceeded with locking.
2) Locking Operator Play By Play
Getting hang of new procedure. Which includes seeing ALS_[X/Y]ARM with the UNLOCKED state selected.
Have significant figure diffs for FSS & SR3 again as noted for previous lock. ACCEPTED these again.
11:18: OBSERVING
Lockloss at 8:16 (lockloss file)
Reverted to 35W per Sheila's alog and noted in my earlier "Back to Observing" alog. Here is the "Operator Play By Play" for locking.
NOTE: This is after two new states:
Play By Play Notes:
Additional NOTE: I wasn't sure whether to mark downtime as "locking" or "commissioning". I did a little bit of both, but mostly "commissioning", because I had to undo some high power stuff and then had to deal with new procedure for ISC_LOCK from Tues work.
10:04 Back to OBSERVING
In regards to the ALS_X/YARM in UNLOCKED, on the main guardian screen for a node, there is a Request area below State and Target. This area has 2 fields, an EPICS readout and a state selector. The state that the state selector reads is not an accurate way to judge what state we are in, hence the EPICS readout right above it. There are only so many states that can fit on the state selector, so if a state is requested from a manager that is not in the selector, it will not be able to show it there. Recent changes have added new states to this node, most likely pushing Unlocked off of the state selector.
Hopefully this clears it up a bit.
Thanks for the explanation, TJ!
I really wanted to note/document what I saw because it was different from what I had been used to & so it had me scratching my head a little. But I do remember there were changes made on Tuesday. Anyway, I've had experience with the new way it works now and think I have it down. THANKS!
As mentioned in my transition summary, noticed that the ~48Hz line happened to be noticeably higher than normal as I walked in this corresponded to a noisier DARM seen via DARM BLRMS (also noticed by Cheryl). Apparently, it looked like 48Hz line would get rung up by certain types of glitches and then have effects on DARM...which looked negative during Cheryl's shift and part of mine.
Then at 7:55:50utc, I noticed a BIG glitch. For a few minutes I watched the DARM spectra with an eye on the big 48Hz line just so I could get some screenshots of that fundamental and its 2nd harmonic. Attached are a couple of them (one where the fundamental/2nd harmonic are "fat" & the other where they are higher in amplitude). See first two attachments.
While looking at these two noisy lines, all of a sudden noticed a big change in DARM: The Bucket Dropped!
- From about 80-200Hz noticed that DARM was well below reference and the range went up to something like 120-140Mpc! (see 3rd attachment)
- Then a few minutes later, higher frequencies followed suit and DARM was below reference from 70Hz well above 7kHz! (I glanced up at the range and saw saw us hit over 170Mpc at one point!). (see 4th attachment)
- All of this weirdness corresponds with a drop in DARM blrms in the 100-450Hz band taking a notable drop (see 5th attachement)
- 6th Attachment is a screenshot of the BNS Range plot where H1 goes off-scale for a few minutes.
It should be noted for the high range observed, there were also many more lines/harmonics appearing in DARM from 200Hz and above. Lasted a few minutes at these high ranges, but then started getting "DCPD" Verbals as well as low frequency (10-40Hz band) bump/glitches. Then H1 just dropped out of lock after a few minutes of this weird behavior.
We should have a look at what the calibration metrics are doing during this time. Was there a hiccup in calcs? The calibration lines don't line up properly in the 2 spectra that have higher reported range.
Though I haven't yet checked calibration metrics, this appears to have been a result of the OMC DCPD's saturating from a PI mode (and converting the entire upper-end of the spectrum into a featureless white noise) -- see LHO aLOG 49704. The investigation continues...
8:16: Lockloss
9:56: NLN (But with a few diffs. Notes below & screenshot attached)
NOTES For Getting Back to OBSERVING:
Because it is the graveyard shift, and since 37W operation was a little interesting at the end of the last lock, I opted to return to 35W for smoother running until the morning.
NOTE On Guardian File Edit to Revert to 35W:
Now to revert to 35W via Sheila's alog, I would need to make a change to the lsc_params.py guardian file. I do not have much experience editing these files, and for my personal login on the workstation, when I open an editor (by clicking an "e" on an ISC_LOCK state, the "e" brings up the guardian file in emacs (emacs is not user/corey-friendly), so I ran over to a differnet workstation and logged in as controls and opened lsc_params.py there via "e" on ISC_LOCK & this brought up file in gedit: Much better.
Made the quick change to the lsc_params.py file. Hit LOAD for ISC_LOCK and LASER_POWER & then went for locking (which was not trivial....I'm guessing due to changes from Tuesday...will alog "play by play" of that experience separately.). As for when H1 returned to NLN, these were the diffs observed:
10:04: Back to OBSERVING
48hz line has been noisy! But then all of a sudden DARM took a dramatic drop in the bucket area of spectrum. Then DARM had a broadband drop in DARM all the way up to several kHz (but also look like many more high-freq lines & now a minute or two later seeing low freq glitches.
NOTE: Observed range which went up to 170mpc briefly!!
And now we just had a lockloss.
Will look into reverting back to 35W.
Will post some screenshots in a bit....after working on locking.
TITLE: 06/06 Owl Shift: 07:00-15:00 UTC (00:00-08:00 PST), all times posted in UTC
STATE of H1: Observing at 110Mpc
OUTGOING OPERATOR: Cheryl
CURRENT ENVIRONMENT:
Wind: 7mph Gusts, 5mph 5min avg
Primary useism: 0.01 μm/s
Secondary useism: 0.09 μm/s
Microseism slowly inching up from the doldrums it was in yesterday. Winds have just died down.
QUICK SUMMARY:
Purely observational note here, but as I scanned the FOMs at beginning of the shift, noticed our pesky ~48Hz line on DARM was noticeably bigger than normal (almost as big as the 60Hz line...atleast getting close/above to 10^-18!).
Looking around can see this corresponds to DARM blrms where there was a glitch and then afterwards we had a noisy 20min segment (around 6:55-7:18utc) where multiple bands were noisy (38-60, 60-100, 100-450Hz). See attached screenshot of DARM blrms & DARM spectra (latter was not during a noisier time, but you can atleast see the 48Hz line clearly).
At any rate, after another glitch, this noisiness calmed down. Perhaps this is related to the new higher power we are running at? Other than that, all looks well with H1 locked for 6+hrs.
TITLE: 06/06 Eve Shift: 23:00-07:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Observing at 107Mpc
INCOMING OPERATOR: Corey
SHIFT SUMMARY: commissioning and then locked
LOG:
Plot attached of the 38-60Hz DARM BLRMS, it has an interesting pattern of noise, coming and going, at about once an hour.
01:24 UTC: Commissioning for today is complete, and H1 is back in Observe at it's new maximum power of 37W. Sheila's alog 49680 has a note about how to revert the power from 37W, back to 35W.
Sheila, Pep, Jenne Georgia Ed
We were able to increase the input power to 37W requested from the PSL, without any big problems. We have also reverted the change to the AS A 36 phases that caused some extra noise last night. We have not yet retuned the LSC feedforward for the new input power, or run calibration sweeps.
Details:
We are going to leave the input power request set to 37W, if there are any problems with this it can be undone by going to userapps/h1/guardian/lscparams.py and changing NLN power to 35 on line 17 and hit load on ISC_LOCK, and on LASER_POWER.
Attached are DARM spectra comparing tonight's 37W lock with last night's 35W lock.
J. Kissel (at the request of J. Driggers & S. Dwyer)
The WFS demodulation MEDM screens rarely get attention regarding their usability and accuracy, but a bit of confusion arouse during attempts to increase the input power today because of a few inaccuracies in the display of the true infrastructure.
I've cleaned it up -- see attached. All changes were related to how the power normalization of the resulting I and Q signals is done; hopefully its a bit more clear now.
Note, though we use the exact same demodulation infrastructure for virtually all WFS, i.e. with the "WFS" library part in the
/opt/rtcds/userapps/release/isc/common/models/WFS.mdl
library, we've had to do special things for AS_A_RF45 signals, which now uses the "WFS_SUM" library part (in the smae library) because the SUM signal is also normalized and eventually used for LSC control, unlike any other WFS.
As such, I had to modify two screens.
The first and second attachment shows before vs. after for the standard WFS.
The third and fourth attachment shows before vs. after for the special AS_A_RF45.
Screens have been committed to the userapps repo, under
/opt/rtcds/userapps/release/asc/common/medm/
ASC_WFS_SETTINGS.adl # standard
ASC_WFS_SUM_SETTINGS.adl # for special AS_A_RF45
LLO should only need to "svn up" the above directory in order receive this goodness (assuming they're using the same underlying infrastructure).
