20:47 at Robert's request, I have closed reamining beam diverters except for POP.
21:06 Anamaria is at EX. BRS has been turned off.
Dr Effler reported to me that she has reverted the -.2 Pit offset that Sheila set recently back to +.2. This resulted in a 400µrad shift in IM4.
I have created a new troubleshooting guide for H1, in preparation for O2. It is available at G1602280 (too big for alog, so won't add it here also). Attached here is the overview page that in the document has clickable links to each section.
My vision for this is that it is a go-to document when things aren't working as smoothly as they should be. I try to provide specific instructions on what to do, but also enough information that Detector Engineers, Operators and anyone else can start looking deeper and debug problems on their own. Text throughout the guide is clickable, so that you can jump to the section that you need.
I encourage others to add to the document, or point me to well-written wiki pages / alogs with detailed instructions so that I can add to the document. For example, both the PI and violin mode damping "instructions" in the guide actually just point you to the very well written wiki pages on those topics. There are 19 sections so far, but I'm sure that I've missed some, and that others will come up, so please let me know or modify the document yourself if you find something!
As of now, H1:LSC-[XY]_EXTRA_AI_[123]_OUT_DQ are in the science frame at 2k, but nobody remembers what they are.
I went to the end stations and found that H1:LSC-X_EXTRA_AI_1 and H1:LSC-X_EXTRA_AI_2 were connected to green REFL A FET demodulator I and Q monitor. Same thing for Y.
Since we haven't looked at these for a long long time, I removed these cables from the DSUB breakout. There's still one cable attached to DSUB breakout which is for Ethercat-frontend interface, but this is not EXTRA_AI channel.
For the moment the output of these channels are turned off to compress the data size. Next Tuesday these are removed from the frame.
We'll also remove jitterFF signal from the frame as we're not using that feature.
Krishna
Summary:
1. Closing the valve for the vacuum can of cBRS makes it very sensitive to pressure through an unknown mechanism. With the valve open, pressure coupling is not visible.
2. cBRS sees signficant ground tilts at wind speeds of ~20 mph, which are coherent with ITMX Stage 1 rY sensor but not very coherent with ITMY ground seismometer. As we suspected, ITMX sees larger ground tilts than the ITMY seismometer. For some peculiar reason, the ITMY seismometer seems to be in a null zone for tilt. This is the reason it works well for sensor correction for the corner.
Details:
As discussed in my SWG log last week, cBRS sees excess ground tilts below ~50 mHz which are not expected given it's 'd' value. It's 'd' value can be inferred from it's translation coupling visible as the height of the microseism peak along x-direction.
1. I wondered if there was some other effect that was causing the increase in the cBRS signal below ~50 mHz. In an effort to understand this, I closed the valve to the vacuum can. I had hoped to supress air pressure/density changes in the beam-path which would give a phase shift signal through index of refraction effect. To my surprise, the pressure sensitivity got signficiantly worse as seen in the first attached file - PZTControl_ValveClosed.png. The second file cBRS_PressCorr_ValveClosed.pdf shows the time trace of the scaled pressure, angle and temperature signal with some scaling. The pressure correlation is quite clear.
Two possible ways it could have gotten worse are - a) mechanical effects, such as pressure change producing stress on the vacuum can base producing tilt etc and b) stress on the vacuum can base affecting the fiber feedthrough.
Since this was clearly worse than before, I reopened the valve fully.
2. The third file (cBRS_Quiet.pdf) shows data under low wind speeds. I have shown cBRS angle data, the inferred tilt from cBRS, the PEM tiltmeter (fancy bubble level) and the ITMY seismometer acceleration/g (to convert to angle). The next page shows coherence between cBRS angle and Stage 1 rY, ITMY STS-X and Stage 1 X. The coherence at the microseism is due to the translation coupling.
The last file shows the same set of data under wind-speeds of ~20 mph. Note that regardless of the cBRS transfer function, the signal above resonance is trustable and shows a large increase in tilt which is similar to what we see at the end-stations. The PEM tiltmeter and the ITMY STS see an increase as well but they are smaller than the cBRS tilt in the 80-100 mHz region by a factor fo ~3. The coherence plot also shows signficiant broadband coherence between cBRS and Stage 1 rY (except at the microseism, likely due to an incorrect scale factor for the T240). The shape of the inferred tilt looks odd below ~50 mHz and indicates an incorrect transfer function. It looks like there is an additional systematic effect that I'm missing (I think it is not temperature or pressure so far). I'll continue to investigate.
18:29UTC a2l done
20:35 running script again
20:48UTC script done
Work Permit | Date | Description | alog/status |
6344.html | 2016-11-22 14:38 | Revert nds2-client software to nds2-client-0.12.2, as the nds2-client-0.13.1 is causing problems with python's getdata. | 31733 |
6343.html | 11/22/2016 9:08 | Re install electronics for infrasound mic. They were shipped off for upgrades | |
6342.html | 11/21/2016 16:23 | Replace HAM 11 AIP controller. First need to pump out annulus volumes two with turbo aux carts. Need laser safe. | |
6341.html | 11/21/2016 16:07 | Bring HAM1 HEPI down and up to test a theory of why the Guardian is still restoring the cart bias's for the pringle modes. | 31725 |
6340.html | 11/21/2016 15:19 | Remove zeroed 64kHz channels from h1susitmpi, h1susetm[x,y]pi models. These are no longer required with later versions of RCG. DAQ restart required. | 31727 |
6339.html | 11/21/2016 14:43 | Move raw trend files from SSD RAID to SATAboy RAID. This is routine maintenance. No restart of NDS servers required as neither is using h1tw0 trend files. | |
6338.html | 11/21/2016 13:49 | Two items: (1) Complete ECR E1600340 for LHO; install infractructure to control DARM with PCAL. Affects CALEX and CALEY and requires a DAQ restart. See LLO aLOG 29623. (2) Bug fix to CAL-CS model to connect up second DARM demodulator, see LLO aLOG 29521. | 31727 |
6337.html | 11/21/2016 13:28 | Reboot the h1guardian0 machine to reduce the load. Last reboot was Nov 1. | 31725 & 31727 |
6336.html | 11/21/2016 13:14 | Add channels to the DAQ broadcaster. ECR1600350. Will be installed during the maintenance DAQ restart. | 31727 |
6335.html | 11/21/2016 12:09 | Re-center corner station BRS, check low freq tilt. | |
6334.html | 11/21/2016 11:12 | Trying to debug Seismon code I need yubikey access to h1hwinj2 | |
6333.html | 11/21/2016 11:06 | Install gstlal-calibration-1.0.8-v1 on the DMT, update the filter files and restart the calibration code. Changes include: * New filter file. * Addition of option --kappas-default-to-median to replace rejected kappas with last good median instead of default. * Bug fix to prevent/reduce latencies due to bad dataValid flags. * Addition of option --demodulation-filter settle-time to allow filter to settle before accepting computed kappas. | 31732 |
6332.html | 11/21/2016 9:04 | Coordinate with Anamaria E. and Robert S. the test running of a rotating shaft pump setup on the LVEA floor near BSC8*Monitor the impact on the IFO while it is locked and in High Sensitivity*Continue running the pump setup while then baking PT180 if activity shown not to be an interference to the IFO, or, shut down pump setup and postpone PT180 bake if shown to be too egregious to IFO*Will require the use of 120VAC variacs positioned overhead on the BSC8 isolation cross beams during the bake period, as well as, ladder access during Tuesday maintenance period | 31723 |
6331.html | 11/21/2016 7:51 | Pressure wash the access ladder to the roof on the east side of E Y to remove the bird nest detritus. We will also clean around the base of the ladder. | |
6330.html | 11/18/2016 9:52 | Create a new seismon EPICS IOC for earthquake prediction. During testing the IOC will run temporarily on zotws2. No DAQ restart. Also testing code to populate EPICS channels. | |
6329.html | 11/17/2016 21:24 | Tour of LVEA (if & only if H1 happens to be down). This is a group of up to 7 from Univ. of WA and Seattle Central Community College visiting Sunday morning. | |
6328.html | 11/17/2016 17:06 | Set IP address for EX RGA and turn filaments on all three powered RGAs - currently valved into BSC5, BSC6, and OMC tube near HAM4. RGA filaments will be left on in faraday mode indefinitely to collect partial pressure scans of volumes. | |
6327.html | 2016-11-16 07:25 | Replace Anemometer on all buildings. Align north along X-arm | 31730 |
PREVIOUS WPs | |||
6317.html | 2016-11-14 09:18 | Turn off the DBB for the duration of ER10/O2. Since the DBB still uses 60Hz mains power, it is our standard procedure to turn it off during runs. | 31717 |
6315.html | 2016-11-14 09:07 | Install the daqd fw build that is on h1fw2 onto h1fw0 & h1fw1. This will bring all the frame writers to a modern libframecpp (the same version that LLO is running) and get the updated leap second table in all the frame writers. | h1fw0 upgraded 2016-11-15, 31517 & 31727 |
The high frequency 5001.3 Hz "roaming" calibration line has been turned back on at 18:54 Nov 23 2016 UTC.
18:45UTC
SEI - Working on getting seismon running. Everything else is ok at this time.
SUS - didn't get charge measurement on Tue. Will try to get this week. FIlter fix, fixed the ITMY issue. SDF updated. (re: causing 20 minute ringdown after locklosses)
CDS electronics - reverted NDS2 software client back to previous version, agaon. Should hold us through 02.
PSL - nothing
VAC - wants to do BSC experiment in corner station. Maybe start bake of PT180
CDS - ane installed and seems to be functional, again. PEM work in vault.
Hardware detection test was done last night successfully. things that did not work: people could not log onto LDAS.LIGO.WA mostly successful mat do it again before '02. In the future, in the case of an alert, immediately stop any injections
Keita wil be investigating unused ADC channels at End Stations
REMINDER - to be cautious around siesmometers when doing maintenance work. (i.e. infrasound mic incident)
18:16 UTC lockloss at CLOSE_BEAM_DIVERTERS. This also tripped HAM 6 ISI. Going to stop at NOISE_TUNINGS and perhaps move manually to NLN?
Here's the transition summary a little late.
Frequently I find that SRC will not lock properly. I then misalign SRM and look at the spot centering on te ASC_QPD to check the alignment of SR2. Usually I find that it's been driven off diagonally in one direction or the other making it difficult for the loops to align and converge SRM into the cavity. After re-centering I will re-align SRM and re-request SRC_ALIGN and it should lock and converge properly. Maybe some hand alignment will be necessary.
Just a minor note---the QPD one looks at is the AS_C QPD. (i.e. Anti-Symmetric port "C" QPD & not the ASC QPD).
Not sure why sequence stuck at FIND_IR. X-arm power not reaching maximum potential. I re-centered as per oplevs because I noticed that after the las op lev align the ITM and ETM seemed a bit off-axis in it's alignment. I got the power back to acceptable range and tweaked TMS a µrad or two and letting WFS 'cook' for a while. Curious as to why there was 'FIND_IR" levels of IR in X arm while doing ALS align?
TITLE: 11/23 Owl Shift: 08:00-16:00 UTC (00:00-08:00 PST), all times posted in UTC
STATE of H1: Lock Acquisition
INCOMING OPERATOR: Ed
SHIFT SUMMARY: First half of the shift Sheila was moving around optics and eventually gained 10 Mpc. Then we took H1 to Observe to be ready for the scheduled injections. Sheila took out SDF from the intention bit condition so SDF differences will not prevent us from going to Observe. INJ_TRANS Guardian had some issues. Whatever the issue was, hitting INIT seems to have fixed it.
LOG:
11:59 Went to Observe to get ready for the scheduled hardware injection. a2l was still running the last couple of lines.
13:35 Gamma Ray burst. Since it's still an Engineering Run I assume the scheduled injection takes priority. Ignored the GRB.
15:18 Lockloss
During ER10, in case we receive ANY EM alert in Nominal Low Noise (NLN) state with the intent bit set while L1 is also running, please immediately cancel all future transient hardware injections.
Also see L1500117. Call me if necessary.
It looks like we had another incident of the POP90 power changing, (circled in the striptool screenshot) similar to what Stefan described in 31181. Is this still a problem with the demod as RIchard found the first time? If its only an intermittent problem with the POPAIR 90 demod, we probably don't need to worry about fixing it before O2 because that is jiust a monitor and won't be used if we are able to close the beam diverters for the run.
Detchar question:
Did we have RF45 glitches around these times? The times are roughly 16:26, 16:30, 16:36 and 16:42 Nov 19th local time, which is 0:26, 0:30, ect Nov 20th UTC time.
We were able to fix some obvious problems with this but the problem with the shield was not changed as we did not test it after the fact. Though this problem should only appear if someone was in the rack. Would be interesting to see if anything is on 9,or 45MHz
I took at look at the auiliary channels we used to create DQ flags monitoring RF45 noise in O1, namely H1:LSC-MOD_RF45_AM_CTRL_OUT_DQ and H1:ASC-AS_B_RF36_I_YAW_OUT_DQ. I created BLRMS of these channels in the same way we did in O1 to threshold on. In all of these plots we see a steady BLRMS over 21 hours from 20th Nov 00:00 - 21:00 UTC, indicating that these channels do not see any form of RF45 noise we are used to:
* Plot 1 - BLRMS of H1:LSC-MOD_RF45_AM_CTRL_OUT_DQ between 10-100Hz in 60 seconds strides
* Plot 2 - BLRMS of H1:LSC-MOD_RF45_AM_CTRL_OUT_DQ between 10-100Hz in 1 second strides
* Plot 3 - BLRMS of H1:ASC-AS_B_RF36_I_YAW_OUT_DQ between 30-170Hz in 1 second strides
Hveto for this day indicated that H1:ASC-AS_A_RF45_Q_PIT_OUT_DQ was a good channel to veto noise with on Sunday. I therefore did a BLRMS of this channel:
* Plot 4 - BLRMS of H1:ASC-AS_A_RF45_Q_PIT_OUT_DQ between 5-100 Hz in 1 second strides
This channel does show excess noise at certain times of the day. If we were to threshold on this BLRMS using the 99.5% BLRMS value during this time period, we would capture the times Sheila mentions and also veto 8/10 top ten pycbc live triggers for this day.
Not conclusive that this noise is RF45 noise similar to what we saw in O1, investigating further...
Seemingly another incident: circa 2016-11-23 20:25:30 Z.