TITLE: 10/26 [EVE Shift]: 23:00-07:00 UTC (16:00-00:00 PDT), all times posted in UTC STATE Of H1: Lock acquisition. OUTGOING OPERATOR: Cheryl QUICK SUMMARY: Lights appear off in the LVEA, PSL enclosure, end X, end Y and mid X. I can not tell from the camera if they are off at mid Y. Seismic appears to have recovered from earthquake in Afghanistan. Microseism is still high but trending down. Winds are back below 20 mph. Jenne put ISI blends back to 45 mHz. Jenne, Evan and Sheila are here and helping with lock acquisition and violin mode damping.
TITLE: Ops Day Summary: 15:00-23:00UTC, 8:00-16:00PT, all times in UTC
STATE of H1: major interference in locking the from earthquake after effects, useism, and wind - locking is getting progressively better
Support: Jenne, Evan, and Sheila
Quick Summary: IFO is locking well up to DRMI, and gotten past DRMI, but not to ENGAGE_ASC, can now get to DARM_WFS
Details:
- Ground motion significantly effecting arms, with arm powers in green and red dipping down to as low as 0.5-0.7
- Stopping at DARM_WFS to damp ROLL mode early - this is the stage where the signal becomes useful
- Switched to Quiet_90 ISI blend filters - alogged earlier
- Needed higher gain to lock X arm in IR, while doing an initial alignment
- Day spent considering and sometimes trying new approaches to improve locking with high ground motion
Shift Activities: work around the site
- Joe and Chris on X arm sealing the beam tube, morning and afternoon
- Richard to Electronics Room, 20:45-21:15UTC
- Kyle on Y arm at the y28, doors closest to End Station
Control Room Display:
- restarted cameras on video4 (camera's froze)
- restarted nuc4's dtt, DRMI_FOM_displacements
- Carlos working on nuc5 (couldn't get to lower screen when logged in)
Currently:
- locking is getting progressively better
- earthquake (~14 hours ago) effects are gone
- useism is slowly coming down
- wind is dying down
J. Kissel We'll be down for the entire maintenance period (8:00a - 12:00p PDT) tomorrow, as there are activities that require laser safety which are expected to take the entire 4 hours. This means we don't expect to get much observing data during the work day tomorrow, especially if the current perfect storm of wind vs. microseism continues. Other than the HAM1 HEPI pier grouting, we don't expect any of these tasks scheduled for tomorrow to impact the IFO's lock acquisition abilities or to change its configuration. However, we must be mindful of how the HEPI pier grouting may affect REFL ASC by changing HEPI alignment. I attach a shot of the whiteboard. Lots of activities, but essentially all of them can be done in parallel. The DAQ restart (to store some SDF diagnostic channels in the frames) is scheduled first, to get it out of the way for those tasks that do need the DAQ (namely PCAL Inv. Actuation Filter characterization after it's installed).
Due to the violin mode problem on 10/25, Sheila has asked me to investigate when this mode really started to rung up. The first plot attached shows that the amplitude of 1008.45Hz were consistant the day before the power glitch and three hours before power glitch (the small difference you see is within the mode normal fluctuation range). The second plot shows the 1008.45 Hz got rung up by an order of magnitude during the first lock acquired after the power glitch just like others. Because this mode didn't have a damping filter at the time, ideally the amplitude should have stayed where it was. However, the final plot shows that the amplitude became worse as time progress while other modes were either stable or being damped until it caused the problem on October 25th. Could anything that happened during the power lost caused the mode to change its phase as it seems to be slowly rung up by ETMY MODE3 that's been existing since before O1? Note that this violin mode had never rung up before. The investigation continues.
To ensure that the 1008.45Hz line hasn't been slowly ringing up all this time, I've looked back at the asd amplitude of this mode until October 1st. First plot attached shows the amplitude/sqrt(Hz) versus frequency of this particular mode, one plot per day. The second plot shows log amplitude versus time. I only plotted one data point per day (10:00:00 UTC if data available, or any time where the BNS range was stable and the ifo was locked for at least an hour). The last data point is today (10/28 02:00:00 UTC). This mode has been fluctuating between 1e-22 and 1e-21 since the begining of the month (10/01) up until 10/20. You can see clearly that the amplitude begins to rise above its nominal on 10/21 after the power outage on 10/20 and continues to grow exponentially until it started to cause problems on 10/25. Indicates that the amplitude grow was causing by a positive feedback, which Sheila found it to be ETMY MODE3.
To conclude this study: This mode wasn't, and hasn't been ringing up before October 20th. Why it started to ring up after power outage is unclear. I can't think of anything else but something must have changed to cause this mode to change phase during the power outage.
Should we worry...?
Was there a significant temperature excursion during the power outage?
Yes.
I've attached the plot of average temperature in the VEAs. After the power outage LVEA average temperature had three big dips of about half degree. Average temperature at EY seems to fluxtuate more often and EX had couple of large drops.
Which turns out to be just a coincidence with the power outage according to John.
In light of the violin mode stuff over the weekend, I wanted to make sure we aren't going to be caught unawares by other harmonics ringing up.
We have paid considerable attention to damping the fundamentals and second harmonics of the violin modes, but in general we leave the higher harmonics undamped. (We have had some isolated cases in which we have had to temporarily apply damping to high harmonics; see, e.g., LHO#18599).
I checked the heights of the third harmonics during the first long lock (25 hours long) that we had after the power outage. I chunked the DCPD sum timeseries into 1000 s segments and then took ASDs of each. To identify the violin modes, I simply looked for peaks in the DCPD spectrum that were higher than some threshold above the noise (in this case, the noise floor is about 1×10−7 mA/Hz1/2, and the threshold I chose was 5×10−6 mA/Hz1/2). This resulted in 25 peaks being identified. For each ASD, I integrated under each peak to get an rms mode amplitude. Since the violin modes are anharmonic, I believe that these peaks (which range in frequency from 1456.18 Hz to 1484.67 Hz) are truly third harmonics, rather than upconverted first harmonics (which range from 498 to 509 Hz, and hence we would expect upconversion to appear from 1496 to 1527 Hz).
All 25 peaks appear to be ringing down, with time constants on the order of 2×105 s (≈50 hours). I fitted each ringdown to an exponential, which resulted in Qs ranging from 0.55×109 to 1.28×109. [For similar results on lower-order mode Qs, see, e.g., Keiko's measurements on L1 IY.]
Similarly, we don't have any damping on the second harmonics of the ITM violin modes. Here, we see 16 modes, and all of them ring down, with Qs ranging from 0.72×109 to 1.43×109.
For a discussion on the behavior of some of the ETM second harmonics after the power outage, refer to Nutsinee's alog.
For fourth and fifth harmonics, there are fewer traces which show clear ringdown; many appear to be complete garbage. Not clear why.
Anyway, for the fourth harmonics, Qs range from 0.6–1.1×109, and for the fifth harmonics, Qs range from 0.6–0.9×109.
Day Shift: 15:00-23:00UTC, 8:00-16:00PT, times listed in UTC
Update on IFO:
- morning alignment issues: LSC to ETMX gain was zero when trying to lock the X arm on IR
--- the gain needed to lock was 0.10, when the usual gain set by Guardian is 0.05
- 18:40-18:41UTC switched ISI platforms to Quite_90s from 45mHz
--- reason: Quite_90s are better in wind, and 45mHz are know to be bad in wind, and wind is between 20 and 25mph
--- result: X and Y ALS arm signals dip to 0.7-0.8 while arms are locked in green (max is 1.09)
--- before switching to 45nMz, arms only dip down to 0.9-0.95
- as of 20:40UTC, IFO is well aligned and locking DRMI, but not making it much past DRIM
--- plan is to continue to let the IFO try for DARM_WFS
When an SEI platform trips and then relocks, there is DC control to zero the error point stearing to a fixed position. Not all ISI platforms/dofs do this but for HEPI, all platforms and all DOFs servo to the 'reference location', except, the pringles modes. These are the potato chip like distortions required to allow the other 'standard' cartesian dofs to be realised given the physics of the hardware. As an example, the four horizontal actuators on a HEPI platform are all situated at 45 degrees to the X & Y direction. For RZ, you can picture that you push or pull on the 4 actuators to easily get RZ (Yaw.) But consider X or Y, to move in X, the 2 far actuators will squeeze the Y dof together and the near actuators will push the Y apart. See the Attached for a visual of a BSC (G1000125.) The green arrows represent the signed directions of the local Actuator coordinates. Maybe better said, to achieve a motion in X, a movement in the Y component must occur. This 'distortion' is what is measured by the Pringle mode.
The pringle mode isolation loops of HEPI are not DC coupled, this is what Kiwamu noted in alog 22719. So despite the other cartesian positions being servo'd and back to their reference position, the pringle difference means that the local positions are not where they were before the trip. The second plot shows the four horizontal IPS positions (local) and the horizontal cartesian Location Mons. Clearly all the horizontal cartesian positions are where we tell them to go and the Pringle ends up where it does. Obviously too, the local sensors are not where they were before the trip. They all indicate about a 0.2mil (~6um) shift across the trip.
1) Kiwamu reports that the HAM2 ISI OpLev (looking at a fixed mirror on the ISI table) is only about a urad different at worst if that can be trusted.
2) It seems extreme to think that these shifts in the HEPI Actuator positions can translate ultimately to the ISI and cause shift requirements in the optics. These pringle distortions are not taken by the real support structure, the HEPI Actuator is designed to deal with that. However, the horizontal shift of the platform is experienced by the Vertical Actuators as a lateral shift of the IPS sensors wrt its flag. There will in fact also be an associated tilt with this lateral slide. These out-of-axis shifts of the sensor/flag can result in real shifts of position. The four vertical local IPS sensors all indicate around a 4um shifts.
3) So despite having restored the cartesian computations to the reference location, the actual actuators are not in their previous positions and these may lie to us at some level, some worse likely more than others, and maybe we are shifting the ISI Stage0 and ultimately the Optical Table.
Maybe this should be considered low priority but we should consider how we can close these loops so we can get repeated positions:
1) Rather than closing these loops all at once, maybe in a particular sequence to achieve a repeatable local position result
2) Maybe we actually put DC authority into the pringle modes and we'll achieve 1) without all the thinking, modeling, and teeth gnashing
GregM, RickS, DarkhanT, JeffK, SudarshanS
This was an attempt to study what the GDS output will look like with kappa factors applied. GregM volunteered to create test data with kappa applied, kappa_C and kappa_tst, on 'C01' data for the days between October 1 through 8. The kappa corrected data is referred as 'X00'. The correction factors are applied by averaging the kappa's at 128s. This was loosely determined from the study done last week (alog 22753) on what the kappa's look like with different time-averaging duration.
Here, comparisons are made between 'C01' as 'X00'. The first plot contains the kappa factors that are relevant to us. kappa_tst and kappa_C are applied and are thus relevant, whereas cavity pole (f_c) varies quite a bit at the beginning of each lock-stretch and is thus significant but we don't have an infrastructure to correct it. The first page contains kappa's calculated at a 10s FFT and is plotted on red and a 120s averaged kappa's plotted in blue. Page 2 has similar plot but has kappa plotted at 20 minutes averaging (it helps to see the trend more clearly).
Page 3 and onwards has plots of GDS/PCal at pcal calibration line frequencies for both magnitude and phase plotted for C01 and X00 data. The most interesting plots are the magnitude plots because applying real part of kappa_tst and kappa_c does not have a significant impact on phase. The most interesting thing is that applying kappa's flattens out the long-term trends in GDS/Pcal in all four frequencies. However, at 36 Hz, it flattens out the initial transient as well but introduces some noise into the data. At 332 Hz and 1 Khz it introduces the transient at the beginning of the lock stretch and it does not seem to have much effect at 3 KHz line. We think that this transient should be flattened out as well with the application of kappa's. The caveat is we don't apply cavity pole correction and we know that the cavity pole has a significant effect in the frequency region above the cavity pole.
DarkhanT, RickS, SudarshaK
After seeing the ~2 % transient at the beginning of almost each lock stretch in GDS [h(t)] trend at around 332 Hz, we had a hunch that this could be a result of not correcting for cavity pole frequency fluctuation. Today, Drarkhan, Rick and I looked at old carpet plots to see if we expect variation similar to what we are seeing and indeed the carpets plot predicted few percent error in h(t) when cavity pole is varying by 10 Hz.
So we decided to correct for the cavity pole fluctuation to h(t) at calibartion line frequency. We basically assumed that h(t) is sensing dominated at 332 Hz and used absolute value of the correction factor that change in cavity pole would incur [C/C' = (1+ i* f /f_c)/(1+ i* f /f_c')] and appropriately multiplied it to the GDS output.
The result is attached below. Applying cavity pole fluctuation gets rid of the transient seen at the beginning of each lock stretch as well as 1 % overall systematic we saw on the whole trend. We used cavity pole as 341 Hz for nominal value which is calculated from the model at the time of calibration. In the plot below, the cyan in both bottom and top left are the output of GDS CALIB_STRAIN/ PCAL uncorrected for Kappas, the green on the top left is corrected for kappa_tst, kappa_C and cavity pole whereas the green on the bottom left is corrected for kappa_C and kappa_tst only ( we only know how to correct these in time domain).
Ops Day Shift: 15:00-23:00UTC, 8:00-16:00PT
H1 current state: relocking after an initial alignment
Help: Jenne
Summary:
Earthquake and useism and wind at EY have caused some locking issues.
Currently IFO has locked DRMI and attempted ENGAGE_ASC once without success.
Relocking continues.
Subsystem Reports:
Kyle - trips around the site, y28 and x28, closest to the end stations.
Hugh - no invasive work
Richard - electronics cabling
Jason - PSL plumbing change
Maintenance tomorrow:
Bubba - HAM1 pier grouting, laser safe, many hours
Richard - temperature sensor install
Kiwamu/Sedarshan - ISS 2nd loop
JeffB - cabling
Jodi - heavy things moved to Mid Stations
Richard - more solar panel work
Betsy - Tip-Tilt inventory
The main report can be found on the detchar wiki, but here are the highlights:
The script had stopped, same connection error as usual. I restarted it in the same screen (pid: 4403).
One stage of heat was added Friday 23rd but the LVEA temperature control is still marginal so we have energized an additional heater. HC3B is now operating on one stage. It appears that the control signal for this heater is not working correctly. It is set to 0ma which should not turn it on but when we energized it the duct temperature rose to 81F which is consistent with one stage.
TITLE: Oct 26 OWL Shift 7:00-15:00UTC (00:00-08:00 PDT), all times posted in UTC
STATE Of H1: Aligning
SUPPORT: Jenne
LOCK DURATION:
INCOMING OPERATOR: Cheryl
ACTIVITY LOG:
07:49 DARM “breathed” between 60Hz and 300Hz with no saturation verbal.
08:00 Wind speeds have picked up, sometimes gusting to 40mph.
08:12 Lockloss. More than likely due to high winds. µSei approaching .9microns/s.
08:30 Wind has dropped down to below 20mph. IFO is locking again.
08:50 Lockloss after being at ENGAGE_ASC_PART3 for a minute or so.
09:01 IFO made it to RF_DARM then lost lock.
09:08 Wind picking up again to almost 30mph. This time the CS is taking the brunt.
10:00 Started switching GS13 gains to LO to get everything as damped as I can.
10:34 Re-Aligned/Isolated HAMs 2&3. GS13s switched back to high gain. IMC relocked. EQ band getting back down to 1micron/s
12:41 I’ve been having trouble trying to find my ALS spots. I’ve trended optic alignments, I’ve cleared ALS WFS histories, I’ve run the dither scripts, I’ve checked shutters for “open” and I’ve called Jenne. I left a message and haven’t heard back yet. Since microseism is still kind of high I don’t want to bother anyone who isn’t on call today until it gets a little better and the time gets a little later.
13:19 GRB alarm. Neither observatory is up.
13:24 PR3 was still in damped mode!!!!! GRRRR! I got my ALS spot back. :)
15:00 Finally got some semblance of an initial alignment happening.
SHIFT SUMMARY: After a harrowing night of wind and earthquake activity, Initial alignment is happening. Ground motion is still high. Handing off to Cheryl.
Apologies. My activity log doesn't reflect the eq activity between 09:08-10:00UTC. This gives the impression that I was switching GS13 gains because of wind.
MID-SHIFT SUMMARY: IFO was locked and holding it’s own, despite the rise in wind speed and high microseism until 08:12UTC. Perhaps it was the building tilt that was eventually the demise of this lock stretch. Then while trying to relock, the 7.7mag quake near Afghanistan started tripping everything. I’ve gotten everything back up, isolated and gained up. The IMC is re-locked but I’m having trouble finding Green ALS Beams. µSei is still up around .7microns and the eq bands are still up in the .3-.4micron range. I don’t think a phone call to wake anyone is warranted until I think I at least have a chance of actually re-locking. I’ll keep plugging away at my problem for now.
08:12UTC More than likely due to high winds. µSei approaching .9microns/s.
This entry was mad by me, not Jim. An alog session had been left open on another workspace that I was unaware of.
The LHO SEI team has known about a .6-ish hz peak on the HAM3 ISI for a long time (see my alog 15565, December of last year for the start, Hugh has a summary of LHO alogs in the SEI log for more). I was working with Ed on a DTT template for the operators when I noticed it was now gone. Very strange. Looking a little closer, it seems to have been decreasing over the last couple of days to a week, and disappeared completely this morning about 7-8:00 UTC. Attached spectra are from ~0:00 UTC (red) and ~16:00 UTC (blue, when I found it was missing). Looking at random times over the last week, it looks like it may have been trending down.
Could someone in Detchar look at this peaks longish term BLRMS, say over the last month, or even over the last year since we found it? Pretty much every sensor in that chamber saw this, but the GS-13s are the best witness.
I checked the coherence of H1:SUS-PR2_M1_ISIWIT_L_DQ with some PEM sensors for frequencies around 0.6 Hz.
Note - it reappeared for a few hours on Oct 13 - picture at https://alog.ligo-wa.caltech.edu/aLOG/index.php?callRep=22775