TITLE: 09/30 Eve Shift: 23:00-07:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Observing at 114Mpc
OUTGOING OPERATOR: Cheryl
CURRENT ENVIRONMENT:
SEI_CONF state: WINDY_NO_BRSX
Wind: 15mph Gusts, 12mph 5min avg
Primary useism: 0.06 μm/s
Secondary useism: 0.12 μm/s
QUICK SUMMARY: Running SEI_CONF in WINDY_NO_BRSX since BRSX is in fault.
In the Morning Meeting, there was a discussion about options for the time frame of 6AM to 8AM Pacific (13:00-15:00UTC).
TITLE: 09/30 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Observing at 117Mpc
INCOMING OPERATOR: Patrick
SHIFT SUMMARY: locked all shift, plans for this week/this month are in alog 52205
LOG:
ITMX is out or the range +/-10, however, since we don't use the oplev in Observe, this may be acceptable.
SR3 SUM was juping between two modes, changing the SUM, however, with the cooler weather, it's chosen one mode and is curently running very smoothly (also not used in Observe).
There are currently 3 Turbopump Stand's at MID-Y requiring modifications/additions relating to future fixturing needs. This work required craning and as such, the times of the craning activities are noted below: (intermittent) Craning FROM 13:11 TO 13:40 (continuous) Craning FROM 14:19 TO 14:24
We replaced the TCSY chiller two weeks ago (alog51996) and looking at the two weeks since it has been running, the flow seems to be slightly more consistent than before. The laser temperature is still somewhat sporadically changing though, something that I hoped the chiller swap would help with.
We will still need to replace the flow meter, scheduled for middle of Oct.
Possibly related, the TCSY PZT makes large movements every once in awhile. This will cause the laser to lose its lock point sometimes, but it can recover other times. The second attachment shows the pzt moving and the laser losing lock, while the third attachment shows a large, and very sudden, swing of the pzt from 30 to 60 (the range is 0-70). The laser temperature, flow, or pressure change with any of these large swings, so I don't believe this is a chiller or flow meter related issue.
More investigation is needed.
Tomorrow 8am the one month long commissiong break will start.
We will have a 1pm Wednesday Vent meeting.
See the white board for entire the Trello plan for the vent, or see the link on G1901314. It is colored for different types of activities. Brief outline for this week, but please see Trello for the full schedule:
Other notes:
Some initial trends for those offsite.
Temps at both end stations have been climbing (see aux channels) and both BRS drifts are approaching their 'red' lines. Notice, X drift has crossed the line and this will be our focus for recovery.
Haven't looked at other temps yet but suspect the sudden cold snap HVAC response.
TITLE: 09/30 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Lock Acquisition
OUTGOING OPERATOR: Corey
CURRENT ENVIRONMENT:
SEI_CONF state: WINDY_NO_BRSX
Wind: 11mph Gusts, 8mph 5min avg
Primary useism: 0.03 μm/s
Secondary useism: 0.12 μm/s
QUICK SUMMARY: on it's way to NLN
TITLE: 09/30 Owl Shift: 07:00-15:00 UTC (00:00-08:00 PST), all times posted in UTC
STATE of H1: Observing at 117Mpc
INCOMING OPERATOR: Cheryl
SHIFT SUMMARY:
Appeared to lose BRSx, so running H1 in WINDY_NO_BRSX (this means we can't use the EARTHQUAKE state!).
Did not run any Initial Alignments for locking.
Candidate (S190930s) for H1/L1 was recieved.
LOG:
Just something I remember last time we had an issue with a BRS (see alog) is that we will not be able to try the EARTHQUAKE state to ride through earthquakes (we need to have functional BRS x & y to use the EARTHQUAKE state).
Took 2 attempts, but H1 is back to Observing.
NOTEs:
1) Running SEI_CONF in WINDY_NO_BRSX state. Since BRSx has issues, you will see it flash issues on Diag Main (but that doesn't matter since we are running without it)
2) Had an SDF Diff for MC2's Drive Align L2L Offset being ENABLED. Not sure why this Offset was ENABLED, but since I was wary of disabling it and keeping H1 locked, I ACCEPTED this diff (see screenshot with the diff, the filter bank and it's time machine from the previous lock.
Before a lockloss (9:29utc/ 2:29amPDT), noticed "BRSx is noisy" on Diag Main.
Then H1 broke lock.
Followed by EX ISI trip & TMSx trip.
BRSx is waivering between DAMPING & "BRS Bad" for the Damping Control Monitor. Will investigate more after I try to restore EX ISI & TMS.
9:51 Took OBSERVATORY MODE to CORRECTIVE MAINTENANCE since this is an out-of-the-norm lockloss & BRSx looks to be in a BAD state.
Follow-up on BRSx:
Not sure why BRSx is in a BAD state, but back in August we had bugs attacking it and this was causing glitching. These issues led Jim & TJ to make a diagnostic to flag BRS system when it has issues (this is notified me there was something wrong with the BRSx).
On the BRSx Overview we have "BAD" State showing along with a red box around BRS ETMX Velocity. Since we have issues with BRSx, will run H1's SEI_CONF in the WINDY_NOBRSX state until this can be addressed during the day by Jim/Hugh. Here are some attachments with info on BRSx:
Notes on EX SEI:
After the lockloss and the Watchdog trips, the ISI would trip immediately after untripping and letting Guardian attempt to take the ISI to FULLY ISOLATED. So I used instructions Jim gave me when I had similar issues recently. Basically, I worked with individual SEI guardian nodes first:
Summary pages are down and my remote connection is hopeless, but Corey's image shows BRSX clearly going out of range on his screenshot of the BRS overview (the lower right drift trace shows smooth curves alternated with discontinuous flat sections, doesn't look like real motion). When I checked driftmons before the weekend, BRSX still hand 4-5000 count margin. I suspect that VEA temps have driven it out of range, as a reaction to the cold over the weekend.
(Attached is a 75day trend of the Avg temp inside the MX VEA and outside temperature.)
All (4) temperature sensors for MX are in the LOW, MAJOR state (i.e. temps below 60degF) today. We also are in the middle of our first cold snap of the season which started 2-days ago. Will send an email to Bubba G.
(And a week ago we had lost connection with FMCS Air Handler channels.)
TITLE: 09/30 Owl Shift: 07:00-15:00 UTC (00:00-08:00 PST), all times posted in UTC
STATE of H1: Observing at 117Mpc
OUTGOING OPERATOR: Patrick
CURRENT ENVIRONMENT:
SEI_CONF state: WINDY
Wind: 10mph Gusts, 6mph 5min avg
Primary useism: 0.03 μm/s
Secondary useism: 0.14 μm/s
Microseism has gone below 50th percentile in last 8hrs & winds are below 10mph.
QUICK SUMMARY:
H1's been locked/observing for ~3.25hrs. We just lost V1, but they're locking and will hopefully rejoin us for triple coincidence for the evening.
Matthew Ball, Adrian Helmling-Cornell, David Shoemaker, Cheryl Vorvick, Sharan Banagiri, Robert Schofield
We found that the 48 Hz peak in DARM was associated with the HAM3 chamber doors using a new technique to localize scattering sites on vacuum enclosure walls. We injected from two shaker locations at slightly different frequencies (e.g. 48 and 48.01). Because the injection locations are separated, the relative phase of the two signals varies with location on the vacuum enclosure. As a result, the phase of the beat envelope varies with position, and different sites experience maximum chamber wall motion at different times. The sites with accelerometer signals that have the same beat envelope phase as DARM are candidates for the scattering sites on the vacuum enclosure walls. We get reasonable resolution in part because propagation velocities on the steel membranes are only hundreds of m/s. The technique is illustrated in Figure 1 and there is a bit more detail in my Warsaw talk: https://dcc.ligo.org/LIGO-G1901683 . We first used a variant of this technique, injecting with two separated speakers, which, because it reaches everywhere in the LVEA, may be the best first step (BTW we could clearly hear the spatial variation in beat envelope). The beating shakers seemed better for finer-resolution localization.
We found that there were often a couple of locations with envelope phases indistinguishable from DARM’s, but when we injected at multiple frequencies, we could eliminate the chance phase alignments (e.g. instead of just 48 and 48.01, we also used 50 and 50.01 and 47 and 47.01 as well as beating frequency sweeps).
In the first set of injections, the best match in the LVEA was to the single accelerometer on HAM3, so we began focusing in that area. Previous studies using single shakers and impulses suggested coupling in the vertex area, but had not narrowed the location specifically to the HAM3 area. In order to increase the resolution of our accelerometer array, we mounted temporary accelerometers at various locations on HAM3, the HAM3-BSC2 spool and HAM3 side of the MC tube. Since we only have two spare accelerometers, we had to inject and move accelerometers multiple times. We also checked the GS13s on the table, H1,2,3 and V1,2,3, but their phases did not match DARM, consistent with other evidence pointing to the chamber walls. The accelerometer locations that had the best beat-envelope match to DARM were on central parts of the HAM3 doors, but not the edge.
We opened the viewports and found a lot of scattered light on the inside of the doors, including two “beams” that came right through two different viewports, so we were able to block them with black glass. It turned out that the “beam” that was mainly responsible for the 48 Hz peak was the beam coming through the illuminator port. We put black glass in this port (lower –X port on the –Y side of HAM3) and found that this eliminated the 48 Hz peak. I could easily see a misshapen “beam” on an IR card so I think it might be a milliwatt-scale “beam”. It was surprising that this worked so well since light was scattered all over the doors, but it may be that the illuminator port was the most important location because it has a flat bulb cover that was oriented in such a way that it may have retro-reflected scattered light back to the source (Figure 2).
The source of the “beam” of scattered light (and also the likely recombination point) appears to be the +Y edge of the PR2 scraper baffle aperture (see photos and diagrams in Figure 2).
Figure 3 shows a comparison of summary-page spectra for the day before and the day after we inserted the black glass (https://alog.ligo-wa.caltech.edu/aLOG/index.php?callRep=52136 ). The 48 Hz peak seems completely eliminated and has not re-appeared in a few days. So, fixing the scattered light source in HAM3 does not now seem to be an immediate priority. However, if we arent going to fix it during O3, we would like to make injections to estimate the remaining scattering noise since light is scattered all over the door. Finally, it is possible that there is a second problem on the table that produces the other “beam” that comes out through the lower +X port on the +Y side of HAM3, though Cheryl thinks it might also be from the PR3 scraper baffle. We also put black glass in this port.
Congratulations on finding and fixing a long-standing mystery noise source!
Nice! Did the little peak at 96 Hz go away as well? Looks promising in your fig. 3.