In the past couple of days, somewhat more than the usual amount of water required to top off the chiller has been added. This morning I added ~200 ml. Yesterday JeffB added a similar amount. I took a look under where the air trap was installed last week and sure enough there are some drops hanging onto the pipes and a wet spot on the floor underneath.
Found a leak between the 1/2" close nipple and the ball valve. Leak was not apparent when first installed because there was, by design, a trapped volume of air in the standpipe. After this air vented, the water leaked next. Tightening the connection between the valve and the close nipple slowed but did not stop the leak. Will replace the ball valve and the close nipple tomorrow morning, which will require the laser be shutdown.
Owing to the much warmer weather and ongoing 3 IFO work at the mid stations, I have restarted the chillers at both buildings. Temperature set points are at 74 degrees [F] for both.
J. Kissel Measurements were driven and the data was recorded successfully, but the log files that are written during the measurement to record what was driven when for offline later analysis were corrupted this time around because of some sort of missing SIStrLog /broken function and/or file access issue. I've got emails out to Dave Barker, T. Shaffer, and S. Aston in hopes that this code (written years ago over months by transient guest stars B. Sorazu, B. Barr, and L. Prokorov) can be fixed. Will try again tomorrow after some expert help debugging (or, rather, after Fil gets finished pulling cables for this new... vacuum gauge thing? I dunno, didn't see an aLOG. Maybe comes as a consequence of the new ESD driver power supplies that were replaced while I was out, and coincidentally while I was out).
Opened FRS ticket 8218.
J. Kissel Now the that LVEA temperature is stable and normal, I've gathered transfer functions will all the "nominal" O2 offsets in place: - Reaction Chain has +150000 V, -5000 P of drive from the test bank, originally installed to alleviate suspected rubbing during our cold January - Main chain has normal*** P and Y alignment offsets in place. *** Although the main chain offsets have been used to align the optic to its optical lever reference position prior to the vent, and we suspect that optical lever is a bad reference more on this in a future comment below. However, with this restoration, we're now using up ~85% of the DAC range on the F1 OSEM (where we had been using 35%). With these offsets in place, preliminary results (i.e. from eyeballing the DTT sessions) show no signs of rubbing. The data lives here: /ligo/svncommon/SusSVN/sus/trunk/QUAD/H1/ITMX/SAGM0/Data/ 2017-05-22_2056_H1SUSITMX_M0_WhiteNoise_L_0p01to50Hz.xml 2017-05-22_2056_H1SUSITMX_M0_WhiteNoise_P_0p01to50Hz.xml 2017-05-22_2056_H1SUSITMX_M0_WhiteNoise_R_0p01to50Hz.xml 2017-05-22_2056_H1SUSITMX_M0_WhiteNoise_T_0p01to50Hz.xml 2017-05-22_2056_H1SUSITMX_M0_WhiteNoise_V_0p01to50Hz.xml 2017-05-22_2056_H1SUSITMX_M0_WhiteNoise_Y_0p01to50Hz.xml /ligo/svncommon/SusSVN/sus/trunk/QUAD/H1/ITMX/SAGR0/Data/ 2017-05-23_0047_H1SUSITMX_R0_WhiteNoise_L_0p01to50Hz.xml 2017-05-23_0047_H1SUSITMX_R0_WhiteNoise_P_0p01to50Hz.xml 2017-05-23_0047_H1SUSITMX_R0_WhiteNoise_R_0p01to50Hz.xml 2017-05-23_0047_H1SUSITMX_R0_WhiteNoise_T_0p01to50Hz.xml 2017-05-23_0047_H1SUSITMX_R0_WhiteNoise_V_0p01to50Hz.xml 2017-05-23_0047_H1SUSITMX_R0_WhiteNoise_Y_0p01to50Hz.xml We should consider these offsets a "worst case scenario." I suspect once we get all optics re-aligned to form some sort of Michelson in the corner and/or we open to the arms that we'll find that the optical lever reference was indeed wrong, and we'll end up removing most of the main chain offsets. Plus, I think we now might be able to run with out any reaction chain offsets. Stay tuned for full matlab assessment using the exported data.
On why I think we can't trust the optical lever to restore ITMX alignment. (Even though it's been done a few times.) Attached are 25 day trends of (1) The Pitch of the optic as measured by - the L3 optical lever (in [urad]) - the M0, R0, L1, and L2 OSEMs (in [urad]) - the alignment offset of the M0 stage (in [urad]) compared against the BNS range. (in [Mpc]) This shows that although the optical lever has been restored to a time around 2017-05-07 (just before the vent) none of the OSEMs agree that this is right position (2) The Yaw of the optic as measured by - the L3 optical lever (in [urad]) - the M0, R0, L1, and L2 OSEMs (in [urad]) - the alignment offset of the M0 stage (in [urad]) compared against the BNS range. (in [Mpc]) Similarly show that the OSEMs completely disagree with the optical lever (3) Because the naive always suspect the ISI system, I show the Z, RX, RY, and RZ ST1 position sensors (in [nm] for Z, [nrad] for Rotation), which show > 0.5 [urad] / [um] of position change (4) The amount of F1 (the pitch) OSEM DAC range in use (85% of the range instead of 35%) and the temperature of the optic / suspension measured in vertical displacement of the main and reaction chains (in [um]) and the FMCS temperature channels. Another damning piece of evidence against the optical lever for ITMX -- The MICH fringes are terrible at the moment. This leads me to suspect something when wrong with the alignment restoration done to the lowest stage OSEMs by Vaishali and Jenne earlier in the week (referenced in LHO aLOG 36276). In support of this, I attach similar 25 day trends of all PRMI optic's lowest stage OSEMs in (5) Pitch (in [urad]) (6) Yaw (in [urad]) so we're not that close with any optic according to *those* references. Also another bit of evidence is that we cannot find a return beam from the Hartman Table (LHO aLOG 36332). I've discussed with Betsy whether it's possible that the balance of the suspension has changed throwing off all the OSEMs, but she says essentially "no way." All that was done to the SUS was to slide in teflon rails between the test mass's lower barrel EQ stops and the optic, and then the optic was gently raised into its upper EQ stops. The process was reverse when finished. No OSEMs were touched, no balance of any mass was adjusted. Since there is still suspicion that the IMC is in the wrong place (LHO aLOG 36329) people are reluctant to launch on a big "restoration of the alignment based on short lever arm local things" adventure before we get the arms and global alignment control back. To clear up potential memory failure leading to confusion, it was ITMY's optical lever that was swapped a few days before the vent (see LHO aLOG 36029), not this one.
This morning Jenne brought the the ITMX suspension back to good values, according to the oplev, so that we could try a to search for a HWSX return beam. I followed Aidan's instructions and did the search just as before:
I ran the measurement from 22:03-23:30 UTC (+/- a few minutes) before I thought it was easy enough to confirm that there was still nothing showing up, and then handed the ITM back to Jeff to complete his measurements. With more time all of the gaps could have been filled, but this does not show any higher intensity points unlike previous tests (alog36179).
After talking with Aidan, he thinks that ITMX is still not within an acceptable aligned range, and the upper periscope mirror may be misaligned. If there is more time later this week we can try again with the amplitude doubled, increasing the search area, but just waiting for the green beam and his arrival might be best.
Note to myself: I left the HWSX fiber in h1hwsmsr
Though I did valve-in the Vertex RGA this morning I took no additional steps listed on WP #6644 as no one is now calling for the premature opening of GV5 and GV7. As such, we will continue to let the Vertex+YBM+XBM "dry out" on the turbo pumps until further notice.
Per request from Jeff K. I have raised the temperature at EX by 2 degrees F. from 64 to 66 degrees.
On what motivated the 2 [deg F] warmer request: Two attachments: 15 day trends of (1) the FMCS temperature sensors vs. PCAL temperature sensors (2) the vertical positions of the SUS The vertical positions of the SUS show a temperature overshoot (i.e. we corrected the too hot with too cool a set point), because the vertical position has been shooting up higher that before the upgrade (about 2/3rds of the way through the graph). The PCAL temp sensors suggest that the XVEA was at 20.3 [deg C] = 68.54 [deg F] before the upgrade, and had settled to 19.3 [deg C] = 66.74 [deg F] with the recently set setpoint of 64 [deg] (which was a copy-and-paste from EY a few days ago). Hence, I requested a 68.54 - 66.74 = 2.16 ~~ 2 [deg F] increase in temperature.
This afternoon while looking into some temperature issues at EY, I found a large puddle of glycol on the roof of the air handler room near the make up tank and on the floor. I traced this back to a PRV that has failed open. The PRV is on the non potable water line which is tied into the glycol return line from the chillers. This appears to be some type of auto fill system that was not used and probably has never worked. Not even sure why it was tied into this system. Anyway, I valved the PRV off and refilled the glycol lines back to normal operating pressure,~30 psi. At this point, I do not intend to replace the PRV because we will not use that system for auto fill.
[Vaishali, Jenne]
We went again to have a look at the IMC trans path on IOT2L, and we think that the main beam is just not coming out of the vacuum.
Kiwamu pointed us to alog 17310 from March 2015, where he notes that something happened in vacuum and the IMC trans beam went from somewhat clipped but still somewhat normal looking to totally abnormal looking (which we've had consistently for the last 2 years). Vaishali is currently looking into Cheryl's IMC spot measurements to see if the beam spot motion is consistent with the beam moving closer to a position where the trans beam is clipped on the IM1 suspension cage.
We still have the ghost beam on the trans camera, but no beam bright enough (by a factor of ~100, as measured by the PD) to be the main beam.
Drawing attached showing the beam on IM1 that is IMC Trans.
That beam was -6.5mm off center before the vent, and is now -7.0mm off center after the vent, making the clearance through the light pipe even worse.
The -0.5mm change in beam comes from using the MC3 OSEM changes from before to after the vent.
1178267478, before vent | 1179071669, after vent | diff | |
urad | urad | urad | |
mc1 p | -25.2 | -29.5 | -4.3 |
mc1 y | -1030.2 | -1039.3 | -9.1 |
mc2 p | 500.3 | 497.0 | -3.3 |
mc2 y | -672.0 | -671.5 | 0.5 |
mc3 p | -808.9 | -816.4 | -7.5 |
mc3 y | -978.9 | -986.8 | -7.9 |
The root cause of this issue with IMC Trans are the positions of the beam spots on the IMC mirrors, and in particular the beam spot on MC3 that I measured recently as -5.7mm from center in yaw along the face of the optic.
The two things that we can control that effect the IMC beam spot positions are the IMC input pointing, and I have an approved ECR to add hardware to the IO beams on the PSL to monitor that beam. That beam can also be corrected from the PSL.
An emerging issue is that the steering mirror to the MC2 Trans QPD may be positioned in such a way that maintaining centering on the QPD with it's servo is not producing centered beams on the IMC mirrors.
This is my final Ops Summary alog.
Activities:
16:16 Carlos -> EY
16:57 Corey to squeezer bay and open roll up door
17:33 Corey out
17:59 Kiwamu to optics lab looking for AM laser
19:20 Richard -> roof
19:33 Richard back
19:36 Fil -> EY
19:49 Kiwamu starting measurement
19:55 Jeff measuring ITMX TF (subbing test)
20:47 Jeff B. to end stations
Kiwamu out
21:22 Jeff B. back
21:43 LVEA transitioned to LASER HAZARD
TJ(HWS table)+Jenne+Vaishali (IOT2L) -> LVEA
21:53 TJ out
22:02 TJ starting SR3 excitation (HWS measurement)
22:13 Carlos to EX
Jenne+Vaishali out
22:36 Carlos back
22:47 PEP-> EY (line hunting)
22:51 Dave -> MY
Added 250ml water to Crystal Chiller to silence low water alarm.
Jonathan, Dave
The DAQ was restarted at 19:06 Thursday 5/18 UTC (for vacuum controls IP6 work). If an NDS1 request for second trend data for certain channels is sent which spans the restart time, then the data preceding the restart is correct, but post data is bogus. In the case of H0:VAC-EY_Y3_410_PIRANI_INTLK, data at the tail end of the 19:00-19:10 second trend file is a static 2.2, and from 19:10 onwards is a noisy 2.2
We conclude that only the second trend frame from 19:00-19:10 is suspect. Both framewriters wrote the same frame, both nds servers show the same issues if spanning this frame. Investigation continues. I've saved the frame and its md5 file under h1nds1 control's homedirectory (dave subdirectory).
-rw-r--r-- 1 controls controls 434M May 22 11:27 H-H1_T-1179169200-600.gwf
-rw-r--r-- 1 controls controls 33 May 22 11:27 H-H1_T-1179169200-600.md5
This may be similar to a known Dataviewer issue when getting trend data that spans a data gap. See LLO FRS 7239.
Possible answer to Thursday's second trend issues:
Patrick noted that the interlock channel looked like it was showing a different channel's second trend data after the DAQ restart. I then remembered that the original NDS code had a performance feature built into it. When asking for second trends, it read the full channel list from the first GWF file, and then applied those channel offsets for all subsequent files it opened. In the initial system second trend files were one minute in length.
To cover the case where the channels configuration was changed during the requested time span, the code looked for any missing second trend files. At one minute per file it was not possible to restart the DAQ and not have a missing file. The file following the gap was read for the channel configuration and that was used from that point onwards.
Fast forward to today, with 10 minute trend files. It is now highly unlikely for a second trend file to be missing when the DAQ is restarted, and so the channel data pointers will be incorrect after a DAQ reconfiguration. We seem to be seeing this, with the interlock channel being replaced with a cold-cathode raw voltage channel (value of about 2.2V).
After reviewing the nds1 server code it looks like Dave's assesment is correct. The nds1 server implements a nice optimization where it reads in the frame table of contents at the start of a span of contiguous frame files. It only reads a new copy of the table of contents in when there is a gap in the frame files (thus saving a large amount of processing time). Now we can restart the daq fast enough that there need not be gaps in the trend frame files, so the nds1 server will not catch a restart of the daq (and thus the need to read a new copy of the table of contents in). I will work on correcting this on the DTS today.
Starting CP3 fill. TC A error. TC B error. Fill aborted. Starting CP4 fill. LLCV enabled. LLCV set to manual control. LLCV set to 70% open. Fill completed in 2620 seconds. TC B did not register fill. LLCV set back to 36.0% open.
Dave, Patrick It appears that the temperature of the CP3 thermocouples started above 40 deg C which the script to perform this fill takes as an error. That is, the script takes any reading from the thermocouples above 40 deg C as an indication that the thermocouple is not working correctly. We have created WP 6649 to increase this upper limit.
Thanks!
Increased CP4's manual LLCV %open value from 36% to 37%. Will let new script run before altering CP3's value.
First run of updated script did not complete after an hour: Starting CP3 fill. LLCV enabled. LLCV set to manual control. LLCV set to 50% open. Fill not completed after 3600 seconds. LLCV set back to 21.0% open. Dave is rerunning it.
Fill completed: Starting CP3 fill. LLCV enabled. LLCV set to manual control. LLCV set to 50% open. Fill completed in 384 seconds. TC B did not register fill. LLCV set back to 21.0% open.
I looked at CP3's exhaust discharge pressure and am convinced that it did overfill following the second running of the new script. I increased CP3's LLCV %open to 22% up from 21%. It is getting warmer this might be too conservative?
Kyle (Tasks part of WP #6644) ~1015 hrs local -> I valved-in the Vertex RGA to the combined vacuum volumes of the Vertex+YBM+XBM while these volumes were being pumped by their respective turbo pumps. PT120's response is attached (1 hour of second data). "From my head" 10-9 torr/10-5 L = 10-4 torr/L of gas accumulated in the unpumped ~5L RGA volume in 14 days? So, no significant air leaks in the RGA joints. ~1045 hrs. local -> I energized the Vertex RGA filament (note: I am confused with the Pfeiffer software's naming of my connect/scan session with the included phrase "Airdemo..". The other versions of this software don't do this and I makes me question as to whether I am actually connected to the Vertex RGA (h0rgacs) or am I connected to their "Simulator" or virtual RGA in this connect/scan session?
Correction to this obvious typo error: (10-9 Torr)(10+5 L) = 10-4 Torr*L of accumulated gas.