cdsfs1 went into read-only mode Friday evening between 18:20 and 19:20. At 17:45 this afternoon I rebooted the machine using the standard recovery procedure. Backups are proceding.
While looking at the system overviews I noticed that h1fw0 is not running. It has been unstable from about 16:45 this afternoon. Unfortunately fw1 is not perfectly stable, it had failed around 3pm and 5pm with short restarts. fw0 cannot run for more than a few minutes without crashing.
The console message on h1ldasgw0 shows many errors of type:
samfs: WARNING: SAM-QFS: ldas-h1-frames: cannot flush pages err=43 ....
I called Greg, he and Dan will investigate further. Sometimes fw0 gets to run for 10 minutes before crashing, so I'll keep both frame writers going to minimize the possibility of complete data loss. Greg will switch LDAS to using h1fw1 as the primary frame source.
Attached is a plot of the following 6 signals of that last 31h of IMC locking:
H1:PSL-FSS_FAST_MON_OUTPUT
H1:PSL-FSS_AUTOLOCK_STATE
H1:PSL-ISS_DIFFRACTION_AVG
H1:IMC-MC2_TRANS_SUM_OUTPUT
H1:PSL-FSS_TEMP_LOOP_OUTPUT
H1:PSL-PWR_NPRO_OUT_DQ
I didn't find a good PSL room temperature monitor, but I guess that all the fluctuations we see are temperature-induced in some form. See also elog 7593 for comparison.
- Corey, Andres, Hugh, Keita, Cheryl TMSX telescope moved by Genie from the TMS lab to the VEA. The telescope suspension wires were attached, and the telescope is now suspended. The upper mass is still locked by EQ stops. The vertical safety wire is installed, and the swing stop assembly is positioned under TMSX. A rod on the swing stop assembly goes through two eye bolts on TMSX, preventing it from swinging into the QUAD. Gymnastics of turning the telescope 180 degrees was documented more thoroughly in pictures today. The telescope safety support beams were installed Wednesday (my idea), which mean the Genie would not fit under the upper structure, so we rotated the front beam out of the way after removing 3 of 4 bolts. I returned to EX after lunch and attached the 4 cables to the ISC table. Cables come from the clamp in the center of the upper mass, attach to raised wire clamps, and connect to the cable brackets on the ISC table. They will need further tweaking after the upper mass is freed, to make sure they are not touching. Pictures added later.
Here are a few action photos from the TMS Table/Telescope install yesterday (& as usual, ALL TMS photo documentation is on ResourceSpace, here).
Pictures showing telescope optics protected by Alpha wipes, the swing top assembly, and cables. 1 - view from the floor - swing stop assembly on the left - telescope optics covered with large Alpha wipes 2 - swing stop assembly attached to TMSX 3 - CB-7 4 - CB-8 cables to upper mass 5 - CB-8 6 - CB-7 cables coming from upper mass 7 - CB-8 cables coming from upper mass For the cables in pictures 6 and 7, the peek zip ties are temporary to ensure connectors don't fall on the TMS optics.
The two attached plots show the following signals over the span of 6h (patch them next to each other): Blue: FSS control signal readback Green: FSS autolocker state Brown: ISS diffraction efficiency Cyan: IMC build-up Yellow: FSS temperature loop control signal Notable: - The ISS diffraction efficiency varied between 10% and 15%. This corresponds to an ISS loop gain variation of sqrt(0.15/0.10)~1.2, or about 20%. - The short lock losses were due to Ethercat reboots, resulting in IMC lock losses. The FSS never dropped below state 2, as is evident from the fact the the temperature loop did not forget its offset. - The IMC held lock through the magnitude 7.0 earth quake in the Aleutians this morning. The peak seismic signal at LHO was on the order of 20u/sec.
With the doors on and cleanroom now moved, I've installed all the sensors & electronics. Quick look at spectra suggest things are OK. The HEPI is still locked and the Vertical L4-Cs still need to be leveled and protected. There remains some cable dressing as well. Please be mindful of the sensors and cables at this location.
09:00 Cory, Cheryl, and Andres at End-X to move and attach TMS OB/Telescope 09:32 Betsy & Travis working on ITM-X 09:41 Richard at ITM-X working on SUS electronics 10:15 Justin check out dust alarm in Bake Out lab 10:15 Bubba/Apollo in LVEA working on cleanroom move 13:15 Thomas Vo Recycled EtherCAT in corner station 13:26 Hugh at HAM6 working on HEPI electronics 14:11 Andres dog clamping SR suspensions to optics table 14:32 Cheryl at End-X working on TMS
I attempted to replace Compressor #1's "noisy" scroll but the new scroll (SLAE05E) is not a direct substitute for the legacy scroll (SLAE05) -> I'll attempt to acquire the old part number or make a custom mount if needed (next week)
We've connected all of the cables on the ITMx QUAD and all 20 OSEMs have some sort of signal coming out to medm. Thanks Richard for some power and rack troubleshooting on QUADTST. There is some strangeness on the medm so Arnaud is looking into what model version is loaded on the QUADTST testbed. The main chain does not damp so we likely have crossed signals or a sign flip somewhere...? More from him to come.
Damping gains were still set to their old values. They are now set to -1, which is the convention for filter gains. Coilouput gains and signs have been checked, and are in accordance with the magnet convention (+ sign = north pole magnet, and vice versa).
The Continued Story of CB3
On Wed, there were unresolved issues with cabling for the TMS. The main issue was a cable bracket (CB3) which, on a drawing, is supposed to be on the east side of the table (i.e. -Y side of TMS), but having it here has limited access, and it would be even worse after we box it in when the Table/Telescope was installed.
Now the cables (TMS OSEMs) which use CB3 eventually run to a feedthru which is quite on the other side of the TMS (i.e. the +Y side of it); on this side of the TMS there is lots of open space & getting to the feedthru is trivial. So, after consulting with Cheryl/Mike/Calum, Jeff & I decided to move CB3 from it's "called out" location on the "edge" of the table to roughly the center of the table. So, the "quadrapuss" OSEM cable D1000234 was run to this feedthru (and excess bundled near bracket), then D1000225 was run straight "north" on the table and out to a spot where Jim W suggested---then he'd be able to dress it on the ISI and on to the feedthru. The beauty of this cable run is that we have elegant access to the cables & cable bracket.
This leaves us ready at the Cartridge to bring in the Table/Telescope.
Table/Telescope Installed On Test Stand: Promise!
Throughout the day, I also prepped the Table/Telescope for the trip form the TMS Lab to the Test Stand. I blew off the Telescope Mirrors, installed the TMS Cover (to protect Table optics), and arranged the Genie and Table/Telescope for the move first thing tomorrow morning.
Attached are photos of how Jeff B. ran the cabling and where he positioned CB3 (the D1000225 has about 14' of cable and it is coiled on the table ready to be grabbed by SEI for final dressing to feedthru).
Attached are plots of dust counts requested from 4 PM August 28 to 4 PM August 29.
Attached are plots of dust counts requested from 4 PM August 27 to 4 PM August 28.
To tie up some loose ends, I went back to the EY drift event of 07/19/2013 00:00 UTC (elog 7146), which was triggered by turning off the clean room.
The attached plot 1 shows 20 days around the event. Marked in dark blue is the turn-off event, marked in bright clue is the turning on of the same clean room a day after. I am not sure where exactly the two temperature sensors are located, but I amsure the actual temperature jump at the BSC was bigger (From John's elog 7149 is was >2degF).
According to the OSEMS the TMSY drifted by about 5 urad in pitch and yaw (ignore sharp steps due to realignments) - I don't think that can be responsible for us falling off the photodiode. so something else must have moved.
The ETMY is even less: ~2urad in pitch and nothing significant in yaw (plot 2).
The PZT actuators (under QPD control, slaving the beam to the TMS) show some shift, but nothing that would be very unusual.
So the bottom line is: I still don't understand why we fell of the diodes on ISCTEY that day.
'
I have been trying to familiarize myself with Rai's "hydrocarbon" RGA (software and apparatus) which is mounted on the East door of HAM3 and have included some preliminary scans taken today. These aren't intended to contribute to the eventual analysis but are interesting nonetheless. It looks to me like the lack of distinct hydrocarbon peaks will necessitate an accumulation on the RGA's 80K "cryo finger". We may need to compare scans with and without CP2. Rai has already used this same setup to quantify the hydrocarbon partial pressures at the Y-end and has commented on the expected impact on aLIGO mirrors (see T1100561, T1100642, T1200021 and T1200395). Note: these .pdfs contain "sticky notes" without which these scans won't have much meaning
- Apollo crew working on the floor of End-Y.
- Apollo installed HAM door in LVEA.
- Arnaud was taking TF of the BS.
- Dust monitors location updated (I guess Patrick did it).
- Richard went to mid-X to check equipment.
- Dave installed firefox on left operator's computer for camera views.
- Dave copy firefox's folder after crash (lost bookmarks) on main operator computer.
- Rick and Christina (her last day) working on PSL.
pablo
Last night, I left the two ISIs at the corner station with the aggressive feedback isolation filters and the feedfoward engaged. HEPIs are just controlled in position (100mHz UGF). Configurations were almost identical (cf item 5) on both ISI (low blend on translations and high blend on rotations to limit noise injection).
I have attached spectra of the stage 2 motion in the Y direction.
Note that:
1- ISIs didn't trip during a long stretch of time. The control is very robust (the ITMY feedback filters were even designed with the HEPI locked)
2- The stage 2 motion in the Y direction meet the requirements for both ISIs
3- The performance are similar for both ISIs
4- The feedforward control allows squashing the HEPI pier resonance on ITMY (feedforward filters clearly need to be reworked on BS)
5- ITMY performs slightly better than BS at low frequency. I noticed that the sensor correction was engaged on stage 2 in the Y direction. It might explain the difference
I have removed the lock and tag on the ALS light pipe entering HAM 1 and opened the shutter.
Ham 6 doors are on with 4 bolts each. Flange joints on hams 4,5,&6 are taped all around, started sealing vacant bolt holes on same flanges. Relocated clean room that was over ham 4 to area in front of hi bay. Tyler in machine shop, talked with Betsy regarding a longer tap, still working on SUS parts. Flooring guys at end y will finish this afternoon, with the exception of approximately 14' of trim. This trim had to be oredered and is 2 weeks out. This trim is where the seams come together and does not pose any contamination isssue.
For future reference, open light values have been taken for ITMX currently at the test stand (QUADTST on medm)
M0F1 22745 1.319 -11372
M0F2 25209 1.190 -12604
M0F3 25497 1.177 -12748
M0LF 25276 1.187 -12638
M0RT 24820 1.209 -12410
M0SD 23964 1.252 -11982
R0F1 25057 1.197 -12529
R0F2 30488 0.984 -15244
R0F3 29015 1.034 -14507
R0LF 29178 1.028 -14589
R0RT 27580 1.088 -13790
R0SD 27366 1.096 -13683
Here are the serial numbers inserted with the measured values:
M0F1 22745 1.319 -11372 SN 091
M0F2 25209 1.190 -12604 SN 494
M0F3 25497 1.177 -12748 SN 482
M0LF 25276 1.187 -12638 SN 509
M0RT 24820 1.209 -12410 SN 490
M0SD 23964 1.252 -11982 SN 111
R0F1 25057 1.197 -12529 SN 470
R0F2 30488 0.984 -15244 SN 467
R0F3 29015 1.034 -14507 SN 127
R0LF 29178 1.028 -14589 SN 115
R0RT 27580 1.088 -13790 SN 664
R0SD 27366 1.096 -13683 SN 080
Lower masses open light values for lower masses for ITMX (currently on quadtst) have been recorded
osem name olv gain offset serial number
L1UL 31509 0.952 -15755 439
L1LL 29214 1.027 -14607 510
L1UR 26506 1.132 -13253 474
L1LR 27979 1.072 -13989 487
L2UL 21877 1.371 -10938 146
L2LL 22829 1.314 -11414 263
L2UR 24994 1.200 -12497 112
L2LR 22583 1.328 -11291 463