Per Work permit 5173 A continuation of the work done yesterday. Today the Seismic and Suspension AA/AI chassis at bot EX and EY were swapped for V6 chassis. All Chassis in the electronics room have been addressed at this time and are V6.
The pressure at XEND has stepped up to 1e-7 due to the leak testing activity. No cause for alarm as this appears to be the purge valve leaking. More info to come later today.
Here are plots from yesterday, again around 0130pdt like previous posts: Monday, Friday (18046), and Wednesday (18007.)
Again (looking at the ASDs,) the Z channel looks like all the others and better for STS2-B (ITMY) compared to reference (old seismo.) Also, the X axes all look good (similar) now (compared to Monday.) Before it wasn't clear, maybe the seismic environment wasn't as benign as I thought. For the Y DOF ASDs, HAM5 & ITMY sensors diverge below 20mHz but more telling, the HAM2 Y DOF goes off track from the other two around 70 or 80 mHz. The traces also look differnt relative to the Noise trace, something has happened to the calibration I need to sus out.
The TFs I've presented before are a little too busy. I'll rework their arrangement for the future.
Added 804 channels. Removed 918 channels. 2 channels remain unmonitored: H1:SUS-SR2_LKIN_P_OSC_SW2R H1:SUS-SR2_LKIN_Y_OSC_SW2R
Scott L. Ed P. Chris S. 4/27/15 the crew worked on crevice and tube cleaning ending 2.3 meters north of HNW-4-016. Tested clean areas (posted on this report) and began moving lights and cords to the next section. Vacuumed support tubes and sprayed bleach/water solution. 4/28/15 continued vacuuming and began tube cleaning. Cleaned 55 meters ending 17.5 meters north of HNW-4-018.
J. Kissel I've accepted a few changes in the SDF system, and chose to not monitor a few more before bringing the end station SEI and SUS systems to OFFLINE, in prep for their AA/AI chassis to be swapped out for v6. They are ready to go for the swap!
After Evan and Sheila made the first steps towards maintenance recovery I stuck around for a few hours working through the Guardian steps towards resonance.
In alignment, the PRC2 loop is not working in the DRMI state. I reverted the input matrix from 1.0*REFLA_9I --> PRC2 to the settings we used last night, 0.5*(REFLA_9I + REFLB_9I) + 0.83*(REFLA_45I + REFLB_45I), but this didn't give us a good error signal either. For this evening I commented out the loop from the DRMI Guardian and tweaked PR2 by hand, it worked well enough to move forward. Oddly, it's only the yaw loop that looks bad.
The AQ45--> DARM matrix element needs to be re-tuned in the late stages of the CARM offset reduction. We are losing lock at the 10picometer state -- the optical gain is increasing and DARM becomes unstable. The DARM loop in the 15pm state is ok (40Hz ugf, 50deg margin).
Looks like the aperture mask on the ITMY green camera can be much more improved. See the attached picture. I removed the aperture for now as it occulted large fraction of the spot. I will tune up the mask tomorrow.
Kyle, Gerardo Our X-end pressure has been higher than nominal following activity in September 2014 which required cycling GV20 -> Data virtually mimics issue LLO had (has?) at their X-end station (see G1500110-v1). As such, we had intended to spray helium on/in/around GV20 today but the VE configuration is not good (we are open to the BT) for the anticipated leak size and we also have been lead astray by the response of tightening the Vent/Purge valve which should have 1-liter of "rough" vacuum on the other side of its metal seal. PT510B responded and the X-end pressure went down after we tightened the Vent/Purge valve -> Tightening this valve a little more resulted in a further pressure reduction -> I don't like looking at the visible "gap" on the conflat joint as I am torquing this valve knowing that we are open to the tube so we stopped and went looking for a torque wrench and the specs. for this valve -> We will continue this investigation as commissioners permit.
815 - Richard, Filiberto to EX
840 - Dick G. in LVEA
852 - Elli in LVEA
859 - Joe in LVEA checking first aid kits
920 - Richard, Filiberto back from EX, heading to EY
929 - Jason, Ed to EY for OpLev laser replacement
939 - Corey in LVEA looking for laser in Squeezer Bay
940 - Kyle, Gerrardo to EX for Vac work
945 - Elli back
945 - Elli, Nutsinee to EY for HWS
951 - Corey out, to EX
1004 - Richard, Fil back
1012 - Karen to EY
1014 - Jason, Ed back
1018 - Jason, Ed LVEA near PSL rack
1032 - Richard, Fil in LVEA to finish SEI work
1035 - Jason, Ed out
1045 - Water delivery
1046 - Elli out
1058 - Joe out
1101 - Bubba out
1119 - Richard, Fil out
1125 - Karen Leaving EY
1141 - Elli in LVEA next to HAM1
1215 - Kyle, Gerrardo leaving EX
1230 - Kyle in LVEA to tighten valves
1230 - Jeff K. in LVEA
1234 - Jeff K. out
1237 - Elli out
1238 - Kyle out
1315 - Elli in LVEA
1334 - Elli out
1430 - Kyle to MX
1448 - Dick G. out
The GDS package has been reverted to the previous gds-2.16.12.4 version due to a bug with ramp down of excitations with the gds-2.16.17.2 version installed earlier today.
J. Kissel I've completed the HAM-HEPI and HAM-ISI model upgrades as described by the step-by-step instructions yesterday (and today) in 18078. It should be noted, that with the amount of top-level model modifications and rearranging, one should definitely reserve two days for this update -- one for making the changes for the models which can be done "offline," and one for the model restarts and debugging. We still have some minor MEDM screen debugging to do, but we should be able to get this done by the end of the day, and commit when ready. However, for LHO, this closes out the following ECRs and integration issues: - Change the count for the saturation counters so that all of the HAMs have the values used now in HAM6. E1500186 :: II 1018 - Put a 10 sec delay on the connection monitor for the SUS Payload WD alerts. E1500196 :: II 1028 - Install parallel feed-forward paths from the Stg 0 L-4Cs and the HEPI L-4Cs; Disconnect the L4Cs from the HAM Sensor correction. E1500179 :: II 1026 - Clean up STS-2 wiring/ update the STS -> HAM inputs so that we don't overwrite last two ADC channels (which are reserved on ADC0). E1400386 :: II 942 - Put the QPD part into the HAM model for the HAM Optical Levers. E1400452 :: II 971 - Add monitor channels to the frames for the STS and displacement sensor signals in the Sensor Correction Path (done for HAM-ISIs and HAM-HEPIs only with this update). E1500216 :: II 1046 - Fix L4C routing bug in HPI template (done for HAM-HEPIs only with this update) (Bug fix -- no ECR) :: II 1045 There were a few hitches including - discovering how the IOP bit logic works, and that it needed inversion before going into the library part - releasing that I needed updates to the isi/common/src directory as well as the model and medm directories - remembering that the BSC HEPIs would be affected by the hepitemplate update - a nasty hiccup in that somehow the version of the h1isiham6.mdl I was editing was in my home directory (where I'd never [intentionally] saved it) instead of in the userapps repo, so I was compiling something I hadn't yet edited - a fix to the HAM3 and HAM5 HEPI models because of the differing cardNum vs block name identification of the ADCs but I persevered in the end. All affected top-level models have been committed to the userapps repo for demonstrative purposes when LLO performs their upgrade.
Jim W had finished his TFs, so we went ahead with a DAQ restart. The restart failed due to duplicated slow channels in the corner station Beckhoff EDCU files. Both H1EDCU_ECATC1PLC1.ini and H1EDCU_ECATC1PLC12.ini had the following two channels defined:
for now I have removed them from both INI files.
h1dc0 actually starting running, with an EDCU channel count of zero. This is unusual, it would normally not start. However the other DAQ systems did not restart.
I restarted h1dc0 via port 8087 as usual, it ran correctly but the other DAQ components did not restart. I used monit to restart the frame writers, NDS and broadcaster. All systems then came online.
The EDCU is now green (I had to manually removed TCS_ITMY from the guardian INI file)
J. Kissel In prep for the HAM-ISI and HAM-HEPI Model upgrade (see SEI aLOG 708, WP #5171, E1500207, and a slew of ECRs and Integration Issues), I've put together a step-by-step instructions of all of the everything that I need to do today and tomorrow. Most of these upgrades were tested at Stanford and MIT, so I don't expect too many problems with the infrastructure, but they can only test/describe so much: - they only have single chambers, not many chambers each with different top-leve needs - those chambers don't have SUS on them that need to stay aligned to anything, - they don't have IOP watchdog systems, - they don't have settings that the really care about in any of their SDF systems, - they don't have multiple STSs to store with distibution chassis, and - they don't have optical levers on their HAM-ISIs So this will be the first full-scale test and I wanted to get my ducks in a row before I touched anything. See attached.
Biweekly crash recovery Logged in. Closed system manager. Said no to save current changes. Restarted computer. Errors in EPICS IOC. Closed EPICS IOC. Restarted computer again to get screenshot of errors in EPICS IOC (see attached). Closed EPICS IOC. Opened LIGO TwinCAT Target Configuration GUI Selected H1ECATX1 -> SYS, PLC1, PLC2, PLC3. Clicked Stop. Clicked Update from source. Clicked Compile. Failed (see attached) Ran svn update on C:SlowControls. No changes at end X. (see attached) Ran svn st on C:SlowControls. No local modifications. Opened LIGO TwinCAT Target Configuration GUI Selected H1ECATX1 -> SYS, PLC1, PLC2, PLC3. Clicked Stop. Clicked Update from source. Clicked compile. This time it completed successfully. Clicked Activate and run. Got message: No boot project created since last download. Exit anyway? Clicked OK. Said it completed successfully. Clicked Restart EPICS database. PLC1 and PLC3 came back. PLC2 did not (see attached) Why is the location Corner? Opened PLC2 from target. Logged in and selected run. PLC2 came back. Went to /ligo/cds/lho/h1/burt/2015/04/28/00:10 Ran: burtwb -f h1ecatx1plc1epics.snap burtwb -f h1ecatx1plc2epics.snap burtwb -f h1ecatx1plc3epics.snap
Work Permit 5175 Per ECR-E1400386 V5 In conjunction with the Model changes made by Jeff K. we have removed the STS-2 signal cables from the STS-2 Fan out chassis for each BSC to the Ham 3 and 5 AA chassis. Since Hams share the front ends we did not need to duplicate the signal path. Six cables were removed.
J. Kissel In order to get the latest and greatest hardware injection infrastructure from LLO (for the ODC part, see LLO aLOG 17911), I've - svn updated the hardware injections library part, /opt/rtcds/userapps/release/cal/common/models/CAL_INJ_MASTER.mdl - added an SHMEM IPC sender called H1:CAL-INJ_ODC_IPC to the top level of /opt/rtcds/userapps/release/cal/h1/models/h1calcs.mdl - svn updated the library part /opt/rtcds/userapps/release/sys/common/models/ODC_MASTER_PARTS_V2.mdl - added an SHMEM IPC receiver called H1:CAL-INJ_ODC_IPC to the top level of /opt/rtcds/userapps/release/sys/h1/models/h1odcmaster.mdl and connected it to the last input - compiled, installed, and restarted both models - committed the top-level models to their respective location in the svn. This closes WP 5172.
Did not in the end move it except vertically a few mm as I cut out holes in the vinyl floor under the feet. It is back under the igloo and stabilizing.
H. Radkins, J. Kissel, Sadly we had to break the very-long-standing RF lock. I've brought all IFO / ISC guardians to DOWN, and I've reloaded the IMC guardian in hope to receive T.J.'s new "OFFLINE" state for the IMC, but there's some bug there. I've left it in INIT and misaligned MC2. Hugh is beginning to bring the platforms to OFFLINE.
IMC_LOCK has been fixed and works now. Hopefully it stays that way. Things changed are:
This has now been committed to the svn.
Evan, Dan, Sheila. Kiwamu
This afternoon we moved the picomotors on both transmon IR QPD sleds, although we moved Y much further than X. This was to reduce the clipping and saturation Keita pointed out in 18077. After that we excited TMSY in both pitch and yaw, and found error signals for the ITMs that are insensitive to TMS again.
We have closed all the loops in full lock and things seem stable, we were able to increase the power at 12:00 UTC on the 28th and saw that the recycling gain stayed stable. We stopped at 6 Watts because the Y QPD is near saturation. We should probably do more picoing (most of the light is on one quadrant), and reduce the whitening gain.
The attached screenshot shows the settings we have now, the DSOFT and CSOFT loops could have gain of 1, but we were cautious.
We had some difficulty turning on the ASC loops earlier tonight. One thing that we noticed was that we were able to close INP1 each time, even when we didn't close PRC2 or CHARD (or SRC). We also had trouble closing the CHARD PIT loop, although YAW was fine. Once we manualy adjusted the alignment to improve the recycling gain and closed the ITM loops, we closed CHARD PIT with no problems.
We have left the ASC engage commented out in the guardian, because it is not working right now. We also attempted earlier in the evening to take Keita's advice and manually aling the green PZT durring a full lock with good recycling gain. We had difficulty getting the build ups high while minimizing the WFS signals, we adjusted the QPD offsets to keep the build ups high but this resulted in a bad recycling gain. Probably a more thorough attempt is waranted.
ETMX ESD tripping
Since saturday, the ETMX ESD has been tripping almost every lockloss. We also need to toggle BO 4 to reset it, which is not required to reset ETMY.
durring this 2.25 hour lock, the recycling gain stayed fairly stable. As the POP LF trend in the attached screenshot shows, the recylcing gain dropped by about 5% in the first half hour of the lock. the Y trans mon QPDs see a drift in yaw on a similar timescale, although the X arm yaw loop control signal (DSOFT Y OUT) doesn't respond to this, which would indicate that the drift is from TMS, which our combination of error signals is chosen to be insensitive to. In conclusion it seems like the QPD loops are doing their job. As keita suggested, it is possible that the change in recycling gain in the first half hour is a thermal effect.
