We only restored PIT of HEPI2 (H1:HPI-HAM2_IPS_RY_SETPOINT_NOW) yesterday, but there was a change in ROLL (RX) on the order of 10 urad that was not restored, which might affect the input pointing.
Also, there was a jump in ROLL in HEPI3, again on the order of 10 urad, which would affect POP and ALS beam position on ISCT1 as well as POP sled in HAM3.
These DOFs were brought back to pre-maintenance levels.
I don't know/remember why, but it seems that a new set of angle setpoints is made for HEPI such that the system starts new with free-hanging position after each restart. Seems like this free hanging position changes over time.
Since any change in HEPI angle (RX, RY and RZ) on HAM2 and HAM3 would affect many things related to the alignment, most notably pointing references like IM4 transmission QPD and POP sled, it's a good idea to keep the set points unchanged over multiple restarts.
I noticed the autoburt VM was behind in patches, so I updated it and rebooted in between snapshot runs. There should be no gaps in the autoburt coverage, though the next run may take longer as the embedded gateway re-caches the PV names.
Replaced the ISI Coil Drivers D0902744 for HAM3. This is a temporary swap to help with troubleshooting the 0.6Hz seen in HAM3. Electronics removed: S1103321 U24-U25 S1103358 U23-U22 Electronics Installed: S1103316 U24-U25 S1103301 U23-U22
model restarts logged for Mon 12/Jan/2015
2015_01_12 08:59 h1nds0
2015_01_12 10:49 h1hpietmx
2015_01_12 10:49 h1iopseiex
2015_01_12 10:49 h1isietmx
2015_01_12 13:41 h1fw0
two unexpected restarts. ADC replacement of h1seiex in preparation for RCG2.9 upgrade.
model restart for Tue 13/Jan/2015
in attached file. RCG2.9 upgrade required all systems to be restarted. Several later unexpected DAQ restarts of frame writers.
Now that ETMX is back to vacuum, I measured a trend of the effective test mass charge voltage over 3.5 hours. The sensor was the oplev, since there is no cavity. I was told by Jeff K, who was told by Richard (or was it Kyle?) that the ion pump is valved out. The pressure was about 1.5e-6 torr, pumping with the turbo for about 2 days since the last vent.
The first pdf shows the charge seen the pitch and yaw oplev signals from excitation in each ESD quadrant. The second pdf shows the deviation from the average of each quadrant - oplev siignal pair, e.g. pitch from UL exc, etc. The largest deviation is -12 V. Most of the data is within +-10 V.
The charge measuring script is
/ligo/svncommon/SusSVN/sus/trunk/QUAD/Common/Scripts/ESD_UL_LL_UR_LR_charge_07-H1.py
The charge analysis script is
/ligo/svncommon/SusSVN/sus/trunk/QUAD/Common/Scripts/ESD_UL_LL_UR_LR_analysis07_H1.m
The charge trend script is
/ligo/svncommon/SusSVN/sus/trunk/QUAD/Common/Scripts/Long_Trend_H1.m
Yes, the ION pump is valved out. You can see on the EX Vacuum MEDM screen that IP12 is registering 4 decades better pressure than PT510. Therefore it is isolated from the BSC volume.
https://lhocds.ligo-wa.caltech.edu/screens/png/H0VE_EX-current.png
Jim, Dave: WP5003
This morning we upgraded H1 FE and DAQ to RCG2.9. First the DAQ was upgraded and restarted. Then FE startup scripts were changed, model startup scripts changed.
Then a "make installWorld" was ran to install the models which were compiled yesterday. The H1.ipc file generated yesterday was re-installed.
We tested a reboot-restart first on a non dolphin FE (h1susauxh2) and then on a Dolphin FE (h1seib1).
We then restarted all frontends in the MSR by power cycling the computers, and end station frontends by rebooting the computers. We ended up having to powercycle h1iscex because it got stuck in reboot. In the MSR when we were restart testing, h1susb123 got stuck on reboot and needed a power cycle which Dolphin crashed the other MSR machines.
We got to a stable system with DAQ and FE reboots proceding correctly. We had 6 front end IOP models with large IRIG-B excursions into the 1000-2000 counts. The h1oaf0 IOP took 4 hours to return to normal IRIG-B range. One IOP went below zero (999998) which caused IPC errors on models receiving its channels. This fixed itself in about 10 minutes.
We noticed a single ADC error on the h1ioplsc0, which was cleared with a DIAG_RESET.
We noticed the DAQ frame size is smaller now. For science frames 670MB to 610MB (10% reduction) and commissioning frames 1.2GB to 1.0GB (20% reduction). We suspect new common models are the reason for this.
I linked Jamie's SDF_OVERVIEW MEDM screen to the SITEMAP under the GRD section. We are configuring the IOP, PEM and PSL systems.
This closes WP5003
Frame size reduction was bogus. I checked too soon after the reboots when systems were not fully engaged and the frame was compressing zeros. After a few hours of ifo operation the frame sizes came up to nominal. Attached plot show commissioning and science frame sizes for the past two days.
Some of the ODC channels now appear to have string values with unrecognized characters. For example: caget H1:PSL-ODC_FSS_BIT21 H1:PSL-ODC_FSS_BIT21 FSS Not Resonant (Loop 300370216d377177 I regenerated the channel list, searched it for channels with ODC in the name, added these to the exclude list, and regenerated the channel list again.
The alog didn't like the backlashes. Here it is with forward slashes in place of backslashes: caget H1:PSL-ODC_FSS_BIT21 H1:PSL-ODC_FSS_BIT21 FSS Not Resonant (Loop /300/370/216d/377/177
(To know the whole story about the Z-RZ coupling, see DCC document #E1400432.)
I turned ON the substraction on ETMY-ISI ST1 with good results. If you look at the first attachement:
- Red curve -> nominal configuration (RZ loop OFF)
- Blue curve -> RZ loop ON. As expected, we amplified the RZ motion around 0.1Hz by turning the loop ON. The loop provides some isolation above 0.5Hz. Z doesn't change.
- Brown curve -> RZ loop ON + Substraction ON. We can see an improvement by almost a factor of 10 in the RZ spectrum around 0.1Hz. The substraction seems to work just fine.
As Ryan suggested, we also wanted to see if we can obtain similar results without the substraction. We do have Z sensor correction on HEPI, so we might be able to relax the actuation on Z (meaning higher blend) and still have reasonable performance in vertical. By relaxing Z, the couping with RZ should be reduce. Unfortunately, the Z motion seems to big for our need if we swich to higher blend (see second attachement).
Last attachement shows different substraction configurations:
- Red curve: Substraction ON with a 90mHz blend on Z and Z sensor correction on HEPI
- Blue curve: Substraction ON with a 45mHz blend on Z and Z sensor correction on HEPI
- Brown curve: Substraction ON with a 90mHz blend on Z and Z sensor correction on ISI
Viewing those results, it seems that switching to a lower blend on Z might be a good idea.
For further analysis, we need to look at more sensors and the optical lever. We might want to look at the other chambers as well.
The attached pdf compares Zach Patrick's BSC-ISI model against measurements of all H1 BSC-ISIs. Overall, the magnitudes are pretty good fits. Some of the TFs have right half plane zeros which throw the phase off. This will be a problem when trying to close loops around the model. Thus, I think we need to refit the model. I think this model was fit with frequency domain data in N4SID, but we now know that time domain is best (for good coherence). We also don't have any ground to stage motion fits, so we could add those in at that time.
I spent some hours trying to correct the right half plane zeros without success. My conclusion is that it is easy to fix in a SISO system by simply reflecting the right half plane zeros around the imaginary axis in the complex plane; but it is not feasible to do this with a MIMO system with dozens of poles and zeros. To do this you have to convert the state space to a zpk, which is fine. The problem is converting from zpk back to ss. For systems with multiple outputs, matlab does not have a numerically stable way to convert from zpk to ss. In fact it warns you about this in the comments to zp2ss.m. You could leave the model in zpk form, but it runs extremely slowly. Generally, state space is the best form for MIMO systems, since it permits the use of a lot of MIMO tools and analysis.
The model is located on the svn at
/ligo/svncommon/SeiSVN/seismic/BSC-ISI/Common/BSC_ISI_Model/N4SID_Model/L1BS/BuildFullL1BSmodel.m
To compare the model to data, run
/ligo/svncommon/SeiSVN/seismic/BSC-ISI/Common/BSC_ISI_Model/N4SID_Model/L1BS/CompareModel_to_data.m
Per work permit 5004, installed DC Power Breaker Switch Boxes D1400007 in-line with HEPI Pier Pods (D080520). The DC power breaker boxes were installed in the CER racks and provide over-current protection. Work was performed around noon and includes: HAM1-HAM6, BSC1, BSC2, BSC3, BSC9, BSC10.
Karen & Cris: Cleaning in the LVEA Hugh: Starting work in MR for WP 4913 07:30 - 07:45 Jodi: Picking up 3IFO optics from mid Y 08:15 Jim B.: Stopping data concentrator. Building data concentrator version 2.9. Restarting DAQ to bring to version 2.9. 08:18 Jeff B. & Andres: Moving suspensions onto platform in LVEA. Going into bifurcated laser safe area. 08:20 Hugh: Retrieving parts from LVEA 08:49 Unifirst: Truck through gate 09:10 Karen: Cleaning at mid Y 09:13 Dave B. & Jim B.: Starting upgrade of frontends to RCG 2.9 09:31 Travis & Danny: Moving welding equipment in LVEA 10:03 Kyle: Checking access for WP 5005 10:11 Brian S. & Mitchel: Putting equipment in LVEA 10:26 Karen: Leaving mid Y 10:46 Travis & Danny: Entering LVEA 11:00 Cyrus: Firmware upgrade on OPS work station 11:04 Praxair: Truck through gate 11:05 Corey: 3IFO work at mid Y 11:10 Travis & Danny: Out of LVEA 11:30 - 11:50 Jodi & Jeremy: Surveying mid stations for 3IFO storage 12:00 Brian, Danny & Mitchel: Out of LVEA 12:04 Jeff B. & Andres: Out of LVEA 12:31 Dave B. & Jim B.: RCG upgrade complete 13:13 Corey: 3IFO work in squeezer bay 13:17 Filiberto & Sudarshan: Checking PSL power supplies 13:26 Kyle: Checking pump at end X 13:53 Richard: Turning on ESD at end X 14:08 Richard: Back from end X 14:12 Kyle: Back from end X 14:19 Jeremy & Brian: Loading equipment outside of LVEA Jim B.: Updating GDS software WP 4997 15:00 Dale: Tour in control room 15:12 Jeff B. & Andres: Out of LVEA. Had been transferring suspensions into LTS container. Started around 13:30. 15:21 Keita: Going to squeezer area to talk to Corey 15:35 Ketia: Out of squeezer area 15:59 Corey: Out of squeezer area
Spent the morning getting all the cable runs sorted and the pressure sensors zero'd. Switched databases so the servo would run on the differential pressure collected at BSC2 in the LVEA. This will address a long standing Integration issue and work permit. Still need to do similar at EndX before these items are fully resolved. The EndY has been running on the differential for several weeks.
Couple issues here at the corner station: 1) As I was doing some final cable coiling etc this afternoon, I lost the controller signal to the South Pump Station. I valved the PS out to stop the back spinning. I'll restart this next Tuesday; I think I can restart it without harm but didn't want to risk it after being off all morning and commissioning now ongoing.
2) Pressure sensor 3 on Pump Station 3 (3rd from South) has an issue so the signal is actually coming from sensor #2. The apparent 25psi differential pressure across the 3u filter is not true.
3) The medm still shows the Differential Pressure coming from the Main Supply Line Sensor, again it is really coming from BSC2.
Again, I'll deal with issue 1) on Tuesday. For issues 2) & 3) I'll rework the medm to correctly reflect reality.
Attached is the current medm.
Okay, adjusted the medm. So the servo pressure appears to be coming from BSC2 which it is. Also annotated the PS3 sensor issue. I've also adjusted the database calc so the red Pressure NOT OK will go green, the next time I restart the process, next Tuesday. Corrected medm attached.
Jeff B & Andres R This morning we loaded the 4 3IFO Quad Upper Structures (Quad-6, Quad-7, Quad-8, & Quad-9) onto the HAM ISI storage container #2. No issues or problems were encountered during the load. We will load the 2 TMS telescopes and Elliptical Baffles into this container during the week as opportunity allows.
Version 3 (svn tag 97) FPGA code was loaded into all timing comparator/frequency counters. This addresses:
There can still be a problem with crosstalk between channels, especially at frequencies above 200 MHz and signals stronger than 0 dBm. This has only been seen in open channels with no input. Channels with an RF input signal don't seem to suffer from it.
The CER and EX units were swapped with units which had a proper heatsink installed for the 5 V regulator. The following units were modified:
Serial | Location | Heatsink | Code |
---|---|---|---|
S1107952 | CER | installed | 3 |
S1201886 | EX | installed | 3 |
S1201222 | EY | installed | 3 |
S1201227 | shop (MSR) | 3 | |
S1201224 | shop (spare) | 3 |
J. Kissel, S. Dwyer, D. Sigg, E. Hall, N. Mavalvala After performing all of the usual alignment position checks (SUS, SEI, MC TRANs QPD, IM4 TRANS QPD), after the RCG 2.9 upgrade (see LHO aLOG 16040) we've confirmed that the IMC REFL path has jumped down in pitch again as it has done several times in the past after major computer reboots or SEI/SUS events (see Integration Issue 854, LHO aLOGs 10335, 11481, and 15650). We didn't preserve camera views prior to the reboot, and I didn't get the chance to capture an UNLOCKED camera view in which the REFL path was similarly low, but I attach the misaligned views after the RCG, then SEI and SUS recovery before realigning the path, and the realigned views after Evan, Sheila, and Nergis have realigned the whole path. We've also confirmed that the MC WS are running happily now as well. Note that the IMC REFL L RFPD and trigger PD are on different image planes and lens focal points than the MC WFSA and MC WFSB and MC REFL GIGE Camera (see D0902284), so it's to be expected that they're differently affected by a misalignment of the MC REFL path. In other words, the IMC LSC path is much less sensitive to misalignments than the IMC ASC path (which contains the MC REFL GIGE Camera View). Finally, for future reference, I attach a screenshot of all of the relevant camera views and screens I can for the IMCASC and surrounding sensors, with 2015-01-13_H1IMC_Status_ScreensandCamera.png
We added an iris to the IMC refl path on IOT2L, in front of the calcite polarizer, in the hopes that we can use this next time to recover the alignment.
Keita, Brett and I also noticed that HAM2 HEPI moved between before and after the bootfest, as can be seen both on the oplevs and the IPSs. (RY, which is pitch, was different by about 15 urad before and after the bootfest) We have restored HAM2 RY, but I didn't finish checking this for the rest of the chambers since the schnupp crew want the michelson locked for now. In the morning we will have to go through the rest of the chambers and redo the alignment procedure.
This isn't a big enough shift to explain why we had to realign on IOT2L earlier, but restoring this did bring the IM4 trans QPD back to almost the same position it was at before the bootfest.
WP 4997 The gds tools have been updated to branch/gds-2.16.12.4. This includes two bug fixes: Bugzilla 288 - add ramp down of excitation to swept sine response measurement. (diaggui) Bugzilla 778 - Make filter module lists iterable in Python (foton) This affects only Ubuntu workstations. The update of nds2-client software mentioned in the work permit will not take place until a bug is fixed affecting collection of trend data from NDS1 servers.
I've done a survey a of all the SEI Guardians with regard to the position dofs that are restored (maintained everytime.)
HAM_ISIs maintain every dof but the HAM_HEPIs only maintain the RZ dof. I'd say this is backwards given the range and more importantly hysteresis of HEPI.
While at it, for the BSCs: their HEPIs all maintain the RZ dof, the ETMs maintain their pitch and the ITMY maintains pitch roll and X dof to deal with ACB install and Initial Alignment. The BSC ISIs maintain NO dof.
It certainly makes sense to me that at least the HAM2 & HAM3 RX & RY dofs are also restored along with the RZ. Don't know why we wouldn't do this with HAMs 4, 5 & 6 too. The X Y & Z dofs are probably okay to let wander but commissioners can comment.