model restarts logged for Thu 22/Jan/2015
2015_01_22 00:06 h1fw0
2015_01_22 00:42 h1fw0
2015_01_22 01:41 h1fw0
2015_01_22 03:05 h1fw1
2015_01_22 03:31 h1fw1
2015_01_22 04:03 h1fw1
2015_01_22 04:29 h1fw1
2015_01_22 05:37 h1fw1
2015_01_22 23:39 h1fw0
X1PLC1 10:30 1/22 2015
X1PLC2 10:31 1/22 2015
X1PLC3 10:30 1/22 2015
Y1PLC2 09:43 1/22 2015
all unexpected frame writer restarts. Recent fw period of instability ended at 5am, could be related to internal RAID diagnostics, investigation proceeding. Conlog frequently changing channels report attached.
In order for people to get some undisturbed DRMI data for glitch hunts or seismic performance evaluation, we've left the DRMI locked with the ETMs misaligned starting at 1:09 AM local time (0909 UTC).
Its been locking fine tonight with only tiny angle tweaks made after locking (but probably unneccesary). Hopefully it will relock by itself overnight. If so, the morning SEI crew ought to be able to be able to switch the seismic filter state and re-eval the control signals with only 0.1-0.2 urad tweaks to PR3, SR3, and BS.
We think that the DRMI CAL channels are out of date, but Kiwamu can probably tell us what they mean in meters units with almost no effort.
Rana, Evan
Tonight, we took some transfer functions from the ETMX/ETMY ESDs to LSC DARM IN1, when controlled by ALS DIFF. Right now we only use ETMX as our DIFF actuator (as we had previously had some trouble using ETMY), but perhaps it is time to revisit this. Using both ESDs will double our range. Additionally, by properly balancing the relative drive strengths between the two ESDs, we can make ourselves more immune to cross-coupling between the ALS-controlled DARM and CARM loops.
For the excitation we drove each L3 TEST L filter bank with 40000 ct from 10 Hz to 40 Hz.
Evidently, ETMX and ETMY have opposite signs and their gains differ by 6 dB or so (at least below 20 Hz, where the coherence is OK). Since we're aiming for a DIFF loop with a UGF of 15 Hz or so, we don't care so much what's going on above 20 Hz, so long as it's not outrageous. So we (hopefully) should be able to implement a balanced ESD drive with little trouble by engaging DIFF with both ESDs (with fiducial output matrix gains), then turning on a line in DARM, and then adjusting the output matrix gains until the line is nulled.
This plot shows the ESD DAC range used by a single ETM to lock the ALS DIFF loop (with the UGF around 15 Hz I think).
The dip between 20 and 200 Hz comes from a filter called "NicLP" which has now been renamed to be descriptive. The RMS of 14000 counts is dominated by the bump at 15 Hz (due to poor phase margin).
We replaced this with another filter (RLP33) which is just a pair of poles and a pair of zeros. This gives us back 12 deg of phase at 15 Hz and only increases the RMS drive by <10%.
We then brought in the ETMY ESD and saw that it does indeed run stably and decreases the load needed on each ESD. It went unstable after ~10 sec due to some low frequency oscillation, but we think that is probably just the L2P being off on ETMY (or maybe that we have not turned on ETMY UIM or PUM drives). We should be able to get this going with ~1 hour of work tomorrow.
Evan, Alexa, Elli, Sheila, Rana,
We think that we are loosing lock due to some 12-15 Hz noise in CARM which is non stationary. We watched the spectrum of CARM locked on ALS with and without DIFF locked, and saw that the non stationary noise is only there when DIFF is locked. We then looked back at our ALS DIFF design (alog 15025) and decided to reduce the gain by 35%. This has reduced the gain peaking, but we still have nonstationary noise in CARM.
lock loss times: 1:44:20 UTC, 3:01:27 UTC Jan 23rd UTC lock loss durring swept sine excitation 3:55 UTC
The first attachment is of a REFLBIAS transfer function when we had partially transitioned between TRXY and ALS COMM (CMB IN1 Gain was 4, and CMB IN2 gain was -11). When we measured this TF we had not engaged the FM5 boost. Still, this does not look correct ...
The second attachment is of ALS DIFF and CARM spectra. It seems we had some gain peaking in ALS diff, which motivated us to lower the gain as Sheila mentioned. This reduced the peak in CARM, but did not eliminate it.
Here are some more lock loss times for last night:
23/1/15 00:03:03 UTC
23/1/15 01:14:01 UTC
23/1/15 01:43:45 UTC
23/1/15 02:17:01 UTC
Attached is lockloss plot at 23/1/15 00:03:03 UTC. LSC-REFL_SERVO_IN2GAIN was turned down at 00:03:03 -26 seconds. Lock was lost 26 seconds later. CARM noise dropped once the gain was turned down. Also attached is a power spectrum of CARM signals before and after turning down the gain. REfs 0 and 1 are before, refs 4 and 5 are after. Puzzlingly, the high frequency noise increased after the ALS gain was turned down.
Here are some other lock loss plots from yesterday. The first one shows that the ALS Y glitch caused one of our locklosses, the Y arm transmission went to 0, with the charachteristic glitch in Y REFL CNTRL, as seen in alog 15242 and 15402. We think this was bad luck, and a reminder that these glitches are a problem.
The other lock losses were less conclusive so far, I've attached a plot that shows that as the gain of the ALS path is ramped down on the common mode board we have large glitches in CARM, however the lock survives this and drops a few seconds later.
Sudarshan K, Rick S, Darkhan T. Pcal beam power measurements were taken at EndY. Measurements inside TX module: OFS PD outputs 3720 counts, 2.25 V. TX PD outputs 1592 counts. Laser power: 2 W. Servo loop is open. Heat sink is not installed. Power measurements before any optics (after the laser shutter) at different voltages: 1.05 W at 3.0 V 1.35 W 3.5 V 1.60 W 4.0 V 1.86 W 4.5 V 2.07 W 5.0 V Power measurements, laser power set at 5 V: OFS PD: 9.06 V 14850 cts. 2.72 mW TX PD: 4.2 V 6930 cts. 10.1 mW Power meter is used with OD1. NE01A-B + NE02A-B attenuators attached to OFS PD. Power reading inside of TX module between two lenses (combined power of two beams going to ETMY): 1.69 W. Measurements of two beams at the TX module: Inner beam: 844 mW Outer beam: 824 mW Measurements of two beams at the RX module: Inner beam: 827 mW Outer beam: 810 mW Both beams combined: 1.63 W RX PD mon (both beams combined) output: 10.3 V Pictures of Pcal spots on ETMY (f/22, 30 sec): LHOetm_0250.jpeg - inner beam only on ETMY (outer beam is blocked) LHOetm_0251.jpeg - outer beam only on ETMY LHOetm_0252.jpeg - both beams We left Pcal laser ON, beams on ETMY.
This morning I focused the ETMx GiGe camera (CAM 25) in IR. I had to zoom in closer to the optic to get the pitcure into focus. For reference, attached are photos of ETMX before zooming in and after zooming in (which is the current camera configuration).
Using Dave's extract from the Guardian logs since the RCG 2.9 restart to generate an exclude file. I've loaded SDF Tables for all LHO HPIs (except HAM1.) Attached is a snap of the donot monitor file; this is Dave's file with filter module core names removed (Guardian uses ezca which doesn't have to use a complete name to manipulate a switch) and the HPI MASTER_SWITCH added (the SEI Manager toggles this switch, not the HPI Guardian.) Also in the image is the difference table from the SDF before the donot_monitor list was added. After updating the Table, the diff list went to zero channels.
All HAM HPI currently have no diffs--let's keep it that way. The BSC HPIs have a few diffs (1-4,) these are a few ISC settings and sensor correction gains that need to be corrected by updating the safe.snap value. Fab tool!
Thanks to Dave & Jonathan.
All HEPI safe.snaps and exclude lists are committed to the svn from /opt/rtcds/userapps/release/hpi/h1/burtfiles
J. Warner, A. Pele, J. Kissel Jim has improved the isolation loop design for the HAM2, HAM3, HAM4, and HAM5 ISIs by increasing the frequency / aggression of the Level 3 isolation filters (along with tweaking the UGF and high frequency rolloff such that the aggressive boost remained stable). As such, we have a great deal more feedback gain in the 0.5 to 5 [Hz] region of all DOFs for these HAMs -- the region where the noise performance is typically loop gain limited -- and therefore the performance improvement has scaled linearly with the gain increase. See "IsoFilterComp" attachments. We didn't have noise budget yet to prove it, but Jim had recognized that HAM6, the most recently designed chamber had much better performance, and much high boost frequency. Arnaud has gathered performance data comparing platform displacement before and after the change, which confirms the goodness locally. See "PerformanceComp" attachements. We didn't grab / compare the rotational degrees of freedom but these have also improved. Now that Arnaud has installed properly-matched, sensor correction, gains (see LHO aLOG 16208), we should expect a little bit better translational performance, so we'll compare again and make sure we capture rotational DOFs. Further, we plan to use calibrated DRMI cavity control signals to ensure that what we've done locally is good for the IFO. The parameter files for these new current designs can be found here: HAM2/Scripts/Control_Scripts/Version_3/Hori_ISO_H1_HAM2_Lv3_HR_2014_10_23.m HAM2/Scripts/Control_Scripts/Version_3/Vert_ISO_H1_HAM2_Lv3_HR_2014_10_22.m HAM3/Scripts/Control_Scripts/Version_3/Hori_ISO_H1_HAM3_Lv3_IMC_HP_2013_01_16.m HAM3/Scripts/Control_Scripts/Version_3/Vert_ISO_H1_HAM3_Lv3_IMC_HP_2013_01_16.m HAM4/Scripts/Control_Scripts/Version_3/Hori_ISO_H1_HAM4_Lv3_HR_2014_09_22.m HAM4/Scripts/Control_Scripts/Version_3/Vert_ISO_H1_HAM4_Lv3_HR_2014_09_16.m HAM5/Scripts/Control_Scripts/Version_3/Hori_ISO_H1_HAM5_Lv3_2014_09_25.m HAM5/Scripts/Control_Scripts/Version_3/Vert_ISO_H1_HAM5_Lv3_2014_09_25.m
The start of the path to the filter parameter files indicated is /ligo/svncommon/SeiSVN/seismic/HAM-ISI/H1/HAM(#)/Scripts/Control_Scripts/Version_3/(filenames)
8:38 Rick to EX to pick up PCal equipment
9:30 Benton PUD on site
9:32 Fil to LVEA working on PEM mics
9:38 Karen to MY
9:40 Rick done at EX, heading to EY for PCal work
9:57 Jeff and Andres to LVEA for cleanup
10:09 Jeff and Andres out of LVEA
10:45 Benton PUD off site
11:00 Pepsi truck on site
11:10 Karen done at MY
12:23 Fil back to LVEA PEM work
14:00 Fil kicked out of LVEA by mean commissioners :(
I have turned up the preamp gain on the LSC-Y_TR photodiode (a PDA100A on ISCTEY) from 0 dB to 20 dB. The BS immediately before this PD is a BS1-1064-50-2037-45P.
The LSC-TR filter banks have been adjusted as follows:
Somehow these values were overwritten with the old values around 2015-01-24 03:37:00 UTC. I've restored the new values.
Reminder, if you wish to edit the site map, please do so in the /opt/rtcds/userapps/release/cds/h1/medm directory.
Please do not do it in the /opt/rtcds/lho/h1/medm directory as this breaks the symbolic link and the change does not get committed to the SVN version control repository.
Over the last couple days I have made a couple of housekeeping updates to the IFO_ALIGN screen:
I repeated the study I did on the L1 ADC hold times ( aLOG entry 16455 ) using date from H1. It is running the tagged RCG 2.9 and has higher thresholds (MAX_ADC_WAIT_CARD_0 of 20, MAX_ADC_WAIT_CARD_S of 3).
Here, I record values from the IOP model proc file (i.e. cat /proc/h1ioplsc0/status on h1lsc0)
FE | Max Hold | ADC#0 Max | Max Delta |
---|---|---|---|
h1lsc0 | 22 | 21 | 1 |
h1susb123 | 23 | 21 | 2 |
h1sush34 | 25 | 21 | 2 |
h1sush56 | 24 | 21 | 2 |
Again, in most cases ADC#0 Max + Max Delta = Max Hold. It does appear that better limits of 23 and 5 for MAX_ADC_WAIT_CARD_0, MAX_ADC_WAIT_CARD_S (from L1 data) would work here. The current CDS bugzilla issue is Bug 791
Last week, sts correction gains were measured and calculated on HAM4/5/6 to improve the CPS sensor correction. We measured them today for the remaning chambers.
For the BSC chambers, we locked the platforms to the ground by increasing the blends to 750mHz (cf alog) and turned off the sensor correction. We measured the ratio between the T240 (stage 1) and the STS (ground) signal (should be 1 with the platform moving with the ground). The same measurement was also carried out between STS and L4C for hepi, since IPS sensor correction is used for the Z DOF.
All the measurements using the corner station STS B as input to the sensor correction (ITMX, BS, ITMY) show a factor of two mismatch with the hepi and platform sensors in the X DOF only. There is certainly a calibration error with the STS B seismometer. I checked the calibration filter in the input filter banks and it looks correct (10.17 nm/s /cts). We should investigate more. The other chambers and dofs need a relatively small correction.
The ISI gains (T240/STS for BSCs and GS13/STS for HAMs) are
X | Y | Z | |
HAM2 | 1.036 | 0.995 | 0.877 |
HAM3 | 0.976 | 0.958 | 0.812 |
ETMX (T240 for gnd sensor) | 1.114 | 1.106 | 1.105 |
ETMY | 0.985 | 0.974 | 0.995 |
ITMX | 1.964 | 0.976 | 0.993 |
ITMY | 1.982 | 0.984 | 0.993 |
BS | 1.99 | 0.986 | 0.994 |
The BSC - HEPI gains (L4C/STS) are
X | Y | Z | |
ETMX | 1.110 | 1.107 | 1.091 |
ETMY | 0.998 | 0.991 | 0.969 |
ITMX | 2.167 | 1.063 | 0.972 |
ITMY | 2.011 | 1.006 | 0.963 |
BS | 2.09 | 1.055 | 0.924 |
Attached are examples of the transfer functions for ITMY
To calculate the gains I wrote a script for the bscs in a similar fashion as Sebastien's : it is called BSC_gain_matching_calculation(IFO,Chamber,start_time,duration) located under /ligo/svncommon/SeiSVN/seismic/BSC-ISI/Common/Misc. The results above were obtained using the following parameters :
start_time=tconvert('01/22/2015 15:35') ;
duration = 45*60 ;
The sensor correction matching gains were changed to their new values for the BS ITMX and ITMY in the X dof. The pdf attached are comparing the X platform motion before/after the change, showing improvement (x~2) in the 300mHz 700mHz band.
I confirmed that the factor of two mismatch described in the log above comes from the STS B, looking at the comparison of the three ground seismometers (X dof) of the corner station, see attached screenshot.
It looks like the problem is in the B STS or associated cabling. I took spectra of the raw ADC signal and the B seismometer shows half as much X signal (ADC channel 26 on attached plot) as the other 2 STS's. Arnaud and I went out to check the cabling at the rack, everything is tight there. We are doing measurements with the ITM's right now, so we didn't go out to check the cable on the pod.
I looked though the trends and science channels, and it looks like the auto center on the B STS was pushed at 8:30 am local on Dec 23rd, 2014 and the X channel didn't recover properly. On the first attached trend you can see that before the alleged button push (alleged because there are no logs, I'm just guessing, second trend shows what looks like an auto-centering sequence though) the X channel showed more signal, then after less signal. Y channel looks roughly the same before and after. Attached spectra (3rd png) shows overall spectra is very different, solid red is before, dashed green is after.
Yesterday I dif the ETMs. Today I did all the remainder. All came back under guardian first time.
Have added and committed HPI guardians to the h1 area so as to not by default edit the common file. All other guardian changes from this also committed.