Two items:
I started updating the slow controls SDF models to the latest version in trunk. This was to fix a bug Daniel saw after the power outage. Unfortunately there were problems with the channel type detection code so we fell back to running the old code again. I have left the work permit open, I will debug, test, and deploy later.
Crashed earlier in the day, I'm guessing with the Beckhoff restart: Jan 26 13:07:00 h1conlog1-master conlog: ../conlog.cpp: 301: process_cac_messages: MySQL Exception: Error: Out of range value for column 'value' at row 1: Error code: 1264: SQLState: 22003: Exiting. Dave updated the channel list to remove the dust monitor channels.
Richard, Dave, Evan
We added infrastructure in the OAF model to take LSC error signals, filter them, and output them to an 18-bit DAC channel in the PEM frontend. This signal is then piped into the control room and can be played on a speaker.
MEDM screeens still need to be made.
1 h1psl PSL.team HPO optic exam x113 00:02:19
2 ISC/IO/ALS Richard add sensors to all tables 202 00:01:19
3 TCSY Alistair startup TCSY laser 202 00:01:19
4 LVEA crane Bubba shim rails - requires lift 202 00:00:48
5 TCS Aidan/Alastair install/modify 202 00:01:21
6 optics lab Betsy work 212 00:02:19
7 ESD drives filiberto/ed EX and EY 212 00:02:38
8 mid-y john/bubba check cryo pumps 202 00:02:43
9 EY Kyle check pumps 202 00:02:48
10 OAF Dave+Evan new oaf and pem, audio testing 202 00:03:44
List of reservation system - some times are longer than the task is expected to take 1 h1psl PSL.team HPO optic exam x113 00:03:40 2 ISC/IO/ALS Richard add sensors to all tables 202 00:02:40 3 TCSY Alistair startup TCSY laser 202 00:02:40 4 LVEA crane Bubba shim rails - requires lift 202 00:02:09 5 LN2 company? delivery on site 202 00:01:03 6 TCS Aidan/Alastair install/modify 202 00:02:42 7 optics lab Betsy work 212 00:03:40 8 ESD drives filiberto/ed EX and EY 212 00:03:59
Before bringing the SEI platforms back up. Checked the diffs with the safe.snaps in SDF. There were a few chages related to GS13 gains and FF ON/OFFs. All updated and committed to svn.
Brought all the SEI Platforms down and spun down the HEPI Pumps to check the Accumulator charge. Most accumulators showed a few PSI drop--likely from pressure gauge loss. No charge dropped to action level. It was mid July last time this was done. This suggests none of the accumulators are leaking. I'll check again in a couple months--checking the pressure sometimes loosens the valve and it can start leaking.
In anticipation of going to Observe last night (which didn't happen due to a lockloss sometime just before 1am), I had accepted some ISI SDF diffs while JimW was still here to approve them. I'm sure he'll look at his data more, and make a final decision on whether we should keep the sensor correction changes or not, but for now the Guardian will probably put the EY ISI back in its old nominal state, so we'll see a bunch of SDF diffs when we get to NomLowNoise. Anyhow, I attach a screenshot of what I accepted, and we can re-accept the old values later today, until Team SEI decides to change the Guardian.
Updated the corner TwinCAT system to the most recent svn. This includes the changes needed for adding the ASC-AS_A/B.RF90 channels.
The new AS_B.RF90 demod channels won't be active until the corner 4 chassis is upgraded. The AS_A.RF90 demod channels have been taken over from the old (now deleted) AS_D. Whitening channels for both AS_A and AS_B are working.
Added whitening screens for AS_A/B_RF90 to ISC-CUST_WHITENING_OVERVIEW.adl
Added related screen buttons for AS_A/B_RF90 to ASC_OVERVIEW.adl
WP 5696, ECR 1500402 The version of GDS software for the control room has been updated to branch gds-2.17.1.3. This release addresses several Bugzilla bugs regarding data access, especially NDS2 data access. A new NDS2 input section has been added which will allow the user to specify an epoch start and end time, which will limit the channel list to those that existed during the epoch. For those channels whose data rate was changed without changing the name, it will be possible to specify an epoch where only a single data rate exists. (This assumes the user knows the time that the data rate changed.) If multiple channel rates for a channel exist during the chosen epoch, those rates will be displayed when channels are selected. Because of the way NDS2 returns data, it is still possible to get incorrect data if you choose a data rate for a time when the data really doesn't exist, so it's best to narrow the epoch to display only a single data rate for the channel. Diaggui now warns you if you don't have a kerberos ticket when attempting to use the NDS2 input. A release note document has been generated for the GDS tools, see T1600007.
Current activities:
1 h1psl PSL.team HPO optic exam x113 00:04:57
2 HAM4/TCS Filiberto/Ed cables 4 hartman 202 00:03:14
3 CPS/PI Richard cabling for CPS, status PI 202 00:03:17
4 HEPI Hugh CS HEPI check - all down 202 00:03:24
5 Sprague Joe check all buildings 202 00:03:26
6 dust monitors JeffB modify pumps 202 00:03:32
7 ISC/IO/ALS Richard add sensors to all tables 202 00:03:33
8 TCSY Alistair startup TCSY laser 202 00:03:33
9 LVEA crane Bubba shim rails - requires lift 202 00:03:54
Alarm Test:
Enclosed are the past 10 day trends. As usual, please refer final analysis to Peter, Jason or Rick.
I again measured the frequency noise coupling into DARM by injecting at the common-mode error point. This time, I did both a broadband injection and a swept-sine injection. The broadband TF is shown in the second attachment.
Compared to the previous measurement (2015-10-08, first attachment), the high frequency coupling seems to have decreased by a factor of 3.
Combined with the expected residual frequency noise (third attachment), this suggests that frequency noise lies a factor of 50 to 100 below the DCPD shot noise (1×10−7 mA/Hz1/2).
The H1_HOFT_C01 and L1_HOFT_C01 hoft frames are now ready for use for all of ER8/O1. The times the C01 hoft cover are:
H1: 1125969920 == Sep 11 2015 01:25:03 UTC to 1137258496 == Jan 19 2016 17:07:59 UTC
L1: 1126031360 == Sep 11 2015 18:29:03 UTC to 1137258496 == Jan 19 2016 17:07:59 UTC
i. The STRAIN channels to use are:
H1:DCS-CALIB_STRAIN_C01 16384
L1:DCS-CALIB_STRAIN_C01 16384
ii. The analysis segments and missing data segments to use are:
H1:DCS-ANALYSIS_READY_C01:1
L1:DCS-ANALYSIS_READY_C01:1
H1:DCS-MISSING_H1_HOFT_C01:1
L1:DCS-MISSING_H1_HOFT_C01:1
iii. State and DQ information is also in these channels:
H1:DCS-CALIB_STATE_VECTOR_C01 16
H1:ODC-MASTER_CHANNEL_OUT_DQ 16384
L1:DCS-CALIB_STATE_VECTOR_C01 16
L1:DCS-CALIB_STRAIN_C01 16384
The search groups, of course, should decide which analysis segments pass their DQ specifications.
The documentation has been updated here:
https://wiki.ligo.org/Calibration/GDSCalibrationConfigurations
https://wiki.ligo.org/Calibration/GDSCalibrationConfigurationsO1
https://wiki.ligo.org/LSC/JRPComm/ObsRun1#Calibrated_Data_Generation_Plans_and_Status
https://dcc.ligo.org/LIGO-T1500502
We have had plenty of things wrong in the IFO since we used bad safe.snaps to restore after the power outage last wensday. (A2L gains, missing ASC notches, ETMY bias was accidentally flipped, plenty of dark offsets changed, some violin damping was on with the wrong filters engaged ect...)
This morning Jenne, Kiwamu and Evan helped me to clean this up so we are nearly back in the observe state from O1, with only intentional changes like the LSC feedforward and sending MICH to PR2, which we have accepted.
There are a few things remaining red:
The bias is flipped on ETMY, and the gain in L3 DRIVEALIGN_L2L (which compensates for the bias flip). It might be that we have messed up the calibration by doing this flip accidentally, but we were planning on doing it soon anyway.
I've also implemented a version of the scripts from LLO that load down.snaps in SDF when the IFO goes down, and observe.snaps when we reach nominal low noise. So far we only have saved a few down.snaps, but if we add more and keep them up to date this should be alot easier in the future since we won't need to be locked to see what differences we have.
After the IFO lost lock Jenne, Trais and I went through SDF and updated the downs that I had made last week after the power outage, and made down.snaps for the models that had differences from the observe state. I also added these the the bash scripts that the guardian uses to load sdf files.
We also flipped the bias on ETMY back.
This means that if we loose lock from nominal low noise, SDF should be all grey and green now. If we loose lock from a different state (like we just did during the CARM offset reduction) SDF will not be green.
Now if we can keep these up to date we can avoid this kind of trouble in the future. We probably need to think about how we are going to keep these up. Maybe it can be a weekly maintence task?
I have cleared the rest of the SDF diffs that we have for today in the Observe state.
For some reason, SR2's setpoint had the M2 Pit and Yaw inputs turned off. This isn't right, so I've accepted the correct state of having the inputs turned on so the signal can go through.
I have also accepted the gain of -1 in the PR2 ISCINF_L filter bank, which is used for the PRCL-MICH decoupling as of last week.
The last thing I accepted was a filter module turned on in the SRM's ISCINF bank that Gabriele created to try to get rid of the 3.2kHz broad peak in DARM that is very very coherent with SRCL.
We're still taking measurements right now, but SDF is green, so we should be able to leave the IFO in Observe when I'm done.
There are small glitches very close to each second boundary in the ETMY drive signal and the ETMY SUS ad SEI rack magnetometers. In order to investigate the half-Hz combs in DARM (see alog 20790), I took an hour of data and folded it with a four-second period. If there is a repeated glitch at any multiple of this period, it should become far more visible. The result is that in several channels, there are glitches very near the boundary of each GPS second. The peak time of these glitches seems to be about 10 milliseconds after the start of the second. The glitch does not repeat identically every second. There is one shape in the first second, then one with an opposite polarity in the next second. The first two attached plots are for the SUS and SEI racks, which are shown with a 40-Hz zero-phase lowpass. The SUS has a narrow spike, and the negative spike is larger than the positive one. The SEI signal is more complicated. The ETMY L3 MASTER signal, which is the DARM output to the ESD, is shown with a 10 to 50 Hz bandpass. These glitches are more like sine-Gaussians, but the even and odd seconds still seem to have opposite polarities. There are more channels with similar glitches. We can make a more thorough investigation, and use more data and more times, to try to track down the origin of these glitches. Hopefully these glitches are responsible for the 0.5 Hz combs such that removing them will improve those.
J. Kissel
Tagging CDS in this entry.
I'd recently taken a look at the requested output of the ETMY L1 stage,
${IFO}:SUS-ETMY_L1_MASTER_OUT_*_DQ
and was interested to find ~1 [Hz] combs in the requested output. Though this isn't the 0.5 [Hz] combs that Andy mentions above, I think it's an excellent place to take the investigation further in a more focused manner -- with the point being that even the SUS's *requested* signal has a comb.
Attached is a 100-sec FFT ASD, of a typical, 1000 [sec] stretch of observation-ready data during the run (2016-01-04 04:00 UTC). Here, to give a feel for the physical amplitude of these signals: at 30 [Hz] the noise amplitude of one of these comb peaks is roughly 1e-3 [ct/rtHz] of requested DAC output, which corresponds to
1e-3 [ct/rtHz] * (20.0 [V] /2**18 [ct]) = 7.6e-8 [V/rtHz] @ 30 Hz ( * sqrt(2 * 0.01) = 1.1e-08 [V_pk])
Potentially verifiable / refutable Crack-pot Theories / Wild guesses:
- Perhaps there is some of this glitching in the inter-process communication (IPC) on the reflected memory (RFM) data transfer from the corner to the end station, that's only exposed for requested drives that have such a huge dynamic range? For whatever channels in the signal chain that are stored, can you reproduce the same combs by filtering those channels offline?
- Recall that the power supplies for the Hartmann Wavefront Sensor (HWS) were replaced some time ago, see Integration Issue 1062. Has anyone made a before-and-after comparison on this searched other sources for such combs in auxiliary channels? Perhaps forming a BruCo-like search where this UIM / L1 stage control signal is the response instead of DARM?
- Keith has already done a long-term study of the analog-to-digital converters (ADCs), looking for combs: see, eg. pg 44-49 (yeah!) of G1300997. He found no-such combs. Perhaps we should do a similar study on the digital-to-analog convert (DAC) side of things? I could also imagine a similar set up for a set of RFM channels that make the 4km journey along the arms.
In response to Jeff: The SUS-ETMY MASTER signal is just a filtered version of DARM, so if DARM has the comb so do the drive signals. I don't think that tells us where in the loop they originate. But you're right, this could be a digital problem or an electronics one involving something synched to GPS. Keith T. and Annamaria both suggested that the power supply of the timing fanout might be involved. That of course can be perfectly synched to the GPS second. Annamaria showed me an ADC in the L1 corner station being used as a temporary monitor of one of these power supplies. That signal (attached) jumps downward one second and upward the next, matching what we see in the magnetometers and DARM. Could we check if there's such an effect at the H1 Y-end?
Note that the small glitches in Andy's post are exactly synchronized to GPS; this makes coupling to many power supply glitches (HWS, or trickle chargers for magnetometers, etc.) an unlikely source.