As part of the recovery effort today we stumbled upon a way that seems to fairly reliable allow us to transition from PRMI to DRMI (we were 3 for 3 today - we'll try again tomorrow).

The procedure is:

- Let Guardian set up for DRMI locking.
- Misalign SRM
- turn off H1:LSC-SRCL OFFSET, INPUT and OUTPUT
- Let PRMI lock
- make sure H1:LSC-SRCL OFFSET, INPUT and OUTPUT are off (guardian turns on OFFSET), and clear history.
- Pause ISC_LOCK and ISC_DRMI guardian
- Align SRM, wait until it damps down. Today the PRMI always stayed locked in this configuration.
- Turn on H1:LSC-SRCL OUTPUT
- Turn on H1:LSC-SRCL INPUT. DRMI is now locked.
- Turn on H1:LSC-SRCL OFFSET, if desired for mode-hopping suppression

As recorded in r11378, I have fixed a bug in the ext_alert.py GRB monitor that prevented it from running.

Can this monitor be restarted at the earliest available opportunity?

16:00-17:00 Kiwamu confesses he has been tuning OMC without realizing the Intent Bit was set to Observing

17:05 Stefan doing some tuning

17:06 Lockloss

17:45 Fil to EY

18:10 Fil back from EY

Locking summary:  IFO was locked when I arrived (5 hour lock stretch), although at poor range.  Lockloss due to commissioning efforts.  After the tuning was satisfactory, I did an initial alignment.  After some Guardian issues were worked out with Kiwamu's help, we were able to bring the IFO back to Low Noise, albeit shortlived locks.  Commissioning continued thereafter.

The lockloss analysis tool at "/opt/rtcds/userapps/trunk/isc/common/scripts/lockloss" is updated. Now it will be triggered only when a lock loss from nomimal low noise state happens; losses during lock acquisation cannnot trigger this tool.

Besides, a "readme.txt file" is added to help you use this tool.

I "destroyed" the TEST guardian node, so that we have only critical nodes active during the run.

Today, I finally committed all of the SUS, LSC, and ASC SAFE.snap files to the svn after a few weeks of updating from SDF has gone by.

[Jamie, Vern, Duncan M]

I have modified the ODC-MASTER EPICS configuration to read guardian state information directly from the IFO top node, rather than the ISC_LOCK node.

This has no impact on the guardian system itself, but means that ODC-MASTER bit 3 is now a readback of H1:GRD-IFO_OK.

The upshot of all of this is that the ODC-MASTER will now not report 'observation ready' (bit 2) until all guardian nodes (monitored by the IFO node) report OK, and there are no test-point excitations on an ODC-monitored front-end. 

conlog failed last night at 18:37PDT, same error. I have just restarted it.


Aug 18 18:37:16 h1conlog1-master conlog: ../conlog.cpp: 301: process_cac_messages: MySQL Exception: Error: Out of range value for column 'value' at row 1: Error code: 1264: SQLState: 22003: Exiting.
Aug 18 18:37:17 h1conlog1-master conlog: ../conlog.cpp: 331: process_cas: Exception: boost: mutex lock failed in pthread_mutex_lock: Invalid argument Exiting.

Betsy, J. Kissel, Nutsinee

Today we restored all the band-limitting and low-pass filters to the violin monitor (alog20660). The band-limitting filter gains have been restored to 100.

Look for a subsequent alog regarding sign change troubles in ASC from commissioners later.  However, because Stefan was lamenting that the alignment of the SRC seemed to be problematic of late, we trended the SR3 PIT pointing during OPLEV damping control epochs and CAGE servo control epochs.  We discovered that the "bad pointing" of SR3 corrected by commissioners on last Thur Aug 13 (20519) was restored somehow on the day the cage servo was implemented Sunday Aug 16 (20571).  So, today we turned the cage servo off, manually restored the SR3 PIT pointing, cleared the offset  and turned it back on.  Note - ON, OFF, and CLEAR were all done via the servo's Guardian.  Hopefully this will help with the unstable ASC matrix problems, but we'll see... 

See attached trend which represents the pointing history of SR3.

Continuing on the task of summarizing the SVN status of CDS code, here is the guardian python user code list:

M       /opt/rtcds/userapps/release/als/common/guardian/ALS_ARM.py
M       /opt/rtcds/userapps/release/als/common/guardian/ALS_GEN_STATES.py
M       /opt/rtcds/userapps/release/isc/h1/guardian/lscparams.py
?       /opt/rtcds/userapps/release/isc/h1/guardian/TEST_BOUNCE_ROLL_DECORATOR.py
M       /opt/rtcds/userapps/release/isi/common/guardian/isiguardianlib/ISI_STAGE/edges.py
M       /opt/rtcds/userapps/release/omc/h1/guardian/omcparams.py
M       /opt/rtcds/userapps/release/sys/common/guardian/ifolib/CameraInterface.py
M       /opt/rtcds/userapps/release/sys/h1/guardian/IFO_NODE_LIST.py
M       /opt/rtcds/userapps/release/sys/h1/guardian/SYS_DIAG_tests.py

to get the list of python files I did the following:

looking at the archive of the guardian logs under /ligo/backups/guardian, I made a list of log files created in the past 19 days (for the month of August). For each log file I grepped for "user code:" to get the source py file. This gave a list of 79 files. For each file I checked its SVN status.

A brute force coherence report can be found here:

https://ldas-jobs.ligo.caltech.edu/~gabriele.vajente/bruco_1123740797/

I’m using data from Evan’s elog 

It took some time to process this since first data was not available on ldas-pcdev1.ligo-wa.caltech.edu due to some maintenance, and then my home folder was not available on ldas-pcdev1.ligo.caltech.edu due to some other maintenance. However, all’s well that ends well

• Below about 15 Hz there is coherence with DCPD_NULL, and this coherence is quite large in the 2-7 Hz region. I don’t know how to interpreter this, but I’m sure I’ve never seen it before (figure 1)
• There’s a bit of coherence below about 14 Hz with ASC-SRC2_Y (figure 2)
• Coherence with SRCL is at about 20%, so not limiting right now (figure 3)
• Coherence with MICH is somewhat higher, but still not limiting (figure 4). We are very close in the 35-45 Hz region.
• Between 15 and 30-40 Hz there is very large coherence with DHARD PITCH: this is limiting the sensitivity between 15 and 25-30 Hz! (figure 5) The same coherence is visible with ITMX and ITMY L2 coil signals (figure 6)
• Two “lines” at 27.25 and 28.25 Hz shows high coherence with PRCL
• In the regions 50-60 Hz there is some (not very high) coherence with ASC-X_TR_B_YAW. Is this beam jitter? (figure 7). Instead, between 77 and 78.75 Hz there is large coherence with ASC-X_TR_B_YAW_OUT (figure 7)
• Figure 8 shows the coherence with OMC-LSC_DITHER: there are few lines here an there that are coherent, even at low frequency (the lowest one is 64 Hz). Is this some kind of non-linear up/down conversion from the main dither lines? 
• At 64, 128 Hz and 256 Hz there is coherence with OMC-LSC_DITHER again but also with PEM_EY_MAG_EBAY_SUSRACK_X/Y/Z. Having coherence at 64, 128, 256 Hz is very suspicious!
• At 64 Hz there is coherence with a microphone (PEM-EY_MIC_VEA_PLUSY) and a magnetometer (PEM-EY_MAG_EBAY_SEIRACK_Z) and the end Y station
• Coherence with AS_A/B/C_SUM is still present at high frequency. This is our 45 MHz amplitude noise (figure 10). The same level of coherence is visible with ASC-OMC_A/B/C_SUM (figure 11)
• Good news is that the coherence with the periscope accelerometer is low (figure 12), although I still can guess the shape of the two bumps in the sensitivity…

Yesterday I completed installing the updated DMT software (gds-2.17.2) and its dependencies on the DMT Machines at both observatories. This includes (but is not limited to) new versions of:

 * h(t) calibration pipeline (gstlal-calibration-0.4.0)
 * SenseMonitor
 * Omega trigger generation from h(t) improved, configured and run continuously.
 * Improved monitoring of the DMT run status.

A few notes:

The latest version of SenseMonitor contains the improved anti-aliasing filtering using the decimation function written long ago by Peter for dtt. This fixes the problem that was noted at the start of ER7 where noise ner the Nyquist frequency was folded down to near DC due to inadequate anti-aliasing.

The version of the h(t) calibration pipeline was introduced because it is packaged with the new gds infrastructure. I am not qualified to comment on this verison of the pipeline, but from what I understand, it continues to run in a mode using the same calibration algorithm as the package it replaces. Further updates will be needed to start makeing the additional corrections under development by the calibration group.

The new monitoring functionality continues to generate the DMT Spi page showing the status of all monitors running. It now does a better job of checking that all monitor processes are reading data from one of the shared memory partitions. This is especially useful for monitors that were reading the calibrated h(t) data.

This non-event almost went unnoticed, but yesterday was the day h1ecatx1 should have done its every-fortnight-on-a-Tuesday crash until Carlos fixed the issue a week ago. Looks like this problem is now resolved.

Fil, Peter K., Nutsinee

Today I tried to hook up the spare IR sensor and the comparator box to the controller. Again it didn't work. I tried swapping the sensor, comparator, controller, I even tried swapping DB9 cable. A set of sensor and comparator box has been sent to EE shop for analysis (one left at the controller so the CO2X can lase). Peter and Fil found that the IR sensor works but the comparator box didn't seem to behave right. Fil has replaced the op-amp and the comparator but the output was still not what we were expecting (reduced set point but the comparator tripping point didn't get any lower). We are going to replace the switch and hope for the best. 

I have taken the transfer function of the coil A, B, C, and D inputs to outputs on the Satellite Box (D0901284).
The measurement was taken with an SR785 sourcing the input through a Coil Driver Chassis to turn the input into a differential, and then fed through the Satellite Box.
We expected the Satellite Box to be have flat phase response and gain.  We have confirmed that to a tenth of a percent. (0.1%)
The Reference TF was taken of the Coil Driver Chassis by itself, and the residual calculation took our response from the Satellite Box Coil A over Reference TF.
The following was calculated using Satellite_Box_TF_Plotter.m located in /ligo/svncommon/CalSVN/aligocalibration/trunk/Runs/PreER8/H1/Measurements/Satellite_Box/2015-08-17/.

While Travis was working on initial alignment earlier today, we found that we were once again having trouble turning on the ASC for the X green arm.  I think the trouble was that the DC centering loops' error signals (from the green ITMX camera) were a little bit too large.  This isn't something that happens too often, but it definitely caused the ASC to pull the arm away from resonance.  Engaging the ASC without the DC centering was fine, so it's definitely something with that pair of loops.

By hand, I turned the gain on the DOF 3 loop for both pitch and yaw (which handles the DC centering) to half of the nominal value.  Once the error signals were closer, I put the gain back up at the nominal value for both loops.

Since this is a particularly tricky problem to troubleshoot, I've tried to handle it in the ALS Arm guardians. The modifications were made in the "generator" states, so they are the same for each arm.  The actual gain values for each arm are stored in the alsconst.py file, which is already loaded by the generator states. 

• In the ENABLE_WFS state, if the DOF3 error signal is large for either pitch or yaw, it'll turn the gains for DOF3 on to half of nominal.  If the error signal is not too large, carry on as usual, with the nominal gain.
• In the LOCKED_W_SLOW_FEEDBACK state, if the gain is not at its nominal value (i.e., it has been turned on at half of nominal), it will wait in the run state until the DOF3 error signals are small-ish, then it will ramp the gain up to the nominal value, and allow the guardian to continue.

I have loaded this new code, but it has not been fully tested, since we have not done initial alignment since I loaded it (and that's where it'll get used).  I've tried to test the logic in the guardian shell and that all seems to be fine, but there's no test like doing it live.  If it is giving errors that seem insurmountable, I did an svn checkin of the as-found code before I started modifying it (as well as another checkin after my edits), so we can go back one svn version. 

This entry is meant to survey the sensing noises of the OMC DCPDs before the EOM driver swap. However, other than the 45 MHz RFAM coupling, we have no reason to expect the couplings to change dramatically after the swap.

The DCPD sum and null data (and ISS intensity noise data) were collected from an undisturbed lock stretch on 2015-07-31.

Noise terms as follows:

• Shot noise
  • For 20.0 mA dc on the DCPD sum, we expect a shot noise of 8.0×10⁻⁸ mA/Hz^1/2.
• Dark noise
  • Dan and I measured this a few months ago. In full lock we use one stage of whitening, which amounts to a dark noise of 1.5×10⁻⁸ mA/Hz^1/2 or so above 100 Hz. We have at times experimented with two stages of whitening, but it requires some vigilance regarding the violin modes.
• Intensity noise
  • Coupling was measured à la Kiwamu by leaving the outer loop off and driving the inner-loop excitation point. For the noise estimation, I used the outer loop out-of-loop sensor. Since Sudarshan et al. previously found that this is artificially high, this estimation is an upper limit (though probably no more than a factor of 3, given Sudarshan's plot).
• Frequency noise [CARM sensing noise only]
  • Previously (i.e., when we locked CARM using the in-air REFL sensor), I estimated the coupling by looking at the TF in-vac REFL to DARM, and for the frequency noise estimation I used the noise of in-vac REFL as an upper limit for the frequency noise. Since we now lock with in-vac REFL, which has more light on it than in-air REFL does, I do not expect that the noise of in-air REFL is a particularly useful upper limit for the frequency noise.
  • Therefore, I have tried to instead estimate the sensing noise in CARM (from in-vac REFL, the summing-node board, the common-mode board, and PDH shot noise) in V/Hz^1/2 and propagate it (using the Izumi/Sigg expression) into a frequency noise in Hz/Hz^1/2. For the CARM optical TF I use 12.7 W/Hz at dc, with a pole at 0.48 Hz.
  • Then I have injected noise into CARM and looked at the TF from the residual of the excitation into DARM. This is the coupling TF needed to propagate the estimated sensing noise to DARM, assuming the CARM gain is high (this isn't quite true at high frequencies; the ugf is 17 kHz or so, and the gain does not exceed 20 dB until 3 kHz).
  • Since this includes sensing noise only (and not, for example, FSS noise), I have labeled it as a lower limit. In fact, the coherence between REFL 9Q and the DCPD sum is about 0.04 around 7 kHz, perhaps suggesting a frequency noise contribution of 2×10⁻⁸ mA/Hz^1/2 or so. More investigation required.
• Oscillator phase noise [not included here]
  • Stefan and I measured the coupling of the 9 MHz RF source into DARM, and using the expected performance of the OCXO estimated the noise in DARM should be 2.8×10⁻¹⁰ mA/Hz^1/2 at 1 kHz. This is quite low (and not shown on the plot).
  • This number should be replaced with Alexa's and Daniel's measurements of the phase noise. An even more satisfying number could be made by trying to measure the 9 MHz and 45 MHz phase noises in situ, closer to the EOM (e.g., at the output of the distribution system on the ISC rack).
• Oscillator amplitude noise [45 MHz only]
  • In this case we know that the 45 MHz RFAM is the dominant contributor (other than shot noise) to the DARM noise above a few hundred hertz. Here I use Stefan's estimate of the RFAM-induced photocurrent (1.8×10⁻⁸ mA/Hz^1/2), which (as he notes) is not enough to explain the excess between sum and null.
  • However, based on tonight's measurements using the new EOM driver readback, it seems that 45 MHz RFAM is enough to explain the excess. Perhaps Stefan's measurement is an underestimate, since it was done at the output of the distribution system and did not include the final rf amplifier before the EOM.
  • As for 9 MHz RFAM, Stefan and I made an estimate of the coupling from the 9 MHz source into DARM. Based on the specified performance of the OCXO, the expected noise in DARM is quite small. However, it would be better to look somewhere further downstream, as Stefan did (although I think he said the measurement was not that clean). This noise is not estimated in the plot.

The downward slope in the null at high frequencies is almost certainly some imperfect inversion of the AA filter, the uncompensated premap poles, or the downsampling filter.