Summary

This is the data analyzed from last week's end-station Pcal readout calibration.  The new calibration factors will be used to compare with the old calibration factors. The calibration factor currently used will only be updated if there is a significant change. 

Details:

The calibration factors obtained from this particular measurement are tabulated below. The number in parenthesis is the weighted mean of our past calibrations that we have been using in our current calibration model. The new factors are comparable with the past measurements in most cases. 

The RxPD at ENDX shows a difference of about ~1% from our currently used calibration. We think this is because there might be some beam clipping issue again that we have seen in the past in this particular unit. The drop in optical efficiency, seen in the attached report, and the trends of the RxPD signal both point in this direction. We don't have major calibration lines running in this unit( there is a high frequency (3KHZ ) line only). Additionally, we also have a TxPD readout that we can use if we show the clipping is happening on the way out, after reflecting off the testmass, which we think is the case. There are ways to check this and we are pursuing those cross-checks.

All the data and plots have been updated to the svn and more details about the intermediate factors used during the calculation, optical efficiency etc is attached with this alog. 

The summary page (T1500252) on DCC that  keeps track of these calibration factors has also been updated.

Leo, Jeff

We try to measure the force acting on the TM from the electrical field goes from ESD blades to the cage and other surrounding. It corresponds to F_3 in Livingston alog 16611 and gamma in LIGO-T1500467. Relation between force due to the ESD-to-Cage field and due to usual ESD differential field may be characterized by c/a (in terms of LLO Log, = gamma/alpha in DCC note).
The measurements was done on Aug, 17 and Aug, 25 , each of 4 charge measurement with additional measurement in script. The additional measurements was done with zero bias voltage and signal offset voltage = +/-4 V. The sine voltage in signal was multiplied by 0.5 against the charge measurements due to voltage should not not exceed the maximum value, while the charge measurements use the amplitude of the signal voltage close to the max.
Measurements was done with ESD_Night scripts with Gamma1 measurements and proceeded by ESD_analysis_gamma.m 

Results of c/a (=gamma/alpha):

ETMX 
Pitch	08/17	st. dev		08/25	st. dev
UL		0.32		0.03			0.30		0.04
LL		0.34		0.04			0.33		0.02
UR		0.31		0.02			0.31		0.03
LR		0.33		0.03			0.32		0.02
Yaw				
UL		0.31		0.03			0.31		0.04
LL		0.27		0.03			0.28		0.03
UR		0.30		0.04			0.31		0.03
LR		0.25		0.02			0.27		0.02
	
ETMY
Pitch	08/17	st. dev		08/25	st. dev
UL		0.32		0.01			0.33		0.01
LL		0.35		0.02			0.37		0.02
UR		0.34		0.02			0.32		0.02
LR		0.37		0.03			0.36		0.02
Yaw					
UL		0.32		0.01			0.33		0.01
LL		0.31		0.02			0.31		0.01
UR		0.34		0.03			0.33		0.02
LR		0.29		0.02			0.31		0.01

In an ideal world we would have been able to systematically investigated each stage of the SR3 chain to find where the glitch was coming from.  But with the pressure of needing to get the system up and running I decided to replace all electronics and do a followup in the lab. So this morning we took down the SR3 Front end and installed a new DAC card.  Replace the coil driver SN100186 with SN 100192.  The AI chassis had been replaced last night.  So aside from cabling replacement the SR3 drive chain is new and after 10 minutes we do not see the glitches.  Of course I could go 10 min this morning and not see them.

Also there was a problem with the Voltage Monitor for T2.  This was in the IO chassis ADC interface so we replaced the ADC the ribbon cable and ADC interface card for the SUS Aux chassis.  This fixed problem.  

We pulled both the power supply and the harmonic generator from the remote rack and tested them in the lab, together with spare supply and spare generator. We used a spare 9.1MHz source on the bench.

In the lab, regardless of the combinations (supply and generator), 45MHz thing was clean.

In the rack, we tested the original generator with spare and original supply, with 9.1MHz source coming from the distributor in the rack. In both of the cases there were huge 45+-2MHz-ish humps.

Evan reported in the past that using IFR at the rack didn't change the results, so we didn't bother to look at 9.1MHz source.

In all of these test cases both in the lab and rack, the only thing connected to the harmonic generator was power supply, 9.1MHz source and the network analyzer on 5x output. Everything else was terminated.

We also looked at the supply voltage in the lab. We opened the chassis and used clips to access gnd and +15V test points. Under the load, both of the units didn't show any huge high frequency noise. RMS voltage measured on the scope was about 1mV, which was dominated by some pickup (it got smaller when I hand held the chassis away from the 9.1MHz source). Fil also measured the spectrum and it was within the spec. And anyway, the harmonic generator didn't show any hump in the lab, so the power supply is pretty much exonerated.

Fil put the original units back in.

J. Oberling, P. King

The investigation

Today we investigated what appeared to be frozen PSL channel: H1:PSL-OSC_DCHILFLOW.  Upon entering the laser diode room and comparing the data on the PSL Beckhoff computer's CHIL screen for the diode chiller to the front panel of the chiller itself, we found that all of the channels were not reading properly (flow rate, temp set point, actual temp, conductivity).  These data are read into the Beckhoff computer via RS-232 connections; the flow rate is read into the PSL Interlock Control Box, and the rest of the data are read directly into the Beckhoff computer via an add-in PCI RS-232 card.  Digging into the Beckhoff code it appeared that the Beckhoff data channels were not properly reading in the data from the diode chiller.

What we think happened

Back in May we performed several tests of the interlock system for the chillers to see how they behaved when certain cables were unplugged.  See Peter's alog here, under "The Evidence," point e where he talks about opening and closing the interlock switch.  We unplugged the cables to open the interlock and plugged them back in to close.  When this was performed, we did not restart the PSL Beckhoff PC; we should have, as RS-232 is not hot-swappable.  Therefore the channels for the diode chiller got stuck (interestingly enough the crystal chiller's channels have all been fine, and we performed the same test with it).

The solution

To fix this, we restarted the PSL Beckhoff PC.  We first called the control room to inform them of our intentions, since restarting the PC shuts the PSL down.  They put the IFO in a safe configuration (in this case they simply unlocked the IMC) and we shut the laser down and restarted the Beckhoff PC.  Everything came back on just fine, the PSL fired right up without issue.  Looking again at the CHIL screen the diode chiller data now matches that shown on the front panel of the diode chiller.  Problem solved!

One caveat

While we are now reading data on H1:PSL-OSC_DCHILFLOW, we do not trust the actual reported flow rate.  The same holds true for the crystal chiller, H1:PSL-OSC_XCHILFLOW.  Back in May we replaced the turbine style flow sensors in the PSL chillers with vortex style flow sensors (no moving parts in the new sensors).  With the sensor change we also had to change the flow sensor calibration, a number we received from LZH, who had tested these flow sensors (we went from ~550 pules/liter to 970 pulses/liter).  Upon this change the flow rate on the crystal chiller "dropped" from ~18 lpm to 9.7 lpm, a change of almost a factor of 2.  There was no associated drop in the HPO laser head flow rates (each of the 4 laser heads has its own flow sensor), therefore we do not trust this calibration number and therefore do not trust the flow rate being reported by the chillers.  That being said, we can still use the reported flow rate to monitor if there is a change in flow and as an indicator that something might be wrong.  The flow rate should not change, and if it does then something is not working correctly regardless of the flow rate being output by the chillers.  We do, however, trust the flow rates being reported by the HPO laser head flow sensors and by the water cooled power meter flow sensors.  To fix this, we will take the next opportunity when we have to swap out the running chillers for the spare chillers.  Before putting the spare chiller into operation, we will hook up an external flow meter on its own water circuit (this is before hooking the chiller up to the PSL water circuit), measure the flow of the chiller and adjust the pulses/liter calibration number until the chiller reports the same flow rate as the external flow meter.

The h1fw1 computer is now only writing raw minute files to it's locally attached SSD RAID file system.  

The h1fw2 computer is writing science and commissioning frames to the /cds-h1-frames file system through the h1ldasgw1 computer.  We will monitor the performance of this configuration and if it works reliably we will change the computer names to reflect their new usage.  

This should have no effect on h1nds1, other than the gap in data that occurred during the reconfiguration and restart of the frame and trend writers.

I am starting work on WP 5471 which will move the DMT to EPICS IOC from a test install to a perminant fixture.  Dave will then configure the EDCU to capture the channel in the frames.   There will be some interruptions on H1:CDS-SENSMON_CAL_SNSW_EFFECTIVE_RANGE_MPC and H1:CDS-SENSMON_CAL_SNSW_EFFECTIVE_RANGE_MPC_GPS.

This is follow up work on previsous work listed in the alog at: https://alog.ligo-wa.caltech.edu/aLOG/index.php?callRep=20925

I have added the following three guardian DIAG nodes:

• DIAG_CRIT: "critical" diagnostics required for OBSERVATION (currently empty)
• DIAG_SDF: check that all SDF_DIFF_CNT for all active front end models are identically zero (0)
• DIAG_EXC: check that all front end models (except calcs) are reporting no active excitations (hardware injections go in through calcs).

DAQ restart required to acquire new channel names.

These nodes will all be folded under the IFO top node, but have not yet.  Waiting for go ahead from ops...

I'm also in the process of updating the GUARD_OVERVIEW screen to add these new nodes.  I will post when that screen has been updated.

At least twice it was just stage 2.  Is something going wrong with the ISI?  or is this a side effect of the SR3 problems, which could cause the MICH ASC to get a kick?

Over the weekend while it was windy Evan noticed that we were close to saturating PRM M3, which is because the offloading I did 19850 didn't do as good a job as the M2 offloading, even though it solved the M2 range issue.  

Tonight I had a second look.  I moved the roll off up in the sus comp filter, moved the complex zeros down a bit to gain some phase, and removed the 3 Hz pole (this design was originally intended for a crossover above the sus resonances and didn't include the low frequency pole we are using.) The filter comparison is the second attached screenshot.  The first one shows the current crossover measurement, the third shows the drives.  The prominent peak at 0.6 Hz could be from an ISI.  The last screenshot shows the current configuration of the filter. 

None of this is in guardian, so the next time things are locked it should revert to the old offloading. 

Tonight was our chance to work with PRMI.  

We had a look at the phasing of AS36, following the same procedure as we used with MICH bright the other day 20961.  (locked PRMI, aligned by hand, used OM1 +2 to align all the light onto one quadrant, maximize the Q signal. )

For AS A the results were satisfying,

• the phases needed to maximize I are similar for all 4 quadrants,
• the signal levels were all similar,
• once I re-engaged the centering the signals were still similar on all 4 quadrants. 
• Exicitng BS pitch (12 Hz 1000 cnts in Drivealign) and measureing a tranfer function gave a signal that has the correct sign for pitch in all 4 quadrants, exciting yaw gave the right sign for yaw in all 4 quadrants.  

For AS B the situation was also reasonable, but everything was a little less good (the phases are similar for all quadrants, but not quite the same, the signal levels varied more, and when exciting the BS the phases were almost right for PIT and YAW.   Overall this seems reasonable to use.  The resulting phases are in the attached screenshot, the new phases are the epics values, the old phases are the setpoints.  AS B quadrant 3 did not change, it is 63 degrees.  

I will revert these momentarily, but the next steps would be to set the matrix for AS A yaw back to something reasonable (the current set up doesn't use segments 1 and 2), set the phases to the values in the screenshots, lock DRMI, excite SRM and move the phase of all 4 quadrants on each WFS in common to minimize the SRM signal in the Q phase.  Since DRMI locking is frustrating tonight with the SR3 glitches this will have to wait.  

[Sheila, Anamaria, Jenne, Cheryl, Dan, Evan.  Richard via phone]

SR3 glitches are still causing us grief.  This is a continuation of the story started last night (alog 21046), and worked on earlier in the day (alog 21062).  After some heroic efforts, we have determined that we cannot lock in this state. 

To prove to ourselves that indeed it was a problem with the analog actuation chain we investigated turning different pieces of the analog electronics off.  The first attached Dataviewer screenshot shows the NOISEMON channels of the SR3 M1 stage throughout this investigation.  When the local damping is on, we cannot see the glitches very clearly in the noisemon channels - but we do see them in the voltmon channels (The second attached screenshot shows that the T2 voltmon channel does in fact see the glitches, so it's not a broken monitor).  When the local damping is off, we only see the glitches in the noisemon channels. 

• A:  04:57 UTC we turned off the local damping (M1 OSEMs and "cage servo"), but left the DC alignment.  After this we see significantly more noise in the M1 T2 NOISEMON than the other M1 channels. 
• B:  05:41 UTC we take SR3 to safe mode (i.e., we remove also the DC alignment bias on the M1 actuators).  Glitches seem to be gone?
• C: 05:45 UTC we power cycle the AI chassis for SR3 (which is shared with the OMC, so the OMC was first brought to safe) and the top coil driver chassis that drives T1, T2, T3 and __. After this we restore SR3 to aligned, but no local damping.  Noise isn't as good as other channels, but looks better than earlier.
• D: 06:07 UTC we turn off the AI chassis for SR3 and OMC.  Glitches and noise seem to be gone.
• Around 06:20 UTC we swap out the AI chassis (removed S1104370) with a spare borrowed from the H2 storage racks (at Richard's recommendation).  We put in S1108081. 
• 06:24 we are back from swapping, and the new AI chassis is turned on.  Glitches seem to still be gone.
• E:  06:26 UTC we restore SR3 alignment and local damping.

Since we do not see the glitches when the AI chassis is powered off, we infer that the noise is not generated in the coil driver board.  (Note that we also borrowed a triple top coil driver chassis from the H2 storage racks, S1100192, but did not swap it in since we don't think the noise is coming from there).  We have not, however, determined whether the noise is coming from the AI board (probably not, since it's still there after a swap?) or the DAC. 

We tried a few times to lock the IFO after the AI board swap, but we were continually losing lock before the CARM reduction is complete.  Lockloss investigations showed that the problem for most of these was SR3 motion. 

At this point, we have determined that we need more expert brains to have a look at the analog electronics.  The owl operator shift has been cancelled, since there will be no more full IFO locking happening until this problem is resolved. 

At LLO there are problems with engaging the PRC2 ASC loop when the recycling gain is low, but we don't seem to have this problem at LHO.  One theorey about the difference could be that we just arrive in lock with a higher recycling gain.  Although I believed this myself, the data I downloaded from the first 13 days of ER8 seem to indicate this is not the difference.  I looked at times when we first arrived on resonance (112 examples), when we transition to DC readout (after the ASC including soft loops has been on for about 1 minute, 85 examples), and after we power up to the maximum available power (60 examples).

• We have a 5% discrepancy between the recycling gains measured in transmission of the 2 arms, so that gives us an idea of the accuracy of this calibration into recycling gain.
• We seem to have a bimodal distrution of recycling gains when we first lock, centered around 34 (35 according to the Y arm) and 36 (38 according to the Y arm). 
• The ASC brings us to a more consistent high recycling gain.
• According to the X arm, we keep the high recycling gain when we power up, while the Y arm indicates that the recycling gain drops a bit as we power up.  This is interesting, it might be nice make the same plot for POP_DC. 
• I was curious if the locks that start with a lower recying galn survive to low noise, so I looked at the same plot made for only locks which reached the nominal low noise state (44 examples). It looks the same, meaning that we make it to low noise even when we arrive on resonance with a low recycling gain.