model restarts logged for Tue 16/Feb/2016
2016_02_16 13:58 h1tcscs
2016_02_16 14:27 h1isietmy
2016_02_16 14:29 h1dc0
2016_02_16 14:31 h1broadcast0
2016_02_16 14:31 h1nds0
2016_02_16 14:31 h1nds1
2016_02_16 14:31 h1tw1
Maintenance day. TCS and ISI-ETMY model changes. H0EDCU_VE.ini change. Related DAQ restart.
Wed 10/Feb/2016 - Mon 15/Feb/2016 No restarts reported.
TCS model change. WP5732
Dave, Aidan:
Aidan made model changes to h1tcscs.mdl
Buried EY Seismometer added to h1isietmy model. WP5733
Jeff, Robert, Dave:
h1isietmy was modifed to read the external seismometer, these channels were added to the science frame.
Adding all Vacuum Controls channels to the DAQ. WP5731
Dave:
All the vacuum controls channels were added to the DAQ. Increased the number of channels from 280 to 1777
DAQ restart
Jim, Dave:
DAQ was restarted to support all the above changes. After several hours h1fw1 became unsynchronized to h1fw0 (we had not seen this before). Restarting h1fw1 fixed this.
GLIBC patching
Dave:
Patched and restarted critical servers and about half of the control room workstations.
I filled the PSL diode chiller with 350ml of water. Attached is a 30 day plot showing where the diode chilller was filled on Feb 4th, and then the alarm level increasing starting around Feb 13th.
How does this alarm? Is it on Verbal? Audible alarm out in the Diode Room? Or is the red light the only way to tell if it's low. In general, the Crystal Chiller is the one which requires action every week. For the Diode Chiller all we have is the red LED to warn us of low levels (I've never seen it). We (operators) just need to remember to also keep checking the Diode Chiller when performing this task.
Conceivably, we check this every Thursday. If the alarm goes off on Friday, is it OK for this chiller to be in alarm until it's checked the next Thursday?
I wonder if we should include trending the alarm level on the FAMIS procedure for this as Cheryl did.
Unfortunately the red LED on the face of the diode chiller is the only alarm we have and checking this LED is part of the FAMIS procedure. Additionally, especially with Peter and I in Germany until mid-March, if anyone is in the diode room for any reason and sees this red LED lit, fill the diode chiller immediately until the light goes out and post in the alog that water was added and how much. If the water level gets too low, the chiller will shut off, which also shuts off the entire PSL. We need a better alarm (especially in the hypothetical situation Corey brings up, this could result in a laser shut down if not caught before the next scheduled check), but this is what we have for now.
I have increased the LLCV valve setting to 18% (from 15%) to try and compensate for changing conditions.
Factors that are changing are:
The supply pressure is falling as the liquid level in the storage dewar falls.
Heat leak to the liquid transfer line is increasing with outdoor temperature. This results in more vapor in the line and less liquid to the pump.
Radiation from the beam tube as the beam tube warms.
The attached plot shows 100 days of the control valve setting. The flat line at 15% is due to the failure of the differential pressure sensing- we have had to resort to manual setting of this valve.
For reference the liquid capacity of the pump reservoir is ~77 gallons when full.
I forgot about the heat into the pump due to radiation from the surrounding vacuum chamber in the mid station. Here is a plot of recent VEA temperatures at MID Y.
That is illuminating, thanks. We take feedback controls for granted around here; don't miss it till it's gone. Hope this sophisticated PIL* control algorithm will tide us over until we can regen & fix the level sensor.
*Primate In Loop
Here is a 45-day trend of HEPI Pump Pressures & Drives (thus closing out FAMIS # 4517).
A few note:
Attached is a time series of TMSX roll osems over the last year, which has been slowly drifting in the same direction the whole time. There was a went in June, wich is the larger jump, you can see when the sat amp started to oscillate a few weeks ago, and me moving roll over the weekend. This drift is mostly seen in F1, I don't know if it is real or not.
Rana has told us that at LLO the TMS QPD positions are sensitive to roll. Before closing the soft loops we moved roll by 250 urad (accroding to the osems) and saw that this had no impact on the error signals of the soft loops.
Here are optical lever 7-day trends; closing out FAMIS # 4392.
Masayuki
The ISS 2nd loop medm screen has been Improved. The digital path has been shown.
I have noticed that h1fw1 is lagging quite a ways behind h1fw0 (they are normally synced to the second for the fast data CRC and within a few seconds of writing frame files). h1fw1 was lagging 1 to 10 seconds on CRC and up to 20 seconds for frame writing.
17:47PST, restart h1fw1 daqd process. h1fw1 is now back in sync. The lack of CRC sync lead me to think this is a daqd problem and not a Solaris QFS slow raid.
1610 - 1700 hrs local -> To and from Y-mid + Y-end Today took nearly 25 minutes to over-fill CP3. It is as if the pump is really low. Like this past Sunday's fill the ambient temperature is unseasonably warm. Also, the BT cleaning crew have been leaving the Y1 BTE doors propped open overnight. Could the extra warm BT be significantly changing CP3's consumption (see Mike Z.'s comments from Sunday's entry)? Also, there has been a slight uptick in Y-arm pressure for the past day or so. I confirmed that IP9 is programmed in FIXED voltage @ 7000V and isn't stepping between voltages. Next scheduled over-fill to be Thursday, Feb. 18th before 4:00pm
J. Kissel, R. Schofield WP 5733 Robert and I moved around cabling for the recently re-installed GND STS that has been buried +X (at ~10 m away from the building) of the ETMY building. This STS (S/N 89921) was formerly in the vault during O1. Now cabling for this STS is such that the BNCs, from the hostbox outside, go into channels 13, 14, 15 of the PEM BNC AA chassis at the very bottom of the TCS-C1 rack. Out of the back of this chassis, we've installed a new (standard L-Comm) DB9 cable which now goes to the front of the HEPI AA chassis, in U30 of the SEI-C1, in the port marked "27-30." This corresponds to the channels ADC3_[26-28], which have been pre-wired up to go into STS_A at the top level of the EY model. In addition, I've unhooked the top-level GND_STS block from the library, and added three new sets of channels, H1:ISI-GND_STS_EYBURIED_X H1:ISI-GND_STS_EYBURIED_Y H1:ISI-GND_STS_EYBURIED_Z. Each above channel prefix set has - test points stored to frames as "_DQ" (stored at 256 [Hz]), e.g. "H1:ISI-GND_STS_EYBURIED_Y_DQ"; - an EPICs "_MON" channel, e.g. "H1:ISI-GND_STS_EYBURIED_Y_MON"; - the full BLRMS infrastructure with EPICs channels "_BLRMS" with associated BLRMS extensions for the given band, e.g. "H1:ISI-GND_STS_EYBURIED_Y_BLRMS_30M_100M." The modified top-level model has been committed to the userapps repo. Because the buried STS readout system has two factors-ot-two differences between the nominal STS readout system (one factor from the fact that only the positive legs of the host box signals are carried in via BNC, and then the other is from reading out this single-ended, half-signal with a differential ADC), I had to create new calibration filters (in FM2 of the H1:ISI-ETMY_ST1_GNDSTSINF_A filter banks, called "Cal_Buried") which multiply the nominal gain (10.1725 [(nm/s) / ct]) by 4 to get the calibration right. Thus, all of the derivatives of the above mentioned channels are calibrated into (nm/s), as is the case for all other GND_STSs. The modified filter file has been loaded, committed to the repo, and the new FMs have been turned on. As of Feb 17 2016 at 00:00 UTC, these channels are calibrated and awesome. Calibrated DTT Template saved as /ligo/home/jeffrey.kissel/2016-02-16/2016-02-16_H1ISIETMY_GND_STS.xml P.S. Robert suspects that the best position for the STS is roughly at ~20 [m] from the building because it balances high-frequency drop off of coherence and coupling to building tilt. I.e. 10 [m] is a little too close, so the building tilt pollutes the ~40 [mHz] signal just as much as the STS on the slab, where as at 40 [m], the ~0.5 [Hz] motion is no longer coherent enough to do viable subtraction. So, he may dig one more hole and move it to 20 m, but it'll likely stay put until the next time Robert's in town.
Jeff, Darkhand, Kiwamu,
We have been meaning to do this, but kept missing a chance to do. Today we have fliiped the sign of the ESD bias voltage both on ETMX and ETMY.
This alog only describes what we did on the digital front end models. We will perform a set of confirmation measurement and verify the H1 DARM model later.
See alog 22135 for a concise summary of the bias flip.
[Bias flip on ETMY ]
[Bias flip on ETMX]
We took open loop and Pcal sweep measurements in nominal low noise after the biases were flipped. They reside at:
/ligo/svncommon/CalSVN/aligocalibration/trunk/Runs/PostO1/H1/Measurements/DARMOLGTFs/2016-02-16_H1_DARM_OLGTF_7to1200Hz.xml
/ligo/svncommon/CalSVN/aligocalibration/trunk/Runs/PostO1/H1/Measurements/PCAL/2016-02-16_PCALY2DARMTF_7to1200Hz.xml
Darkhan will process the data later.
P.S.
I have updated the SDF accordingly for SUS-ETMX and ETMY and CAL-CS when the interferometer was in nominal low noise.
Kiwamu, Darkhan,
Summary
Our analysis of a DARM OLG TF and a PCALY to DARM TF measurements taken after the ESD bias sign flip on Feb 16, 2016 showed that:
Disscussion (actuation function)
The ETM ESD drivers' electronics and front-end FOTON filters associated with the ESD responses were recently updated, see LHO alogs 25468, 25485. Since the front end filter's do not perfectly cancel the ESD response, in the Matlab parameter files we used a better approximated ZPK responses that are based on the old ESD electronics measurements. Since currently we do not have TF measurements for the new ESD driver electronics, we have removed all of the compensations (for the non-perfect front-end filters) from the Matlab analysis of the DARM and PCALY to DARM TF measurements.
Measurement files, scripts
DARM OLG TF and PCALY to DARM TF measurements have been committed to CalSVN:
Runs/PostO1/H1/Measurements/DARMOLGTFs/2016-02-16_H1_DARM_OLGTF_*.txt
Runs/PostO1/H1/Measurements/PCAL/2016-02-16_PCALY2DARMTF_*.txt
Up to date filter files have been copied to
Common/H1CalFilterArchive/h1omc/H1OMC_1125881488.txt
Common/H1CalFilterArchive/h1susetmy/H1SUSETMY_1126240802.txt
DARM parameter file and comparison script have been committed to
Runs/PostO1/H1/Scripts/DARMOLGTFs/CompareDARMOLGTFs_O1andPostO1.m
Runs/PostO1/H1/Scripts/DARMOLGTFs/H1DARMparams_1139722878.m
Results (plots) are in
Runs/PostO1/H1/Results/DARMOLGTFs/2016-02-16_*.pdf
I've added a block to the TCS model to estimate the real-time thermal lenses in each of the test masses and the surface curvature of each of the test masses.
The main new library block, shown in the attached figure, is OPTIC_LENS which takes as inputs:
Each of these inputs goes into a filter that contains (or will contain) the transfer function for power to diopters for that specific actuation. These are:
which are summed together to yield. These are outputs for the model:
There are four of these blocks in the main TCS model h1tcscs.mdl - one for each test mass. Although the ETMs contain inputs for CO2 lasers, these are grounded. See the attached figure.
The outputs from the four blocks are currently terminated. I plan on updating the model to create channels for COMMON and DIFFERENTIAL MICHELSON and ARM lenses and others, as required. This will be after I've verified that the model and filters are all working correctly.
Title: 2/16 OWL Shift: 16:00-00:00UTC (08:00-16:00PDT), all times posted in UTC
State of H1: Locking
Shift Summary: Typical maintenance day
Incoming Operator: Corey
Activity Log:
At 23:03UTC (3:03pmPST) I reset the PSL Watchdog this afternoon & CLOSED FAMIS Request #3585.
Bubba and I were able to crane the Genie manlift over the Y beam manifold this morning in order to continue the crane rail work.
GV5 and GV7 were soft closed for the duration from UTC 16:15 to 16:53.
The crane behaved normally.
{Rana, Evan}
This evening we looked at the coupling of DCPD bias voltage into DARM.
Each DCPD is biased with +12 V from a linear regulator with a 1 Ω output resistor. We wanted to inject extra bias noise, so we exposed the bias lines with a breakout board at the DCPD chassy in the HAM6 rack (see D1300502). Then we summed in the output of an SR785 across an impedance of 10 kΩ in series with 20 µF (see diagram; green shows the normal DCPD electronics and red shows the addition). The 10 kΩ gives a 1:104 ratio of bias fluctuation to drive voltage, and the capacitor ac-couples the drive so that it does not pull on the bias at dc.
With the SR875 we drove a 1 Vpk line first at 187.3 Hz and then at 62.4 Hz (i.e., the bias fluctuation was 70 µV rms). We looked at the response in DARM (at 2 W dc readout, with DARM still controlled by EX). We had 10 mA dc on each PD, in the 400 Ω transimpedance configuration.
For 187.3 Hz (from 02:03:00 to 05:22:00 Z), the magnitude in DARM was 1.13×10−18 m rms, which implies a coupling of 1.6×10−14 m/V. The noise of the regulator at this frequency was 0.9 µV/Hz1/2, which implies a DARM noise of 1.4×10−20 m/Hz1/2.
For 62.4 Hz (from 05:33:00 to 05:56:30 Z), the magnitude in DARM was 1.4×10−18 m rms, which implies a coupling of 2.0×10−14 m/V. The noise of the regulator at this frequency was 1.9 µV/Hz1/2, which implies a DARM noise of 3.7×10−20 m/Hz1/2. Note that there is some room for error here because the DARM control configuration here is not exactly the same as the low-noise configuration. However, the ugf and phase margin should not be too different in the two configurations.
This is very close to the current low-noise DARM noise floor. However, if DARM is limited by voltage noise of the regulators, we should either expect that the DCPD null stream is equal in magnitude to the DCPD sum (it isn't), or the regulator noises are coherent (they aren't).
I am not sure if there are any contradictions in these numbers. DARM calibration at DC is C [W / m] = 4 * pi * G_arm / lambda * sqrt(G_prc * P_in * P_as / G_src) = 1.5e9 and 5e9 for input power 2W and 22W and 26mW at the AS port. This means that the response you measured is ~2e-5 A/V. If the input power is 22W, 1uV/sqHz projects to 4e-21 m/sqHz around 100Hz. This number is what you have in your noise budget plots.
The freerunning DARM channel may not be properly calibrated in the low-power state with EX control, so the numbers I gave originally could be too high.
We directly measured 1.2×10−6 mA rms in DCPD B at 187.3 Hz, which implies a coupling of 0.017 mA/V, which implies the bias noise shows up in DCPD B at 1.5×10−8 mA/Hz1/2. The shot noise with 10 mA on the PD is 5.7×10−8 mA/Hz1/2. So it is a factor of 4 or so below the DARM shot noise, assuming DCPD A is similar. That's close to the dark noise, but the dark noise is flat down to a few tens of hertz while the regulator noise has some negative slope.
Voltage noise of bias (pin #8 of the d-sub), measured with BNC clip doodle and SR785 (with AC coupling).
Sheila, Rana
We continued the investigation. Today we hooked up a DAC channel to the bias to make sweeps and noise injection. No surprises relative to yesterday. Still seems so close to DARM as to be unbelievable. Maybe the 1 Ohm resistor in the bias circuit is not stuffed?
We also injected random noise into PZT2 to look for upconversion there. We saw the usual kinds of quadratic coupling; noise at a level ~500x above the quiesscent HVmon noise level was able to be just visible in DARM in the baseband as well as by injecting noise close to the 4100 Hz dither frequency.
0554 UTC, leaving it in 'low noise mode'. The BS coil driver is in some mixed low noise state giving us increased DAC noise below 50 Hz, but the noise above there is as good as ever.
Can't get the photodiode D's off the DCC at the moment (outage???), but surely there must be a biga$$ cap (ideally several in ||) from the pd cathode to ground??
Yes, in the in-vacuum preamp, there is a 1 uF capacitor from the Bias to ground. At the LM317T (voltage regulator that sets the bias), there is a 10 uF cap to ground, on the regulator side of the 1 ohm resistor (it's not shown in Rana's sketch above). In any case, the projection given above relies on the 1e4:1 scaling of the injected signal - can't you just measure directly the drive on the bias line, rather than assuming the 1e4:1 ratio?
Bigger caps are unlikely to help much. The noise is due to the internal burried zener diode which serves as the voltage reference. The BW of the regulator is about 10kHz here. A bigger C may reduce this, but may also increase gain peaking. Just use a low noise supply such as the one here. Noise performance: T1000025.
Den, Rana
We directly measured the voltage noise on pin8 and confirmed that our injection through the 9.09k resistor (not 10k as indicated above) scales as expected. It seems that the 1 Ohm resistor is really 1 Ohm (+/- 20%).
From MZ, we have http://www.edn.com/electrical-engineer-community/industry-blog/4422750/2/Simple-circuits-reduce-regulator-noise-floor.
This indicates that some larger caps may reduce the regulator output noise at 100 Hz by a factor of 5 or so. Could be easy to try if we have a spare whitening chassis sitting around.
Rana, Sheila, Den We have measured the bias noise with a better resolution and found that this noise is almost flat below 60Hz. Then made a projection of this noise to DARM when the input power was 22W and power on each OMC PD was 10mAmps. For PD A the coupling is 1.5e-15 m/V while for PD B is 3e-15 m/V. Attached plot shows the projection of bias noise to DARM. It is factor of ~4 below the current sensitivity.
According to the noise budget and measurement of the dark noise, this noise is above the dark noise level. This means that the noise is present only when the DC photocurrent is present.
Suggests QE compression due to bias reduction. For higher P it may be wise to not only improve bias voltage noise, but to fix anode-cathode potential. Either a true transimpedance preamp, or a tracking supply at the rack.
We drove PD_B bias and measured the coupling of the bias noise to OMC NULL channel for different current levels. Plot is attached. The coupling is close to zero when there is no power. The coupling scales linear for small currents and stays almost constant for large currents (>10mA on each diode). We have also double checked that the optical signal scales linear with the current by driving an intensity line.
Kiwamu, Den We have digitized bias channels (LSC:EXTRA_AI_2 channel) and double checked coupling to DARM. Attached plots show coupling of PD_B bias to DARM. There is a calibration line at 62.4Hz of this coupling.