Shortly after 1 P.M. local time, LHO held it's annual fire drill. Response was great, THANKS TO EVERYONE.
This was done at John's suggestion so as to reduce the risk of over pressurization of the LN2 inner vessel during dewar deliveries (Due to obstructed sensing line, the LN2 level for CP3 is determined by a manual setting of the fill valve which maintains the level adequately only for static conditions and must be adjusted as a function of dewar liquid height and vapor pressure)
following a lengthy summer break, and with ER10 approaching, the daily restart report is back.
model restarts logged for Wed 07/Sep/2016
Per FAMIS #7070, Reset HAM4's HEPI L4C Accumulated WD Saturation Monitor (it was at 774 counts). While clearing this Hugh was looking over my shoulder and noticed that HAM1 is missing from the medm screen (he put it on his "To Do" list).
Attached is a plot of IM4 Trans qpd and ISS 2nd Loop qpd signals.
First plot - top - shows that IM4 Trans Pitch and ISS 2nd Loop Pitch don't match well
First Plot - bottom - shows that IM4 Trans Pitch is a good match to the ISS 2nd Loop Yaw signal (yaw modified: multiplied by -0.7 and an added offset of 0.005)
Second Plot - top - shows the IM2 Trans Yaw and ISS 2nd Loop Pitch - raw data
Second Loop - bottom - shows the IM2 Trans Yaw and ISS 2nd Loop Pitch (pitch modified: multiplied by -0.3 and an added offset of 0.8)
The DCS Disk2Disk scripts at LHO, which copy data from CDS to the LDAS archive and scratch disks have been restarted to use the CDS h1fw1 framewriter as the primary source of frames. The backup is h1fw0. This will prevent LDAS from picking up "bad" h1fw0 frames, except in the very unlikely event h1fw1 is down and h1fw0 writes a bad frame at the same time. (See: https://alog.ligo-wa.caltech.edu/aLOG/index.php?callRep=29521 for the latest on the h1fw0 issue.)
Began 15:22 UTC, finished 15:24 UTC
As soon as the restart finished IPC was red. I hit diag reset button and it went away. OAF_L0_MADC1_EPICS_CH25 started to overflow pretty much as soon as the model restart was done.
TCS chillers and lasers are recovered.
The laser tripped this morning. From the laser MEDM screen it looked like the NPRO had tripped which would pull
the entire system down. However the Beckhoff screen indicates otherwise; that the fault is associated with the
crystal chiller. It should be noted that the crystal chiller has 10468 hours on it.
Logs indicate that it tripped around 3 am. Unfortunately I did catch the trip.
Further inspection shows that the laser tripped not because of the crystal chiller (see AMPFlow.png).
Might be due to flushing of the newly deployed (old) manifold.
Looks like the power meter circuit caused the trip this time around. Doesn't appear to be the crystal chiller, however I am uncertain how often the chiller flow is updated via the RS-232 link.
The PSL trips over the last couple weeks have all been due to the Front End flow glitching. Suspecting the FE flow sensor might be faulty we swapped the PSL water manifold for the spare, with new flow sensors installed on the spare. As Peter says above this latest trip was due to the Power Meter water circuit (first time we've had an issue with this water circuit since the end of July). Unclear at this time what the cause was, although it's likely to be junk stirred up by our manifold swap on Tuesday working its way through the system (as Peter mentions above); Peter has some pictures of the output water filter for the PSL cooling system and there are a few new occupants taking up residence.
Filed FRS #6169.
Sheila, Terra, Jamie
Today I set up an OSA on ISCT6 to look at sideband assymentry at the AS port. Mark stayed late to get an MHV cable together, and we were able to look at the asymmetry in some of the 50 Watt locks that Jenne and Stefan had. Roughly, we saw that the asymmetry of the 45 MHz sidebands is 66% in power, while the 9 MHz sidebands are about a factor of 3 different in power. We have left the set up in place for now, and we will make some plots tomorrow.
Jenne, Sheila, Matt, Stefan, Terra, Jamie, Robert, Kiwamu commissioning Still needing to stop at LOCK_DRMI_1F to minimize ASC error signals before engaging control loops. The TCS chillers tripped from a DAC error and were reset. 23:18 UTC Sheila to ISCT6 00:19 UTC Sheila back 00:37 UTC Sheila to ISCT6 02:00 UTC Sheila back 03:23 UTC Restarted frozen video2
Sheila, Jenne, Terra, Patrick It appears that we have the same situation as described in alog 27435.
We ran through the instructions in the referenced alog and brought the TCS chillers and lasers back.
The OAFIOP Dac has gone bad again. We're just going to leave it at this point. The TCS lasers will need to be turned on in the morning, after this has been addressed.
Keita Marc Fil Daniel
We realized that we can implement the required compensation for the second loop by using the whitened monitor signals (TP10) from the transimpedance board as the inputs to the sum of PD1-4 and PD5-8, respectively—instead of the unwhitened outputs from the transimpedance amplifiers.
The required modifications are:
During testing we noticed that the input transimpedance amplifiers (eight AD797s), were all oscillating between 10-15 MHz with an amplitude of about 500 mVpp. Adding capacitance to the feedback network seemed to have little effect, so we swapped all of them with TLE2027. This solved the oscillation. Using 400 Ohms transimpedance, the input referred noise of a channel is about 25 pA/√Hz between 10 Hz and 10 kHz. This is maybe a factor of 2 below shot noise at high laser input power. The electronics noise is dominated by the Johnson noise of the first 4.87K resistor in the whitening stages.
The output SMA connector which was shortening the negative leg to the chassis has been removed. Instead, we drilled a new hole for an isolated TNC connector.
With the upfront whitening gain the fast monitor points now have too much gain and are saturating. We removed the gain of 50 from ERR1 and ERR2 by replacing R60 with a 4.53K (from 220K). This also removed the 2.7 kHz pole in this path. The OUTPUT channel was also modified for a flat transfer function with a fixed gain of 1. It now looks like the other two. In detail, C52 and C53 were shorted out, and R60 and R61 were changed to 4.53K (from 45.3K).
The transfer function of the ISS outer loop AC coupling is attached. As implemented it should be unconditionally stable with a ugf of 10 Hz. With a gain of ~500 at 10 Hz in the overall outer loop servo, the AC coupling point would be around 0.1 Hz.
Here is a spectrum of the outer loop readbacks at 2W and 50W input power, respectively, with the ISS second loop open. The AC coupling is on. At 50W the third loop is also on. The ERR readbacks are very near saturation at the higher power. Since the error signal is followed by a fixed gain of 3, the output was saturating.
I stopped and reran the Hartman code for ITMX at around 21:30 local.
As suggested by Aidan, I restarted the Hartman code with a fresh template. This time, I restarted both the ITMX and ITMY codes at around 9:40 local.
Note that the last restart before yesterday was done on Aug.8 (28947). It was done for both ITMX and ITMY. They have been running without errors until I stopped one of them (namely ITMX) yesterday.
Kiwamu, Nutsinee
A code is scheduled to run at 9 pm tonight. The code will power up CO2 to 1.3W and power it back down to 0W after 6 hours. Please do not touch ITM optics.
I updated the HWSY centroids with low variance versions.
I also reset the HWSY magnification to 7.5x
controls@h1hwsmsr:~/temp/HWSY$ caput H1:TCS-ITMY_HWS_MAGNIFICATION 7.5
Old : H1:TCS-ITMY_HWS_MAGNIFICATION 17.5
New : H1:TCS-ITMY_HWS_MAGNIFICATION 7.5
Here is the time series for the CO2 heating from last night. Analysis to follow but rough numbers are as follows.
There are two measurements per ITM of the lensing: when CO2 is turned on and again when CO2 is turned off. Hence:
| CO2 | HWS | Lens per Watt | |
| X | 1.25W | 63 +/- 2 uD | 50.4 +/- 2 uD/W |
| Y | 1.1W | 55 +/- 7 uD | 50 +/- 6 uD/W |
One thing that I forgot to mention is that I make the power step down half way and stayed there for an hour before going to 0W. As you can see from the timeseries.
Every time when we restart the HWS code for ITMY, the code sets the MAGNIFICATION value back to default of 17.5 and this had been annoying. So instead of manually changing the value for every restart, I made a hack so that when the code is executed for ITMY, it automatically changes MAGNIFICATION to 7.5, 60 seconds after the execution of the command. This was done by editing the alias setting in ~/.bashrc. Now the alias for Run_HWS_H1ITMY is written as follow.
alias Run_HWS_H1ITMY='/opt/HWS/Run_HWS_0/distrib/Run_HWS_0 & sleep 60; caput H1:TCS-ITMY_HWS_MAGNIFICATION 7.5'
This was tested once and was successful.
Last night, we ran a quick TCS test where we attempted to minimize the intensity noise coupling to the DCPDs by changing the CO2 differential heating.
It seems that the following CO2 setting gives a much better intensity noise coupling when the PSL power is 25 W:
This did not improve the recycling gain so much. It seems to have increased by 2% only.
According to a past measurement with a lower power PSL of 2 W (alog 26264), a good differential CO2 power had been found to be P_{co2x} - P_{co2y} = 270 mW (or probably less than 270 mW because I did not explore the lower differential power).
This could be an indication that ITMY has a larger absorption for the 1064 nm light such that the differential self-heating linearly changes as a function of the PSL power. We should confirm this hypothesis using the HWS signals.
[The test]
No second or third loops engaged, DC readout, no SRC1 ASC loop.
Drastic reduction of the intensity noise coupling was observed mostly between 4:24 and approximately 5:00, indicating that reducing the CO2Y power helped improved the coupling. After 5:00 UTC, we did not see a significant reduction. This may mean that we might have been already close to an optimum point where the coupling is minimized. The attached shows DARM spectra from various time during the test.
A broad peak at around 400 Hz is my intentional excitation to the first loop with band-passed gaussian noise in order to check the coupling to the DCPDs. As shown in the spectra, the reduction from the beginning to the end of the test is about a factor of 5. As reported in 27370, broad noise above 100 Hz up to several kHz is indeed intensity noise and therefore we see the noise floor in this frequency band decreasing too.
Because of the error in picomotor assignment, there's a good chance that the CO2Y laser is severly clipped and not properly aligned to the test mass. The result of this would be strong higher spatial order lensing (non-quadratic) on ITMY. We're certainly seeing an excess in lensing as measured with the HWS but the exact nature is unclear.
If there is signifiant higher order mode lensing then the best differential effect will be from having the CO2Y laser set to zero. However, this will not be the optimum lensing for the power recycling cavity.
So, poor CO2Y lensing is at least consistent with and a plausible explanation for requiring 0W on CO2Y to minimize intensity noise coupling while observing reduced PRC gain.
I looked back the intensity coupling of this particular day. See the attached.
The coupling at 100 Hz, even though the coherence is not high, seems to be too high by a factor of two or so comparing agaist the measurement from this February (25476). When dCO2 was sort of adjusted (red curve), the coupling at around 400 Hz and above seems comparable to what it was in this February.
