Per https://alog.ligo-wa.caltech.edu/aLOG/index.php?callRep=26806 CP5 LL has stabilized to its normal conditions. Still has a jittery output.
Lots of lock losses and FSS oscillations. Sheila and Jenne investigating. 14:15 UTC Chris S. working in LVEA moving barrels from bottom of stairs into the high bay 14:30 UTC Jeff B. to HAM6 and PSL rack 15:00 UTC Chris S. done in LVEA, working in high bay until 17:00 15:17 UTC Kyle to LVEA 15:17 UTC Peter to H1 PSL enclosure 15:28 UTC Jeff B. back 16:24 UTC Filiberto to LVEA to terminate cables for PT170 and PT180 BPG402 gauges 16:54 UTC Kyle back from climbing on HAM11, HAM12, BSC4 17:05 UTC Jamie restarting h1guardian0 for code upgrade 17:12 UTC Christina and Karen to mid Y Peter done 18:20 UTC Turned on and off PSL noise eater in PSL rack 18:21 UTC Christina and Karen done at mid Y 18:33 UTC Jeff B. to LVEA to check dust monitor cabling 18:56 UTC Dave restarting OMCPI, SUSPI models, DAQ restart 18:58 UTC Jeff K. changing SUS ESD bias 19:19 UTC Filiberto out for lunch 19:21 UTC Jeff B. out of LVEA 19:26 UTC Sheila to LVEA to investigate FSS oscillations 19:39 UTC Filiberto to LVEA to continue cable work 19:53 UTC Gerardo to end Y to replace electric LLCV motor fuse 19:55 UTC Sheila back, made no changes 20:11 UTC Gerardo changed CP7 electric LLCV motor fuse, going to mid Y to overfill CP3, then end X 20:17 UTC Joe to LVEA to track down view port equipment 20:19 UTC Jeff B. to turn on dust monitor at PSL 20:48 UTC Gerardo changed CP8 electric LLCV motor fuse 20:48 UTC Filiberto done 21:00 UTC Jeff B. done 21:47 UTC Gerardo to BSC4 to look at PT140 cabling 22:20 UTC Gerardo back 23:03 UTC Gerardo to LVEA to measure PT140 voltage at vacuum rack
This upgrade includes a couple of minor bug fixes and a new guardian log server/client:
After these changes were applied, all nodes were re-created to get the new changes to the logging, and the h1guardian0 machine was rebooted. A couple of small issues were encountered during the reboot:
This version of guardian includes a new and improved log client/server. The new server running on the h1guardian0 machine:
See "guardlog -h" for more info.
Tega, Ross, Dave WP5850
We added two Dolphin IPC senders on the h1omcpi model (running at 64kHz) and two corresponding receivers on h1susitmpi. These send the OMC DC_PD signals for A and B PDs. A new common part in PI_MASTER.mdl was created.
Both models were restarted, followed by a DAQ restart.
model restarts logged for Tue 26/Apr/2016 DAQ restart for Vacuum LX and LY upgrade to Beckhoff. h1fw1 instability.
2016_04_26 01:24 h1fw1
2016_04_26 03:13 h1fw1
2016_04_26 11:05 h1dc0
2016_04_26 11:05 h1nds0
2016_04_26 11:05 h1nds1
2016_04_26 11:06 h1fw1
2016_04_26 11:07 h1broadcast0
2016_04_26 11:07 h1fw0
2016_04_26 11:07 h1tw1
2016_04_26 20:34 h1fw1
model restarts logged for Mon 25/Apr/2016 DAQ restart for vacuum MY upgrade to Beckhoff. ASC and ETMXPI model changes (with DAQ restart). both FW instability.
2016_04_25 01:04 h1fw1
2016_04_25 05:53 h1fw1
2016_04_25 06:23 h1fw1
2016_04_25 08:34 h1fw1
2016_04_25 08:54 h1nds1
2016_04_25 09:23 h1fw1
2016_04_25 12:08 h1broadcast0
2016_04_25 12:08 h1dc0
2016_04_25 12:08 h1fw0
2016_04_25 12:08 h1fw1
2016_04_25 12:08 h1nds0
2016_04_25 12:08 h1nds1
2016_04_25 12:08 h1tw1
2016_04_25 13:03 h1fw0
2016_04_25 13:20 h1asc
2016_04_25 13:27 h1susetmxpi
2016_04_25 13:30 h1broadcast0
2016_04_25 13:30 h1dc0
2016_04_25 13:30 h1fw0
2016_04_25 13:30 h1fw1
2016_04_25 13:30 h1nds0
2016_04_25 13:30 h1nds1
2016_04_25 13:30 h1tw1
2016_04_25 20:33 h1fw1
2016_04_25 21:03 h1fw1
2016_04_25 21:05 h1fw0
2016_04_25 22:23 h1fw1
model restarts logged for Sun 24/Apr/2016 No restarts reported
model restarts logged for Sat 23/Apr/2016 No restarts reported
model restarts logged for Fri 22/Apr/2016 No restarts reported
model restarts logged for Thu 21/Apr/2016 FW0 instability. ETMXPI model change (no DAQ restart required)
2016_04_21 01:17 h1fw0
2016_04_21 01:25 h1fw0
2016_04_21 16:52 h1susetmxpi
After the chiller problems noted in the LLO aLOG #25879, I did a visual inspection of both the LHO chillers. There was no evidence of damage or deformation to the compressor or the crossbar. The coolant reservoir is not leaking. When the chiller is shut off there is a big back surge of water, which has enough force to blow the filler plug across the room. If the filler plug is screwed in too tightly and there is no other pressure relief in the coolant system, it is possible the rapid pressure increase due to the water surging back into the reservoir could burst a seam in the reservoir. We leave our filler plug loosely secured so as to act like a pressure relief valve.
PEM: All new dust monitors installed except in PSL enclosure. VAC: Replacement of fuse for CP1 electric fill valve. Investigate failure of BSC4 Pirani gauge. Investigate fill control of CP5.
Phil is replacing the fuse at CP1 with one larger than 1 A since the new electronic valves draw at least that much current. CP1 has overfilled as a result. Current fill level at 5%.
Chris B., Jamie R.
The hardware injection guardian node has been setup at LHO. The node should be ready to perform injections for the engineering run. Many thanks to Jamie.
The node is called INJ_TRANS. I have paused it.
Code is in: /opt/rtcds/userapps/release/cal/common/guardian
States that can be requested
A graph of the guardian states is attached. There are two states that can be requested:
* INJECT_SUCCESS: Request this when you want to do injections
* INJECT_KILL: Request this to cancel an injection
You should request INJECT_SUCCESS to perform an injection. The node will move to the WAIT_FOR_NEXT_INJECT will continuously check for an injection that are going to happen in the next five minutes (so if there are no injections for a long time, the node will spend a long time in this state). Once an injection is soon, it uploads an event to gracedb, reads the waveform data, and waits to inject. Eventually it will move into the injection state and inject the waveform. It will move back to the WAIT_FOR_NEXT_INJECT state and begin waiting for the next injection.
While the node is preparing to do an injection, eg. gracedb upload, etc., there will be a USERMSG letting the operator know an injection is about to occur. See MEDM screen below.
How to schedule an injection
This is just some short hand notes for how to schedule an injection with the guardian node until a document is in the DCC.
There are three steps:
(1) Update the schedule file and validate it
(2) Reload the guardian node
(3) Request INJECT_SUCCESS if its not already
The current schedule file at the time of writing is located here: https://redoubt.ligo-wa.caltech.edu/svn/cds_user_apps/trunk/cal/common/guardian/schedule/schedule_1148558052.txt
The location of the schedule file is defined in https://redoubt.ligo-wa.caltech.edu/svn/cds_user_apps/trunk/cal/common/guardian/INJ_TRANS.py, search for the variable schedule_path.
An example line is:
1145685602 INJECT_DETCHAR_ACTIVE 0 1.0 /ligo/home/christopher.biwer/projects/guardian_hwinj/test_waveforms/box_test.txt None
Where:
* First column is GPS start time of the injection.
* Second column is the name of the guardian state that will perform the injection. Choices are INJECT_CBC_ACTIVE, INJECTION_BURST_ACTIVE, INJECT_DETCHAR_ACTIVE, and INJECT_STOCHASTIC_ACTIVE.
* Third column says whither you want to do the injection in observing mode. If this is 1, then do the injection only if the IFO is in observing mode. Otherwise set this to 0.
* The fourth column is the scale factor. This is a float that is multiplied with the timeseries. For example, 2.0 makes the waveform's amplitude twice as large and 0.5 makes the waveform's amplitude twice as small.
* The fifth column is the path to the waveform file. Please use full paths.
* The sixth column is the path to the meta-data file. Please use full paths. If there is no meta-data file, then type None.
Do not schedule injections closer than 300 seconds apart. If you want to do schedule injections closer than 300 seconds, then you will want to tune imminent_seconds in INJ_TRANS.py.
You should validate the schedule file. To run the script on a LHO work stations do:
PYTHONPATH=/opt/rtcds/userapps/release/cal/common/guardian/:${PYTHONPATH}
python /opt/rtcds/userapps/release/cal/common/scripts/guardian_inj_schedule_validation.py --schedule /opt/rtcds/userapps/release/cal/common/guardian/schedule/schedule_1148558052.txt --min-cadence 300
Note you need the glue and gracedb python packages to run this script - currently an FRS to get this installed.
Failure states
There are a number of failure states, eg. waveform file cannot be read, etc. If you validate the schedule the node shouldn't run into any failures. If a failure state is entered, the node will not leave it on its own. To leave a failure state identify the problem, resolve the problem, request INJECT_SUCCESS, and reload the node. Places where a failure could occur will print a traceback in the guardian log.
GraceDB authentication
I write this for anyone not familiar with the process.
Running this guardian node will require a robot certificate because the node will upload events to GraceDB automatically. To get a robot certificate follow the instructions at https://wiki.ligo.org/viewauth/AuthProject/LIGOCARobotCertificate.
We created a robot certificate for the controls account at LHO for the h1guardian0 machine.
We had to ask the GraceDB admins (Alex P.) to add the subject line from the cert to the grid-map file.
In the hardware injection guardian node env we set X509_USER_CERT to the file path of the cert and X509_USER_KEY to the file path of the public key.
Tested gracedb API with: gracedb ping.
Successful injections on GraceDB
Injections on GraceDB are given the INJ label if they are successful. There is a success message also printed in the GraceDB event page, with the line from the schedule file. For example H236068.
Test injections
At the end of the night I did a 2 hour series of CBC injections separated by 400 seconds, I've attached plots of those injections as sanity checks that everything looks alright.
Command line to bring up MEDM screen: guardmedm INJ_TRANS
TITLE: 04/27 Eve Shift: 23:00-07:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Planned Engineering
INCOMING OPERATOR: None
SHIFT SUMMARY:
LOG:
To continue the ASC input matrix stability testing from last night, I restored the AS_A / AS_B RF36 WFS combinations we were using (-1 / 1 for PIT, and -0.8 / 0.5 for YAW). Apparently these haven't made it into the guardian yet. Ramping them on by hand appeared to improve the alignment. I also enabled a length dither, which we plan to use to monitor the DARM pole (LSC-OUTPUT_MTRX_1_9 set to 1). Leaving H1 locked in this state overnight.
DIAG_MAIN notification shows that HWSX had bad peak counts for a moment. Timeseries shows that this happened between 4:10 and 4:13 UTC but got better on its own. The glitches is probably caused by me stopping and restarting the code to look at stream images. They look fine. The output power off HWSX sled is ~0.8W.
I think you mean 0.8mW
Oops yes. Sorry.
Chris, Keita, Evan
Today we were able to lock the outer ISS loop with the modecleaner at 20 W (and no interferometer). We looked at several PSL/IOO PD signals (the FSS transmission PD, the ISS inner-loop PDs, the IM4 transmission PD, and the ISS outer-loop PDs) and tried to understand their behavior in different ISS configurations.
Naively one would expect all these signals (except the in-loop ISS PDs) to agree with each other, since they should all be out-of-loop sensors for the RIN leaving the PMC. Together, these signals monitor three of the four PMC ports: the FSS transmission sees the RIN of one port, the out-of-loop inner-loop ISS PD sees the RIN of another port, and IM4 trans and the out-of-loop outer-loop ISS PD sees the RIN of yet another port.
These are the behaviors we observed (see attached pdf):
We think that a possible explanation for these effects is that both ISS PDs are seeing some correlated noise that is not seen by either the FSS PD or the post-IMC PDs. In this scenario, the inner-loop ISS would suppress the HPO noise but impress this correlated noise on the light entering the PMC.
Briefly we entertained the idea that the light circulating in the PMC could be multimoded (either from the NPRO or the HPO), but judging from the RIN before and after the IMC, this seems to not be the case (png attachment).
One other idea is that some of the 808 nm light is getting through the PMC and onto the ISS.
Is this really incompatible with jitter? There are a lot of variations visible on the PMC reflected camera. The finesse of the PMC isn't that great (~100), and neither is jitter supression. If there is a static misalignment into the PMC, there would also be a linear term for the jitter to intensity conversion. The two inner loop detectors see rather different signals at 10Hz, if the inner loop is engaged but not the outer one.
Certainly the jitter seen on the IMC WFS is worse than before the HPO turn-on.
Before the turn-on, the jitter below 100 Hz was 1 nrad/Hz1/2 or so (LHO#21212). Now it is 10 nrad/Hz1/2 at 10 Hz, with a 1/f slope.
The attachment shows IMC signals with the inner ISS loop off (dashed) and on (solid).
Update: BS alert. Read the next entry.
Jitter is much larger than before, but the jitter alone doesn't seem to explain all of our observations at the same time when the 1st loop is closed but the 2nd loop open.
PDA=P+a*J+Sa, PDB=P+b*J+Sb, IM4=P+x*c*J+Sim4
P is the intensity noise leaving the AOM. When the loop is open it's just the free running noise P0.
J is the beam jitter (01 amplitude relative to 00) coming out of PMC.
a, b and c are the jitter to intensity coupling at PDA, PDB and IM4 trans due to clipping or diode inhomogeneity or whatever.
x is the attenuation of 01 mode amplitude by IMC, which is about 0.3%.
Sa, Sb and Sim4 are the sensing noise.
When 1st loop is closed, J is imprinted on P:
P=P0/(1+G) - G/(1+G) *(b*J + Sb) ~ P0/G - b*J - Sb,
PDA ~ P0/G + (a-b)*J +Sa-Sb,
IM4 ~ P0/G + (x*c-b)*J + Sim4-Sb ~ P0/G -b*J +Sim4-Sb. (note x=3E-3.)
where G is the OLTF.
Allowing some conspiracies but not extreme ones, lack of coherence between PDA and IM4 is explained in either of the following:
The first case is false because swapping PDA and PDB makes no difference in IM4.
In the second case, PDA spectrum should look like all sensing noise, but this "sensing" noise in reality is big at 10Hz.
So, even if the clipping effect is common in PDA and PDB so the PDA and IM4 becomes incoherent, we need another noise that is not unlike big sensing noise, i.e. of about the same amplitude on PDA and PDB, is incoherent between PDA and PDB, and does not appear on downstream sensors.
I take my words back about PDA-downstream coherence.
I was looking at the coherences from this morning, and it seems like when only the first loop is on, 1st loop out of loop sensor is coherent with downstream sensor before and after the IMC (attached, bottom red and blue). The plot is calibrated in RIN.
Note that we switched the control photodiode from PDB to PDA last night, so in this plot the out of loop sensor is PDB. I switched them back again at 17:49:10 UTC.
Anyway, out of loop sensor is more coherent with downstream sensors than HPL monitor is at f<10Hz (bottom red|blue VS brown|pink), but HPL is more coherent from 10 to 200 Hz. Difference between bottom brown and bottom pink probably doesn't mean much, just the noise floor difference between IMC-PWR and MC2_TRANS.
Some thinking necessary, but at the moment I cannot say that jitter cannot explain everything.
John and I visited CP5 before lunch today to investigate why the liquid level is so noisy (compared to CP6). We verified wires were tight at the controls rack, and eventually made our way to the LL transducer. We closed the exhaust at transducer and the flow stabilized suggesting the instability is caused by a real pressure differential and not electric. We did not check the LLCV pneumatic actuator. After trending the numbers this evening, looks like we did a real number on the system. See plots attached. The % valve open is ranging full scale and LL full spans 90-95%.
Could this be related to the midstation air compressor replacement?
Richard just reset the PID values (same values). The LL seems to have stabilized (well, to its prior stability which is still relatively noisy). We will watch it throughout the day.
C. Cahillane I have produced statistical uncertainty spectrograms for all of O1. For the most part the uncertainty doesn't change much over all of O1. The biggest concern is a couple days around Nov 17th with large kappa variations. Again, Jeff has suggested that I detrend all of the kappas to eliminate this spike in uncertainty. I believe that this is good evidence that the statistical uncertainty over all of O1 is fairly constant. For the LLO statistical uncertainty spectrograms, please see LLO aLOG 25652.
C. Cahillane I have detrended the kappas and reproduced the above spectrograms without the massive spikes of uncertainty.
Michael, Hugh, Krishna
A lot of progress was made with the installation of BRS-2 today. In the morning, we cleared out a space on the VEA floor. We had to slightly reposition the SEI ground seismometer (STS2) and the PEM Guralp seismometer. We then cleaned the BRS-2 vacuum can, foam box and other parts and moved them in. Upon opening the vacuum can, we noticed that the epoxy (TorrSeal) on one of the capacitor plates had come off, likely during the drive. We found a suitable alternative (Loctite 1c) and reattached the capacitor plate.
We the proceeded with the assembly. The beam-balance was pulled out, the flexures were carefully attached and the beam-balance was then reinserted into the can and suspended from the flexures. We did a crude adjustment of the horizontal center of mass. The vacuum can was then partially closed up and the autocollimator attached to it. We then had to realign the optics to get the reflections from the reference mirror and the main mirror algined well. Tomorrow we will continue with autocollimator adjustments to get good reflection patterns on the CCD.
In the meantime, Jim B and Carlos installed the BRS-2 Beckhoff computer at End Y. Also, Filiberto and Peter laid out the GiGE, Ethercat and Fiber-Optic (For the autocollimator light source) cables going from the VEA to the computer room.
Plan for tomorrow:
1. Get the C# code to read the CCD data and start measuring the tilt signal. At this point, the instrument is still in air, so will have excess noise and drifts.
2. Hook up the electronics for the capacitive control and the DACs for writing out the signals. Once the cables to go from these to the SEI Frontends is complete, the tilt data will be accessible to SEI.
3. Close up the rest of the vacuum can.
I attach a collection of pictures I took during this day's installation. I was primarily focused on pictures of the electronics readout system that's new for BRS 2.0, but there are also some pictures of the balance and vacuum can. Let me know if you like and need any of the originals. For my record, the originals live on my laptop, in the folder /Users/kissel/Desktop/scratch/2016-03-24/
