The last day LHO has been suffering from high microseism, so we've had lots of time to ponder ground motion. JeffK noticed a seeming inconsistency on the DetChar summary pages, that maybe someone can clear up for us. The ASD for H1 ground motion last night seems has a microseism peak in Z at about 5 microns, but the BLRMS for that period indicates that the motion is about a micron, a factor of 5 disagreement. Attached pictures are screenshots of the Detchar page for last night, first is the ASD, second is the BLRMS. This could be partly because the two measures are calculated for different measures of amplitude, peak vs peak-to-peak vs RMS. But I think that at best explains a factor of 2 difference, not 5.
The ASD has units of m/rt(Hz), so to get RMS value, you have to multiply by square root of the frequency of interest. So 5 micron/rt(Hz at 0.1 Hz gives an RMS of ~1.5 micron.
There is also the problem of velocity versus displacement measurement, but since f is nearly 0.16 Hz, it doesn't make much difference.
This is just a quick listing of the tests that each IFO has in their DIAG_MAIN Guardians and their docstrings.
LHO
ALS: ALS checks, This can be seen when the beatnote drops down to 0-2 MHz and when (in dBm) it gets above 5. Checks to see if the ALS SHG Temperature Control Servo is on Nominal is 'ON'
BEAM_DIVERTERS: The Beam divereters should be closed after the ISC_LOCK state of CLOSE_BEAM_DIVERTERS #550. 1 == closed, 0 == open.
BECKHOFF: Beckhoff status check
Check some random Beckhoff channels to see if the system in running.
COIL_DRIVERS: Coil drivers functioning, If they stay at 0, it will bring up a notification
ESD_DRIVER: ESD driver status
HWS_SLEDS: HWS SLEDS are on
HW_INJ: Hardware injections running
ISI_RINGUP: ISI ring up motion as seen in alog24245, Only looking at the ETMs and non rotational dofs for now.
MC_WFS: MC WFS are engaged
OMC_PZT: The OMC PZT2 high voltage should be on, otherwise the OMC_LOCK Guardian will fail at the FIND_CARRIER step
OPLEV_SUMS: Optical lever sums nominal, Check only when the suspension is aligned.
PEM_CHANGE: Major environmental changes
PMV_HV: PMC high voltage is nominally around 4V, power glitches and other various issues can shut it off.
PSL_FSS: PSL FSS not oscillating
PSL_ISS: ISS diffracted power
PSL_NOISE_EATER: PSL noise eater engaged
RING_HEATERS: End station ring heaters on
SEI_STATE: SEI systems nominal, Also checks if the WD saturation counts are are greater than somepercent of the limit.
SEI_WD: SEI watchdog not tripped
SERVO_BOARDS: These are Beckhoff controlled channels and cannot be monitored by SDF yet.
SHUTTERS: Monitor the Beckhoff controlled shutters
SUS_PUM_WD: RMS watchdogs
SUS_WD: SUS watchdogs and OSEM input filter inmon on the quads
TCS_LASER: TCS laser OK, Switched ON, Power above 50W, No Flow Alarm, No IR alarm, Only for TCS X for now.
TIDAL_LIMITS: Tidal limits within range
LLO
ALS_LASER: ALS laser power above threshold
ALS_PLL: None
ALS_SERVO_POLARITY: ALS servo polarity monitors
BECKHOFF: BECKHOFF system monitor
BSOPLEV: Beamsplitter OpLev Monitor
IOP_DACKILL_MON: Monitors IOP DACKILL status and reporting faults
ISS_DIFFRACTED_POWER: ISS Diffracted Power monitor
OMC_PZT_HV: OMC PZT HV monitor
PARAMETRIC_INSTABILITY: PI not rung up
PRC_GAIN: PRC gain above threshold
SEI_ISI_T240_SATMON: SEI ISI T240 saturation monitors
SUS_QUAD_ESD_BMON: QUAD BIO ESD driver digital monitors
SUS_QUAD_ESD_DMON: QUAD ESD driver digital monitors
SUS_QUAD_L3_BIO_CHECK: QUAD L3 BIO switch consistency check
SUS_QUAD_PUM_WD: QUAD PUM RMS watchdog
static_vars: None
To get these I opened a python shell and imported inspect and the desired module, then...
for name, obj in inspect.gemembers(module_name):
if inspect.isfunction(obj):
print '{}: {}'.format(name, inspect.getdoc(obj))
Hey, TJ, here's a simpler way, that just prints registered tests and not other functions, for future reference:
servo:~ 0$ guardian -i DIAG_MAIN
--------------------
aLIGO Guardian Shell
--------------------
ezca prefix: H1:
system: DIAG_MAIN (/opt/rtcds/userapps/release/sys/h1/guardian/DIAG_MAIN.py)
In [1]: SYSDIAG.print_tests()
Registered tests:
SEI_STATE: SEI systems nominal
SUS_WD: SUS watchdogs and OSEM input filter inmon on the quads
SEI_WD: SEI watchdog not tripped
HWS_SLEDS: HWS SLEDS are on
PSL_ISS: ISS diffracted power
SHUTTERS: Monitor the Beckhoff controlled shutters
PSL_NOISE_EATER: PSL noise eater engaged
BECKHOFF: Beckhoff status check
SERVO_BOARDS: These are Beckhoff controlled channels and cannot be monitored by SDF yet.
HW_INJ: Hardware injections running
ESD_DRIVER: ESD driver status
RING_HEATERS: End station ring heaters on
ISI_RINGUP: ISI ring up motion as seen in alog24245,
BEAM_DIVERTERS: The Beam divereters should be closed after the ISC_LOCK state of
PEM_CHANGE: Major environmental changes
MC_WFS: MC WFS are engaged
COIL_DRIVERS: Coil drivers functioning
TIDAL_LIMITS: Tidal limits within range
PMV_HV: PMC high voltage is nominally around 4V, power glitches and other
PSL_FSS: PSL FSS not oscillating
TCS_LASER: TCS laser OK
SUS_PUM_WD: RMS watchdogs
OMC_PZT: The OMC PZT2 high voltage should be on, otherwise the OMC_LOCK Guardian will fail
ALS: ALS checks
OPLEV_SUMS: Optical lever sums nominal
In [2]:
1535 -1555 hrs. local -> To and from Y-mid Next manual over fill to be Sunday, Jan. 29th before 4:00 pm
The ASC model has been modified, and is ready for the 2f AS "WFS" centering.
The ADC inputs were made to be consistent with T1100472-v16. The spare AS_D (also called AS_X in the model) channels were used, as well as 8 more that had previously been unused. This means that any channels previously called AS_D no longer exist, as they have been replaced with the AS_A_RF90 channels.
So, the model now has 2 WFS demodulator blocks, AS_A_RF90 and AS_B_RF90. The I outputs (pitch and yaw) are sent to the ASC input matrices. Since these are 2f sensors, we can rotate all the signal into the I-phase, so the Q outputs are terminated in the model. These have been put at the end of the matrix so that no channel numbers / names will have changed.
It looks like the DC5 centering loop has always existed, and we just weren't using it, so I didn't add any filter banks for the control loops, nor did the output matrix change in the model. Daniel has already added the DC5 control filters and the DC5 matrix column to the medm screens.
The model compiles, and has been checked in to the svn. While I still have other ASC model modifications, this version should be ready for install during Tuesday maintenence if I don't get the other modifications done in time.
Medm screens have also been modified. They have white boxes for now that will be populated when we restart the model on Tues. I think we're ready to roll software-wise for the 2f centering.
State of H1: trying to lock, but has only made it to DRMI 1f one time in the last 90 minutes, useism and wind are high
Summary:
It looks like somehow Gabriele's DHARD Pitch injections (alog 25218) didn't get stopped properly on Wednesday night, and they've been going ever since. Ooops.
Gabriele thought that he stopped everything, and the workstation he was on has been logged out of and re-logged into by someone else, so I would think that awggui would have definitely stopped, but somehow it was still going. Anyhow, I cleared the test points for ASC and the excitation has actually stopped now.
Attached screenshot shows the excmon for the last 5 days. I don't know that this has prevented locking (Cheryl + commissioners had a 10+ minute lock earlier today), but it probably hasn't been helping.
Ed Merilh, Nick Anderson (TJ's cousin)
The final 17, spare, AOSEMs that we have in inventory are soldered and measurements have been taken to populate the spreadsheet in the DCC (pending). Betsy will complete assembly and Joe will follow with cleaning. The final count is approximately 78.
1320 - 1355 hrs. local -> To and from X-end Moved controller to X2-8 and powered ion pump using gas generator and spare (short) HV cable to demonstrate that issue is with the existing HV cable and not the controller or the pump itself -> cable connector items needed to shorten the existing cable and finish the permanent install of the controller at the X-end station will be here early next week and will, at that time, take steps to locate short. X2-8 BT ion pump will remain out of service until further notice.
There was a report of GPS glitch as large as 13 microsec (http://www.itnews.com.au/news/satellite-failure-caused-global-gps-timing-anomaly-414237) starting around "January 26 12.49 am local time", presumably Mountain Standard Time, so it's probably around Jan 26 07:49 UTC, and was fixed by "6.10 am Mountain Standard Time".
I looked at the timing comparator for the cesium clock (H1:SYS-TIMING_C_MA_A_PORT_2_SLAVE_CFC_TIMEDIFF_1) as well as X end GPS receiver (H1:SYS-TIMING_X_FO_A_PORT_9_SLAVE_CFC_TIMEDIFF_3) and Y end GPS (H1:SYS-TIMING_Y_FO_A_PORT_9_SLAVE_CFC_TIMEDIFF_3).
(It seems like end station GPS units are connected to the second input of timing comparators at each end station at LLO, but here they are to the third input.)
Right plot shows 6-days, and the left one is 4 hours around the reported glitch, and I see no timing jump.
State of H1: locking but not making it to DRMI_Locked
Summary:
Update as of 21:12UTC (13:12PT)
We are pretty sure this added motor is a non issue as far as noise goes. But, I'll let this run for a week before the configuration changes. It is running under servo control and the level switch functions so it should shut itself off should a bad leak occur. I have a nuisance drip on one of the hoses but am capturing that.
Otherwise, if you are in the Mechanical Room and see some major problem at the flushing stand, just hit the red button on the North facing enclosure cabinet.
Re WP 7505
Making these gains unity greens the hardcoded medm making the state of the ISI look better by reducing the reds seen on the Overview screens. There are still other reds seen in Sensor Correction and Blend sections of the Overview but greening or graying these out will be more of an medm programming pain. I'm not sure I'll be able to 'fix' these...
The foton files have been committed. I've acceptted the changes in the SDF OBSERVE.snaps (down.snap for the BS.) I still need to accept these for the safe.snaps & the BS OBSERVE.snap. I'll complete that after the meeting currently attending.
Meanwhile--Let's not restart any ISI FEs.
Here are conlog reported differences to record the gains. If it is deemed a good idea to change the gain back just be sure to turn off the Match_G filter under the appropriate bank doing so in a reasonable order. Gains changed to zero are on banks not being used so there is no reason to mess with them--Jim and I like to turn paths off as much as possible.
The foton files are committed. JeffK grepped to extract the gains in the files and they are attached here too. I just looked at them again and I very sure I did not make any typos in generating the fotons. That is, the old gains (conlog) match the new gains in the fotons.
I've captured all the ISI SDFs for safe, observe, & down.snaps. These are committed to the svn except for the down (BS) as I don't know where that one is at the moment.
I've moved the ISI BS down.snap from the target area to the userapps area as h1isibs_down.snap and symbolic linked it back to the target area down.snap; it has been committed to the svn.
0915 -0955 hrs. local -> To and from X-end Pump won't start on either HV channel, gets to 500V then limits at 500ma -> Tempted to increase the default arc limit of 10 per minute but will consult with GammaVacuum first
1020 - 1040 hrs local -> To and from X-end Consulted with Gamma Vacuum -> They advised not exceeding 20 arc/min setting -> I tried 15 - no help -> I also unplugged the HV cable from the controller but left the SafeConn connector connected and then enabled the HV which then came up to 7500V without a problem - meaning the controller is likely OK - more to come
What type of HV cable is in service? Gamma's cable or RG58 RED? Did you pull both types of cables to have one as a back up?
H1 has 30 sender RFM channels on each arm, of which only 26 have corresponding receiver(s). So 4 are being sent and no model is using the data.
L1 has 29 senders on the X-ARM (of which there are 26 receivers), and 30 senders on the Y-ARM (of which there are 26 receivers).
So the two sites are very close in number of sending channels.
Analysis details: The base number of potential senders was derived from the main IPC file, looking for RFM0 and RFM1 ipc types. This resulted in 30 for H1-X and H1-Y, and 42 for L1-X and L1-Y. Because the ipc file is only appended to during compilation, if it has not been cleanly regenerated recently it may overcount the number of sending channels.
For each channel, I searched the models' RCG generated IPC_STATUS.adl medm file for the channel name (e.g. H1LSC_IPC_STATUS.adl). Assuming that no two ipc channels share the same name, if I found the channel name in the adl file this means it is a running sender with a receiver. For the remaining possible senders without receivers (H1-X=4, H1-Y=4, L1-X=16, L1-Y=16) I looked for the channels in the top level simulink source files (e.g. /opt/rtcds/userapps/release/*/l1/models/l1*.mdl). This showed that all four channels on H1-X and H1-Y do have sending models, and for L1-X 3 of the 16 had sending models, and for L1-Y 4 of the 16 had sending models.
If we can possibly remove some of the RFM channels which are not being received, additional RFM channels can be added to the loop with no risk.
For H1 the sending channels with no receivers are:
Tagging people interested in adding new (or rather, trading for) RFM channels. SEI: IFO Basis SEI channels CAL: Sending PCAL excitations to the corner.
Dave, Thanks for the count. For the SUSpoint motion in the IFO basis, (see ECR E1600028 , or Integration Issue 1193 , or Tech Doc T1500610 ) we need 2 RFM channels, 1 per arm for the ETMX SUS-WIT and ETMY SUS-WIT each to OAF in the corner. For completeness, I note that we also need some PCIe channels from the top level of other 12 suspensions (3 IMCs, 3 SRMs, 3 PRMs, BS, ITMX, ITMY). These can replace the GS-13 X/Y signals now being used by OAF. Evidently the RFM senders for these are living in the PEM model at LLO. I do not know why it is done this way, but it may be related to the configuration of the FE machine for ISI at LLO. ALSO (1) : for more complete monitoring, it would be useful to also send the STS-2 X/Y signal from the end to OAF. ALSO (2): For Earthquake common mode control (still hypothetical) we would need to send the End X or Y STS-2 to the corner, and ALSO send the corner X/Y out to the ends. Summary of RFM: 1 per arm from SUS to OAF (high priority) 1 per arm from ISI-GND to OAF (med priority) 1 per arm from GND-ITMY X to End X and ITMY-Y to End Y (med priority) NOTE- these signals don't need to be 16k. We want accurate data at 1 Hz and below, so 512 sample/sec would be fine. Thus, it is not crazy to think about ways to de-stress the RMF system (e.g. interleave several slow channels on one fast RFM connection, or something like this.)
I spent some time looking at how we can reduce the DARM residual.
In the attached plot (template lives in evan.hall/Public/Templates/LSC/DarmResReduce_2015-01-23.xml), yellow shows the nominal DARM residual and estimated freerunning displacement.
Blue shows the residual after the addition of a 3 Hz resonant gain. I put this in LSC-DARM FM10, so the freerunning estimate should still be correct. [However, since the DARM filter bank is full, I had to overwrite some other filter. After this test was over, I reverted my changes and placed this RG filter in the LSC-OMC_DC filter bank for safekeeping.]
Red shows the residual after the addition of the 3 Hz resonant gain, as well as a microseism boost, which I put in LSC-OMC_DC. This affects the freerunning estimate, but the amplitude and phase changes at and above 10 Hz are minimal (less than 2 ° of phase change at 10 Hz).
Here is a plot similar to the one in 25053.
It shows the DCPD spectra from last saturday when I had a 6 Hz line injected, with the predicted upconversion from the DC readout quadratic term explaining the line at 12 Hz as well as the noise in the DCPDs at around 10 Hz. The red trace is the spectrum evan calls nominal above, where there were no changes to the DARM loop but the feed forward was retuned, as you can see the predicted upconversion is reduced by nearly 2 orders of magnitude at 10 Hz, and the upconversion around the calibration lines is reduced.
The black trace is once Evan had added both a boost and a resonant gain to the DARM loop, so we expect the DCPD specrtum to be reduced at low frqeuency simply because of the change in the DARM loop shape. You can see that there is also a reduction in the predicted upconversion around the calibration lines as well as at 10 Hz.
Incidentally I was looking at the residual spectra of LLO and LHO (from Jan 1 2016 00:00:00 UTC) for some other reason. I post the plot for the record.