It looks like somehow Gabriele's DHARD Pitch injections (alog 25218) didn't get stopped properly on Wednesday night, and they've been going ever since. Ooops.
Gabriele thought that he stopped everything, and the workstation he was on has been logged out of and re-logged into by someone else, so I would think that awggui would have definitely stopped, but somehow it was still going. Anyhow, I cleared the test points for ASC and the excitation has actually stopped now.
Attached screenshot shows the excmon for the last 5 days. I don't know that this has prevented locking (Cheryl + commissioners had a 10+ minute lock earlier today), but it probably hasn't been helping.
Ed Merilh, Nick Anderson (TJ's cousin)
The final 17, spare, AOSEMs that we have in inventory are soldered and measurements have been taken to populate the spreadsheet in the DCC (pending). Betsy will complete assembly and Joe will follow with cleaning. The final count is approximately 78.
1320 - 1355 hrs. local -> To and from X-end Moved controller to X2-8 and powered ion pump using gas generator and spare (short) HV cable to demonstrate that issue is with the existing HV cable and not the controller or the pump itself -> cable connector items needed to shorten the existing cable and finish the permanent install of the controller at the X-end station will be here early next week and will, at that time, take steps to locate short. X2-8 BT ion pump will remain out of service until further notice.
There was a report of GPS glitch as large as 13 microsec (http://www.itnews.com.au/news/satellite-failure-caused-global-gps-timing-anomaly-414237) starting around "January 26 12.49 am local time", presumably Mountain Standard Time, so it's probably around Jan 26 07:49 UTC, and was fixed by "6.10 am Mountain Standard Time".
I looked at the timing comparator for the cesium clock (H1:SYS-TIMING_C_MA_A_PORT_2_SLAVE_CFC_TIMEDIFF_1) as well as X end GPS receiver (H1:SYS-TIMING_X_FO_A_PORT_9_SLAVE_CFC_TIMEDIFF_3) and Y end GPS (H1:SYS-TIMING_Y_FO_A_PORT_9_SLAVE_CFC_TIMEDIFF_3).
(It seems like end station GPS units are connected to the second input of timing comparators at each end station at LLO, but here they are to the third input.)
Right plot shows 6-days, and the left one is 4 hours around the reported glitch, and I see no timing jump.
State of H1: locking but not making it to DRMI_Locked
Summary:
Update as of 21:12UTC (13:12PT)
We are pretty sure this added motor is a non issue as far as noise goes. But, I'll let this run for a week before the configuration changes. It is running under servo control and the level switch functions so it should shut itself off should a bad leak occur. I have a nuisance drip on one of the hoses but am capturing that.
Otherwise, if you are in the Mechanical Room and see some major problem at the flushing stand, just hit the red button on the North facing enclosure cabinet.
Re WP 7505
Making these gains unity greens the hardcoded medm making the state of the ISI look better by reducing the reds seen on the Overview screens. There are still other reds seen in Sensor Correction and Blend sections of the Overview but greening or graying these out will be more of an medm programming pain. I'm not sure I'll be able to 'fix' these...
The foton files have been committed. I've acceptted the changes in the SDF OBSERVE.snaps (down.snap for the BS.) I still need to accept these for the safe.snaps & the BS OBSERVE.snap. I'll complete that after the meeting currently attending.
Meanwhile--Let's not restart any ISI FEs.
Here are conlog reported differences to record the gains. If it is deemed a good idea to change the gain back just be sure to turn off the Match_G filter under the appropriate bank doing so in a reasonable order. Gains changed to zero are on banks not being used so there is no reason to mess with them--Jim and I like to turn paths off as much as possible.
The foton files are committed. JeffK grepped to extract the gains in the files and they are attached here too. I just looked at them again and I very sure I did not make any typos in generating the fotons. That is, the old gains (conlog) match the new gains in the fotons.
I've captured all the ISI SDFs for safe, observe, & down.snaps. These are committed to the svn except for the down (BS) as I don't know where that one is at the moment.
I've moved the ISI BS down.snap from the target area to the userapps area as h1isibs_down.snap and symbolic linked it back to the target area down.snap; it has been committed to the svn.
After months of 24/7 work, had an opportunity to gather a majority of the LHO Operator Team during our Ops Meeting (Jim is photoshopped in since he was working as Commissioning Help during the evenings this week).
Thank you for all your work!
You need to add Nutsinee's name to the list
WOW! Thank you for catching that Keith!! OK, it has been updated.
FYI - I've been holding off on taking the ETM charge measurements since Cheryl has been hard at work to relock both yesterday and now today. We assume it is better to keep plugging away at lock recovery since we've been on a dry spell for a few days. We'll continue to look for an hour of dead time today through Tuesday.
The cooling system for the H-2 electronic building was repaired yesterday, (see FRS 4271), and is functioning correctly.
The TCS AOM drivers had tripped. To reset them, I turned the AOM driver power supply (mezzanine in the mechanical room) off and then on again.
As much as I enjoyed the walk ou there, we need to be able to do this remotely.
Day Ops: 16:00-23:59UTC (08:00-16:00PT)
State of H1: unlocked, locking and making past DRMI, but not to Low Noise
Shift Summary:
Work Today:
Issues encoutered today - locking the X arm in IR:
Environmental deturants to locking:
0915 -0955 hrs. local -> To and from X-end Pump won't start on either HV channel, gets to 500V then limits at 500ma -> Tempted to increase the default arc limit of 10 per minute but will consult with GammaVacuum first
1020 - 1040 hrs local -> To and from X-end Consulted with Gamma Vacuum -> They advised not exceeding 20 arc/min setting -> I tried 15 - no help -> I also unplugged the HV cable from the controller but left the SafeConn connector connected and then enabled the HV which then came up to 7500V without a problem - meaning the controller is likely OK - more to come
What type of HV cable is in service? Gamma's cable or RG58 RED? Did you pull both types of cables to have one as a back up?
H1 has 30 sender RFM channels on each arm, of which only 26 have corresponding receiver(s). So 4 are being sent and no model is using the data.
L1 has 29 senders on the X-ARM (of which there are 26 receivers), and 30 senders on the Y-ARM (of which there are 26 receivers).
So the two sites are very close in number of sending channels.
Analysis details: The base number of potential senders was derived from the main IPC file, looking for RFM0 and RFM1 ipc types. This resulted in 30 for H1-X and H1-Y, and 42 for L1-X and L1-Y. Because the ipc file is only appended to during compilation, if it has not been cleanly regenerated recently it may overcount the number of sending channels.
For each channel, I searched the models' RCG generated IPC_STATUS.adl medm file for the channel name (e.g. H1LSC_IPC_STATUS.adl). Assuming that no two ipc channels share the same name, if I found the channel name in the adl file this means it is a running sender with a receiver. For the remaining possible senders without receivers (H1-X=4, H1-Y=4, L1-X=16, L1-Y=16) I looked for the channels in the top level simulink source files (e.g. /opt/rtcds/userapps/release/*/l1/models/l1*.mdl). This showed that all four channels on H1-X and H1-Y do have sending models, and for L1-X 3 of the 16 had sending models, and for L1-Y 4 of the 16 had sending models.
If we can possibly remove some of the RFM channels which are not being received, additional RFM channels can be added to the loop with no risk.
For H1 the sending channels with no receivers are:
Tagging people interested in adding new (or rather, trading for) RFM channels. SEI: IFO Basis SEI channels CAL: Sending PCAL excitations to the corner.
Dave, Thanks for the count. For the SUSpoint motion in the IFO basis, (see ECR E1600028 , or Integration Issue 1193 , or Tech Doc T1500610 ) we need 2 RFM channels, 1 per arm for the ETMX SUS-WIT and ETMY SUS-WIT each to OAF in the corner. For completeness, I note that we also need some PCIe channels from the top level of other 12 suspensions (3 IMCs, 3 SRMs, 3 PRMs, BS, ITMX, ITMY). These can replace the GS-13 X/Y signals now being used by OAF. Evidently the RFM senders for these are living in the PEM model at LLO. I do not know why it is done this way, but it may be related to the configuration of the FE machine for ISI at LLO. ALSO (1) : for more complete monitoring, it would be useful to also send the STS-2 X/Y signal from the end to OAF. ALSO (2): For Earthquake common mode control (still hypothetical) we would need to send the End X or Y STS-2 to the corner, and ALSO send the corner X/Y out to the ends. Summary of RFM: 1 per arm from SUS to OAF (high priority) 1 per arm from ISI-GND to OAF (med priority) 1 per arm from GND-ITMY X to End X and ITMY-Y to End Y (med priority) NOTE- these signals don't need to be 16k. We want accurate data at 1 Hz and below, so 512 sample/sec would be fine. Thus, it is not crazy to think about ways to de-stress the RMF system (e.g. interleave several slow channels on one fast RFM connection, or something like this.)
I spent some time looking at how we can reduce the DARM residual.
In the attached plot (template lives in evan.hall/Public/Templates/LSC/DarmResReduce_2015-01-23.xml), yellow shows the nominal DARM residual and estimated freerunning displacement.
Blue shows the residual after the addition of a 3 Hz resonant gain. I put this in LSC-DARM FM10, so the freerunning estimate should still be correct. [However, since the DARM filter bank is full, I had to overwrite some other filter. After this test was over, I reverted my changes and placed this RG filter in the LSC-OMC_DC filter bank for safekeeping.]
Red shows the residual after the addition of the 3 Hz resonant gain, as well as a microseism boost, which I put in LSC-OMC_DC. This affects the freerunning estimate, but the amplitude and phase changes at and above 10 Hz are minimal (less than 2 ° of phase change at 10 Hz).
Here is a plot similar to the one in 25053.
It shows the DCPD spectra from last saturday when I had a 6 Hz line injected, with the predicted upconversion from the DC readout quadratic term explaining the line at 12 Hz as well as the noise in the DCPDs at around 10 Hz. The red trace is the spectrum evan calls nominal above, where there were no changes to the DARM loop but the feed forward was retuned, as you can see the predicted upconversion is reduced by nearly 2 orders of magnitude at 10 Hz, and the upconversion around the calibration lines is reduced.
The black trace is once Evan had added both a boost and a resonant gain to the DARM loop, so we expect the DCPD specrtum to be reduced at low frqeuency simply because of the change in the DARM loop shape. You can see that there is also a reduction in the predicted upconversion around the calibration lines as well as at 10 Hz.
Incidentally I was looking at the residual spectra of LLO and LHO (from Jan 1 2016 00:00:00 UTC) for some other reason. I post the plot for the record.