Upgrade end station SUS front end computers with faster models [WP5728]
Robert, Carlos, Hugh, Dave, Jim:
The front end computers h1susex and h1susey were upgraded to the faster models. A brand new IRIG-B card which was installed in h1susey did not function correctly, and the card from the old computer was used instead. At both end stations the swap caused the other machines on the Dolphin fabric to hang, and therefore all models on h1seie[x,y] and h1isce[x,y] were restarted. The IRIG-B time on h1iopsusey dipped below zero for a short time (giving bad DAQ data and IPC receive errors). On h1iopsusex there was no IRIG-B time slew.
Hugh investigated the ISI payload watchdog as we tried to get SEI to ride through the SUS outage. We found several interesting things about this watchdog, and the main message was that it is not possible to bypass this watchdog before the sus mode is stopped. Turns out this was moot anyhow, since when the sus computer was powered back up the Dolphin fabric was disrupted.
All models were set back to OBSERVE sdf reference. The PEM ADC spare channels which were activated yesterday for Robert went back to their inactive SAFE state, using OBSESRVE state and REVERT I re-activated these.
With the faster computer and the original 2.6 kernel comes the periodic timing and adc errors on the sus machines. Since these errors usually latch on until a DIAG_RESET is issued, I am running a script on opsws16 which presses the DIAG_RESET button every minute on h1iopsuse[x,y] and h1susetm[x,y]. This will permit trending of these errors.
New PSL ISS model
Dave, Kiwamu:
Kiwamu installed his latest h1psliss model, this required a DAQ restart
New Beckhoff corner station PLC2 code
Daniel, Dave, Patrick:
Patrick installed new h1ecatc1plc2 code. I updated the DAQ INI and autoburt.req files. This required a DAQ restart
DAQ restart
Jim, Dave:
We restarted the DAQ once today for the above changes. The restart was clean. One interesting note is that h1fw0 started sufficiently before h1fw1 that it wrote our a full frame before h1fw1 got going.
Conlog
Dave, Jim, Patrick
After the autoburt .req file was regenerated, Patrick rescan the conlog channels. On restart he found that LAB DUST channels were missing. Jim tracked it down to a crashed IOC and restarted it.
SVN code cleanup
Dave:
I ran my ''check_h1_files_svn_status'' and committed all outstanding modifications. This resulted in updates for: front end mdl file (h1psliss), many filter files, many SAFE SDF snap files, many OBSERVE SDF snap files, many Guardian python scripts.
Partial filter loads full loaded
Dave:
I performed a full filter load on the two models which were running with possible partial loads: h1psliss, h1susitmy
Under WP #5710
I applied another modification on the PSL ISS front end model as planned in alog 25316.
I tested the automatic engagement and adjusted the parameters so that it locks smoothly. The IMC_LOCK guardian code is updated to incorporate this new automation and has been tested multiple times.
I have not tested the engagement as part of the full locking sequence.
[The change in the h1psliss model]
As shown in the above attached screenshot, I added two functions last time (see alog 25316). Today, I have refined the automatic reference adjuster a bit by placing signal conditioning filters and etc. The above screenshot shows the final configuration I have implemented.
The concept is that when the second loop is open we set the reference coarsely using the manual slider (i.e. REF_SIGNAL_ANA). Then a fine tuning is done by feeding SECONDLOOP_SIGNAL (shown as a green tag) to the reference point. This will let the second loop operating point close enough to close the loop. Once the loop is closed, one can then optimize the diffraction power at the AOM by feeding the diffracted power back to the reference of the second loop. This choice can be done by a mtrix.
[The screen]
The latest screen now looks like this.
Regenerated NEG pump with the aid of an aux cart.
Valved out NEG from Y-End, then started regeneration of NEG pump ~18:30 utc.
2 hours later heating was complete, waited for temperature to drop down, valved in NEG pump to Y-End volume at 23:48 utc @ 34 deg C.
Dewhitening filters for LVESDAMON were made and coefficients were loaded to H1SUSAUXEX and H1SUSAUXEY.
For the moment it's a simple zpk([40;40;965;965],[2;2],1,"n").
Today I attempted to find the short in the HV cable for the X2-8 BT ion pump (before proceeding with cutting off the extra cable length-unrelated task) as follows: I cut off the connector at the controller end and left the pump end connected to the pump. I then connected the "Megger" (5000VDC @ 12mA) to the controller end of the cable and demonstrated that the short was still present. I visually inspected the 100'+ excess cable for any obvious nicks or cuts (I did note one feature that looked like a clean incision or sharp slice). I disconnected the pump end connector from the pump and let it hang in space -> No change. Cable still shorted. Finally, I cut-off the connector at the pump end and separated the conductors -> No change. Cable short still present. Tomorrow I'll double check/inspect the run of cable that is in the HV raceway in the VEA and then proceed with cutting of the 100' extra cable length which will eliminate the incision "feature" previously noted and which I was going to do anyway prior to the appearance of the short.
Modified the ETM LVLN ESD Drivers (ECR E1500341): 1. Changed C32 in the monitoring amplifier from 1uF to 10nF to put the pole frequency at 4.244kHz 2. Changed C36 in the summing node from 1uF to 0.047uF to increase the dynamic range EY Unit SN S1500066 EX Unit SN S1500073 Replaced the ESD AI chassis in SUS-C1 (EX/EY). New AI Chassis contains the PI Band Pass Filter D1500177. Cable H1:SUS_ESD-05 had to be re-terminated with a DB9 male connector to accommodate new AI front panel interface. EY Old D1100815 Unit SN S1103824 EY New D1500177 Unit SN S1500300 EX Old D1100815 Unit SN S1103820 EX New D1500177 Unit SN S1500299
Evan, Keita
The above means that the actuation function changed due to C36 change.
That capacitor, together with R50 and R51, used to provide zpk([152], [3250]), but now this should be zpk([3225], [67725]).
Correction: used to provide zpk([3250], [152]), but now this should be zpk([67725], [3225]).
New digital compensation filter was made and the coefficients loaded to SUSETMX and SUSETMY.
Old compensation filter in L3 ESDOUTF antiAcq: zpk([152], [3250], 1, "n").
New one: zpk([3225], [], 1, "n"). The 67kHz pole zero in analog is ignored.
Attached is the transfer function from L3 ESDOUTF_LL_IN1 etc. to the LVESDAMON_LL_OUT_DQ etc.
LVESDAMON channels are with correct-ish dewhites mentioned in alog 25471, so they are supposed to be transparent for f<1kHz.
If the new antiAcq filters in the actuation are good, the measured transfer function should be flat, and indeed all TFs measured look flat for 1<1kHz.
The analog and digital filters seem to be the same? I guess the analog TF changed from zpk([3250], [152]) to zpk([67725], [3225]).
This should give a non-red state now for the VE unless something else is problematic so please respond to vacuum alarms.
The charge measurements for ETMX went well, but not great for ETMY. The measurement will loop through 5 times and we had almost finished the 2nd time when the SEI WD tripped and forced me to exit the running script, but I am not sure how good the data will be before the trip anyway. There wasn't enough time to run them again, so I tried to put back all of the settings differences, but I'm not 100% I did that correctly. I took screenshots, trended channels, and tried to mirror restored settings seen from the ending of the ETMX script. Hopefully it is all back where it should be.
I will post the plots in a few minutes from another computer.
(All Times UTC)
Turned out to be a fairly long Maintenance Day, but hope to be locking soon. Maintenance Activities ended just before 0:00UTC (4pmPST).
We decided to forego the ETMy Charge measurement this week, and Nutsinee will proceed with locking H1.
WP 5728, Dave, Jim, Carlos Install newer "Fast" computers for h1susex, h1susey. These are the computers which were briefly installed in July of 2015 and were removed because they caused timing, ADC, and IPC errors. The BIOS parameters for the computers are set to the optimal default with the exception of hyperthreading is turned off, the computers are set to stay off on power loss, the CPU power profile is set to maximum performance, and the boot order is changed to only look for DVD or PXE boot. Initially, a much more restrictive set of parameters was tried, which disabled nearly all of the features settable from the BIOS, but it was found the models wouldn't run. We have done nothing to correct the timing, ADC, and IPC errors which were observed in July. We are monitoring the error rate so we have a good baseline for possible fixes in the future.
Updated channel list for h1conlog1-master. Channel changes attached.
The slow controls SDF IOCs have been updated to a newer release. It is running RCG from trunk at revision 4114.
The SDF IOCs are configured to load straight to the OBSERVE.snap file (skipping the safe.snap which we do not use).
Changes:
With the precision changes I have accepted all numeric differences smaller than 10^-16. These should not need to be accepted again until there is an actual change.
I ran BruCo on half a hour of data starting at GPS 1138960800. The full report is available here:
https://ldas-jobs.ligo.caltech.edu/~gabriele.vajente/bruco_1138960800/
Some interesting things:
Related: https://alog.ligo-wa.caltech.edu/aLOG/index.php?callRep=25449
This morning Richard swapped the whitening chassis with the correct variant. Whitening seems to be working for all quadrants of 90MHz AS_A and AS_B.
WP 5727 Yesterday Daniel and I changed h1ecatc1 PLC2 to replace IscRfCSpareAmp[1] of IscRfCSpareAmp[1..5] with IscRfCAsAmp90m. This morning I rescanned PLC2 in the system manager and added the links for IscRfCAsAmp90mIn and the lost links for the remaining IscRfCSpareAmpIn (now IscRfCSpareAmpIn[1..4]). IscRfCAsAmp90mIn.OutputMon linked to Corner Chassis 6 L1 Channel 4 IscRfCAsAmp90mIn.PowerOk linked to Corner Chassis 6 L3 Channel 4 IscRfCSpareAmpIn[1..4].OutputMon linked to Corner Chassis 6 L2 Channel 1..4 IscRfCSpareAmpIn[1..4].PowerOk linked to Corner Chassis 6 L4 Channel 1..4 The wiring diagram also needs to be updated to show that IscRfCTcsAom40mIn.OutputMon is linked to Corner Chassis 6 L1 Channel 3 and IscRfCTcsAom40mIn.PowerOk is linked to Corner Chassis 6 L3 Channel 3. Since everything was restarted I burtrestored PLC1, PLC2 and PLC3 to 6:10 this morning local time.
I updated the channel link list E1201049-v9.
I reset the PSL 35W FE power watchdog at 17:55 UTC (9:55 PST).
BS had saturation count of 712 & was RESET.
Per FAMIS request 4391, attached are oplev 7-day trends.
Everything here looks normal, nothing out of the ordinary. All active oplevs are within acceptable operating ranges.