LVEA: Laser Hazard Observation Bit: Undisturbed 07:45 Cleared TIM, ADC, & IPC errors at both end stations 08:00 IFO Locked at 24.3W and 65Mpc 08:20 Hanford Fire department on site 08:35 IFO LockLoss – Commissioning activities 09:00 Jason & Peter – In the PSL 09:30 Sudarshan – Going into LVEA to remove ISS Outer Loop Servo 09:36 Bubba – Moving items in and out of LVEA Hi-Bay 10:12 Kyle & Gerardo – Going to End-Y beam tube to work on new Ion pump 10:44 Sudarshan & Kiwamu – Going to LVEA to install repaired ISS Outer Loop Servo 11:00 Sudarshan & Kiwamu – Out of the LVEA 11:05 Sudarshan & Travis – Going to End-X for PCAL clipping investigation 11:03 Vinnie – Going to Mid-X 11:05 Kyle & Gerardo – Going into End-Y to recover an Ion pump 11:30 Rick – Going to End-X for PCAL work 11:45 Kyle & Gerardo – Back from End-Y 11:50 Jeff – Deliver tools to PSL Enclosure 12:15 Jason & peter – Out of the PSL enclosure 12:18 Rick, Sudarshan, & Travis – Out of End-X 12:24 Kiwamu – Going into LVEA to take TF on ISS Outer Loop Servo 12:36 Jim, TJ, & Cheryl – Going into the LVEA to reset the NPRO RRO 14:13 Kiwamu & Sudarshan – Out of LVEA
Sudarshan, Kiwamu,
This is just a brief update of what we did with the ISS during the maintenance window today.
Yesterday, Sudarshan and Matt came up with another idea of a quick hack in order to increase the gain margin of the ISS 2nd loop. The idea is to modify the unused filter path of the ISS 2nd loop servo box to obtain more poles and zeros without making a change in the main servo filers. We pulled out the servo box out of the PSL rack, soldered some components on the board in the EE shop and put the box back to the rack. The above plot is the open loop transfer funciton of the 2nd loop after the modification. As shown, the slope of the open loop is now steeper than it used to be, indicating a more gain margin. We will post more details about the circuit modeling, today's mofication and noise model.
The open loop measurement was done with a PSL power of 2.4 W. PD1-4 were used as in-loop sensors, the second loop gain was at the maximum of 40 dB, the boost and integrator engaged, and the additional gain was also engaged. The raw data is attached -- the first file is the magnitude in dB and the second for the phase in degrees.
J. Oberling, P. King
Power Budget
We measured the power at several points around the PSL:
There are 2 PDs that need to be recalibrated: PD_AMP (which monitors the power out of the frontend) and PMC_TRANS (which monitors the power transmitted by the PMC). Due to time constraints and interference with IFO recovery and commissioning activities (have to turn the ISS off to recalibrate PMC_TRANS), the recalibration has not been performed yet. Will be done ASAP.
HPO Contamination Check and Green Light Inspection
Since we had to take the lid off the HPO box to measure PD_AMP, we also inspected the inside of the box for contamination. This is something that had been on our radar to do next time we opened the HPO box. Matt had noticed back in April (see LLO alog 17972) that the LLO HPO box had a large amount of contamination, specifically underneath the holes where the lid is screwed down to the box itself. We also noticed some particulate underneath the screw holes, but not to the extent that was seen at LLO. Peter took pictures and will post them. We went ahead and wiped the screw holes, lid holes, and screws themselves with wet IPA wipes.
We also tood one of the green flashlights and looked at a few optics to get a feel for general PSL contamination. All and all it wasn't too bad, once again nothing like was seen at LLO. Peter took some pictures of mirror M26 (in the FSS path), and the top and bottom mirrors from the IO periscope. We used a couple small puffs of air to see if we could dislodge some of the dust on M26; this was successful (see pictures). We also looked at the PMC mirrors with the green light; these looked clean, we couldn't see any dust.
Modified TTFSS Box Round 2
Peter had made some more modifications to one of the spare TTFSS boxes (same one we used last week, see LHO alog 19872), so we installed it again. Unlike last time, the FSS would not lock with the modified box installed. It would seem like it was about to lock, we could see flashes of the 00 mode as the NPRO frequency was changed, but it wouldn't lock onto it. At this time we aren't sure as to why. We reinstalled the original TTFSS box and the FSS locked without issue. We measured the UGF at 2 different Common Gain settings (Fast Gain was constant at 22.2 dB):
Peter has the data for these measurements and will post it as a comment. We restored the Common and Fast gain settings to their original values of 20.7 dB and 22.2 dB, respectively. Will investigate why the modified TTFSS box was not working.
The measured TTFSS open loop transfer function with the common gain set to it's "default" of 20.7 dB and when set to maximise the unity gain frequency. (ttfssoltf.png) The gain sliders were left how we found them. Namely common gain 20.7 dB and fast gain 22.2 dB. In the default condition, the unity gain frequency was around 200 kHz. With fast gain at 27 dB, the unity gain was in excess of 600 kHz. One of the mirrors in the beam path after the frequency shifting AOM, before and after using compressed air. The bottom and top mirrors of the IO periscope. For the top mirror, most of the dust was on the back surface - not surprisingly.
Cheryl, JIm W, TJ
When Jim started locking, I noticed that SYS_DIAG was reporting the PSL Noise Eater NPRO RRO was out of range. To fix this we went out to the floor and flipped a switch just like in alog19876. This knocked the ISS around but just adjusted the ISS Ref Signal to -2.11.
The directions to fix this are being added into the SYS_DIAG wiki located Here.
The full procedure can be found in the RCG release notes https://dcc.ligo.org/LIGO-T1400570 section 5.7.3, I'll detail what I did yesterday to rebuild and restart the front end models here.
as user controls on h1build:
backup and empty the H1.ipc file in /opt/rtcds/lho/h1/chans/ipc
in /opt/rtcds/lho/h1/release/: perform two run throughs of the make, the first populates the H1.ipc file, the second is a full make and should have no errors. No build errors were seen, so the installWorld was performed.
unalias make
make -i World
make -i World | tee /tmp/make_world_28jul2015_11_15
make installWorld | tee /tmp/install_world_28jul2015_11_39
Then shutdown all the front end models, as controls on h1boot
grep "^h" /diskless/root/etc/rtsystab |awk '{print $1}' > /tmp/front_end_list.txt
for i in 'cat /tmp/front_end_list.txt';do echo $i;ssh $i /etc/kill_models.sh;done
I then editied /tmp/front_end_list.txt and removed h1pemmx and h1pemmy (they have to be manually started). We also removed h1sush2a and h1sush34 since they were showing DAC drive errors on previous restarts.
Then start all models
for i in 'cat /tmp/front_end_list.txt';do echo $i;ssh $i /etc/start_models.sh;done
and manually started h1pemmx, h1pemmy, h1sush2a and h1sush34 (did not see a repeat of the DAC drive issues)
All systems started correctly and there were no IPC errors.
Yesterday I modified the end station cal models to add the FPGA duotone signal to the commissioning frame. When analysing the data, I noticed that the non-filtered loop back duotone signal has a low amplitude. In the attachment below, the red plot is the FPGA duotone signal zoomed at the zero crossing point. This signal has a maximum amplitude of 4000 counts over the whole second. The green plot is the IOP loop-backed duotone, again zoomed in. This plot has a max amplitude of 250 counts, so at the zero crossing the bit noise is apparent. The reduction of the amplitude is due to the difference in the voltage range and bit range of the DAC compared to the ADC (ADC is 16bit, +/- 20V, DAC is 18bit +/- 10V). There is a missing factor of 2, perhaps the differential signal is converted to single ended in the loop-back circuit?
Rick thinks we dont need to do anything for O1 to boost the non-filt amplitude. One solution would be to use additional DAC and ADC channel and perform the loop-back external to the IO Chassis, sending the signal out through a gain to compensate for the amplitude loss.
Ryan F., Duncan M. Today we have updated the EPICs settings for ODC channel configurations that affect the following channels: ODC-MASTER ODC-X ODC-Y These changes follow the changes described in the ECR 1500289. We accepted the changed in the SDF system, and svn committed the change to the safe.snap for ODC-MASTER. The h1iscex and ey _safe.snap files were backed up prior to the modifications, modified, but not committed. We also changed the ODC-MASTER EPICs to complete the configuration for the INJection bit reporting.
I have also fixed some problems with uninitialised EPICS settings for the h1calcs model. These have been initialised and accepted in the safe.snap file (SDF) but not committed to cds_user_apps. We did back up the old safe.snap with a date string.
These EPICS settings, and those mentioned by Ryan above, have no affect on interferometer controls, and are only readbacks for the ODC.
I have completed ODC EPICS mofications to more models, including all HPI, ISI, SUS models, and h1omc. These changes only affect ODC EPICS strings, and will have no effect on interferometer operations. All changes have been captured in the SDF safe state.
The Tuesday maintenance day was a bit rough but successful. CDS will be working on PEM wiring DtEng commissioning work: PSL – TTFSS testing and commissioning PD power budget ISS Outer Loop electronics PCAL – End-X clipping investigation ALS – Diff calibration measurements PEM – Cabling work on the roof Railed sensor at Mid-X Safety Meeting: John asked the GSA cars be kept available especially during the Tuesday maintenance window for onsite work. All car use must be entered in the log, and the keys returned to the key box when finished with the car. It was stressed NOT to eat or leave food and drinks in the cars, as this attracts rodents to the cars, with rather unpleasant and expensive repairs.
h1susey BIOS change
Mike Thomas, Dave, Jim:
Mike took detailed photographs of l1susey's BIOS screens. After reviewing them and those of the slower front end computers Jim had some changes to try on h1susey. These were installed and it looks like we have not seen an EY IOP glitch since then.
RCG 2.9.6 upgrade
Rolf, Jeff, Betsy, Jim, Dave:
All front ends were upgraded to RCG 2.9.6. The IPC file was created from scratch. All front end models were restarted. Detailed alog on its way.
PCAL duotone channel to DAQ
Rick, Dave:
the models h1calex and h1caley were modified to write the FPGA duotone channel to the DAQ commissioning frame.
Cosmic Ray Channel to DAQ
Richard, Dave:
h1pemcs was modified to write the TRIG channel from the cosmic ray detector to the science frame.
OBSERVATORY MODE install
Vern, Dave, Stefan:
h1odcmaster was modified to add the H1:ODC-OBSERVATORY_MODE EPICS channel. The associated MEDM screen was added to the SITEMAP under the OPS tag.
ISC IOP model settings
Dave:
The PCAL models at the end stations require the DUOTONE loop back path to allways be engaged. I modified the safe.snap for the models h1iopisce[x,y] to ensure this is so. SDF will altert if they are turned off.
New SUS PI models and DAQ configuration:
Matt, Sheila, Joe B, Dave:
Installed Monday, DAQ configued Tuesday. New h1susetm[x,y]pi models were started in the end station SUS front ends. To avoid the "DAQ too small" error two channels were added at full rate but filled with zeros to minimize impact on commissioning frame size. DEMOD channels were added to DAQ at 4kHz rate.
Other New user models
Hugh, Jeff, Kiwamu, Daniel:
Many user model changes other than those detailed above went in: New ISI models from HAMS and BSC, new ASCIMC model, some SUS models showed a DAQ change (investigation needed), PSL PMC finally applied its 32kHz to 16kHz DAQ channel transition.
DAQ Restart:
Jim, Dave:
The DAQ was restarted to support the above changes. Both frame writers continue to be completely stable.
Restart Log:
The models restart log is attached, due to the RCG upgrade it is lengthy.
Occasionally, we ring up the BS butterfly mode (2.5kHz), the BS violins (300 Hz) and something at 41Hz. What is that thing at 41Hz (which is always present, though usually not so big), and how are we exciting the other BS modes?
Is the line at 75Hz from the ring heater fans, as in LLO?
The highest Roll mode of the HSTS is modeled to be at 40.3 [Hz] -- I suspect SR2 or SRM as you drive it for lock acquisition, or they could have been rung up during the low-frequency sawtooth waves from the earth we saw last night.
At 75 Hz there is coherence with magnetometers and microphones at the Y end. So I guess it's the ring heater.
We noticed that we can't really turn off the ETMX ESD. We aren't sure why.
This causes the guardian to hang, in the state LOWNOISE_ESD_ETMY because it checks that the ESD is off. This can be worked around using manual, but the operator has to be carefull not to skip steps if we force the gaurdian to move on.
Not sure why this is. I was able to cycle the power to the High Voltage ESD Amp multiple times from the Binary MEDM screen. I was also able to turn each channel On and Off individually from this screen as verified at the output of the unit not just the read-back.
We need to update the channel that is used to turn the ESD on and off. I verified again this morning that the Binary switch does in fact work.
I second what Richard says -- the Beckhoff binary signal that *was* used to control the HV driver on ETMX has now been replaced by similar "fast front-end" binary I/O that's at ETMY. So, for each quadrant, to turn off the high voltage, set the EPICs records H1:SUS-ETMX_BIO_L3_${QUADRANT}_VOLTAGE_SW to 0 and H1:SUS-ETMX_BIO_L3_${QUADRANT}_HVDISCONNECT_SW to 0. This has been true since Richard cabled up the LVLN driver yesterday afternoon (see LHO aLOG 19994).
[Jenne, StefanB, Cheryl]
We were having some trouble getting the IMC WFS to converge (WFS came on, looked okay for a while, then started dragging the MC transmitted power down), so we implemented and tested the new global burt restore state, accessible from the ISC_LOCK guardian.
In the end, the specific problem with the IMC WFS was that the ISC input filters on the M3 stage of the MC mirrors (definitely MC1, maybe others) had gain of zero, so the WFS signals weren't getting through to the optics' outputs.
The solution we created, which should solve all kinds of problems, is a guardian state that does a global burt restore to a recent set of autoburt snapshots. This state has a date and time hard-coded in the guardian script, although it is easily change-able if we find a more preferable time. Currently, it is restoring to the autoburts of 28 July 2015, 07:10. To select this state, you need to go to "manual", since there are no edges to get there from any other state. When it has finished the restores, it will immediately go to and run the Down state.
The list of snapshots that are restored is:
Perhaps though, we should just restore *every* snapshot that is captured by autoburt?
J. Kissel, J. Driggers Use this feature with caution, as we found yesterday evening during the rest of recovery (LHO aLOG 20011), BURT restoring to a single time with the IFO was at low-noise isn't necessarily the right time to BURT restore to, especially until we've caught all of the settings that have fallen between the guardian and SDF cracks.
I really don't like this as a solution to whatever problem you're trying to solve. Is there something wrong with the SDF system that is causing you to resort to such a sledgehammer approach?
For Maintenance this morning I needed to break the IFO lock, so recorded the steps used today.
- Besty, Jeff, TJ, Cheryl
The reason that requesting ISC_LOCK to down doesn't work anyore is because I made down not a goto state. This means that there is no path from NOMINAL LOW NOISE to DOWN. (We were having problems with unintentially going to DOWN.) So when you request DOWN, the guardian doesn't actually go there. To force it to do that, you can go to manual mode and select DOWN.
I understand more now, and was not aware that you had changed that on purpose.
No suggestion for a change, just reporting what I was asked to try and the results I got.
Per ECR E1500322, I've added the Wiener filter sensor correction path to the BSC and HAM ISI master models. Pretty simple change (even if it took ~3 hours to do), as there were already terminated STS paths on both models that just needed connection to a bus and a few filter blocks needed to be added to the Senscor paths. I've commit the changes to the SVN and checked to that HAM2 and ITMX would "make" on the new models. We should be able to install and use on all chambers tomorrow, the MEDMs should be pretty easy, too.
Here is the new and edited medms for this filter path addition.
Modified these medms again. Arnaud reminded me that of course the inputs to the WNR filter and the FIR & IIR filters are distinct. While the input is actually the same instrument they come from different paths through the STS2CART Matrix. So they could conceivably be scaled there but more importantly, they could be off at the matrix and one wouldn't know that as I had depicted it on the medm before (above.) I did not catch a before view but you can see that above. Here you can see the HAMISI chamber overview, the STS2CART matrix and the new HAM SENSCOR OVERVIEW and BSC Senscor overview as well.
Will commit and notify LLO to update.