New PI code
Kiwamu, Dave:
Kiwamu installed new h1omcpi and h1susprocpi code. DAQ was restarted
New NGN code for CBRS
Krishna, Chris, Jenne, Jim, Dave:
Jenne added a CBRS component (Compact BRS) to h1ngn. We discovered a conflict with DAC channel allocation, so for now the new code is not driving DACs. Tomorrow we will swap DAC channels between h1pemcs and h1ngn. DAQ was restarted.
Timing FPGA upgrade at EX
Daniel, Ansel, Jim, Dave:
Daniel installed the latest FPGA code on the fanout at EX. This required a power cycle of the front end computers. We did not power cycle irig-b, comparator or RF amps.
Investigation into strange signal behaviour seen during timing upgrades
Daniel, Hugh, Jim W, Ansel, Patrick, Jim, Dave:
A reminder, during the timing system upgrades on 19th May 2015 and 19th July 2016 we saw surprising behaviour of HEPI fluid pressures (they appeared to quickly increase) and the vacuum cold cathode gauges (they saw a slight increase over a long time base of several hours).
To investigate this further, we powered down the EY front ends and went through the same power sequence as we followed during the 19 July upgrade. We quickly found that when the IO Chassis for h1seiey was powered up (with the h1seiey computer powered down) the hepi pump controller reported an increase of the hydraulic fluid pressure from 80 to 107 (as was seen May 2015 and July 2016). We powered down the IOC chassis and all AA and AI chassis. We powered up the IO Chassis and then, one at a time, each AA and AI chassis. When we powered up the AI chassis for the first DAC, the problem reappeared. This DAC is used by h1hpietmy model to drive the HEPI Valve Controller chassis. At this point it was looking more like a real fluid pressure increase. This was confirmed at EX during the timing upgrade by viewing the mechanical fluid pressure gauge attached to the pump station, it reported the increase in pressure from 80 to 115.
The sequence is (from a fully powered down state): the IO Chassis is powered up, the fluid pressure increases over several seconds from 80 to 105. When the h1seie[y,x] computer is powered up, the pressure drops immediately at the time the power button is pressed (before the OS is loaded and any models are started). Specifically, when the One-Stop expansion card in the IO Chassis is activated the fluid pressure drops.
To prevent any run-away issues, the hepi pump controller at both end stations were set into manual mode during these tests and returned to auto PID control later.
On the test stand, we found that a 16bit DAC card, when the IO Chassis is powered up, outputs 9.97V. It stays at this voltage until the front end computer is powered up, at which time its output voltage drops to -0.1V. Hugh thinks that this large voltage drive to the valve controller could be causing a back pressure to the pump station. One way to prevent this would be to power down the AI Chassis prior to the frontend/IO-Chassis power cycle, and only power the AI back on when all the code is running and the DAC channels are being fully controlled.
We saw evidence of different power up behaviour with 18bit DAC cards, and some variation depending upon which PCI slot they are in. Investigation is continuing.
We suspect the cold cathode signal change is associated with chamber cooling when the ISI and SUS drives are deactivated for a long period of time (many hours). We see daily variations of CC signals which match the room temperature's daily variation.
Bottom line, these signals appear to be real and not caused by software or electrical issues linking dissimilar systems in unknown ways. There is no connection to the reprogramming of the timing system, this activity just provided the power sequence which allowed these issues to emerge.
1605 hrs local -> Dew point measured at HAM6 check valve exhaust <-31C. 1630 hrs. local -> Kobelco running colder than normal (also noticeably quieter?) As found "LOADED" temps (manual coolant valve at the exit of the 2nd stage was 1/4 turn open) 1st STAGE DISCHARGE TEMPERATURE 276F 2ND AIR SUCTION 79F 2ND AIR DISCHARGE 257F LUBE OIL 91F 10 minutes after closing closing manual coolant valve 1/8 turn "LOADED" temps 1st STAGE DISCHARGE TEMPERATURE 288F 2ND AIR SUCTION 81F 2ND AIR DISCHARGE 280F LUBE OIL 108F Corner Station chilled water set point @ 44F (as per Bubba - normal is 48F) Compressor UNLOAD-LOAD cycle time = 1 minute 17 seconds Drying Tower switching cycle time = 4 minutes 56 seconds
Forgot to mention -> Compressor loads when receiver tank pressure is @ 80 psi and unloads @ 120 psi
UNLOAD-LOAD cycle time is time between beginning of UNLOADs
Mystery solved! Bubba double checked and discovered that the current Corner Station chilled water is 38F! This now explains everything. I closed the manual valve mentioned above a "smidgen" more this morning and the LUBE OIL TEMP is now ~120F. The other (air) temperatures are still too low but I believe this isn't a threat to the machine but more likely makes the water removal that occurs in the compressor less effective. The measured dew point down stream of the external drying towers is still good regardless.
[Betsy Koji Keita Corey Hugh]
For the OMC removal, we did the following actions along with the procedure E1600164
- Hugh removed the some of the table ballasts to have only the first layer
- Locked the top vertical blades.
- Locked the upeer mass with the screws
- Locked the upper mass vertical blades
- Removed a part of the black glass shroud
- Unhooked the suspension wires from the OMC breadboard
- Removed the breadboard from the OMC SUS frame
L4C seismometers number 23 and 25 have been removed due to their proximity to the during-vent storage location for the HAM6 doors.
I have not removed #21, but if it is at all in the way, please feel free to do so. It is a passive instrument, so all that is needed is unplugging the DB9 connection that is within ~12 inches of the sensor, and then lever the sensor off the epoxy. The sensor and cooler can be stored next to any other out-of-the-way sensor, and I will re-install them once the vent is complete.
A supplymental observation in addition to LHO ALOG 28820 was that we found (at least) one more epoxy delamination in the decomissioned OMC. The sympton is similar to the one found in the 3rd OMC last week ([LHO ALOG 28714])
Looks like a problem started around 9am. Could not find anything glaring at the chamber but a CPS rack may have been disturbed with the activity nearby. HEPI is ISoalted and the ISI is DAMPED. Don't see anything unexpected in the Spectrum. We'll start with CPS rack cycling and then more explicit wiring chasing.
Attached is the in-the-loop frequency noise spectrum with the common gain at 16 and fast gain at 9. With these settings, UGF = 418 kHz, with a phase margin of ~48 deg. If the common gain is pushed higher, the UGS is ~860 kHZ with a phase margin of ~15 deg. However at this point the loop is very touchy. Jason/Peter
Summary:
High power beam path in HAM6 was carefully inspected using flashlights and so-called green lanterns. Some of the OMC shroud panels were removed and the OMC was pulled out of the chamber.
Some damages and contaminations were found.
HAM6 didn't look particularly clean, there were some particulates on OM mirrors for example but they were all far from the center.
Note that all of the above except 2. could have been prevented if the fast shutter was working.
No other problem was identified.
Corey will post some of the pictures.
Fixing things:
We will proceed to replace the OMC as planned.
OMC TRANS steering mirror was removed from the Siskiyou mirror holder, particulate was removed on the table using Top Gun, and First Contact was applied. Later the mirror will be installed back in the mirror holder. This way we don't need to worry about the alignment.
V-dump glasses were replaced on the spot.
We might replace the damaged shroud panel if we have a new one and if the cleaning can be done fast, otherwise we might just leave it there.
Other things to note:
We also visually inspected low-power beam path and found nothing suspicious.
AS for the OMC mirror damage, speculation of me and Koji is that we had a particulate contamination of some kind when the large power hit the OMC as the fast shutter was not working.
Using the Fieldfox N9912A we inspected both ends of the Y2-8 HV Cable. We used a Velocity of Propagation (VP) of 0.59 and turned on Low Pass (LP) filter for our testing. We did not see anything with the LP filter off. From Y2-8, we measured 256.5m to the end of the cable at the Y end station, no aberrations on the trace. From the Y end station, we measured 256.5m to the end of the cable at Y2-8 but we also noted a blip at approximately 17.9m from the Y end station connector. This blip did not exist on the scan from the Y2-8 connector. I believe there may be a kink in the cable at the junction box leading out of the Y end station. We also spotted some wildlife trying to get into the clean room area. Gerardo M., Marc P.
J. Kissel, K. Venkateswara, M. Ross, J. Driggers We've built the control infrastructure for the newly installed Compact BRS. This stuff now lives in the h1ngn (the Newtonian Noise model on the h1oaf computer) front-end model. The control scheme follows the diagram as described in T1600325. Screen shots of the various models attached. Also, because the DAC_0 card on the OAF machine is shared with the (we think the H1 Only) AUDIO infrastructure in the h1oaf model, we've also modified that model aesthetically to indicate that this DAC card is shared. The models have been successfully compiled and committed to the userapps svn repo here, /opt/rtcds/userapps/release/isc/h1/models/ h1ngn.mdl h1oaf.mdl
Measured 167.3W at the output of the high power oscillator. 162.5W at the input to the pre-modecleaner with the ISS off. The diffracted power versus offset slider position was measured. Combined with the power out of the laser, this gave the percentage diffracted power. The free-running peak-to-peak power fluctuations measured by the ISS photodiodes as reported by a trend of the PDB EPICS was 0.2Vpp with a mean value of 8.3V. So the 0.1Vp fluctuations represent ~1.2%. Allowing for some margin, we wish to diffract 2%. 2% of 162.5W is 3.25W. With the digital support loop on, this corresponds to an offset slider of ~2.5. The polynomial fit calculating the percentage diffraction was: 220.159x**2 - 97.169x + 11.293. This was changed to match the measurements done earlier to day to: 97.6751x**2 - 63.548x + 11.458. The MEDM screen now correctly reports the amount of light diffracted by the ISS AOM. Jason/Peter
Recently the datasheet for the RF amplifier inside the AOM driver was looked at. It suggested a continuous maximum RF input of 5dBm for absolutely no damage to one of the chips used. Thus a 5dB attenuator was added, bringing the total attenuation to 6dB.
Attached are plots of the ISS transfer function and the relative power noise for the corresponding gain settings. The spectra is of the out-of-loop photodiode.
Bubba, Chandra, Gerardo, Kyle HAM 6 took one hour to vent to atmosphere, The north & south doors were removed before lunch. GV 5 & 7 were soft closed prior to vent. Attached is vent pressure plot from today and also from April, for comparison. NOTE: in addition to powering off all HV in HAM 6, we also powered down the ITM HV.
Y2-8 ion pump not operational
[Betsy, Koji] @Bonding lab
The OMC is sit in the bonding lab for curing of the epoxy. Otherwise, it is ready to be moved to the chamber side.
- 3rd OMC optics cleaning
Attachment 1:
We applied FirstContact cleaning of the optical surfaces. The optical side of the breadboard was wiped with IPA-soaked cloth (without touchting the optics).
The FCs will be left until the OMC is brought to the chamber side.
- PD replacement
Attachment 2:
The original OMC DCPDs were replaced with the new high QE DCPDs. The PDs at the BS transmission and reflection sides are from the PD cage A slot 3 (A3) and A4, respectively. They correspodns to the PD serials B1-01 and B1-16. The final testing at Caltech showed the QEs of 0.980 and 0.981 respectively.
The FirstContact seals are attached on the PD apertures to prevent particulates come into the PD surfaces.
- Mounting blacket bonding reinforcement
Attachment 3:
We added a glass prism to reinforce the bonding of one of the mounting blackets on the top (suspension) side of the OMC breadboard. A small amount of glue residue form the existing bond was removed by a razor blade to clean the place for the new prism. As small amount of glue as possible was applied to have round glue foot print, particularly on the glass-glass joint. The glue on the Invar-glass joint looks round. The glue on the glass-glass joint looks square inspite of our effort (We really don't think it is an issue). A steel block was added to hold the prism until the epoxy is cured.
The test data for B1-01 (DCPDA, that in transmission of the OMC TRANS BS) and B1-16 (DCPDB, that in reflection of the OMC TRANS BS) can be found here in the 40m eLOG 255: https://nodus.ligo.caltech.edu:8081/OMC_Lab/255 The serial numbers are indicated in the OMC DCPD wiring chain, D1300502
I was bit again by buzilla bug 1013. The changes I made yesterday (alog28775 =) did not take effect since the change was in common code and I forgot to restart the nodes. I ended up restarting all of the SEI configuration nodes, but really only had to do all of the BSCs. No harm, everything came back nicely.
Illuminators for ETMx and ETMy have been turned off for PCal camera work.
Oplev |
Data segment (1Hr long) |
Sum (counts) | Glitchy? | Remarks |
---|---|---|---|---|
ETMX | 01-08-2016 13:00 UTC | 41000 | No | Seems good. One more stage of whitening may also be ok. |
ETMY | 01-08-2016 10:00 UTC | 28000 | No | All Good. Could copy these settings to ETMX. |
ITMX | 01-08-2016 10:00 UTC | 33000 | No | The periodic pulses seen in the SUM is probably due to the temperature controller in the diode laser having railed or become unstable. |
ITMY | 01-08-2016 10:00 UTC | 30000 | No | Does not seem to have whitening filters since we can see in the SUM spectra that it is being limited by ADC noise beyond 5Hz. |
I dont think the ITMX laser is mode hopping. I did not notice any step like changes in the SUM signal, though it is hard to tell due to the periodic pulses.
The latter can be fixed by tweaking the PID setttings of temperature controller and its set point, I think.
Both ITM oplevs have large resonances at ~60 and ~90 Hz. These are much attenuated in ETMs. The mounting of ITM oplevs could perhaps be examined to see if there is something out of place.
At the laser wavelength 632 nm, Si QPD efficiency is ~0.35 A/W; Transimpedance=10kOhm according to D1100290 ; Differential Ouput gives x2 ; 16bit-ADC gives (2^16)/20 counts/V;
Using these factors : 1mW of incident light on the QPD would give us ~23k counts. Since ETMX, ITMX and ITMY oplevs have 18dB, 21 dB and 15 dB whitening gain,
the incident power on these QPDs is
ETMX | ETMY | ITMX | ITMY |
0.32 mW | 1.22 mW | 0.13 mW | 0.23 mW |
And 1/4 of that per quadrant !!
Note: I'm not sure over what time duration Suresh made his analysis, but Jason has recently tweaked the gain and whitening settings on the ITMX, ITMY, and ETMX optical levers last week Tuesday (July 28). See LHO aLOG 28645. Likely, Suresh's suggestions to improve the whitening were based on data prior to this change, and the setting are now acceptable. See Keita's post-change assessment in LHO aLOG 28661.
The data segment analysed is from 1st Aug 2016 at about 10:00 UTC.
And yes ... I had initially analysed data from before the change and had to redo the job after I saw the post by Jason on the 26th Aug (and also the request from Keita). This analysis is after all those changes.
And just in case I goofed up something, I rechecked the time stamp on the data plotted in the attached pdf file. All data is from 1st Aug.
For the NGN part of today's notes:
I had forgotten that h1pemcs was "using" the DAC channels that we wanted for the new compact BRS in the LVEA, even though Robert doesn't need constant access to these channels, which is why we had troubles compiling the h1ngn (NN) model today when we tried to utilize those channels.
After confirming with Robert, changes have been made to h1pemcs and h1ngn so that the cBRS can use the first 4 channels (numbers 0-3) of DAC0 on the OAF computer. These models have been compiled, and are ready for restarts tomorrow. Unfortunately, since this will move some DAC channels from one model to another, the entire OAF IOP will need to be restarted.