as of 7:100UTC:
Summary: Tonight we are able to lock reliably and stay locked with 60 Mpc range.
The noise lump that is coherent with the PSL Bullseye sensor is back. We hadn't seen this since the vent, and I am wondering if it could be related to the PSL work today.
Notes for whomever next locks:
The guardian can be used as is to get to REDUCE_RF9_MODULATION_DEPTH. For now we need to manually skip SRM_ASC_HIGHPOWER, do NOISE_TUNNINGS as it is written in the guardian, and skip CLOSE_BEAM_DIVERTERS. The next things I would try are to explore a little more with POP spot position (usig PR3 spot move) and soft offsets to see if we can reduce the jitter lump, or improve the recycling gain. The 6dB increase in CARM (LSC-REFL_SERVO) IN1 gain is not in the guardian, it could be done manually about 20 minutes after locking. It might be necessary to run an initial alignment after all the alignment changes I have tried.
I have crashed 3 computers in a row tonight. Is this because of sometime that happened during maintence day?
Sheila, Jim
We had some issues with one of the PI modes tonight. Mode 26 rang up and took some careful wrangling to get damped back down. The first issue we found was that the BP and PLL frequency were both off, so we selected the 15009hz (FM1) and tuned the PLL frequency appropriately. However, the mode was still unresponsive to the normal phase tweaking and broke the first lock. When we got back up, we missed waiting at DC readout for the PI to settle back down, so we ended up having to damp 2 or 3 modes simultaneously. Fortunately everyone but mode 26 were easy customers. Mode 26 required careful tweaking of the phase and Sheila eventually increased the gain to 15000 from 4000. The mode was also kind of slow to respond so zooming in on the rms strip tool helped differentiating good from bad phase tweaks.
[John, Chandra]
Closed WP 7021
We tested CP4's LN2 vaporizer for leaks by first blanking the exit of vaporizer with 15 psig pressure relief valve (works) and gauge. Pressure held at around 10 psi with LN2 bottom draw valve fully open. Then we connected the new flow meter at outlet and let LN2 flow freely for ~30 minutes. We tried to increase the flow rate by increasing Dewar pressure but realize this can take days. The highest flow rate we saw on meter was 5,000 SCFH, but it mostly ran closer to 2,000 SCFH. PSI procedure calls for 5,300 SCFH for regenerating short CPs. No cracks found in CP4 vaporizer. No visual cracks in CP3 vaporizer (not pressure tested yet). CP 5,6 do have cracks in the aluminum at inlet and outlet. Bubba thinks he can fix the welds: FRS 8282.
CP4 regeneration line now has a new flow meter and is properly reassembled.
Since last week, we have been applying 40-60 psi of GN2 through rotameter to the bottom sensing line on CP4 in an effort to "erode" the blockage. It appears we have a passage, but the blockage is not completely gone. Today John and I played around at the pump and found the differential pressure transducer reads back 100% on CDS (Beckhoff limit). The Magnehelic had been overpressurized, and wasn't matching CDS, so I swapped it with a new one. When valved in, it slowly creeps up to ~ 30 in. of water (38" = 92% full), along with CDS read back, but there is a leak in the fitting. So we left it valved out. When we combine the two sensing lines, CDS reads 0%. When we isolate them, it reads 100%, but the reading slowly increases from 0-100% rather than an instantaneous jump - even with Mag valved out, so we think we have a partial pathway across the blockage yielding poor conductance.
We canceled the auto overfill today and set the LLCV to 20% open for about six hours (nominal is 41%) hoping to see it come on scale. At around 5pm local we raised it to 38% to sustain over night. We will watch it periodically and revisit tomorrow. We may try to apply up to 200 psi of pressure on sensing line (bypass rotameter) to fully clear the clog.
CP4 was last overfilled yesterday at 11:16 am local.
I fixed the leak at Mag. It now reads 47" which correlates to 114% full. CDS reads 100% full (Beckhoff limit).
Leaving Mag valved in overnight.
J. Oberling, P. King, E. Merilh
Today we swapped Diode Box 1 (DB1) for the PSL HPO with a spare shipped up from LLO. The swap was relatively painless, although we did have a delay in attempting to extract a rounded screw from the lid of the "new" diode box. Once that was taken care of the DB was installed in the PSL LDR. We then turned only DB1 on with 20A of current. Using an IR camera we confirmed that none of the fiber connections were overheating (all were <23 °C). The PSL was then restarted.
For the new DB, we used an operating current of 49.3 A and set the temperature setpoints for the individual diodes to 29 °C. The HPO came up with zero issues. We then rotated each fiber connecter in the DB to maximize the power output by the HPO (unscrew the connector just enough so the fiber can rotate, rotate to maximize power, re-tighten connector). Once this was done DB1 was closed and we slid it into place in the rack. The 35W FE was restarted and injection locking engaged; the PSL was left to warm up in this configuration for ~1 hour.
After the warm-up period, the pump diode operating current and temperatures were adjusted to incorporate the new DB. The temperatures of the diodes in DB2, DB3, & DB4 were unchanged; the temperatures of the diodes in DB1 were changed to 28 °C. The operating current of each of the 4 DBs is summarized below (a screenshot is attached for future reference):
The PSL is now outputting ~158 W, and the PMC is transmitting ~60 W. The PSL is now fully recovered from the DB swap and functioning as it was before work began. The SN of the old DB is OBS2-DB1, the SN of the new DB is OBS1-DB1.
This closes LHO WP 7019. Incidentally, this work also completes FAMIS 3653 (Weekly PSL Power Watchdog Reset) and FAMIS 8425 (PSL Weekly HPO Pump Diode Current Adjust).
TITLE: 06/06 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Planned Engineering
INCOMING OPERATOR: Jim
SHIFT SUMMARY:
Maintenance Day went until about 22:00UTC (3pmPDT) with biggest activity was the PSL going down for HPO Diode swap out. See other activities below.
After Maintenance, ran an INITIAL ALIGNMENT and had no issues. Started locking and made it to DRMI with no issues (aligned PRMI first). Handed off to Jim & Sheila for evening commissioning.
LOG:
Notes:
WP 36629
This morning the ALS common mode spare chassis S1102633 was pulled and original chassis S1102632 was re-installed. Units were swapped last week to help with ALS glitch hunting. See alog 36622.
The IQ FET demodulator chassis S1000779 was pulled for repairs. Alog 36629 reported CH1 having a 5V offset on the I-signal output. Unit was repaired, AD829 IC's had to be replaced on both boards to remove offset.
WP 7017 FRS 6142 Chandra, Dave, Filiberto, Gerardo, Patrick, Richard There were a few issues: - For some reason there were two EPICS IOCs running on h0vacmx when I logged into it to update the code (first screenshot). - I ran into a number of errors trying to create the h0vacmx TwinCAT project that were resolved after I restarted the computer. - The autoBurt.req file for h0vacmy was using the autoBurt.req file for h0vacmr and hence I could not burtrestore h0vacmy. Gerardo trended back the needed values for the CP settings and set them manually. - I had to reforce the Pirani Torr High Limit for PT120. I set it to 0.01 (second screenshot). I have updated the autoBurt.req files for h0vacex and h0vacey and checked them into svn. The HV has been untripped at end X, end Y and the CER. The new channels for the BRS ion pumps will probably not get added to the DAQ until next week. I will leave the WP open until this is done.
Most activities complete. Waiting on a couple of items:
CP4 log file DOES NOT exist!
This morning's headlines:
J. Kissel, D. Barker II 4705 WP 7010 Dave and I restarted the OMC SUS model to incorporate the changes described in LHO aLOG 36658 and the above mentioned work permits and integration issue. What remains: create a receiver in the and payload flag generator in the h1isiham6 top level model. Somehow, I was reassured that we only needed to make a change to the SUS model, but alas! I'll update the integration issue.
[Sheila, Jenne, JeffK]
Frustratingly, we spent much of the day struggling to transition from ALS COMM to IR Trans. We dug through old alogs, we measured various loops and crossovers, and started figuring out a new order of operations for doing the transition, and then finally discovered that it just worked the original way when we forgot to load the guardian after coding in there our new-fangled sequence. Incredibly frustrating. Even half an hour or so before we were not able to do the original sequence and keep the lock. This is clearly something that we need to address eventually, but it's tricky if it seems to have just gone away. We've always had occasional locklosses around this transition and just trying again would work, so maybe if we solve the greater problem that plagued us this afternoon, we'd also solve that.
Anyhow, once we were able to get past the ALS->Trans handoff, we focused on trying to get the rest of the IFO up and working, since the problem seems to have mysteriously disappeared.
The first big lock we had, we let the guardian take us all the way to NomLowNoise. At first the new AS36 SRM ASC input matrix elements from alog 36578 seemed okay, but something was clearly funny since the control signal kept increasing even though the error signals were centered around zero - the zero point of the error signals was moving. That seemed to go away, and we had several minutes of nice lock. Then, in the last 2 minutes or so, the SRC ASC control signals got very large, and we lost lock before we could open those loops. So, re-doing the SRM ASC input matrix is back on the to-do list. The rest of the locks tonight we stayed on the dither loops for the SRM alignment.
The next 2 big locks were lost after the noise tunings state, with an oscillation that seems to originate in the INP1 ASC loops at a teeny bit over 1Hz. One of those locks we had skipped the 9MHz modulation depth reduction, and one we didn't, and we can stay indefinitely long on LownoiseESD_ETMY, so I think it's something in the Noise_Tunings state that is causing trouble. The 2nd of these two locks, when we did do the 9MHz reduction, we also saw oscillations in DHARD and MICH pitch. So, we need to figure out why we can't hold the lock, but it does not seem to be related to the CSOFT loop (ex. when I had the gain up too high, and we lost lock with a 2.75 Hz oscillation) or the dP/dTheta, since this is around 1Hz which is too high for that. Also, we don't see it in the CSOFT loop until much later than other loops.
We ran A2L - we ran it 3 times because ETMX was so far off that it needed help by hand to get close, otherwise it would have taken forever. I'll look at what this means for our spot positions tomorrow.
Jeff is writing a separate log post about all the CARM loop measurements taken tonight, since we still have lots of freq noise at high frequency. We're suspecting that we need to retune TCS, and perhaps that will help some of these other problems (at least SRM ASC)?
Sadly, after losing our 3rd big lock for the night, we seem to be back at having trouble doing the ALS->Trans CARM handoff :( . Sheila tried doing the nominal guardian state, but with an LSC-TR_CARM offset of -0.8, but we lost lock, so now we've gone back to the nominal guardian order, and we'll have to look into this further tomorrow. Actually, we tried again doing the order listed below by hand, but without the 16% gain matching and later reallocation, and we lost lock. So, either we need to include that gain, or this order doesn't actually work.
----------------------
Notes to self about the order for the ALS->Trans handoff that seemed to work (although it was probably just getting better on its own) :
* ALS-C_REFL_DC_BIAS lower than normal, 20 or 24 rather than nominal 28 (although the max gain we could use seemed to increase toward nominal as time went on......). Do not let LSC-REFLBIAS FM4 come on. FM6 still okay.
* LSC-TR_CARM offset to -1.0 rather than the nominal -0.5, to put us closer to IR resonance.
* Increase LSC-TR_CARM gain by 16% to make up the difference between gain of 24 and gain of 28, but putting the gain in front of the integrator in this series of 3 filter banks.
* Turn on LSC-REFLBIAS FM4.
* Jump to guardian's CARM_ON_TR state (skipping PREP_TR_CARM, which was just done by hand in a different order).
* Reallocate gain for rest of guardian states: LSC-TR_CARM back to nominal of 1 and ALS-C_REFL_DC_BIAS gain to nominal of 28.
* Allow guardian to go forward as normal.
J. Driggers, S. Dwyer, J. Kissel We were finally able to get past the CARM reduction and recover some semblance of nominal low noise this evening (in that ISC_LOCK guardian state, but running on SRC1 / SRM on dither alignment, having not reduced the 9 MHz modulation depth, beam diverters not closed; call this "abnormal low noise"). We've been able to get there a few times (no clue yet on why the CARM reduction has started to work). Having done so, we measured the CARM loop in a few configurations: (1) 2.0W input power, DC readout (literally, the ISC_LOCK state DC_READOUT), with the IMC boost disengaged (2) 29.5W input power, abnormal low noise, (without NOISE_TUNINGS so) with no IMC boost, just arrived at high power (3) 29.5W input power, abnormal low noise, (without NOISE_TUNINGS so) with no IMC boost, after the IFO has cooked for ~10 mins (4) 29.5W input power, abnormal low noise, (without NOISE_TUNINGS so) with no IMC boost, after the IFO has cooked for ~20 mins (5) 29.5W input power, abnormal low noise, with IMC boost, after the IFO has cooked for ~20 mins Attached are the results, and below are the observations: (1) UGF at 18.8 kHz (2) 12.3 kHz (3) 7.97 kHz (4) 6.36 kHz (5) 6.46 kHz There is no place where there is less phase margin than 45 deg. Here's *a* supposition: - Once we get up to high power, we're losing CARM plant optical gain. CARM plant optical gain is defined by the beat of the 9 MHz modulation and the carrier's "audio" frequency sidebands. While 9MHz is fine because it doesn't enter the IFO, we are losing REFL carrier (and corresponding audio sidebands) as the IFO heats up. This is because we're in an alignment / loss space in which we're closer to the critical coupling point, where all carrier power gets sucked up (and lost in) the IFO. That means there's less CARM signal side bands, which means lower CARM gain. We correspondingly / corroboratively have a lower recycling gain (29, where it used 31). See previous aLOG about critical coupling point: LHO aLOG 29474. Measurement Notes: (1) 30 mVpk excitation for all above measurements (2) still captured on ASCII because GPIB isn't working.
The critical coupling point is irrelevant for the REFL optical gain. The error signal is carrier light in the orthogonal phase produced by the arms, which then beats against the 9 MHz sidebands. The amount of DC light on the REFL port only changed the shot noise. The real culprit is most likely the 9 MHz sidebands which are affected by the point absorber. In the past, we lost roughly a factor of 2 in the 9 MHz build-up—indicating significant losses in the PRC.
J. Kissel Since we've put the End-Station VEAs on the wall, we noticed that today at 14:12 UTC (Jun 05 2017 07:12:00 PDT) the noise in the XVEA temperature sensor we've posted (H1:CAL-PCALX_RECEIVERMODULETEMPERATURE) dropped by a factor of two. Then, at 23:45 (Jun 05 2017 16:45 PDT) the higher noise state returned. Today's OPS Report suggests that no one went to any end station. More detailed trending shows that this decrease in "temperature" noise did *not* show up in any other VEA temperature sensor. Given the rash of what appears to be faulty Beckhoff ADCs (see FRS Tickets 6024, 8250), I wonder if there may be a problem here... In other news, the very peaky, higher amplitude diurnal temperature oscillations are back. I'll talk with Bubba tomorrow.
Confirmed still noisy on June 26 2018 (as suspected, no one was informed of the problem beyond this aLOG.) Opened FRS Ticket 10987.
The CP4 autofill did not run this morning from crontab because I had checked its pending changes into SVN earlier and it became a non-executable file. The file was originally non-executable when it was first committed to the repository. I subsequently made it executable so I could call it directly and not as an argument to python. Inadvertently I did this incorrectly by just changing the working-directory file's permissions (chmod 775 cp4_fill.py
).
I should have made it executable with the svn command:
svn propset svn:executable on cp4_fill.py
this command makes the local file executable, and changes the file's metadata ready for a commit to the repository.
By the way, normally a commit like I did this morning will not change the file permissions, but in this case the file has the $Header$ keyword substitution enabled, so when committed both the file's contents and its permissions were "corrected".