Evan and Kiwamu,

We have been unable to lock the interferometer since this Tuesday (alog 25431). We finally identified and fixed the problem.

It was the cage servo of SR3 which for some reason had a too high control gain and therefore kept feeding sensing noise to the suspension up to 1 Hz. We are happy now.

[Symptoms]

The symptoms were described in Evan's original alog.

In addition, today we noticed that the lock loss in DRMI often happened with a mode hop, indicating that some kind of misalignment was involved. Since the mode hop can be easily triggered by the large optics (i.e. BS, ITMs, PR3 and SR3), we suspected the large optics. Then, we found that we could stably lock PRMI which pointed us to SR3.

[Something happened in this Tuesday]

Trend data showed increase in the pitch motion of SR3. See the attached. The increase of noises occurred at around 4 pm local time of this Tuesday. We have no idea what triggered this situation.

By the way, the UR OSEM seemed as if it had reduced noise, but it turned out that this was due to the 60 Hz power line for some reason decreased at the same time. We then switched off the cage servo to see if it improves the situation. Indeed it did. In fact DRMI was able to lock more stably.

We have checked the damped spectra of the top and bottom stages without the cage servo comparing against the past ones (alog 17781). We did not find any obvious issues with the suspension when the cage servo was off. At this point, we concluded that the problem is in the cage servo code rather than the suspension or its damping system.

[The fix]

We lowered the control gain in the guardian by a factor of 100 which may sound big to some readers, but this gave us a good control time scale of 3 minutes according to a coarse impulse response test.

The resulting spectra at the top and bottom stages seemed as quiet before. So this is good. The code is checked into SVN.

We still don't understand why this happened.

Cao, Ellie, Dave O and Aidan

From llo alog 21927, Aidan found that the ratio between s an p polarizations is higher than expected.  The polarization at the output port has been observed to change over time during lock stretches. In particular the s polarization reduces by 30-50% over the course of 1-2 hours and p polarization slowly increases. Due do various polarization-depenent optical elements in the interferometer, the s-polarization pick up different phase shifts. Excess of p-polarized light may couple to DC readout and introduces excess of noise at the output. Given the time dependent behaviour of the s-polarisation we would expect the DARM noise to change with time if  polarization noise was contributing in a major way to the DARM spectrum. The DARM power spectra were inspected a number of times during post power up to  investigate the dependency of the noise on the polarization  the state of the interferometer.

DARM power spectra from 19Dec15, 26Dec15 and 9Jan16 lock stretches show no apparent changes over time after power up. These lock stretches were during the O1 run and were chosen because the interferometer went to observing mode very quickly after the power was increased. Power spectra are recorded at 0.19 Hz bandwidth, 10 averages. The interferometer went to observing mode at 7, 5 and 2 minutes after power up on 19Dec15, 26Dec15 and 9Jan16 respectively.

There is no clear correlation between the polarization drift and the DARM spectra. Whereas there is a 30%-50% decrease in s-polarization over the course of 1-2 hours after power up, the DARM noise spectra remains stable in the 10 to 70 Hz range.

The DARM spectra also indicates no effect due to thermal lensing as the DARM spectra is very stable during self-heating of the interferometer. Thermal lensing relaxation time is approximately 20 minutes after power up. DARM power spectra after stabilization (occurring at minimum 2 minutes after power up) remains constantst during thermal lensing relaxation period and beyond.

We will be looking into the same problem at Livingston, which has a stronger polarization drift.

After today's commissioning meeting, JoeB pointed me to LLO alog 19578, where he describes the change that LLO made that seems to have solved the BS coil driver switching problem.

For whatever reason, it seems that the analog output is not zero until some time after the digital output has been zeroed. So, LLO added an extra 7 second wait time after the matrix has finished ramping.  I have now implemented this here, and we'll see what happens next time we try to lock. 

model restarts logged for Thu 04/Feb/2016
2016_02_04 11:13 h1lsc
2016_02_04 11:13 h1omc
2016_02_04 11:15 h1dc0
2016_02_04 11:17 h1broadcast0
2016_02_04 11:17 h1nds0
2016_02_04 11:17 h1nds1
2016_02_04 11:17 h1tw1

LSC/OMC model change and associated DAQ restart

model restarts logged for Wed 03/Feb/2016 No restarts reported

model restarts logged for Tue 02/Feb/2016
2016_02_02 11:24 h1susauxb123
2016_02_02 11:30 h1asc
2016_02_02 11:54 h1isietmx
2016_02_02 11:58 h1susauxex
2016_02_02 12:00 h1susauxey
2016_02_02 12:33 h1dc0
2016_02_02 12:35 h1broadcast0
2016_02_02 12:35 h1nds0
2016_02_02 12:35 h1nds1
2016_02_02 12:35 h1tw1
2016_02_02 13:18 h1psliss
2016_02_02 13:32 h1psliss
2016_02_02 13:36 h1isietmy
2016_02_02 13:38 h1isiitmx
2016_02_02 13:38 h1isiitmy
2016_02_02 13:40 h1isibs
2016_02_02 13:46 h1dc0
2016_02_02 13:48 h1broadcast0
2016_02_02 13:48 h1nds0
2016_02_02 13:48 h1nds1
2016_02_02 13:48 h1tw1
2016_02_02 14:51 h1psliss
2016_02_02 16:55 h1broadcast0
2016_02_02 16:55 h1dc0
2016_02_02 16:55 h1nds0
2016_02_02 16:55 h1nds1
2016_02_02 16:55 h1tw1

Tuesday maintenance.  SUSAUX model rate change and DAQ reconfigure. ASC new model. ISI bug Fix. PSL ISS model change. Associated multiple DAQ restarts.

model restarts logged for Mon 01/Feb/2016 No restarts reported

(Jenne, Corey)

Following Evan, Richard, & Dave's work of adding the ability to listen to LSC Error Signals in the Control Room (via the big speaker in the front/right of the Control Room), I built a rudimentary medm screen to put all of the relevant interfaces needed for this on one screen (basically following the snapshot Evan posted in his alog. 

This was also an medm learning experience for me--thanks for the tutorial from Jenne!  I went through and made sure a signal from each of the LSC error signals indeed makes it through a matrix and out to the AUIDO output filter bank. 

The medm (H1OAF_AUDIO_OVERVIEW.adl) has been saved & committed to the svn.  Snapshot of the medm screen is attached.

Let me know if anything should be added (I wanted to have a boombox image as the background image, but medm didn't seem to like that idea.). 

Look forward to hearing this in the Control Room soon!

(Deleted this task from the Operator Commissioning Task wiki.)

[Alastair, Nutsinee]

We did a power budget of the Y-arm table, which Nutsinee is tidying up and we'll post soon.  The gist of this was that we are losing a significant fraction of power due to clipping on one of the 2" mirrors, partly due to alignment and partly because the mount is so deep that the mirror sits very far back resulting in a smaller clear aperture than we had intended.  

The beam is also off-center on the lens that sits in front of this mirror, suggesting that at some point this has moved and caused this clipping.  We still have the beam aligned to the irises on the final part of the table so it can be realigned through this section wthout redoing the alignment to the CP (although this will be needed after the table is moved for TMDS anyway).  I'm not changing the alignment right now, but this probably needs done during my next visit.

The faulty in-loop ISS photodiode has been removed and replaced temporarily with a beam dump.  I'll take this back to Caltech for Ben to look at.  It is serial number 003, with S-number 1400286.

The SUS medm-screen-modifying-team (Kissel, Weaver, Sadecki?) recently made a change to the ETM ESD's in "medm land" by taking an older screen stand-alone medm and splitting it between the SUS OVERVIEW & SUS AUX Channel Monitor screen (which is located via button on upper right of the SUS OVERVIEW).  (perhaps this comes from this alog?). 

Since we no longer need the ESD stand-alone medms, I removed both from the SUS/ETM ESD pull-downs on the sitemap.  The sitemap was svn-committed afterward.

This was an item on the Operator Commissioning Task wiki, and this task is now complete and will be removed.

The actual (now-moot) ESD stand-alone medm screens were not touched and are still located at:   $(USERAPPS)/sus/common/medm/quad/SUS_CUST_QUAD_MONITOR_E[x/y]_OVERVIEW.adl

      Transition Summary:
02/05/2016, Day Shift 16:00 – 00:00 (08:00 – 16:00) All times in UTC (PT)
	
State of H1: IFO is unlocked. The wind is a light breeze (< 7mph). Seismic and microseism are still elevated, as they were last night, which may make locking a bit difficult.      

This is the first diagnostic breadboard scan conducted since the end of O1.  The last scan was conducted
back on August 10th, 2015.

The power noise looks about the same as before.  Slight decrease in power on the photodiode, which is
not surprising given that alignment may need some tweaking.

Frequency noise: The control signal is now lower from 1-10 Hz.  The noise is lower between 100-1000 Hz.
Slightly higher elsewhere.  Lower than the reference measurement however.  I would say that the frequency
noise is slightly higher now that the NPRO is in its dotage.

Pointing noise: Something is wrong with the measurement.  This might also be an alignment problem, although
nothing wrong was reported by the MEDM screens.  The control signal is off a little.  This might be due to
the PZT mirror mounts not quite coming back to where they were after electrical power was restored.

Mode scan: The number of higher order modes has decreased, which is surprising (to me at least), but the
higher order mode power has increased by about 0.5%.  Which doesn't seem too bad to me.

ISS power noise: The noise on one of the photodetectors (PDA) is slightly higher than before.  The horizontal
beam pointing has increased by just over a factor of 2.  At low frequencies, less than 5 Hz, the beam pointing
noise is worse by a factor of ~5.  This might be due to the re-alignment of the beam onto the quadrant photodiode
inside the ISS photodiode box.

It is good to know that the diagnostic breadboard seems to work, modulo some tweaking, after its O1 hibernation.

The LVEA has transitioned back to LASER HAZARD.

Summary:  Another rough night for locking.  Commissioners were doing unlocked commissioning tasks for the first half of my shift.  For the latter half, we have been struggling to get much past DRMI locked, only making it to ENGAGE_ASC once.  Microseism is still elevated, but should be borderline passable for locking.  All else seems normal.  On a bright note, I did manage to witness the elusive PRMI_2_DRMI transition work!

Activity log in UTC:

0:34 John and Bubba done with crane work in LVEA

0:51 Kyle back from EY

Again Masayuki.Nakano reported with Stefan's account

Kiwamu, Masayuki

We measured spectrum of the OMC DCPD signals with a single bounce beam. It would help a noise budget of a DARM signal.

What we did

1. Increase the IMC power
IMC power was increased up to 21W. Also H1:PSL-POWER_SCALE_OFFSET was changed to 21.

2. Turn of the guardian of isc-lock
Requested 'DOWN' to the isc-lock guardian to not do anything during the measurement.

3.Miss align the mirrors
For leading the single bounce beam, all of mirrors were misaligned by requesting 'MISALIGN' to guardians of each mirrors except for ITMX.

4.Aligned the OM mirrors
When we got single bounce beam from IFO, there was no signal from ASC-AS-A, B, C QPDs initially. We aligned OM1,OM2,OM3,OMC suspensions with the playback data of OSEM signals

5.Locked the OMC
The servo gain, 'H1:OMC-LSC_SERVO_GAIN', was set to 10 and master gain of the OMC-ASC was set to 0.1.
The DCPD output was 34 mA.

6.Measurement (without a ISS second loop)
The power spectrum of below channels are measured. Measurement frequency was 1-7kHz and BW was 0.1 Hz. The measured channel was as below.
H1:OMC-DCPD_SUM_OUT
H1:OMC-DCPD_NULL_OUT
H1:PSL-ISS_SECONDLOOP_SUM58_REL_OUT
H1:PSL-ISS_SECONDLOOP_SUM58_REL_OUT was used as the out-of-loop sensor of the ISS.

7.Closed the ISS second loop
The ISS second loop was closed. The sensors used to gain error signal was PD1-4.

8.Measurement (with a ISS second loop)
Same measurement as step5. In addition to that, the coherence function between DCPD-SUM and SECONDLOOP_SUM was measured.

Discussion

I scaled out-of-loop sensor signals of ISS, i.e. the residual intensity noise after the ISS second loop, to the same unit as OMC-DCPD signals. The scaling factor was estimated by dividing the H1:OMC-DCPD_SUM_OUT spectrum (without ISS) by H1:PSL-ISS_SECONDLOOP_SUM58_REL_OUT spectrum (also without ISS) at 100Hz.
I scaled those spectrum both (hereafter 'both' means with and without closing ISS) by same scaling factor.
You can see the DCPD-SUM spectrum, DCPD-NULL spectrum and scaled second loop ISS out of loop sensor signals in attached plots.

The both NULL signals agree with the shot noise of a PD with 34mA signal (cyan curve) above 30Hz, and below that it would be limited by ADC noise.
About the SUM signals, it seems to consistent with the scaled intensity noise above 300 Hz. Also they have some coherence between the intensity noise and the OMC PD signal upper than 300Hz(see another plot). On the other hand, there seems to be some unknown noise below 300 Hz when the second ISS loop was closed.

Possibly this unkown noise might come from the length motion of the OMC. I attached another plot. This plot is the one of same channel(upper) and the OMC error signal with a different servo gain of OMC LSC loop. The error signal and DCPD-SUM signal seem to have similar structure around 100Hz. I haven't any analysis yet because these plots are measred after whitening filter had some trouble and we are planing to do same measurement again with whitening filter.

Masayuki and I noticed that ADC noise in OMC was too high. It turned out that the whitening filters turned off for no obvious reason at around 17:00 pm local according to trend.

Looking at the control screen, we noticed that some channels were not accessible. See the attached. We seem to be able to toggle the analog whitenings and digital anti-whitenings independently now. The rest of the PDs seem OK -- they don't have inaccessible channels.

Kiwamu, Jim, Dave:

New h1lsc and h1omc models were created and installed this morning. The DAQ was then restarted.

The change was to expand a single filter module to two in series (providing the potential for 20 filters) for the DARM, MICH, PRCL and SRCL paths.

models were chaged at 11:11 PST and the DAQ at 11:13 PST.

Updated userapps: isi/common/guardian/isiguardianlib/watchdog:

hugh.radkins@opsws1:watchdog 0$ svn up
U    states.py
Updated to revision 12546.
 

guardctrl restart ISI_ETMX_ST2:  this caused the guardian to deisolate stage2.  Not what I experienced Tuesday when I did similar..  When I restarted stage1 guardian, it did not touch the platform, as I expected.

Tripped the platform and now as expected, the damping loops were turned off when a state4 trip occurs.  Okay, so update fixed issue.

At ETMY, restart Stage1 first, then Stage2 and Guardian did nothing to the platforms...

Restarted HAM3, no problem.  Tested (tripping PR2 in the process) and the DAMPing path was turned off.  So good.

When restarting ISI_HAM6, this ISI guardian switched its request state to NONE but did not do anything, chamber manager was throwing messages.  Toggled ISI to HIGH_ISOLATED and everyone became happy.

TCS crew is working on SRC so I'll not touch them.

Guardian restart & test ETMX, HAM3.

Guardian restart ETMY, HAM2, HAM6.

I'll complete the remainder of the restarts and maybe test a couple more when TCS is clear.

Vern, Jim, Dave:

Several times in the past few weeks we have seen an invocation of tconvert produce the output:

tconvert NOTICE: Leap-second info in /ligo/apps/linux-x86_64/ligotools/config/public/tcleaps.txt is no longer certain to be valid; We got valid information from the web, but were unable to update the local cache file: No permission to write in directory /ligo/apps/linux-x86_64/ligotools/config/public
 

This afternoon we spent some time tracking this down. tconvert uses a web server in france to access the IERS bulletin-C information page which alerts us to upcoming leap seconds. It appears that this server was pure HTTP until January 11th this year. It is now an HTTP server which redirects the request to a HTTPS page. The TCL code inside of tconvert interprets the redirection return message as an error. It then marks the expiration of the leap second information for 24 hours in the future.

The procedure we used to maintain future leap second notification was to twice a year run tconvert as user controls, which updated a leap seoncds database file (owned by controls) which was good for the next six months. With the recent HTTP issues, controls now produces a file which is good for only a day.

This afternoon we ran a modified tconvert which access a different leap seconds HTTP page and the LHO CDS database file is now good through 8th August. In the mean time we will raise an FRS and work with Peter S for a long term solution.

There seem to be two new rf oddities that appeared after maintenance today:

• AS90 seems to jump back and forth between two different offsets, maybe about once every hour or so.
• POP90 and AS90 both show irregular low-frequency motion, similar to what we see when an in-loop OSEM (or its sensing chain) goes bad. Ease of locking doesn't really seem to be affected, unlike the previous experience with bad OSEM electronics.