Microseism still ~0.6 um/s, but we're locked anyhow.

Evan G., Jeff K.

Jeff and I got to talking about the problematic 1 Hz comb (and associates) below 100 Hz, and he thought maybe we could have a look at the timing system because there are certainly a lot of 1 PPS signals running around. Our goal was to get a sense of the system and to see where we might start looking for anything broken or misbehaving (not that there is an obvious problem, but it is a place to start).

Summary:

We looked at channels on the timing comparitors to check out timing difference between the GPS signal from the master/fanout with comparitors. The timing fanouts looks very stable aside from a small (~40 ns) shift at EY compared to the EY Symmetricom GPS antenna after a power outage. Another comparitor, the Time Code Converter (TCT), shows a constant, consistent drift at CS, EX, and EY. The timing difference currently for the TCT is around -1.6 us for CS, EX, and EY.

Details:

From the Timing MEDM screens we looked at the comparitor signals to understand the time difference. We had to verify where the cables were plugged in because it is not obvious from the MEDM screen. We used T070173 as a reference for the timing system. We made 60-day trends of the mean values for the different comparison values. Our legends are labeled as follows:

H1:SYS-TIMING_C_MA_A_PORT_2_SLAVE_CFC_TIMEDIFF_1 = "Timing solutions flywheel vs timing master"
H1:SYS-TIMING_C_MA_A_PORT_2_SLAVE_CFC_TIMEDIFF_2 = "Atomic clock? vs timing master"
H1:SYS-TIMING_C_MA_A_PORT_2_SLAVE_CFC_TIMEDIFF_3 = "Symmetricom GPS Antenna vs timing master"
H1:SYS-TIMING_X_FO_A_PORT_9_SLAVE_CFC_TIMEDIFF_1 = "EX Time Code Converter (TCT) vs EX Timing Fanout"
H1:SYS-TIMING_X_FO_A_PORT_9_SLAVE_CFC_TIMEDIFF_3 = "EX Symmetricom GPS Antenna vs EX Timing Fanout"
H1:SYS-TIMING_Y_FO_A_PORT_9_SLAVE_CFC_TIMEDIFF_1 = "EY Time Code Converter (TCT) vs EY Timing Fanout"
H1:SYS-TIMING_Y_FO_A_PORT_9_SLAVE_CFC_TIMEDIFF_3 = "EY Symmetricom GPS Antenna vs EY Timing Fanout"

It appears that things are ok, but we are not sure at what values "we should get worried about this." The only other notable issue was looking at the MEDM screen, the channel H1:SYS-TIMING_C_FO_A_PORT_11_SLAVE_CFC_FREQUENCY_4 would periodically drop to zero and oscillate much more than any of the other channels. We do not know if this is a problem.

Most likely related to transmitted power drop reference in aLog 23941.

[Sheila, Jenne, Travis, JeffK, EvanH]

The transmitted powers through the IFO are dropping on a several-hour timescale, and we don't know why. It looks like this was also happening at the end of yesterday's 31 hour lock.  (POP_LF is shown for the last day or so in the attached plot.)  These are the only 2 locks in the last 10 days that have this trend - for all the others the powers stay nice and steady.

The power into the interferomter as measured by both IMC_Trans and IM4_Trans is steady, so it's not anything from the PSL or IMC. 

We have looked at all the alignment and length control signals that we can think of, as well as the witness channels on the bottoms of the optics, and we aren't seeing anything that jumps out at us as a cause of this power drop.

Intriguingly, the REFL power is dropping as well as the transmitted powers, so perhaps we're losing our mode matching throughout the lock?  Sheila found that the TCS CO2 power is different after Tuesday maintenence this week, although it's not changing throughout these locks.

Anyhow, we're not sure what is wrong, so we're not sure what we would tweak if we could, so we're leaving the IFO alone.  But, I suspect that once POP_LF gets down near 15,000 counts we'll lose this lock. 

Currently ~6m wave activity off the Pacific Northwest coast causing elevated microseism. Omicron glitch rate correspondingly rises, especially in the low frequency region. I attach an image showing microseism trend and Omicron glitchgram. Can we get clues to upconversion mechanisms from IFO data during this time?

Activity Log: All Times in UTC (PT)

16:00 (08:00) Take over from JT
17:48 (09:48) Kyle – Going to X2-8 to recover equipment
19:29 (11:29) Kyle – Returning from X2-8
20:01 (12:01) Jodi – Going into High Bay to check for parts/equipment
21:18 (13:18) Kyle & John – Going to X2-8 to bring equipment back to CS
21:59 (13:59) Kyle & John – Back from X-Arm
22:26 (14:26) Kyle – Moving equipment to VPW. Then driving to Y2-8
23:31 (15:31) Kyle – Returning to CS from Y2-8 


End of Shift Summary:

Title: 12/03/2015, Day Shift 16:00 – 00:00 (08:00 – 16:00) All times in UTC (PT)

Support: None needed 
 
Incoming Operator: Travis

Shift Detail Summary: Overall a good observing shift. The IFO has been locked in Observing mode for the past 11.5 hours. Winds remain calm to light breeze (0-7mph). Seismic remains quiet. Microseism has flattened out and is showing a slight downward slope, but is still around 0.7um/s. 

There is a timing error flagged on H1SUSETMY. Will wait for the next time IFO is out of Observing mode to reset it.     

1. Approval Processor now checks for SegDB overflows, looking for overflow flags 30 seconds prior to and 5 seconds after the event gpstime. The program waits 180 seconds before it performs these checks to ensure that the proper SegDB files have been copied to the emfollow machine.

2. We've doubled the far thresholds respectively for each pipeline.

Prompted by a plot I showed at todays JRPC call of the forest of narrow lines visible below 100 Hz in a typical
daily FSscan normalized spectrum (based on a day's worth of 30-minute FFTs), Jeff asked me 
how much data one needs to see these lines when trying to investigate them. Below are some
ldvw spectra from this week for a variety of total observation times and FFT coherence times,
starting with long times and moving toward shorter times. From these spectra, I would say
that 15 minutes of observing time with 1- or 5-minute FFTs could be adequate to see if a
particular change has made the stronger lines go away completely, but one would need
to go to longer times to quantify small relative changes in strength.

To see weaker lines, one can go to longer observation times and longer coherence times
(see last figure).

Figures:
1 - 2 hours of 15-min FFTs
2 - 1 hour of 15-min FFTs
3 - 30 minutes of 15-min FFTs
4 - 30 minutes of 5-min FFTs
5 - 15 minutes of 5-min FFTs
6 - 15 minutes of 1-min FFTs
7 - 5 minutes of a 5-min FFT
8 - 5 minutes of 1-min FFTS
9 - 8 hours of 30-min FFTs

   The IFO has been in Observing mode for the past 8 hours. Range has been between upper 70s to 80Mpc. Wind is calm to a light breeze (0 - 4mph). Seismic activity is quiet and well below 0.1um/s. Miscroseism has been building for the past 6 hours, but is starting flatten out at around 0.8um/s. There are storms along the North Pacific coast (data buoys reporting waves 12 to 16.5 feet, winds 22 to 37 knots, pressure at 29.2 inches and falling). 

   There have been two ETM-Y saturations.  

When looking at the optical lever diodes recently I noticed that the ITMX diode sum seemed to be dropping and rising very regularly, with a period of approximately 6 seconds.  The attached image shows two minutes of data for the four indivual ITMX diode segments along with their sum.  It was not obvious to my eyes that the four segments would add up to equal the sum (it seemed unlikely) but Keita added the four signals together and it did in fact result in the sum as we see it on the plots.  So the sum channel is being added properly, however we still do not know why it is rising and dropping as it is.  I looked at old data to try and see when this behavior started it and it looks like this has been happening for a long time.  This pattern was visible in the data for all of 2015.  It can be clearly seen in data back from January of this year.  I did not see this behavior in the other optical lever diode channels.

Kyle and Gerardo now have the beam tube ion pump open to the tube. The pressure at XEND has fallen by ~ 1/2.

Two days shown on the plot.

           Transition Summary:
Title:  12/03/2015, Evening Shift 16:00 – 00:00 (08:00 – 16:00) All times in UTC (PT)
	
State of H1: 08:00 (16:00), The IFO locked at NOMINAL_LOW_NOISE, 22.2w, 75Mpc.  

Outgoing Operator: TJ

Quick Summary:  IFO locked in Observing mode for the past 4 hours. Environmental conditions are mixed – wind is calm (0-3mph), seismic activity is quiet. Microseism has been intensifying for the past 4 hours, and is now at 0.8um/s.     

• Title: 12/3 OWL Shift: 08:00-16:00UTC (00:00-8:00PDT), all times posted in UTC
  

• State of H1: Observing at 77 Mpc for 4 hours

• Shift Summary: One lockloss from an unknown cause, the alignment was not good after so I had to run an initial alignment. After some fiddling with the green in IA, it went to NLN with no problems. Cruising since then.

• Incoming Operator: Jeff B

• Activity Log:

• 10:17 - Lockloss

• Observing

Peter King arrived and couldn't log in so I tried my account on two different machines with no luck. LLO isn't having this issue so must be a local problem, but I don't think it should be related to the full file system.

As per TJ's note on some hard disc space being filled.  The remote screens
and control room shots are amiss.  Having said that a number of the HEPI and
ISI ones are still available, if that helps debug the issue.

I'm seeing signs very similar to last time this happened (alog23006).

I first noticed something was up when I couldn't get the Lockloss tool to work due to an IOError of no space left on drive, and then the Lock Clock died. The clock works similarly to the reservation system where it will repeatedly write a new file with the updated times and delete the old one, so when the system gets full it can't write a new one.

I have deleted some stuff in my folder in hopes that it would help, but I don't really have any large files that would make a big dent.

Operators: VerbalAlarms has also crashed because it cannot write its notifications. I started it up on the Alarm Handler computer without the "-w" option so it will still run for now, but none of its notifications are being recorded. Please stop this process and start a new one when the issue is fixed. Same startup as before, type "VerbalAlarms" into the AH computer terminal (the -w and -l options are already aliased in).

Rick, Evan

Summary

This evening we went into the PSL and examined OLTF of the FSS.

Since we want to increase the FSS gain, but cannot turn the common gain slider up any further, we looked for other ways to squeeze more gain out of the loop.

Rick had the idea to try to increase the error signal slope by adjusting the demod phase using the delay line. Indeed, we were able to increase the loop gain uniformly by 3 dB. The phase remained more or less unchanged below 700 kHz.

We now have a UGF of about 350 kHz with 50° of phase. The gain margin is about 3 dB.

Details

Delay line switch positions (up/down) are as follows:

So the phase change at 21.5 MHz is 56°. That seems like quite a lot, so perhaps we should take a closer look at the error signal with the FSS unlocked to make sure it's reasonable.

Also, on the manual FSS MEDM screen, we found that the TEST2 enable/disable button didn't really work; we seemed to get a sensible transfer function no matter what.