model restarts logged for Wed 27/Jan/2016
2016_01_27 08:39 h1iopsusauxey
2016_01_27 08:39 h1susauxey
2016_01_27 08:41 h1iopsusey
2016_01_27 08:41 h1susetmy
2016_01_27 08:41 h1susetmypi
2016_01_27 08:41 h1sustmsy
2016_01_27 09:05 h1iopsusey
2016_01_27 09:05 h1susetmy
2016_01_27 09:05 h1susetmypi
2016_01_27 09:05 h1sustmsy
2016_01_27 09:56 h1iopsusey
2016_01_27 09:56 h1susetmy
2016_01_27 09:57 h1iopsusey
2016_01_27 09:57 h1susetmy
2016_01_27 09:58 h1susetmypi
2016_01_27 09:58 h1sustmsy

h1susey discovered to have bad 18bit-DAC card, was replaced. h1susauxey accidentally restarted.

model restarts logged for Tue 26/Jan/2016
2016_01_26 10:42 h1tcscs
2016_01_26 10:43 h1broadcast0
2016_01_26 10:43 h1dc0
2016_01_26 10:45 h1tw1
2016_01_26 10:47 h1nds0
2016_01_26 10:47 h1nds1
2016_01_26 11:10 h1sysecaty1plc3sdf
2016_01_26 11:16 h1sysecaty1plc3sdf
2016_01_26 11:18 h1sysecaty1plc3sdf
2016_01_26 11:20 h1sysecaty1plc2sdf
2016_01_26 11:24 h1sysecaty1plc2sdf
2016_01_26 11:25 h1sysecaty1plc1sdf
2016_01_26 11:43 h1sysecaty1plc1sdf
2016_01_26 11:48 h1sysecaty1plc2sdf
2016_01_26 11:52 h1sysecaty1plc2sdf
2016_01_26 12:09 h1sysecaty1plc2sdf
2016_01_26 12:14 h1sysecaty1plc3sdf
2016_01_26 14:40 h1pemcs
2016_01_26 14:42 h1oaf
2016_01_26 14:44 h1dc0
2016_01_26 14:44 h1nds0
2016_01_26 14:44 h1nds1
2016_01_26 14:44 h1tw1
2016_01_26 14:45 h1nds0
2016_01_26 14:46 h1broadcast0
2016_01_26 14:46 h1nds0
2016_01_26 14:47 h1nds0
2016_01_26 14:48 h1nds1
2016_01_26 14:51 h1nds1
2016_01_26 14:52 h1nds1
2016_01_26 14:53 h1nds0

Tuesday maintenance. New TCS model, testing new Beckhoff SDF code (was backed out), pem-oaf change to drive audio to control room. Associated DAQ restarts, multiple NDS restarts due to monit/testpoint.par problems.

model restarts logged for Mon 25/Jan/2016 No restarts reported

Using the noise injections I performed yesterday night, I could check that the coupling of DHARD picth to DARM is mostly linear and stationary.

In the attached plot:

• Blue is the DARM sensitivity without any noise injection
• Green is the DARM sensiivity when DHARD pitch noise was injected with maximum amplitude
• Red is the projection of DHARD pitch noise, using the measured transfer function, while the noise was on
• Cyan is the projection of DHARD pitch noise, using the measured transfer function, without noise

What can we learn from this plot?

1. The fact that the red and green curves match means that the measured linear and stationary transfer function is a good estimate of the noise coupling: this is a good indication that the noie coupling is indeed dominated by a linear and stationary term
2. The cyan curve is quite close to the measured sensitivity below 20 Hz: DHARD noise is a significant contribution to the total noise in this region, confirming what already found with a coherence analysis

The second attached plot hows the transfer function between DARM_OUT and DHARD_P_OUT. It's quite flat and feature-less, except for the bounce and roll mode notches. This suggests to me that we could reduce the DHARD pitch coupling by adjusting the differential gain of ETMX and ETMY P2L to minimize the noise coupling.

Alastair, Aidan, Patrrick, Carlos, Jim, Dave:

Wednesday afternoon we ressurected the TCS FLIR cameras which reside on the X and Y arm TCS tables in the LVEA. We re-discovered that TiVo and Cyrus had done most of the installation work already on this system and we just needed to complete the network hookup and permit remote access to the software running on the windows server.

We verified that the Beckhoff control of the camera's power supply was working correctly.

The cameras have the network names h1flircam[x,y] the windows server is h1tcsflir. The server is running Windows7 and the flir camera software provided by the vendor (licenced via a usb dongle).

In the MSR, the gigabit ethernet cable runs from the camera to a punch through on the wall of the table enclosure. From there both X and Y cables run over to the rack by HAM4 and plug into the first two of the four port ethernet patch panel. The other end of the patch panel are ports 49-52 in the CER. We installed two short patch cables from patch ports 49,50 into the Cisco switch using switch-ports 12,14 for camera X and Y respectively.

Carlos worked on allowing user controls to remote desktop from any CDS workstations to the h1tcsflir server.

0915 -0955 hrs. local -> To and from X-end 

Pump won't start on either HV channel, gets to 500V then limits at 500ma -> Tempted to increase the default arc limit of 10 per minute but will consult with GammaVacuum first

While in the CER preparing for some oscillator coupling measurements, I noticed that the spare EOM driver had its input turned off, and the power supply was in standby mode. Since the input switch was turned off, I assume this was intentional and followed the instructions on the sticky note for unplugging the input from the distribution amplifier.

Injected some broad-band noise in DHARD pitch, with a shape tailored to match closely the error signal above few Hz. Analysis will follow tomorrow.

Noise shape:
zpk([0;0;-6.2832-i*25.1327;-6.2832+i*25.1327;-6.2832-i*25.1327;-6.2832+i*25.1327;37.6991-i*62.8319;
    37.6991+i*62.8319;37.6991-i*62.8319;37.6991+i*62.8319;-157.0796],
    [-3.1416-i*5.0265;-3.1416+i*5.0265;-6.2832;-4.3982-i*18.8496;-4.3982+i*18.8496;-4.3982-i*18.8496;
    -4.3982+i*18.8496;-6.2832-i*18.8496;-6.2832+i*18.8496;-6.2832-i*18.8496;-6.2832+i*18.8496;
    -251.3274],1)
gain(0.0115462)cheby1("LowPass",4,1,100)

ampl 300
     1137997278 Wed Jan 27 22:21:01 PST 2016
     1137997409 Wed Jan 27 22:23:12 PST 2016
ampl 600
     1137997432 Wed Jan 27 22:23:35 PST 2016
     1137997556 Wed Jan 27 22:25:39 PST 2016
ampl 900
     1137997581 Wed Jan 27 22:26:04 PST 2016
     1137997706 Wed Jan 27 22:28:09 PST 2016
ampl 2000
     1137997738 Wed Jan 27 22:28:41 PST 2016
     1137997884 Wed Jan 27 22:31:07 PST 2016
 

We just had another lockloss due to coil driver switching.  This is a problem that has been with us since ER8 (also 20305)  I filed my first FRS ticket, which I accidentally sent to LLO Detector engineering 4303

We also have had a number of other locklosses tonight.  

One interesting one was that when we locked with a low recycling gain, our roll mode damping seemed to cause the mode to ring up.  As far as I know, this is the first time this has happened here, although LLO people have complained about this. It is plausible that our spot position moved and the coupling to DARM changed sign.  After Jim and I redid inital alingment Jenne was able to damp this mode using the old gains, which we have been using for almost a year now.

We also had 2 locklosses tonight due to wrong settings on ETMY ESD, we used our new down.snap to hopefully avoid any more wrong settings.

We also had 2 locklosses due to ASC coming on when the recycling gain was low.  (both fixed by doing inital alignment) 

H1 has 30 sender RFM channels on each arm, of which only 26 have corresponding receiver(s). So 4 are being sent and no model is using the data.

L1 has 29 senders on the X-ARM (of which there are 26 receivers), and 30 senders on the Y-ARM (of which there are 26 receivers).

So the two sites are very close in number of sending channels.

Analysis details: The base number of potential senders was derived from the main IPC file, looking for RFM0 and RFM1 ipc types. This resulted in 30 for H1-X and H1-Y, and 42 for L1-X and L1-Y. Because the ipc file is only appended to during compilation, if it has not been cleanly regenerated recently it may overcount the number of sending channels.

For each channel, I searched the models' RCG generated IPC_STATUS.adl medm file for the channel name (e.g. H1LSC_IPC_STATUS.adl). Assuming that no two ipc channels share the same name, if I found the channel name in the adl file this means it is a running sender with a receiver. For the remaining possible senders without receivers (H1-X=4, H1-Y=4, L1-X=16, L1-Y=16) I looked for the channels in the top level simulink source files (e.g. /opt/rtcds/userapps/release/*/l1/models/l1*.mdl). This showed that all four channels on H1-X and H1-Y do have sending models, and for L1-X 3 of the 16 had sending models, and for L1-Y 4 of the 16 had sending models.

If we can possibly remove some of the RFM channels which are not being received, additional RFM channels can be added to the loop with no risk.

For H1 the sending channels with no receivers are:

• H1:OAF-PEME[X,Y]_RFM
• H1:ASC-[X,Y]_TR_A_SUM_RFM
• H1:ALS-[X,Y]_ARM_RFM
• H1:ALS-[X,Y]_REFL_B_LF_RFM

The cooling system failure at the H-2 electronics building (FRS 4271) was due to a defective flex joint in the line set from a outdoor unit compressor to the building. The flex joint has been replaced and the line set and compressor are being drawn down. There will be a small vacuum pump running near the outdoor units throughout the night. If there are no more leaks, the unit will be recharged with gas and back running tomorrow. The vacuum pump will run unattended, however, if there are any issues with the running of the pump, please do not hesitate to call me. 
The commissioners have been apprised of this also.   

1545 - 1605 hrs. local -> To and from Y-mid 

Next CP3 over fill to be Friday, Jan. 29th before 4:00 pm

State of H1: realigned and relocking and currently between DRMI and Engage ASC

Shift Details:

• Richard and Filiberto worked at EY this morning, to recover the ETMY ESD drive
• They used the ETMX ESD drive temporarily to compare the logic
• Bubba started working on shimming the West Bay crane - more time needed
• Richard and Filiberto worked on cabling for TCS - more time needed, scheduled for tomorrow morning
• No outstanding issues on H1 at this time.

The picomotors for the TCS are as follows:

• PICO_F = CO2X
• PICO_G = CO2Y
• PICO_H = HWS

The following screens have been modified to fix links to the TCS picomotors so that they are correct:

• TCS_HWS.adl
• TCS_CO2P.adl
• TCS_MASTER.adl (for passing arguments to the previous two screens)

All files have been committed to the SVN

Justin's web page contains a lot of very useful links, I have linked it at the bottom of the CDS home page.

Looking at the problems from last night I began looking at the ESD this AM.  Noticing that the main read back for driver was flickering on and off I by-passed the LV driver and looked only at the HV driver.  It was obvious the DC bias was not present.  Hoping it was not the Driver itself looked at the incoming DAC channels and all functioned except the DC bias channel.  Tracked it down to the output of the IO chassis.  Filiberto replaced the 18bit DAC, ribbon cable and interface card.  This cleared up the issue.  While working on this noticed the ESD can get into some odd states when the on/off switch is pressed.  I think we still have some binary issues that latch the unit up.  Was able to clear the problem by going to the BIO screen and changing states.

Jim, Evan

Does this behavior of the ETM ESD DAC outputs make sense? Here a large offset has been applied to the UL coil output filter. However, the screen suggests that it's being applied to the UR DAC channels.

This behavior is seen on both ETM ESD DACs, but not the ITM ESD DACs.

New lab dust monitors

Jeff B, Jim, Dave [WP5697]

As part of the install of the new OSB lab dust monitors:

generated new autoBurt.req file. Generated new DAQ EDCU INI file. Recanned conlog against autoBurt.req file.

New GDS version

Jim [WP 5696]:

completed this morning.

Clearing partially loaded filter modules

Was able to do full loads on susbs, susitmx, susitmy, susprm, sussrm. Still have partial loads on lsc, omc, suspr2 and sussr2 which are waiting for the system experts to clear.

DAQ reconfigure

DAQ EDCU was reconfigured for the missing h1tw0 and new DUST channels. Was applied during today's two DAQ restarts.

DAQ Restarts

In both DAQ restarts the dataconcentrator, tw1 and both framewriters started with no issues. In the first restart the broadcaster started by both NDS daqd process were no longer controlled by monit. A simple monit restart of the daqd processes cleared the error. On the second restart the broadcaster started, stopped and later restarted. Again both NDS processes did not start, and when they were restarted they failed. The error was that high dcuid nodes for the Beckhoff SDF system has been mistakenly added to the testpoint.par file, and only the NDS machines objected to this. The file was repaired and monit was able to restart the NDS daqd processes.

TCS model changes

Aidan and Dave [WP5688]:

Installed new h1tcscs model onto h1oaf0. DAQ was restarted.

New Beckhoff SDF code release

Jonathan [WP5695]:

Jonathan tested the new code on EY Beckhoff. He found an issue with enumerated binaries and backed this version out.

RACCESS monitoring/control of h1hwinj1

Jonathan, Jim, Dave:

Jonathan added the hardware injection machine h1hwinj1 to his RACCESS control system. MEDMs were updated accordingly.

H1OAF change to drive audio in control room

Richard, Evan, Dave [WP5704]:

h1oaf and h1pemcs models were changed to permit oaf to drive the 8th DAC channel of PEM's 18-bit DAC card. This analog signal is then routed from the CER to the control room using the audio channel in the digital video fiber link between the rooms.

New operator table install in the control room

Richard, Carlos, Jim, Dave:

The new powered, adjustable-height operator table was installed in the control room at the operator station. The access control PC, gate cam and alarm machines were shuffled to the left and a new workstation was installed on the new table.

HPI and ISI safe.snap work

Hugh, Dave:

Hugh worked on the safe.snap files for HPI and ISI. I was able to offer some help by creating a new script to convert sub-systems to safe SDF mode and converted all HPI and ISI to safe. Later I converted them back to OBSERVE mode.

New ECAT EPICS channels for corner station

Daniel, Dave:

Daniel made Beckhoff changes in the corner station. I updated the ECAT DAQ INI files and the target autoBurt.req files.

Server patching and rebooting

Carlos, Dave:

Carlos applied security patches and rebooted cdsssh, cdslogin and cdsadminctrl. We restarted the vacuum alarms system on cdslogin.

Updated the corner TwinCAT system to the most recent svn. This includes the changes needed for adding the ASC-AS_A/B.RF90 channels.

The new AS_B.RF90 demod channels won't be active until the corner 4 chassis is upgraded. The AS_A.RF90 demod channels have been taken over from the old (now deleted) AS_D. Whitening channels for both AS_A and AS_B are working.

tl;dr: Alignment of IMs and RMs should be checked, then re-run an initial alignment.

Xarm green alignment

Ed was struggling with the Xarm green alignment.  We set the guardian to locked no slow, no wfs.  I then turned on the loops one at a time.  Usually the camera centering loops are the problem, but they were the easy ones this time.  Eventually it was DOF2 that was causing trouble, so I had DOFs 1 and 3 closed and touched TMS by hand to get the error signals for DOF2 closer to zero.  I was able to close all the loops, and let the alignment run like normal after that.

Xarm IR alignment

Not really sure what the problem is here, but it's getting late and I'm getting frustrated, so I'm going to see if I can move on with just hand aligning the input. 

I suspect that the IMs need some more attention, so if Cheryl (or someone else following Cheryl's procedures) could check on those again in the morning, that would be great. 

Also, I'm not sure if the RMs got any attention today, but the DC centering servos are struggling.  I've increased the limits on DC servos 1 and 2, both pitch and yaw (they used to all be 555, now they're all 2000).  I also increased H1:SUS-RM2_M1_LOCK_Y_LIMIT from 1000 to 2000. 

Allowing the INP ASC loops to come on is consistently causing the arm power to decay from 0.85ish to lockloss. I didn't touch the ITM or ETM, but I'm not getting any more power by adjusting PR2 or IM4. 

MICH Dark not locking.  Discovered that BS's M2 EUL2OSEM matrix wasn't loaded, so no signal being sent out.  Hit load, MICH locked, moved on.

Bounce needed hand damping (guardian will prompt you in DARM_WFS state), roll will probably need it too.  This isn't surprising, since it happens whenever the ISI for a quad optic trips.  Recall that you can find the final gains (and the signs) for all of these in the ISC_LOCK guardian.  I like to start with something small-ish, and increase the gain as the mode damps away.  No filters need to be changed, just the gains.  My starting values are in the table below, and I usually go up by factors of 2 or 3 at a time. 

I lost lock while damping the bounce mode (in DARM_WFS state), and the DRMI alignment is coming back much worse than the first few DRMI locks I had.  

I don't actually have a lot of faith in my input beam alignment, so I probably wouldn't be happy with any ASC loop measurements I take tonight even if I got the IFO locked.  Since we have an 8am meeting, I'm going to call it a night, and ask the morning operator to check my alignment and fix anything that sleepy-Jenne messed up.

Evan, Kiwamu,

As some of us have already noticed, there is a broadband noise with a 1/f^{0.5} shape in frequency from 60 to 200 Hz. This noise is unidentified.

Do not believe any statements in this report until futher analaysis. Something is fishy with the claibration of the cross-spectrum.

We are planing to check how stable this noise level is over the course of the entire O1.

The below shows an example spectrum of DARM.

Blue curves are twenty spectra of DARM (aka C01 frame, converted into displacement), each of which is made by the Pwelch with Hanning, detrended, 50% overwrap for a 12 minutes time series. The data starts at a GPS time of 1134604817. Green curves are the square-root of twenty cross-power-spectra of DCPD A and B which are reconstructed from the sum and null streams of the DCPDs. The DARM suppression effect was removed from the sum signal. The cross-spectra are then calibrated to the displacement using the latest O1 DARM model of the calibration group. No time varying correction (i.e. kappas) is applied. Red line is a 1/f^{0.5} line to show how steep the slope of the green curves is. I also attach the fig file.