Ops Day Shift Summary

 

 

ITMx IAS setup and alignment

J. Oberling, T. Sadecki

Today, Jason got the IAS equipment moved into the biergarten cleanroom and setup to begin IAS of ITMx for upcoming welding scheduled to start sometime tomorrow.  We ran through the first round of alignment of the Quad lower structure and PUM and TM optic.  The only issues we encountered were with our recollection of the process since it has been 3 years since we have done it last.  Tomorrow, we'll run through another round of alignment checks before starting welding.

New userapps SQZ system directory created

Sheila, Dave:

I have created a new system directory in the SVN cds_user_apps repository called trunk/sqz.

I have checked this out as a working directory on the CDS file system under the path /opt/rtcds/userapps/release/sqz. Like all systems under the release path, each are individual working directories so there is no possibility of someone at the highest level accidentally updating every system.

I am in the process of adding SQZ to all the search paths for matlab and RCG.

OpLev 7-Day Trends (FAMIS #4748)

   Posted below are the OpLev 7 day trends plots for the past week. Results are a bit off the mark. Spoke to Jason. He will advise if there is value in running these checks while the IFO is in it's current maintenance/upgrade state.  

STS & T240 Centering Checks (FAMIS #6091)

Ran STS and T240 centering checks (FAMIS #6091). The STS's are reporting all within range. 

The ETMX, ITMX, ITMY, and BS T240s report DOFs that are out of range. (Due to vent?) All other T240 DOFs report OK. Logs are posted below.  

SUS RM1 and RM2

Issues reported with some of the volt monitor readbacks for RM1 and RM2, alog 38883. The tip-tilt coil drivers were pulled out and the output buffer IC's OP27 (IC5) were replaced. 

S1200608, IC5 replaced on all channels.
S1200609, IC replace on CH2,CH3, and CH4.

 

J. Kissel, F. Clara

Put in offsets that the COILOUTF to double check whether the VOLTMONs were functional. 
Didn't see a reaction out of the VOLTMONs on the MEDM screen. 
Went out to CER with Fil.
Double checked that the VOLTMONs were functional by inserting 0.5 [V] into each coil driver's DAC input with an independent voltage supply, and saw reaction in VOLTMONs on MEDM screen.
Now suspicious of the AI / DAC chain, Fil inspected the pins (which were clear) then reseated / better seated the DAC SCSI cable at the back of the IO chassis.
Functionality restored: offsets in the COILOUTF make the VOLTMONs light up.

So, apparently the damping loops I'd turned on yesterday weren't doing anything. 

Turned on the damping loops and they now appear unstable, will investigate further.

Resolution of problem described in LHO aLOG 38893.

CDS model restart report: Friday 22nd September - Tuesday 3rd October 2017

model restarts logged for Tue 03/Oct/2017
2017_10_03 10:12 h1ascimc
2017_10_03 10:12 h1asc
2017_10_03 10:29 h1isiham2
2017_10_03 10:36 h1dc0
2017_10_03 10:38 h1broadcast0
2017_10_03 10:38 h1fw0
2017_10_03 10:38 h1fw1
2017_10_03 10:38 h1fw2
2017_10_03 10:38 h1nds0
2017_10_03 10:38 h1nds1
2017_10_03 10:44 h1tw1
2017_10_03 11:28 h1susmc2
2017_10_03 11:28 h1sussrm
2017_10_03 11:30 h1isiham3
2017_10_03 11:30 h1isiham4
2017_10_03 11:30 h1isiham5
2017_10_03 11:35 h1broadcast0
2017_10_03 11:35 h1dc0
2017_10_03 11:35 h1fw0
2017_10_03 11:35 h1fw1
2017_10_03 11:35 h1fw2
2017_10_03 11:35 h1nds0
2017_10_03 11:35 h1nds1
2017_10_03 11:35 h1tw1
2017_10_03 15:26 h1susauxh2
2017_10_03 15:27 h1dc0
2017_10_03 15:29 h1broadcast0
2017_10_03 15:29 h1fw0
2017_10_03 15:29 h1fw1
2017_10_03 15:29 h1fw2
2017_10_03 15:29 h1nds0
2017_10_03 15:29 h1nds1
2017_10_03 15:29 h1tw1

Cleanup of shmem-IPC on h1asc0. Removal of oplev ipc senders on ham-sus, receivers on ham-isi. Additions of filters on sus-aux ham2. Associated DAQ restarts, reboot of h1tw1.

model restarts logged for Mon 02/Oct/2017
2017_10_02 11:46 h1broadcast0

Restart of broadcaster after crash on Saturday.

model restarts logged for Sun 01/Oct/2017 - Thu 28/Sep/2017 No restarts reported

model restarts logged for Wed 27/Sep/2017
2017_09_27 10:16 h1iopsush2b
2017_09_27 10:18 h1sushtts
2017_09_27 10:23 h1sushtts
2017_09_27 10:24 h1sushtts
2017_09_27 10:27 h1susim

completion of moving sushtts to h1sush2b

model restarts logged for Tue 26/Sep/2017
2017_09_26 14:20 h1susauxh2
2017_09_26 15:18 h1iopsush2b
2017_09_26 15:18 h1susim
2017_09_26 15:35 h1susim
2017_09_26 16:40 h1asc
2017_09_26 16:47 h1sushtts
2017_09_26 16:49 h1sushtts
2017_09_26 16:54 h1broadcast0
2017_09_26 16:54 h1dc0
2017_09_26 16:54 h1fw0
2017_09_26 16:54 h1fw1
2017_09_26 16:54 h1fw2
2017_09_26 16:54 h1nds0
2017_09_26 16:54 h1nds1
2017_09_26 16:54 h1tw1

Squeeze install. Moved sushtts from asc0 to sush2b. Moved RM aux to susauxh2. Add shmem-ipc to asc. Associated DAQ restart.

model restarts logged for Mon 25/Sep/2017
2017_09_25 12:42 h1ascimc
2017_09_25 12:42 h1asc
2017_09_25 12:42 h1iopasc0
2017_09_25 12:42 h1ioplsc0
2017_09_25 12:42 h1lsc
2017_09_25 12:42 h1omc
2017_09_25 12:43 h1ioplsc0
2017_09_25 12:43 h1lscaux
2017_09_25 12:43 h1lsc
2017_09_25 12:43 h1omc
2017_09_25 12:43 h1omcpi
2017_09_25 12:53 h1ascimc
2017_09_25 12:53 h1asc
2017_09_25 12:53 h1iopasc0
2017_09_25 12:56 h1broadcast0
2017_09_25 12:56 h1dc0
2017_09_25 12:56 h1fw0
2017_09_25 12:56 h1fw2
2017_09_25 12:56 h1nds0
2017_09_25 12:56 h1nds1
2017_09_25 12:56 h1tw1
2017_09_25 12:57 h1fw1

Squeezer install. New ADC in LSC. ASC model changes. Associated DAQ restart.

model restarts logged for Sun 24/Sep/2017 - Sat 23/Sep/2017 No restarts reported

model restarts logged for Fri 22/Sep/2017
2017_09_22 12:24 h1ascimc
2017_09_22 12:24 h1asc
2017_09_22 12:24 h1iopasc0
2017_09_22 12:34 h1pemcs
2017_09_22 12:38 h1dc0
2017_09_22 12:40 h1broadcast0
2017_09_22 12:40 h1fw0
2017_09_22 12:40 h1fw1
2017_09_22 12:40 h1fw2
2017_09_22 12:40 h1nds0
2017_09_22 12:40 h1nds1
2017_09_22 12:40 h1tw1

Squeezer install. Removal of susauxasc0 and sushtts from asc0. Added filters to quadrature sum on pemcs. Assocated DAQ restart.

CDS model restart report: Tuesday 5th - Thursday 21st September 2017

model restarts logged for Thu 21/Sep/2017 - Wed 20/Sep/2017 No restarts reported

model restarts logged for Tue 19/Sep/2017
2017_09_19 11:18 h1iopsusauxey
2017_09_19 11:18 h1susauxey
2017_09_19 11:20 h1iopsusey
2017_09_19 11:20 h1susetmy
2017_09_19 11:20 h1susetmypi
2017_09_19 11:20 h1sustmsy
2017_09_19 11:22 h1hpietmy
2017_09_19 11:22 h1iopiscey
2017_09_19 11:22 h1iopseiey
2017_09_19 11:22 h1iscey
2017_09_19 11:22 h1isietmy
2017_09_19 11:22 h1pemey
2017_09_19 11:23 h1alsey
2017_09_19 11:23 h1caley
2017_09_19 14:08 h1broadcast0
2017_09_19 14:08 h1dc0
2017_09_19 14:08 h1fw0
2017_09_19 14:08 h1fw1
2017_09_19 14:08 h1fw2
2017_09_19 14:08 h1nds0
2017_09_19 14:08 h1tw1
2017_09_19 14:11 h1nds1

EY front ends restarted due to  IRIG-B unit power connector replacement. DAQ restart for new dust monitors and safety system.

model restarts logged for Mon 18/Sep/2017
2017_09_18 12:21 h1alsex
2017_09_18 12:21 h1calex
2017_09_18 12:21 h1iopiscex
2017_09_18 12:21 h1iscex
2017_09_18 12:21 h1pemex
2017_09_18 13:32 h1ioppemmy
2017_09_18 13:32 h1pemmy
2017_09_18 13:35 h1ioppemmy
2017_09_18 13:37 h1pemmy
2017_09_18 14:58 h1ioppemmx
2017_09_18 14:58 h1pemmxf

power glitch recovery continues, timing issue with h1iscex.

model restarts logged for Sun 17/Sep/2017 No restarts reported

model restarts logged for Sat 16/Sep/2017

Many restarts as part of recovery from 08:12 PDT power glitch. See attached text file for full list.

model restarts logged for Fri 15/Sep/2017 - Thu 14/Sep/2017 No restarts reported

model restarts logged for Wed 13/Sep/2017
2017_09_13 10:05 h1nds1
2017_09_13 10:06 h1nds1

issues with nds, required restart.

model restarts logged for Tue 12/Sep/2017 - Sat 09/Sep/2017 No restarts reported

model restarts logged for Fri 08/Sep/2017
2017_09_08 09:25 h1nds1

issues with new, required restart.

model restarts logged for Thu 07/Sep/2017
2017_09_07 09:43 h1iopsusex
2017_09_07 09:45 h1susetmx
2017_09_07 09:45 h1susetmxpi
2017_09_07 09:45 h1sustmsx
2017_09_07 14:42 h1asc
2017_09_07 14:44 h1broadcast0
2017_09_07 14:44 h1dc0
2017_09_07 14:44 h1fw0
2017_09_07 14:44 h1fw1
2017_09_07 14:44 h1fw2
2017_09_07 14:44 h1nds0
2017_09_07 14:44 h1nds1
2017_09_07 14:44 h1tw1

model restarts logged for Wed 06/Sep/2017 - Tue 05/Sep/2017 No restarts reported

Ops Day Shift Start

Control Functionality Restored on RMs 1 and 2

J. Kissel, D. Barker, F. Clara

Since we started rearranging the SUS analog sugnal chains to incorporate / make room for the new SQZ SUS (i.e. VOPO, OFIS, ZMs 1&2) -- see LHO aLOG 38827 -- and subsequently fractured off the RMs into their own model run on the h1sush2b computer, and installed -- see LHO aLOGs 38796, 38805 -- the RMs digital control infrastructure has been empty. 

Today I restored all filters and EPICs records necessary for local damping, and confirmed that there wasn't anything else. There were some confusing discoveries, as there always are with these poorly maintained suspensions, but I elected to not chase the rabbit down the hole and just restore the SUS exactly as it was at the close of O2.

I've not yet committed all changes to the repository because I want to wait for the svn package to be complete, but things effected in making the RMs functional again:
- Copied the userapps repo version of RM filters from
    /opt/rtcds/userapps/release/sus/h1/filterfiles/H1SUSHTTS.txt
  to the (now unhooked from the repo) chans version
    /opt/rtcds/lho/h1/chans/H1SUSHTTS.txt
  which *only* has RMs in it. 

- Generated and populated the OSEM2EUL and EUL2OSEM matrices using
    /ligo/svncommon/SusSVN/sus/trunk/HTTS/Common/MatlabTools/make_sushtts_projections.m
  and 
    /ligo/svncommon/SeiSVN/seismic/Common/MatlabTools/fill_matrix_values.m

- Restored open light current OFFSETs and GAINs in the OSEMINF filter banks using conlog.
  This was one point of unsurprising confusion: while LHO aLOG 8007 reports values for open light current and corresponding OFFSETs and GAINs -- even if they were poorly measured then because the ADCs were saturated -- those that had been installed for O2 were not at all those numbers in OFFSET, and the GAIN was set to 1.0, with no easily findable aLOG reporting the rationale. If we go into HAM1, we should just remeasure these numbers and install correct OFFSETs and GAINs.

- Turned off the off-diagonal elements of the DRIVEALIGN matrix (making it an identity matrix), and made the SENSEALIGN matrix also an identity

- Turned ON the OSEMINF sensor calibration filters, the COILOUTF driver compensation filters, DAMPing filters, watchdog's band-limiting filters.

- Made both RM1 and RM2 WD threshold be 25000 [ct], though the average output in ambient damped state is in the low 10s, so this should probably be reevaluated.

- Modified OVERVIEW medm screens such that the DAC and IOP DAC outputs account for the new DAC card arrangement, as well as the name change of the coil driver monitors (VOLTMON_??_MON to VOLTMON_??_OUTMON) to show the "upgrade" of those monitor channels to a full filter bank, which impacts the IMs as well
    /opt/rtcds/userapps/release/sus/common/medm/haux/
        SUS_CUST_HAUX_OVERVIEW_all.adl
    /opt/rtcds/userapps/release/sus/common/medm/hsss/ 
        SUS_CUST_HSSS_OVERVIEW.adl
        SUS_CUST_RM_OVERVIEW_ALL.adl

- Modified the macros for those screens to report similar changes, again impacting the IMs as well
    /opt/rtcds/userapps/release/sus/common/medm
        susim1_overview_macro.txt
        susim2_overview_macro.txt
        susim3_overview_macro.txt
        susim4_overview_macro.txt
        susimall_overview_macro.txt
        susrm1_overview_macro.txt
        susrm2_overview_macro.txt
        susrmall_overview_macro.txt

All compensation and control filters were and now are identical between RM1 and RM2.

Also, in regaining MEDM the functionality of the controls, we discovered that the AA chassis for h1susauxh2 was busted.

I'm sure there will be more things to fix (lots of non-functional links to ADC and DAC information, the coil driver MONITOR screen will need updating), but that's all I have steam for at the moment. I leave the RMs fully functional and DAMPED. I have not bothered to restore any alignment offsets.

I attach several screenshots which capture this restored configuration. 

J. Kissel

After discovering that the DAC outputs from the I/O chassis were not getting to the coil driver (see LHO aLOG 38889), I turned on he damping loops again and they went unstable. I found a few more things I'd forgotten:

- The COILOUTF gain signs needed to be restored to compensate for magnet polarity, 
    UL = -1
    LL = +1
    UR = +1
    LR = -1
    (they had been all positive 1.)

- Even though the analog circuit is jumpered to the "Low Pass ON" configuration, I still needed to set the digital request of the coil driver's BIO state to be 2. 
    (It had been at 0)


This was enough to get RM1 up and running as designed.

However, the other confusing thing, was that RM2's signal / control chain has a different sign than RM1. I didn't discover this until I took a full suite of M1 to M1 transfer functions, and looked at the phase (which we rarely do). I think conlog'ed the gains back to an O2 time, and found indeed that RM1's damping gains are all +1.0, where as RM2's damping gains were -0.5, -1.0. and -1.0. I had put in all +1's yesterday, assuming the SUS would be the same.

Eh, well. Found all the problems, and the RMs are now *confirmed* functional. 

Still need to now create a new set of SDF files for it so that these settings stick upon front-model reboot.

First SLiC baffles installed on HAM5

TJ, Jim

This morning TJ and I got parts together and installed the first set of "beard-cover" baffles on the HAM5 ISI. These hang off the +Y side of HAM5, below the beamline and off the table so we think they will be out of the way. We also pulled the old table top baffles and left them in the staging CR next to the chamber. Install for these seemed pretty straight forward. We centered up the middle baffle on the table, using the ISI wall counter bores for guidance. We then installed the outer baffles as far out as we could while still overlapping the center baffle, as seen from HAM4. We put the center baffle as far out from the side of the table as the slots in the mounting brackets would allow and put the side baffles as close to the table as they would go. I would like to get a picture from in the beam tube, but the attached images were the best I could get for now.

Ops Day Shift Summary

Removed iLIGO RGAs from BSC7 and BSC8

Kyle R., Rakesh K. 

Installed 10" CFF blanks in place of iLIGO RGAs at BSC7 and BSC8.  Storing removed RGA assemblies on racks along the North wall of the West Bay.  

WP 7174 updating nds2 client to 0.14.3

Nds2 client 0.14.3 has been deployed on the Debian 8 boxes.

This is a  bug fix release to fix problems when you are streaming nds2 data from disk, in the presence of gaps.

angle change of HAM2 and HAM3 SEI

HAM2, RZ = -246urad

HAM3, RZ = -166urad

SRM M2 and M3 AOSEM dark offsets measured

WHile the lower stage AOSEMs were pulled back over the last few days on SRM, I did a quick look at the dark offsets.  Of course, the "pretty" OSEM offset script wouldn't work due to many years of CDS software tool upgrades since it's last uese (thanks Jim for tyring), so I just ran them by hand, and plugged them into the associated MEDM screens:

   M2

	NEW OLV	OFFSET WAS	OFFSET NOW	GAIN WAS	GAIN NOW
UL	24350	-13573	-12175	1.105	1.232
LL	23240	-13887	-11620	1.08	1.291
UR	24350	-13423	-12175	1.118	1.232
LR	21780	-12388	-10890	1.211	1.377
M3
UL	22240	-13741	-11120	1.092	1.349
LL	23100	-12966	-11550	1.157	1.3
UR	23160	-13527	-11580	1.109	1.295
LR	20850	-12420	-10425	1.208	1.439

The OSEM strengths look to be decaying a bit.

Also, I set the pitch and yaw alignment sliders both to 0 since it is getting a fresh start.  (Bias sliders were P=-666.3, Y = 927.4)

For the record, I've accepted these new open light current values for SRM into the safe.snap of the SDF system this morning, such that they survived the h1sussrm front-end process reboot to remove the HAM5 optical lever (see LHO aLOG 38868). See attached screenshot.