CDS model and DAQ restart report, Tuesday 29th April 2014

 

2014_04_29 10:06 h1susbs

2014_04_29 10:07 h1susbs

2014_04_29 10:54 h1susitmx

2014_04_29 10:55 h1susitmx

2014_04_29 11:16 h1susitmy

2014_04_29 11:29 h1susetmx

2014_04_29 11:44 h1susetmy

2014_04_29 11:57 h1susetmx

2014_04_29 11:58 h1asc

2014_04_29 12:00 h1isiitmx

2014_04_29 12:10 h1susetmx

2014_04_29 12:12 h1asc

2014_04_29 12:12 h1susetmy

2014_04_29 12:22 h1isiitmx

2014_04_29 12:26 h1isiitmx

2014_04_29 12:29 h1sustmsx

2014_04_29 12:33 h1isiitmx

2014_04_29 12:34 h1sustmsy

2014_04_29 12:35 h1isiitmx

2014_04_29 12:49 h1isiitmx

2014_04_29 12:57 h1broadcast0

2014_04_29 12:57 h1dc0

2014_04_29 12:57 h1fw0

2014_04_29 12:57 h1fw1

2014_04_29 12:57 h1nds0

2014_04_29 12:57 h1nds1

2014_04_29 13:10 h1isiitmx

2014_04_29 13:25 h1isiitmy

2014_04_29 13:34 h1iscex

2014_04_29 13:36 h1iscey

2014_04_29 13:41 h1iscey

2014_04_29 13:41 h1isietmx

2014_04_29 13:47 h1isietmy

2014_04_29 13:55 h1isibs

2014_04_29 14:44 h1iscex

2014_04_29 14:45 h1iscey

2014_04_29 15:19 h1susomc

2014_04_29 16:03 h1susitmx

2014_04_29 16:07 h1susitmy

2014_04_29 16:11 h1iscex

2014_04_29 16:12 h1susetmx

2014_04_29 16:19 h1susetmy

2014_04_29 16:25 h1susbs

2014_04_29 16:32 h1suspr3

2014_04_29 16:42 h1susmc1

2014_04_29 16:43 h1susmc3

2014_04_29 16:45 h1susprm

2014_04_29 16:51 h1susmc2

2014_04_29 16:53 h1suspr2

2014_04_29 16:57 h1sussr2

2014_04_29 16:59 h1sussr3

2014_04_29 17:01 h1sussrm

2014_04_29 17:02 h1sussrm

2014_04_29 17:04 h1broadcast0

2014_04_29 17:04 h1dc0

2014_04_29 17:04 h1fw0

2014_04_29 17:04 h1fw1

2014_04_29 17:04 h1nds0

2014_04_29 17:04 h1nds1

2014_04_29 17:10 h1sussr2

2014_04_29 17:53 h1susetmy

 

No unexpected restarts. A very busy maintenance day.

ETMX L2L measurement for UIM

Kiwamu, Arnaud, Sheila

We spent this evening trying to get UIM to test mass length to length measurements.  We do this with no slow feedback to the top mass.  With optical lever damping on, we saturate the PUM at high frequencies, so it has been easier to make the measurements with OpLev damping off.  We have reasonable coherence from 5Hz down, saved as 2014-04-29_H1SUSETMX_L1_L2LPY_SwpetSine_OLDamp_Off_5Hz_to600mHz.xml

We are leaveing one measuremetn running on opsws6 with an envelope that we have tuned to get the low frequency part, saved as 2014-04-29_H1SUSETMX_L1_L2LPY_SwpetSine_OLDamp_Off_1Hz_down.xml.

We briefly tried using the UIM for feedback, with a theoretical plant inversion filter, with no obvious sucess. 

Kiwamu fixed the calibration of REFL_Y_CTRL, makiing it identical to REFL_X_CTRL. 

issue of ETMY M0 OSEMs solved

Sheila told me that a couple of OSEMs on the ETMY M0 stage were reading almost zero values. Since ETMY is a key for the upcoming DARM locking, I tried fixing them.

I went down to EY and spent some time trying to understand what is causing it. Finally I found out that this was due to a loose connection at a DB9 connector on a QUAD coil driver (D0902747).  Since I was standing in front of the electronics rack with a flat top screw driver in hand, I tightened every DB9 connector which was in the same rack. This should increase a chance to detect gravitational waves.

 

Some detailed notes

• I used D1002741-v6 for tracing the cables
• The loose cable is CAB:H1:SUS_ETMX_10
• The malfunctioned channels are M0_F1, F2, F3 and SD. They are exactly the channels that the loose cable is supposed to send to the ADC0.
• TOP coil driver S1000300.
• LED MON was very useful to determine if the signal arrive at the coil driver.

H1 PSL PMC and FSS investigations

JustinG, PeterK, RickS

We went out to investigate issues with the PMC that might have caused it not to lock when the room temperature varied (Reported by Robert Schofield and others).  The first thing we noticed was that the resonance threshold upper limit was set above the unlocked state level, which seems to indicate that the loop would consider itself locked even when it was not.  We changed the upper level to 0.65 which was half of the unlocked level.  I suspect that this was the cause of the problems with the automatic locking.

We then adjusted the alignment into the PMC to minimize the reflected light level while the loop was locked.  We reduced the reflected light monitor level from about 1.3 to about 1.0.  However, we later noticed that the sum of the reflected and transmitted levels had dropped significantly.  We re-visited the alignment and it seemed to be correct, with a calculated visibility (using the RFPD DC out, [unlocked -locked]/unlocked) of 92%.  Something doesn't seem right with the PMC loop or the PMC itself.  I suspect contamination windows and/or cavity.  We measured the incident and transmitted light levels using the water-cooled power meter, 12 and 8 W respectively.  Thus only 67% transmission with 92% visibility.  

We also looked at the FSS.  We found that the beam was too high on the AOM and raised the AOM to significantly improve the diffraction efficiency.  Then, we adjusted the retro-retlecting curved mirror to bring the beam up at the EOM position to achieve a single-pass efficiency (sqrt of the overall double-pass efficiency) of 86%.  We then aligned into the Reference Cavity and measured a transmission efficiency of about 50%, consistent with earlier measurements.

We then proceeded to measure the Open-loop transfer function of the Frequency Stabilization Servo but found that it was too low, about 150 kHz, even with the gain slider all the way up at 30 dB.  The gain margin was only about 35 deg.  This after increasing the power directed to the reference cavity and the reference cavity alignment.  The phase of the FSS loop has a "bubble" centered around 400 kHz so the phase margin is about 25 deg. larger at 400 or 500 kHz than at 150 kHz.  I'm somewhat surprised the performance was acceptable at all with what must have been a sub-100-kHz UGF.  Don't have a simple idea for increasing the UGF.  We need about 10 dB more gain.  Increasing the light level incident on the PMC to 35 W, which we plan to do in June, will help.  Otherwise, we will have to increase the modulation depth or increase the gain in one of the OpAmps on the FSS board.

It appears that the Test2 input on the FSS is always enabled, even when disabled on the MEDM screen.

Notes from today's work included below for future reference.  

Notes:

April 29 2014
PeterK, Justin, Rick

Mac mini OS 10.7.2

"medm_f" is the command to get fixed font sitemap

PMC as found 
PowerRefl 1.3 W
RFPD DC -0.138 V
unlocked
PowerRefl 11.3 W
RFPD DC -1.25V
89% visibility
resonance threshold 1.346 changed to 1.000

Tweaked alignment into PMC.  Not much improvement 0.138 -> 0.130
Changed upper limit on resonance threshold from 1.0 to 0.65, half of the unlocked level

FSS
as-found locked state 0.028 mV at RFPD DC
measured power after the AOM and iris 18.5 mW
measured power after AOM and before iris 26 mW

adjusted AOM height up and measured power now 23.35 mW after AOM and iris
27.35 mW total power of all beams

double pass - started 14.3 mW...by adjusting beam upward return path increased to 20.35 mW
double-pass efficiency = 74%
 single-pass (sort double-pass) = 86%

measured power after EOM 19.9mW
by dropping EOM .5mm meas pwr now 19.45
better centered on apertures

Slow setting start point -0.2511
starting common gain 30.0 dB, fast gain 16.0 dB
FSS RFPD dc level locked 19 mV
FSS RFPD dc level unlocked 107 mV
visibility 82%

power incident on reference cavity ~14 mW
power transmitted by reference cavity ~7 mw

Measured power incident on PMC 12.3 - 12.5 W
Measured power transmitted by PMC 8 W

1.289 reflected when PMC unlocked
0.099 reflected when PMC locked

Note that the TEST2 enable does not appear to work for the FSS S/N S1107453, D040105

ETMX ISI trip

at 1082859581 could have been due to my kicking the suspension around

Some H1SUSETMY M0 OSEMs Reading Bitnoise

J. Kissel

We can't have a days worth of model changes, recompiles, reinstalls, restarts, and restores without at least one mystery. Today's casualty is H1SUSETMY's, M0, F1 F2 F3 SD cluster of OSEMs, which appear to be dead, reading out bit noise. I suspect this is an analog electronics failure that is merely coincident today's reboots, because the OSEMs that have failed are all on one satellite amplifer / coil driver chain. All other OSEM sensors appear fine. We'll have to check the analogue electronics in the morning. 

The timeline of the day is as follows:
15:50:11 UTC / 08:50:11 PDT Event -- DC shift in OSEM values, with some ringing appears to be indicative of a ISI trip
18:42:35 UTC / 11:42:35 PDT Signals go dead -- corresponds with h1susetmy model restart at ~11:44 pm 00:02 PDT
19:56 UTC / 12:56 PDT       Signals come back -- corresponds with h1dc0 framebuilder restart at 12:57 pm 00:02 PDT
20:47 UTC / 13:47 PDT       h1susey model restart -- note the large temporal separation between the restart and the signal death.
~21:17 UTC / 14:17 PDT      Signals die
23:19 UTC / 16:19 PDT       h1susey model restart -- note the large temporal separation between the restart and the signal death.
00:04 UTC / 17:04 PDT       h1cd0 frambuilder restart --- signals DO NOT come back
00:53 UTC / 17:53 PDT       h1susey model restart, because I've noticed the problem, found the third bit of ADCO as red, assumed it was a too-speedy "make" and "make-install" 
The last reboot did fix the ADC0 bit error, but it did not restore the OSEM values. 
             
Note I'm retrieving the model restart times from the very useful model_start.log here:
/opt/rtcds/lho/h1/data/startlog/2014/04/29/2014_04_29_model_start.log 
which is a daily log of all model restarts (a new folder and file is automatically created each day).

model updates on h1iscex, h1iscey and h1asc

I was asked by Keiko and Adam from LLO to update the iscex, iscey and asc models in order to keep them identical between two sites. So I updated them under WP#4609. Here are the status:

• h1iscex: updated successfully, running fine (now at rev.7858)
• h1iscey: upadted successfully, running fine (now at rev.7858)
• h1asc: I was not able to compile. Currently it runs with the previous version of ASC_MASTER.mdl (i.e. rev.7418)

h1iscex and h1iscey got two new RFM sender blocks so that they can send LOCKIN clock signals over to the ASC model. Also they got a LOCKIN block so that they can assess the green WFS sensing matrix with the LOCKINs. I haven't updated the screens yet and therefore the LOCKINs are hidden from the point of view of users at this point. I will update the screens as well once the compilation issue in h1asc is resolved.

Prior to the compilation of h1asc, I updated the master model; common/models/ASC_MASTER.mdl which had been updated by Livingston in order to have the RFM receivers for the new LOCKINs. However I was not able to compile h1asc. It seems that there is some bugs in ASC_MASTER. So I reverted ASC_MASTER back to the previous revision. At the moment, h1asc is running with the previous version of ASC MASTER. I am checking with Keiko to make sure that the new version of ASC_MASTER is healthy.

Routine cleaning in H1 Laser Area Enclosure

Chris Murphy, JustinG, PeterK, RickS

We went into the LAE today for some Tues. morning maintenance on the PSL.  

Chris took the opportunity to do routine cleaning of both the Ante-room and the Laser Room.

Maintenance Day Summary

Many user model changes were made today by SUS and SEI groups. Two DAQ restarts were performed to support these changes at 12:55 and 17:03PDT.

I created a mini CDS overview MEDM to display on the TV monitors for the operators. I regenerated the detailed CDS overview screen, it still needs some updating. I created a new MEDM called H1CDS_FE_FMDAQ_STATUS_STRINGS_CUST.adl which is launched from CDS on the sitemap (FE FiltDaq entry). It shows if individual filter modules have been loaded, if changes are pending and other possible configuration mismatches.

I hand edited the H1EDCU_GRD.ini Guardian file to "green" up the EDCU by removing systems which are not running.

My EDCU greening did not last long, around 3pm PDT the conlog was restarted but has not come back. Patrick will investigate tomorrow.

The preparation for individual logins was completed and an email sent today to encourage users to log into the control room workstations using their own accounts. Main change to the CDS was the adding of common files to the controls groups and making them group writable (was done end of last week).

Some RFM0/1 IPC mismatches were found in the H1.ipc file and were corrected by removing the incorrect entries and recompiling the models (ASC and ISC).

Fiber Polarization

EY PLL gave an error for the fiber polarization again. The fiber trans in the wrong polarization had drifted back to 36%. I turned the controller back on and this kicked the EX polarization to 40%, and brought EY down to 25%. I then adjusted both channels and brought EY to 10% and EX to 13%.  This is similar to the behavior seen in alog 11505. I am wondering whether the controller only holds one channel when we turn it off...

BSC-ISI changes (Model + WD c code) propagated

I've installed the new changes made at LASTI last week here at LHO on all the BSC chambers. This installation involved model update and watchdogs C code update.

This update could be sum up in 3 main news:

1) T240 simultaneous gain switching

The analog and digital gains of the T240s are now switched simultaneously by clicking on the gain of the X channel.

2) T240s ignored when ST1-ISI isolation is off

Modify the Watch Dog code to ignore the T40s when Stage 1 isolation is off. This avoid the ISI to trip when we restore offsets on HEPI. It also avoid un-necessary trips when the ISI in damping mode.

3) T240 Monitor added

Add a T240 monitor that indicates the instruments have settled after they have been shaken. It's necessary for proper functioning of the SEI guardian and for operators as they both have no reliable means of knowing that the T240s have settled. The monitor will indicate whether the Stage 1 isolation can be turned on.

I've changed the overview MEDM screen to diplay this monitor (see pictures attached). It will do for now, but we can think of something better

 

For more details about the changes, see DCC #E1400214

Ops Shift Summary

Day Shift Summary
LVEA Laser Safe

08:30 Sprague – On site to spray outside buildings
08:48 Patrick – Going to Mid-Y to pick up parts
08:53 Justin – Shuttering PSL & ALS lightpipes
09:00 Gerardo – Soft close GV5 & GV7
09:00 Rick, Peter, Chris, & Justin – Working in H1-PSL enclosure 
09:15 Praxair – Delivery to LX-CP2 
09:28 Patrick & Thomas – Working on TSC-X 
09:29 Filiberto & Andres – Checking electrical connection to HAM6 dust monitor
09:34 Cyrus – Recovering parts from LVEA test stand
10:00 Jeff – Restart BS model
10:08 Hugh – General HEPI maintenance work in LVEA 
10:08 Andres – Moving 3IFO Vibration Absorbers to the West Bay storage racks
10:26 Betsy & Jason – Alignment setup at HAM6
10:30 Bottled water delivery
10:45 Jeff – Restarting Quad Models
11:12 Hanford on site to take water samples
11:52 Sebastien – Installing new BSC-ISI master model/MEDM screens and WD code
12:30 Gerardo – Reopen GV5 & GV7
13:22 Justin – Transition LVEA to laser hazard
13:30 Matt, Dave, Thomas, & Co – Starting CO2 laser on TCS-X
13:40 Kiwamu – Restarting ISC, ASC, h1scex, h1scey, h1asc models
14:20 Gerardo – Working in the H2-PSL enclosure
15:05 Check diode chiller water level – Level OK, no water added to system 
15:15 Craig – Working in the H2-PSL enclosure
15:55 Jeff - Restarting all SUS models
16:00 Dave – Will restart DAQ when ASC model update is complete

Alignment Dither Front-End Code Mods for SUS Complete

J. Kissel, D. Barker, J. Batch

I've finished the front-end model modifications for adding independent alignment dither paths, as per ECR E1400105. Today's focus on the BSC suspensions, i.e. the QUADs, the BSFM, and the collateral damage on the TMTS. In addition to adding the dither paths, because I was already modifying top-level, I've cleaned up the ordering of the output ports such that all HAM and BSC SUS types now spit out similar signals in the same order. This includes piping the LOCK control signal out to the top-level model, for the eventual consumption of some auxiliary calibration front end (currently still called OAF) as is already done for the HAM SUS. I've also added any missing SVN $Id$ and $HeadURL$ tags.

When finished, after noticing a peculiar error with the new end-station IPC signals, Dave and Jim reminded me that the different RFM network fabrics for the end-stations require one to add the card number to the IPC part. 
For LHO, EX = card 0, and EY = card 1. 
Last Wednesday, I had incorrectly installed all the RFM IPC parts to both end stations with cardnum=1. This resulted in the two ETMX channels having duplicate IPC numbers, and a receiving error found on the GDS_TP screen. This has now been fixed. It required clearing out all the RFM channels from ASC in the IPC file (/opt/rtcds/lho/h1/chans/ipc/H1.ipc), fixing the card number in the h1asc model, recompiling the sender (h1asc) and the two receivers (h1susetmx, h1susetmy), and a clearing of the RT NET STAT errors by hitting diag reset.

I also terminated the dither inputs to the SIXOSEM_T_STAGE_MASTER library part used in the OMCS_MASTER, which I'd forgotten to do last weekend.

I'll now begin adding this dither path to all SUS MEDM overview screens (and hopefully not uncover any bugs!).

Details:
------------
Affected models:
/opt/rtcds/userapps/release/sus/h1/models
M       h1susbs.mdl
M       h1susetmx.mdl
MM      h1susetmy.mdl
MM      h1susitmx.mdl
MM      h1susitmy.mdl
MM      h1sustmsx.mdl
 M      h1sustmsy.mdl
M       h1susomc.mdl

/opt/rtcds/userapps/release/sus/common/models/
M       BSFM_MASTER.mdl
M       TMTS_MASTER.mdl
M       QUAD_MASTER.mdl
M       OMCS_MASTER.mdl
M       SIXOSEM_F_STAGE_MASTER.mdl

Tips for Stuart:
----------------
- After an svn up of the common/models/ directory, you should only need to make changes to the top-level models.
- The TMTS and OMCS do not need any top level changes, all extra unused ports have been terminated inside the TMTS_MASTER / OMCS_MASTER block, since we know they'll never receive any dither signals.
- Pay close attention to the input and output connections at the top level, especially the lower half of them. This is the only thing you should need to change, but I've added the dither inputs and LOCK outputs in the middle of the input and output list, so most of the lower half of the connections need re-ordering.
- As mentioned above, when adding the RFM IPC to both the ASC and ETM models, be sure to call out the appropriate card number for each end station. You shouldn't need to clear out the IPC file, as long as you don't make the same card numbering error.

J. Kissel, D. Barker

I didn't even start *editing* the MEDM screens before I realized I'd forgotten to add any sort of input read-back of the alignment dither signals before they entered the new DITHER2EUL distribution matrix. In *all* of the models, both HAM and BSC SUS. So, I diverted for an hour, installed two new filter banks,
DITHERINF_P
DITHERINF_Y
in the highest level library part (where the dither signal is first digested) in all the core optic masters, 
/opt/rtcds/userapps/release/sus/common/models
M       HSTS_MASTER.mdl
M       QUAD_MASTER.mdl
M       HLTS_MASTER.mdl
M       BSFM_MASTER.mdl
M       MC_MASTER.mdl
and then recompiled, reinstalled, restarted, and restored every model that used it, which includes
h1susmc1
h1susmc2
h1susmc3
h1susprm
h1suspr2
h1suspr3
h1susbs
h1sussrm
h1sussr2
h1sussr3
h1susitmx
h1susitmy
h1susetmx
h1susetmy
Dave rebooted the DAQ / h1dc0 / frame-builder when I finished, around 5:10p PDT. The updated master parts have been committed to the repository.

HEPI Z-RZ sensor correction testing at BS chamber

The ISC crew has been complaining of excessive yaw on different platforms, which some seismic people have suspicions of actuator/T240 interaction. rDr at Livingston attempted to deal with this by trading some stage 1 isi isolation (by using higher frequency blend filters) for some increased HEPI isolation using STS to HEPI sensor correction. I tried to duplicate this here, but didn't have much success, as my attached plots attempt to show. The first screen shows the T240 spectra, where red is the Ryan's (as I understand it) configuration with T750 blend on st1 Z and his senscor filter on HEPI, and probably isi stage 1 sensor correction turned off (I don't remember cause I did this Friday, and Sheila had a couple this she wanted me to try that didn't get into my notes). Blue is our current configuration with low frequency blends, level 3 isolation and isi senscor,  no HEPI senscor, with HEPI position loops on. Light green is the blue configuration with stage 1 RZ turned off. The dark green is the red configuration, but no HEPI senscor. Looking at the T240's, it looks like we don't really win with HEPI senscor.

Looking at the oplev (second picture, same color sheme (red =hepi senscor, blue = baseline, light green= RZ off)) doesn't really make this simpler, though. It looks like as far as the oplev is concerned, turning off RZ gets a little better at about .1hz (likely because we're not re-injecting actuator/T240 cross-talk?), and otherwise no big changes. Sebastien is here, so I'll get his help to see if I'm missing anything here.

TCS CO2X output power vs temperature and PZT voltage

Aidan, David, Heintze, Thomas, Greg.

Yesterday, to map out the response of the CO2X laser as a function of PZT voltage and temperature, we drove the PZT on the CO2X laser with a 30V amplitude, 50mHz triangle wave whilst increasing the temperature slowly. The resulting phase space is mapped out in the attached density plot. By noting when the laser mode hops (at a given length of the cavity), we mapped out lines of isolength. They have slope of 1.25mK/V - this implies that a 1.25mK change in temperature of the laser yields the equivalent length change to a 1V change in the PZT voltage.

The PZT voltage has a range of 70V. Therefore, the PZT can compensate for a change in temperature of 87.5mK, or +/- 44mK.

Contamination Control Update

Contamination Control Supplies Area Reorganized:
The CC Supplies area in the entrance to LVEA has been re-organized and supplies clearly labeled. 
The heavy use supplies like wipes and foil are in the most obvious and easy to grab location. 
NOTE: The different types of pre-saturated and dry wipes are not interchangeable. General rule of thumb is only use the red pre-saturated or dry Vectra Alpha wipes on Class A parts. (Rest are for "gross cleaning" and floor mop.) The supply shelf is labeled with which one to use, if in doubt ask Jeff B. or Margot.   
The more advanced cc tools like the witness wafers, PET swipe tool, 1'' optics or Top Gun ionizers should only be used by experienced operators. If you think your work in the LVEA cleanrooms/chambers would benefit from using these, contact Jeff Bartlett or Margot Phelps first for assistance. 

Contamination Control work in HAM chambers with Jeff B.:
We are utilizing the new PET swipe tools to track surface contamination during install in HAM4-6. Yesterday Jeff B. took swipe samples of the tables in HAM4, HAM5 and HAM6. We will replicate this at the start of different in-chamber activities to track surface particulate through different install tasks. 
Also plan on chamber inspection and any necessary cleaning of these HAM chambers this week, whenever we can get in there to do so.    

OPS: reset of HEPI L4C Accumulated WD Counters Tuesday 29th April 2014

Reset WD counters on HAM3, HAM4, BS, & ETMX

ETM-X ESD Problems with Power Supplies

The ESD was non-functional this morning when we tried to operate it.  I went down to EX and found the negative supply in the off state (indicating a trip probably not someone shutting it off).  Turned it on went out to ESD drive to turn it on.  As soon as I turned on ESD amp the Neg. Power supply tripped off.  Went back out to power supply to see if I had not set the over current.  The unit was off again. This time it would not turn on.  Took this opportunity to swap out bot the positive and negative supplies.  The positive due to problems noted last week with sync. and the neg. do to this failure.  Power the system back up.  The system once again was in an odd state all of the supplies present but sitting at a neg rail.  I verified voltage everything was properly connected but still at a neg. rail.  This was the same state the unit was in on Friday when I had Kiwamu come out to help.  I disconnected the DAC inputs and reconnected them.  Everything came up just fine.  This is a very troubling state as the DAC was not sending out a voltage and the unit was behaving as though it was.

The unit is now running and seems to be working again.