Displaying reports 58701-58720 of 83002.Go to page Start 2932 2933 2934 2935 2936 2937 2938 2939 2940 End
Reports until 17:47, Thursday 14 January 2016
H1 SUS (CDS)
david.barker@LIGO.ORG - posted 17:47, Thursday 14 January 2016 - last comment - 15:42, Friday 15 January 2016(24950)
SUS watchdog resets stopped working this afternoon

Jeff K, Jeff B. , Jenne, TJ, Jim, Dave:

Around 4pm PST TJ reported that OMC had tripped and the watchdog could not be untripped. Jeff K. recommended a model restart. Unfortunately due to a communication problem we first mistakenly restarted the OMC model on the LSC front end (sorry OMC). Then we restarted the correct SUS-OMC model on SUSH56. This did not fix it. We then restarted all the models on SUSH56 (including the IOP). This did not fix it. We then stopped all models and only started IOP and SUS-SRM to do further debugging. (in the mean time the SWWD on the IOP had tripped SEI for HAM5 and HAM6). After some debugging we found that the PERL script sus/common/scripts/wdreset_all.pl was throwing an error about not finding the PERL CA LIBRARY. Jim tracked this down to a missing CaTools.pm perl module in the userapps/release/guardian directory. Turns out this file was removed from the SVN repository way back on 2nd March 2015 and the LHO working directory was only updated this afternoon by Jenne and TJ. This all nicely ties in with the watchdog resets working last night but not this afternoon.

In the mean time we had manually reset the watchdogs for SUS-SRM/SR3/OMC and SEI HAM5,6 and set the SDF back to OBSERVE for SUSH56IOP, SUSSRM/SR3/OMC and OMC.

For now we have manually copied the CaTools.pm file into userapps/release/sus/common/scripts to get the watchdog reset script working again. 

This raises an FRS:

A perl module which is used by the watchdog systems has been deprecated. The watchdog system should be changed to no longer use PERL and instead use PYTHON (or perhaps BASH for exceptionally simple scripts).

Comments related to this report
david.barker@LIGO.ORG - 17:51, Thursday 14 January 2016 (24951)
stuart.aston@LIGO.ORG - 07:06, Friday 15 January 2016 (24962)CDS
We experienced a seemingly identical occurrence of this issue at LLO last Wednesday (see LLO aLOG entry 24156). However, as well as the SUS/SEI Watchdog reset scripts our initial alignment script was also affected, since it has Perl dependencies.

It is still unknown how the symbolic-link to CaTools.pm became broken at LLO, see #4180.
thomas.shaffer@LIGO.ORG - 15:22, Friday 15 January 2016 (24974)

Stuart, it was broken because I updated the same the same folder when I was visiting LLO. I am at fault for both of these CaTools.pm links being broken at both sites, though I had no idea that simply updating the SVN could cause this.

stuart.aston@LIGO.ORG - 15:42, Friday 15 January 2016 (24975)
Thanks for shedding light on this mystery! I would suspect that svn'ing up pushed the changes to deprecate the Perl module sooner than was intended.
H1 INJ (DetChar, INJ)
adam.mullavey@LIGO.ORG - posted 17:02, Thursday 14 January 2016 - last comment - 17:54, Thursday 14 January 2016(24949)
More Burst (and some CBC) Injections schedule for tonight
Last nights burst injections were scheduled for times when H1 was down, so we're attempting to do these again. This time I've schedule 5 lots of, the first lot starting at 22:00 CT (06:00 UTC), spaced 2 hours apart, with the last lot starting at 06:00 CT (14:00 UTC). Each injection is spaced 20 minutes apart.

We're also including a BNS CBC injection in between the 5 lots of burst injections. All up there will be 30 injections, one every 20 minutes starting at 22:00 CT (06:00 UTC) and ending at 07:40 CT (15:40 UTC).

I'll also mention that transient hardware injections only go ahead when the IFO is in observation mode, so they won't interfere with any PEM measurements that may be happening at the time. 

Here is the updated schedule:

1136872817 2 1.0 burst_GPS_76.259_
1136874017 2 1.0 burst_GPS_76.262_
1136875217 2 1.0 burst_GPS_76.263_
1136876417 2 1.0 burst_GPS_76.264_
1136877617 2 1.0 burst_GPS_76.266_
1136878817 1 1.0 coherentbns1_1135135335_
1136880017 2 1.0 burst_GPS_76.259_
1136881217 2 1.0 burst_GPS_76.262_
1136882417 2 1.0 burst_GPS_76.263_
1136883617 2 1.0 burst_GPS_76.264_
1136884817 2 1.0 burst_GPS_76.266_
1136886017 1 1.0 coherentbns1_1135135335_
1136887217 2 1.0 burst_GPS_76.259_
1136888417 2 1.0 burst_GPS_76.262_
1136889617 2 1.0 burst_GPS_76.263_
1136890817 2 1.0 burst_GPS_76.264_
1136892017 2 1.0 burst_GPS_76.266_
1136893217 1 1.0 coherentbns1_1135135335_
1136894417 2 1.0 burst_GPS_76.259_
1136895617 2 1.0 burst_GPS_76.262_
1136896817 2 1.0 burst_GPS_76.263_
1136898017 2 1.0 burst_GPS_76.264_
1136899217 2 1.0 burst_GPS_76.266_
1136900417 1 1.0 coherentbns1_1135135335_
1136901617 2 1.0 burst_GPS_76.259_
1136902817 2 1.0 burst_GPS_76.262_
1136904017 2 1.0 burst_GPS_76.263_
1136905217 2 1.0 burst_GPS_76.264_
1136906417 2 1.0 burst_GPS_76.266_
1136907617 1 1.0 coherentbns1_1135135335_
Comments related to this report
evan.hall@LIGO.ORG - 17:54, Thursday 14 January 2016 (24952)

06:00 UTC = 00:00 CT = 22:00 PT

LHO General
thomas.shaffer@LIGO.ORG - posted 16:29, Thursday 14 January 2016 (24945)
Ops Day Shift Summary

TITLE: 1/14 Day Shift: 16:00-00:00UTC (08:00-16:00 PDT), all times posted in UTC"

STATE Of H1: Calibration on going

SHIFT SUMMARY: Calibration work for the majority of the day. Lockloss at 23:15 tripped the OMC WatchDog. It would not untrip so Dave had to restart the IOP, but this also tripped HAM5/6. Now SRM will not UNtrip. Dave and Jeff B are currently working to figure it out.

INCOMING OPERATOR: Jeff B

ACTIVITY LOG:

LHO General
thomas.shaffer@LIGO.ORG - posted 16:18, Thursday 14 January 2016 (24947)
Lockloss 23:15 UTC OMC WD Tripped

OMC WD would not UNtrip, Dave had to restart IOPSUSH56 to clear this. By restarting these, HAM5,6 also tripped.

We are currently untripping everything.

H1 DAQ (CDS)
james.batch@LIGO.ORG - posted 15:25, Thursday 14 January 2016 - last comment - 08:36, Friday 15 January 2016(24946)
Trend Writer disk array failure
The SSD RAID for h1tw0 has failed, so h1tw0 is down until we can get it fixed.

FRS 4228
Comments related to this report
corey.gray@LIGO.ORG - 08:36, Friday 15 January 2016 (24967)

For operators, you will notice that h1tw0 will be WHITE-boxed on the DAQ Detail medm (which you should be checking every shift in the Ops Checksheet).

LHO FMCS
john.worden@LIGO.ORG - posted 13:56, Thursday 14 January 2016 (24944)
LVEA temperatures

I reduced one heater, HC1B, from 10ma to 9ma control current at 1:47 local time.

H1 OpsInfo
thomas.shaffer@LIGO.ORG - posted 11:53, Thursday 14 January 2016 (24943)
Initial Alignment Tools

I have revamped my initial alignment lazy script and made a new medm to help operators and commissioners bring up what they need a bit faster.

The Script:

Location: /userapps/.../isc/h1/scripts/Init_align_lazy.py

Action: This will bring up 5 StripTools(XARM_GREEN_WFS.stp, YARM_GREEN_WFS.stp, PITCH_ASC_INITIAL_ALIGNMENT.stp, YAW_ASC_INITIAL_ALIGNMENT.stp, initall_alignment.stp) as well as the new medm I made (INIT_ALIGN.adl). The script will arrange them on the left monitor in a 3x2 grid. They can then be moved if so desired. If any of these windows are open in another workspace, the window manager will get confused and move the other one, so keep that in mind of they aren't in their proper positions.

I have aliased this in for "ops" as 'initial_alignment'

The medm:

Location: /userapps/.../isc/h1/medm/INIT_ALIGN.adl

Combines the ALIGN_IFO, X and Y ALS Guardians with the alignment sliders for ETMX, ETMY, ITMX, ITMY, BS, and PR3. These are the most commonly adjusted optics for IA.

I have not linked this medm off of the sitemap yet, but if/when I do it will most likely be under OPS.

 

Comments adn questions are always welcome.

Images attached to this report
H1 SEI (OpsInfo)
thomas.shaffer@LIGO.ORG - posted 11:24, Thursday 14 January 2016 (24942)
Changed the ISI ETMX X blend from 45 to 90

Jenne noticed the control room symptoms of the ETMX ISI starting to ring up (H1:IMC-F_OUT16 on our Tidal.stp will be gin to oscillate [shot attached] along with the ASC control signals). I brought up the template to watch it and sure enough, ISI ETMX X was ringing up. I switched the X direction blend to the 90mHz and it imediately settled down.

I'm attaching screen shots of the environment control room tools to hopefully help.

Images attached to this report
LHO General
thomas.shaffer@LIGO.ORG - posted 08:04, Thursday 14 January 2016 (24941)
Ops Day Transition

TITLE: 1/14 Day Shift 16:00-00:00UTC (08:00-16:00 PDT), all times posted in UTC"

STATE Of H1: Observing at 78Mpc for 2hrs

OUTGOING OPERATOR: Travis

QUICK SUMMARY: wind <10mph, useism 0.4um/s, CWinj running, More calibration measurements today to come.

H1 General
travis.sadecki@LIGO.ORG - posted 08:00, Thursday 14 January 2016 (24940)
OPS Owl shift summary

Title: 1/14 Owl Shift 8:00-16:00 UTC (0:00-8:00 PST).  All times in UTC.

State of H1: Observing

Shift Summary:  Unlocked at the beginning of my shift.  After an IA, it locked up pretty quickly.  Then there was an lockloss due to EQ.  After EQ ringdown, we stumbled through the CARM section a few times before getting back to NLN.

Incoming operator: TJ

Activity log:

9:27 Locked at NLN

9:30 cleared ETMx TIM error

9:32 Observing

10:06 Set to Commissioning to tweak TMS in an attempt to get POP power higher (it wasn't drifting down, just not as high as usual,15800).  Unsuccessful.

10:13 Observing

11:42 Lockloss.  EQ?

13:50 Locked NLN, attempting to tweak POP power again

14:08 Another unsuccessful POP attempt.  I did however manage to ring up the ASC loops like crazy!  Waiting for them to ring down.

14:21 Observing

H1 General
travis.sadecki@LIGO.ORG - posted 06:24, Thursday 14 January 2016 (24939)
Observing at 14:21 UTC

Finally back to Observing after a few failed attempts.

H1 General
travis.sadecki@LIGO.ORG - posted 03:47, Thursday 14 January 2016 (24938)
Lockloss 11:42 UTC

According to our seismometers, it appears to be an EQ.  The only thing showing on Terramon or USGS at the moment is a 5.1 near Wallis and Futuna with Terramon prediciting 0.17 um/s R-Wave.  However, our seismos are already at ~1 um/s a few minutes before the predicted arrival time for this EQ.

H1 General
travis.sadecki@LIGO.ORG - posted 01:34, Thursday 14 January 2016 (24937)
Observing at 9:32 UTC

After running through an initial alignment, we are back to Observing.  The issue with PRM align mentioned in Jeff's summary seems to have been due to PRM alignment being far off (hundreds of urads).  I restored them to earlier values and it locked without issue. 

H1 General
jeffrey.bartlett@LIGO.ORG - posted 00:01, Thursday 14 January 2016 (24936)
Ops Evening Shift Summary
Activity Log: All Times in UTC (PT)

00:00 (16:00) Take over from TJ
00:19 (16:19) Kyle – Going to Mid-Y to overfill CP3
00:22 (16:22) Robert & Vinny – Going to End-X
00:54 (16:54) Kyle – Back from Mid-Y
01:50 (17:50) Robert & Vinny – Back from End-X
02:39 (18:39) Robert – Swept LVEA
03:11 (19:11) Lockloss – Commissioning activities
03:25 (19:25) Mag 6.7 EQ near Japan – R-Wave 12.9um/s estimated arrival at 04:00 (20:00)
03:25 (19:25) Mag 6.0 EQ near Bolivia  
03:57 (19:57) Reset WD trips on BS ST1, ST2, and SUS
04:00 (20:00) Put IFO into DOWN until the ground settles down
08:00 (00:00) Turn over to Travis


End of Shift Summary:

Title:  01/13/2015, Evening Shift 00:00 – 08:00 (16:00 – 00:00) All times in UTC (PT)

Support: Jenne, Evan H.,   
 
Incoming Operator: Travis

Shift Detail Summary:  Hardware injections scheduled to start at 08:10 (00:10). 

   While doing an initial alignment after a commissioning lock, were hit with two earthquakes at same time (6.7 near Japan, 6.0 near Bolivia). Seismic has rung up to 5um/s. Reset WD trips on BS. Put the IFO in DOWN until the earth calms a bit.  

   Seismic calmed down enough to try initial alignment and locking. The initial alignment took longer than normal, but was mostly OK. Tried a couple of times but could not get PRM_ALIGN to move past REFL_WFS_CENTERING_PRX. I skipped this step. Trying to relock. 
H1 General
jeffrey.bartlett@LIGO.ORG - posted 20:17, Wednesday 13 January 2016 (24935)
Lockloss and Earthquakes
   Evening Mid-Shift Summary
   
   While doing an initial alignment after a commissioning lock; hit with two earthquakes at same time (6.7 near Japan, 6.0 near Bolivia). Seismic has rung up to 5um/s. Reset WD trips on BS. Put the IFO in DOWN until the earth calms a bit. 

    
H1 General
jeffrey.bartlett@LIGO.ORG - posted 18:40, Wednesday 13 January 2016 (24934)
LVEA Sweep
   Robert did a sweep of the LVEA. 
LHO VE
kyle.ryan@LIGO.ORG - posted 16:58, Wednesday 13 January 2016 (24932)
Manually over-filled CP3
1615 - 1650 hrs. local -> To and from Y-mid 

Next CP3 manual over fill scheduled for Friday, Jan. 15th before 4:00 pm
H1 General
jeffrey.bartlett@LIGO.ORG - posted 16:20, Wednesday 13 January 2016 (24930)
Ops Evening Shifdt Transition
     Transition Summary:
Title:  01/13/2016, Evening Shift 00:00 – 08:00 (16:00 – 00:00) All times in UTC (PT)
	
State of H1: IFO locked in Commissioning mode for calibration work.  

Outgoing Operator: TJ

LHO General
thomas.shaffer@LIGO.ORG - posted 16:09, Wednesday 13 January 2016 (24929)
Ops Day Shift Summary

TITLE: "1/13 Day shift: 16:00-00:00UTC (08:00-16:00 PDT), all times posted in UTC"

STATE Of H1: Nominal Lo Noise, Calibration ongoing

SHIFT SUMMARY: Locked for entire shift while Robert and commissioners did their calibration measurements.

INCOMING OPERATOR: Jeff B

ACTIVITY LOG:

X1 DTS
jonathan.hanks@LIGO.ORG - posted 17:13, Tuesday 12 January 2016 - last comment - 13:38, Friday 15 January 2016(24908)
Bad power supplies on x1seiex io chassis

While investigating an issue with x1seibsctim04 Jim Batch and I noticed that the power supply on the x1seiex IO chassis was dead.  We replaced the power supply with a spare unit from the self.  The new power supply appears to be getting 5v (some leds where lit), but it would not turn on (the fans would start to spin up and then stop).

Further investigation is needed.  For now x1seiex has been disconnected from the dolphin network and turned off.

Comments related to this report
jonathan.hanks@LIGO.ORG - 16:24, Wednesday 13 January 2016 (24931)

FRS 4218

https://services.ligo-la.caltech.edu/FRS/show_bug.cgi?id=4218

jonathan.hanks@LIGO.ORG - 13:38, Friday 15 January 2016 (24971)
From the FRS:

--- Comment #1 from richard.mccarthy@LIGO.ORG ---
Turns out the an ADC interface board had a short dragging the power supply
down.
Replaced the board and returned the unit to service.
Displaying reports 58701-58720 of 83002.Go to page Start 2932 2933 2934 2935 2936 2937 2938 2939 2940 End