Displaying reports 62121-62140 of 85644.Go to page Start 3103 3104 3105 3106 3107 3108 3109 3110 3111 End
Reports until 16:00, Saturday 12 December 2015
LHO General
thomas.shaffer@LIGO.ORG - posted 16:00, Saturday 12 December 2015 (24158)
Ops Day Shift Summary
LHO General
thomas.shaffer@LIGO.ORG - posted 15:02, Saturday 12 December 2015 (24157)
Went out of Observing for 15min for PEM inj

A simple in and out for 15min, but the low frequency noise in DARM is still affecting us.

LHO General
thomas.shaffer@LIGO.ORG - posted 14:11, Saturday 12 December 2015 (24153)
Observing 22:08

There is still some noise seen in DARM <70Hz, but not as bad as before. I am also not seeing the oscillations like before, or at least not yet. Something is not healthy though...

Range: 71Mpc

LHO General
thomas.shaffer@LIGO.ORG - posted 13:34, Saturday 12 December 2015 - last comment - 14:52, Saturday 12 December 2015(24151)
Lockloss 21:32

H1:LSC-POP_A_LF_OUTPUT started to get a few hz oscillation in it, and the noise in the DARM spectrum < 100Hz picked up a bit.

Comments related to this report
thomas.shaffer@LIGO.ORG - 13:52, Saturday 12 December 2015 (24152)Lockloss

SRC2 P also had this same oscillation, and PR2 and SRM looks like they were affected by it.

Images attached to this comment
thomas.shaffer@LIGO.ORG - 14:52, Saturday 12 December 2015 (24155)

Vern told me to keep an eye on the BS Oplev, so here is a quick trend of that with some ASAIR powers. I am not sure if it is a glitch seem in the BS or if it is just from the lockloss itself.

Images attached to this comment
LHO General
thomas.shaffer@LIGO.ORG - posted 12:56, Saturday 12 December 2015 (24150)
Observing 20:52

Had to run through an initial alignment, and then lost it a few times on the way up. The wind is still staying below 20mph, the useism is still trending down, CW inj running, lights are off, LLO is down.

H1 PSL
thomas.shaffer@LIGO.ORG - posted 10:41, Saturday 12 December 2015 (24149)
Added 200mL to Diode Chiller bottom

I was having issue keeping the IMC locked, I looked at the PLS status and the chiller alarm was red. I went in to check it out and the bottom of the chiller was in error with "Low Level", the top seemed to be hovering around the Max line. I am not sure if the bottom is seperate from the top, but I added 200mL of water to the bottom. Alarms all good now.

LHO General
thomas.shaffer@LIGO.ORG - posted 10:13, Saturday 12 December 2015 - last comment - 14:54, Saturday 12 December 2015(24148)
Lockloss 18:09 UTC

Not sure of the cause yet, the control signals didn't show any signs of struggle, winds <20mph, useism 0.8um/s.

Comments related to this report
thomas.shaffer@LIGO.ORG - 14:20, Saturday 12 December 2015 (24154)

Generic Lockloss tool set of plots, not showing anything useful (at least not to me).

Images attached to this comment
thomas.shaffer@LIGO.ORG - 14:54, Saturday 12 December 2015 (24156)

Vern told me to keep an eye on the BS Oplev, so here are some plots with the ASAIR powers and the Oplev sum out. Not sure which one caused the other one, if any.

Images attached to this comment
LHO General
thomas.shaffer@LIGO.ORG - posted 09:22, Saturday 12 December 2015 - last comment - 10:00, Saturday 12 December 2015(24146)
Out of Observing for inj

LLO is down due to high wind, Robert is jumping on this opportunity to do some PEM injections.

Comments related to this report
thomas.shaffer@LIGO.ORG - 10:00, Saturday 12 December 2015 (24147)

Back to Observing for a bit. Robert wants to get another test ready, in the mean time we will observe. LLO is still down and out from wind.

LHO General
thomas.shaffer@LIGO.ORG - posted 08:05, Saturday 12 December 2015 (24143)
Ops Day Transition
H1 General
travis.sadecki@LIGO.ORG - posted 08:00, Saturday 12 December 2015 (24142)
OPS Owl shift summary

Title: 12/12 Owl Shift 8:00-16:00 UTC (0:00-8:00 PST).  All times in UTC.

State of H1: Observing

Shift Summary: Locked for my entire shift, 12+ hours total.  A handful of ETMy saturations.  Microseism is slowly coming down, currently ~0.6 um/s.  Wind is picking up a bit to ~15 mph.  There is a H1SUSETMX timing error on the CDS overview screen that can be cleared next time we are out of Observing.

Incoming operator: TJ

Activity log: None

H1 General
travis.sadecki@LIGO.ORG - posted 04:21, Saturday 12 December 2015 (24141)
OPS Owl mid-shift summary

Locked in Observing for ~9 hours.  A couple of ETMy saturations.  No other issues.

H1 General (CDS)
travis.sadecki@LIGO.ORG - posted 00:37, Saturday 12 December 2015 - last comment - 08:46, Saturday 12 December 2015(24139)
nfs server cdsfs0:/ligo errors

I noticed a couple of errors that showed up in the VerbalAlarms window running on the Alarm Handler computer.  These were not announced verbally by VerbalAlarms.  See attached screenshot.

Images attached to this report
Comments related to this report
travis.sadecki@LIGO.ORG - 02:33, Saturday 12 December 2015 (24140)

There were 2 more of these errors (one set of not responding/alive again) ~10:00 UTC.

david.barker@LIGO.ORG - 08:36, Saturday 12 December 2015 (24144)

I see three errors on cdsfs0 for this morning which resulted in the raid card being reset.

Dec 12 00:17:51 cdsfs0 kernel: [306069.208085] sd 0:0:0:0: WARNING: (0x06:0x002C): Command (0x8a) timed out, resetting card.

Dec 12 01:23:42 cdsfs0 kernel: [310009.835370] sd 0:0:0:0: WARNING: (0x06:0x002C): Command (0x2a) timed out, resetting card.

Dec 12 02:57:34 cdsfs0 kernel: [315627.297842] sd 0:0:0:0: WARNING: (0x06:0x002C): Command (0x8a) timed out, resetting card.

 

Here is a full set of logs for the 01:23 event:

 

Dec 12 01:23:42 cdsfs0 kernel: [310009.835370] sd 0:0:0:0: WARNING: (0x06:0x002C): Command (0x2a) timed out, resetting card.

Dec 12 01:23:45 cdsfs0 snmpd[1716]: Connection from UDP: [10.20.0.85]:53320->[10.20.0.11]

Dec 12 01:24:13 cdsfs0 kernel: [310040.313376] 3w-9xxx: scsi0: AEN: INFO (0x04:0x005E): Cache synchronization completed:unit=0.

Dec 12 01:24:13 cdsfs0 kernel: [310040.433071] 3w-9xxx: scsi0: AEN: INFO (0x04:0x0063): Enclosure added:encl=0.

Dec 12 01:24:45 cdsfs0 snmpd[1716]: Connection from UDP: [10.20.0.85]:54550->[10.20.0.11]

Dec 12 01:25:01 cdsfs0 CRON[13730]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1)

Dec 12 01:25:03 cdsfs0 kernel: [310090.356268] 3w-9xxx: scsi0: AEN: INFO (0x04:0x0029): Verify started:unit=0.

Dec 12 01:25:03 cdsfs0 kernel: [310090.385280] 3w-9xxx: scsi0: AEN: INFO (0x04:0x0029): Verify started:unit=1.

Dec 12 01:25:03 cdsfs0 kernel: [310090.385714] 3w-9xxx: scsi0: AEN: INFO (0x04:0x0029): Verify started:unit=2.

david.barker@LIGO.ORG - 08:46, Saturday 12 December 2015 (24145)

Note that this problem is not related to the recent /ligo is 100% full error (it has about 220GB of disk free). 

Looks like most NFS clients rode through these restarts without logging any system errors. Presumably they were short outages and perhaps only computers which were actively trying to access the file system at the time are reporting the error.

Please contact me over the weekend if the errors show up again.

Displaying reports 62121-62140 of 85644.Go to page Start 3103 3104 3105 3106 3107 3108 3109 3110 3111 End