Displaying reports 57641-57660 of 78012.Go to page Start 2879 2880 2881 2882 2883 2884 2885 2886 2887 End
Reports until 08:37, Wednesday 26 August 2015
H1 CDS (DAQ)
david.barker@LIGO.ORG - posted 08:37, Wednesday 26 August 2015 (20912)
CDS model and DAQ restart report, Tuesday 25th August 2015

ER8 Day 9. Maintenance Morning

model restarts logged for Tue 25/Aug/2015
2015_08_25 11:06 h1oaf
2015_08_25 11:17 h1oaf
2015_08_25 11:19 h1odcmaster
2015_08_25 11:20 h1lsc
2015_08_25 11:22 h1asc
2015_08_25 11:22 h1calex

2015_08_25 11:26 h1broadcast0
2015_08_25 11:26 h1dc0
2015_08_25 11:26 h1fw0
2015_08_25 11:26 h1fw1
2015_08_25 11:26 h1nds0
2015_08_25 11:26 h1nds1

2015_08_25 23:17 h1susprm
2015_08_25 23:22 h1lsc
2015_08_25 23:24 h1asc
2015_08_25 23:28 h1suspr2

No unexpected restarts. Model changes followed by supporting DAQ restart. Later model restarts in attempt to fix n+0.5 Hz noise comb.

H1 General
edmond.merilh@LIGO.ORG - posted 07:59, Wednesday 26 August 2015 (20907)
Shift Summary - Owl

ALL TIMES IN UTC

Post-Maintenance: IFO is actively locking; caveat-> excess noise that wasn’t present prior to maintenance (1Hz comb and some high freq). Calibration going on. Wind calm. Seismic calm. Patrick on Evening duty.

Activity Log: Not much activity. Folks here from the day eventually left.

LOCK LOG:

Summary:

H1 PSL (DetChar)
edmond.merilh@LIGO.ORG - posted 07:44, Wednesday 26 August 2015 (20911)
PSL Weekly Report - Past 10 day Trends
Images attached to this report
H1 General (DetChar)
edmond.merilh@LIGO.ORG - posted 07:03, Wednesday 26 August 2015 (20909)
Observation Intent Bit/ Observatory mode

Set OIB/OOM to Undisturbed/Observing after noise successfully removed from PRCL. 

H1 General (DetChar)
edmond.merilh@LIGO.ORG - posted 07:00, Wednesday 26 August 2015 (20908)
Observation Intent Bit/ Observatory mode

Set OIB/OOM to Commissioning/Maintenance for Richard to go into CER and check cables for 1Hz comb in PRCL

H1 General
edmond.merilh@LIGO.ORG - posted 04:29, Wednesday 26 August 2015 - last comment - 04:39, Wednesday 26 August 2015(20902)
Mid-Shift Update - Owl

Wind calm. Sharp rise in ITMY_Z_BLRMS_30M_100M possibly due to 4.7mag in Central America an hour ago but it doesn't seem to be fazing us at all. IFO still locked and in good shape. In fact, in the last half hour it appears that the range may have taken a very slght upward trend from 60 to 62Mpc. This segment has been glitch free, so far. We've had about 3 hours of coincidental locking with LLO. Steady as she goes. Full steam ahead.

Comments related to this report
edmond.merilh@LIGO.ORG - 04:39, Wednesday 26 August 2015 (20903)

11:38UTC Glitched down to 36Mpc. It doen't appear to be related to the typical ETMY saturation. Should have kept quiet about it!

H1 General (DetChar)
edmond.merilh@LIGO.ORG - posted 01:43, Wednesday 26 August 2015 - last comment - 01:56, Wednesday 26 August 2015(20896)
Observation Intent Bit/ Observing Mode

Set at 8:43UTC. 60Mpc

Comments related to this report
edmond.merilh@LIGO.ORG - 01:56, Wednesday 26 August 2015 (20898)

The ODCMASTER in SDF will show 1 diff during this segment due to a change that had to be made that was blocking the Observation Ready bit to Green and keeping he INtent bit from being set. CALCS will also show diffs during this segment. Commissioners aren't certain what these mean.

H1 ISC
jenne.driggers@LIGO.ORG - posted 01:23, Wednesday 26 August 2015 - last comment - 15:20, Wednesday 26 August 2015(20895)
1Hz comb is everywhere

[Sheila, Dan, Evan, Jenne, others]

All afternoon (basically, since we started relocking after maintenence day) we have seen a 1 Hz comb (offset by 0.5 Hz) in the DARM spectrum, as well as in many, many other places.  This seems like the same thing as the final bullet point that was reported in aLog 20790 a few days ago, but with much taller peaks.

Much of the day was spent looking into this, by several people.  The lines are very clear in all of the length error signals, as well as the ASC error signals. 

One thing that we tried, just to exonerate it, was swap the PRC length actuation from the PRM to PR2, and then turn off the PRM M3 coil driver.  As expected, PR2 affects the cavity length twice as much as the PRM, so we needed a gain of 0.5 in the M3 LOCK filter bank for PR2, rather than the 1 that PRM has.  We were able to turn off the PRM M3 coil driver, but saw no change in the length signals' spectra.  We put the actuation back on PRM, but when we tried to turn off the PR2 coil driver, PR2 tripped and we lost lock.  So, the new PRM coil driver is certainly not to blame for this new noise.  (PR2 was untripped, and its coil driver turned back on)  Also, we haven't seen the glitches due to the old PRM coil driver at all today, so hopefully they're gone for good, and not just hidden.

At Richard's suggestion, we tried restarting the models that were restarted earlier today in case it is some kind of synchronization problem.  We restarted h1prm, h1pr2, h1lsc, h1asc.  This didn't change anything - the lines are still there loud and clear after relocking. 

The 1 Hz comb is still present, and we are running low on ideas for what it could be, and what to do about it.  DetChar friends, and anyone with ideas, please let us know if you have a suggestion of where to look.

--------------

I don't know if this has the same cause as the 1Hz comb, but there are also some unusually large lines at ~838 Hz and ~855 Hz.  These lines are a few Hz wide, and are coherent with magnetometer signals.  But, the magnetometers have seen these lines at the same (or larger) amplitude, in the past, without seeing the lines in DARM.  Why would the coupling from magnetic fields have changed so drastically??

Comments related to this report
daniel.hoak@LIGO.ORG - 02:56, Wednesday 26 August 2015 (20900)DetChar

Tagging DetChar so they're aware of this entry.

The various noise features that appeared after maintenance today -- the 1Hz comb below 70Hz, some broadband noise above 80Hz, and the peaks around 850Hz -- are all coherent with the DRMI length error signals, and are much louder (compared to quiet references) in PRCL and MICH.  Something is different in the corner station...hopefully all three things have the same source.

keith.riles@LIGO.ORG - 05:40, Wednesday 26 August 2015 (20906)DetChar
The 1-Hz comb with 0.5-Hz offset is prevalent in magnetometer channels.
Looking at a typical magnetometer FSscan spectrogram is not for the faint
of heart, but there are plenty of examples to be found here.

One example from EY_MAG_EBAY_SUSRACK_Y is attached. It's not unusual.

Images attached to this comment
richard.mccarthy@LIGO.ORG - 07:04, Wednesday 26 August 2015 (20910)
Since the reports stated it shows up in PEM channels I disconnected the LO to the antenna Demod board that resides in the PEM rack and was coincidentally connected yesterday and the comb disappeared.  WE need to verify the RF level needed for these radio signals.  Pick some attenuators and re-installed the LO signal.  I put terminators back on the outputs from the distribution chassis.
daniel.sigg@LIGO.ORG - 09:34, Wednesday 26 August 2015 (20917)

This could also have been due to a ground loop. A balun should be added to the lines.

aidan.brooks@LIGO.ORG - 10:25, Wednesday 26 August 2015 (20919)

I thought this might be the ITM HWS. It looks like the filter on the power supply (which is filter out the 1Hz comb) was turned off yesterday.

evan.hall@LIGO.ORG - 15:20, Wednesday 26 August 2015 (20924)

A small point about the PRM/PR2 experiment: when we switched to PR2, there was a huge 45 Hz resonance (visible in DARM and the vertex LSC signals) that was rung up. It was distinct from the usual 41 Hz HSTS roll mode resonance that we usually see when passing through the noise tuning step in full lock. It did not ring down on its own.

We made sure that the PRCL OLTF did not change before and after the swap, and anyway the loop appeared to have enough phase (more than 30°) at this frequency, so I don't think it was gain peaking. By increasing the loop gain, we were able to bring the resonance down a little bit. I then added 4 dB of resonant gain around 45 Hz, which somehow squashed the resonance completely. (I put it in the PR2 M3 lock L SFM).

It's not clear to me what was going on here.

H1 INJ (INJ)
eric.thrane@LIGO.ORG - posted 22:57, Tuesday 25 August 2015 - last comment - 17:17, Thursday 27 August 2015(20893)
CBC Injection Test For Saturation + First Blind Injection Test
Patrick, Sheila, Jenne, Eric

For the first part of the test, we injected our fiducial CBC waveform (same one used in ER7) and tried raising the LIMIT value on the hardware injection block in order to address saturation problems observed in ER7. During ER7, the LIMIT was 200. We raised it to 400. The first injection did not go through:

1124601535 1 1.000000 cbctest_1117582888_		intent bit off, injection canceled

Patrick, Sheila, and Jenne tried to turn on the intent bit, but there was some sort of problem, which will be alog'ged separately. As a temporary work-around, we turned off the tinj intent-bit check and injected again:

1124602724 1 1.000000 cbctest_1117582888_		successful

Patrick determined that the injection produced a maximum |amplitude| of 15 counts coming out of the injection block, which seemed to indicate that the original LIMIT value of 200 was sufficient. However, an alarm went off to indicate that there was saturation at ETMY. Thus, the saturation problem cannot be solved by tinkering with the INJ block in MEDM. Rather, the problem is occurring downstream on the ETM actuators. We request that Jeff K, Adam M, et al. look into options for avoiding saturation at the ETMs.

Next we tried a blind injection using the new blind injection code. The blind injection code does not log injections in EPICS so they are not automatically picked up in the segment database.

1124603111 1 1.000000 cbctest_1117582888_		successful

The blind injection was clearly visible. The ETM saturation warning went off again. The injection was logged correctly in the blind injection blindinj_H1.log:

current time = 1124603049...
Attempting: awgstream H1:CAL-INJ_BLIND_EXC 16384 /ligo/home/eric.thrane/O1/Hardw
areInjection/Details/Inspiral/H1/cbctest_1117582888_H1.out 1 1124603111
Injection successful.

All of these injections were carried out with scale factor = 1; (that's the 1.000000). The injection file, described in a comment below, is a 1.4 on 1.4 BNS, optimal orientation, at D=45 Mpc. It is the same waveform used in previous ER tests.
Comments related to this report
andrew.lundgren@LIGO.ORG - 00:49, Wednesday 26 August 2015 (20894)DetChar, INJ
It looks like the injection actually does hit the 400 count limit (plot 1). It saturates right at the end when the injection chirps up to high frequency. There's some kind of ringing as well (plot 2). From the spectrogram (plot 3) and the zoom (plot 4) this looks like a feature at just above 300 Hz. I thought it might be a notch for the PCal line, but that's 331.9 Hz. So someone will have to check the inverse actuation filter and see what's happening at that frequency.

It's possible to see the overflow from the first injection in the ETMY L3 MASTER channel (plot 5). It happens at -131072 counts, and the injection is trying to push it past -200000. The blind injection caused an overflow as well, but since this channel is only recorded at 2048 Hz, it looks like it falls short of overflow (plot 6). There's a faster readback whose name escapes me at the moment.

Unless the blind injection is made a factor of about 10 smaller, or rolled off at high frequency, it will be trivial to detect it by looking at the drive to the ETM.
Images attached to this comment
eric.thrane@LIGO.ORG - 04:24, Wednesday 26 August 2015 (20901)INJ
FYI, the injected waveform was fiducial waveform from ER7:

https://alog.ligo-la.caltech.edu/aLOG/index.php?callRep=16125

It's a 1.4-1.4 BNS at 45 Mpc, optimal orientation.
duncan.brown@LIGO.ORG - 05:06, Wednesday 26 August 2015 (20904)

There are a couple of things to watch out for when performing CBC hardware injections, based on iLIGO experience:

  • In iLIGO, the actuation function had a notch at the violin-mode frequencies. If you just invert A(f) without smoothing this out, the notch will get inverted and you'll give the the mirrors a hard kick as the inspiral sweeps through the notch frequency. (My guess is that Adam and Jeff have already dealt with this, though.)
  •  If the end of the inspiral signal is not smoothly rolled off and the signal drops from some loudish amplitude to zero then you put a step function in. This caused the iLIGO test mass to swing at its pendulum frequency, putting a huge ringdown in the data (called the "whooper" in iLIGO). In iLIGO, the CBC group generated waveforms in counts, by applying 1/A(f) ourselves using the adiabatic inverse calibration, so we smoothed 1/A(f) ourselves before applying it. In aLIGO, 1/A(f) is: (i) more complicated, and (ii) applied in the front end by IIR filters. It is possible that the sharp turnoff of the CBC injection at the end is causing the filters that apply 1/A(f) to ring and that's what's giving us garbage at the end of the signal.

For the ER7 injection we used an SEOBNRv2 waveform that has a ringdown at the end, hoping that this turn off would not trigger an impulse. However, for BNS masses, the turn off and ringdown is pretty sharp. I've asked Chris check that there are no "whooper" effects with the SEOBNRv2 waveform, but we haven't had chance to do this yet. For a SpinTaylorT4 waveform (the other waveform CBC wants to inject), there will definitely be a step, so this needs to be checked and rolled off carefully.

duncan.brown@LIGO.ORG - 05:08, Wednesday 26 August 2015 (20905)

One other comment on the test: what scaling in awgstream did you use? That waveform looks monstously loud (eyeball SNR > 20). That's much louder than would be useful for a blind injections, but good for helping us find whooper effects.

eric.thrane@LIGO.ORG - 16:44, Wednesday 26 August 2015 (20928)INJ
Duncan, the scale factor is 1.
andrew.lundgren@LIGO.ORG - 14:27, Thursday 27 August 2015 (20959)DetChar, INJ
Just for completeness, because I didn't see it posted, here's an Omega scan of the injections in h(t). The first is the non-blind injection, the second is the blind injection. I think the glitch ten seconds after the blind injection is unrelated. I thought it might be a filter turning off or being reset, but it's not on a GPS second (it's at 1124603210.28). It does cause an overflow of the ETMY ESD DAC.
Images attached to this comment
peter.shawhan@LIGO.ORG - 17:17, Thursday 27 August 2015 (20966)INJ
I verified that the blind injection was correctly recorded in the raw frame file.
H1 SYS (CDS, DetChar)
sheila.dwyer@LIGO.ORG - posted 22:56, Tuesday 25 August 2015 - last comment - 09:32, Wednesday 26 August 2015(20892)
unable to set intent bit

Sheila, Patrick, Jenne, Eric Thrane on the phone.  

Eric wanted to do an injection, and since his code requires the intent bit to be set, Patrick attempted to set it.  This however proved impossible.  This is probably related to the work described in 20870.  According to the CDS overview, there are no excitations on except those from calibration, which will need to be on while the intent bit is set.  (screenshots attached of the CDS overview and some ODC screens). 

Side note (sheila's opinion only):

I found the ODC screens confusing to navigate, and it is very unclear to me which bits go into calculating other bits to find out what the problem is.  It seems to me like it is a bad idea for us to make ourselves dependent on ODC to set the intent bit, since people on site don't have control over it.  

Images attached to this report
Comments related to this report
sheila.dwyer@LIGO.ORG - 01:44, Wednesday 26 August 2015 (20897)

Evan deciphered the ODC screen that was apparently telling us that TCS was not observation ready.  We put a 0 in the   H1:ODC-MASTER_TCS_2_MASK, and this made the observation bit ready, so Ed has set the intent bit.  

ryan.fisher@LIGO.ORG - 08:49, Wednesday 26 August 2015 (20913)
I think this has uncovered a bug/issue:  There may be a problem with the connection between the TCS ODC and the ODC-MASTER models at LHO.  I'm debugging it now and will reply with what needs to be fixed.  

On a separate note:  I believe site leads (at least Fred) said that operators would be trained on how to read and use the ODC with the changes made in https://alog.ligo-wa.caltech.edu/aLOG/index.php?callRep=20870 and Duncan M. and Stefan B. are the experts on site to teach people how to read this screen, but I'll walk through the way to read it for this example:

Evan correctly read off the bit that was preventing obs-ready from being green from the ODC MASTER overview screen.  To do this, you just trace across the row that you care about (OBS-READY) and find the red bit, then count across the top list of subsystems to find which subsystem that is coming from (I would appreciate it if anyone can show us how to turn those text boxes so we can align them with the columns better).  The next step would be to look down one step further by clicking on the TCS button to see that the 30th bit of the TCS ODC is the only bit that can affect the OBS-READY bit.  From the comment, this was done correctly as well, and you can read off the bitmask for the relevant row in the matrix on the right.   You can also see this by looking at the large black/green matrix to find the green bits that affect each row, which tell you which bits of the subsystem ODC are mapped to which row in ODC MASTER.  At this point, you will see that the whole TCS ODC at the top of that screen is red, however, which I believe is a bug somewhere in the FE code.  If it were behaving normally and you wanted to learn what the bits meant in the vector at the top:  The last step would be to click on the relevant subsystem from the ODC-SITE-OVERVIEW screen, and in this case you would learn that bit 30 is the Excitation bit, as expected.   
ryan.fisher@LIGO.ORG - 09:23, Wednesday 26 August 2015 (20915)
I have found the issue:  The h1odcmaster.mdl was locally edited and then committed to SVN on Aug. 18, as recorded in alog:  https://alog.ligo-wa.caltech.edu/aLOG/index.php?callRep=20622

This occured after we had circulated the new h1odcmaster.mdl that was used for alog: https://alog.ligo-wa.caltech.edu/aLOG/index.php?callRep=20870.

Duncan M. is going to ensure that everything is up to date with the SVN, then re-implement the changes he made on Aug. 18 in a copy of h1odcmaster.mdl and then generate the ECR to fix the bug.

For the time being, I have set all the bitmasks of form H1:ODC-MASTER_TCS_XX_MASK to 0x0 so the bad connection cannot affect ODC-MASTER.
jameson.rollins@LIGO.ORG - 09:32, Wednesday 26 August 2015 (20916)

Ryan, this does not adequately explain the situation. Why does the bit mask in TCS affect READY? We were told that READY was just GRD-IFO_OK and NO EXC. How was it that TCS was somehow wired into that logic?

H1 SUS (ISC, SUS, SYS)
richard.mccarthy@LIGO.ORG - posted 12:38, Tuesday 25 August 2015 - last comment - 09:09, Wednesday 26 August 2015(20865)
PRM M3 LL coil investigation
Following up on Evan's investigation into the glitches from LL coil on the PRM M3.   I tried to monitor the output with a scope attached.  I was not looking for very big signals as the the noisemon indicated roughly 30mV signal and it has some gain associated with it. So I connected to scope probes to the LL coil drive signal and looked at it on a scope with a trigger set fairly low.(10s of milivolts).  I also looked at UL just for comparison.   I was able to get the signal to trigger on some events  on LL that were not seen at as high a level on UL.  This was with the DAC both connected and disconnected with small offset (20 counts) applied and a large offset (20,000 counts) applied.  After a couple of hours of investigation I decided there were some glitches present and seemed to be from the coil driver not the DAC.  I also looked at the DAC input to the coil driver and did not see the glitches.  In the end I swapped out the coil driver.  I am not 100% certain this will fix the problem as it was difficult to verify the glitches were present at all times.

Comments related to this report
filiberto.clara@LIGO.ORG - 09:09, Wednesday 26 August 2015 (20914)
Unit removed S1100045.
Unit installed S1100025.
H1 SUS
jeffrey.kissel@LIGO.ORG - posted 23:07, Monday 24 August 2015 - last comment - 02:29, Wednesday 26 August 2015(20848)
QUAD Overview Screen Rocker Switch Death Bug Fixed
J. Kissel, P. Thomas

Given that we've had so many examples of rocker switch death this evening (see LHO aLOG 20839), I was able to identify that my new(ish) warning message about this failure mode on the MEDM overview screen (see LHO aLOG 20281) had a bug. The MEDM logic was watching L2 for all three stages, L2, L1, and M0. I've committed the bug-fix to the SVN such that LLO can receive its benefits.
Comments related to this report
corey.gray@LIGO.ORG - 10:47, Tuesday 25 August 2015 (20861)

What is the MEDM Overview Screen?  Is this the overall QUAD Overview?  If so, where is the new(ish) warning message?  Are you talking about the Guardian message window (upper right corner) on the QUAD overview?

evan.hall@LIGO.ORG - 02:29, Wednesday 26 August 2015 (20899)

On the top-level screen for each quad, you should see a red rectangle appear around the top, UIM, or PUM coil output filters that says "rocker switch death", as in this screenshot from Jeff's alog.

Displaying reports 57641-57660 of 78012.Go to page Start 2879 2880 2881 2882 2883 2884 2885 2886 2887 End