Displaying reports 20381-20400 of 87736.Go to page Start 1016 1017 1018 1019 1020 1021 1022 1023 1024 End
Reports until 09:46, Monday 08 May 2023
H1 SQZ
camilla.compton@LIGO.ORG - posted 09:46, Monday 08 May 2023 - last comment - 13:43, Monday 08 May 2023(69395)
30 minutes of no SQZ time this Morning

Austin and I took SQZ_MANAGER from FREQ_DEP_SQZ to SQZ_READY_IFO for 30 minutes of no-squeezing time 16:02UTC to 16:34UTC. 

The IFO had been locked for 18 hours. Range dropped from 133MPc down to 113MPc. Both of these range values seem lower than earlier in the week but there was a lot of changes on Friday (69365). 

Images attached to this report
Comments related to this report
victoriaa.xu@LIGO.ORG - 13:43, Monday 08 May 2023 (69417)
Link

Just checking -- and it seems not a huge difference in squeezing levels over ~10 min with squeezer ASC's ON/OFF. Low frequency differences likely just due to noise technical noise drifts. GPS times:
FDS FC+SQZ ASC OFF (blue): 1367613320 + 10 min
FDS FC+SQZ ASC ON (black) 1367612238 + 15 min

Images attached to this comment
H1 SEI
austin.jennings@LIGO.ORG - posted 08:37, Monday 08 May 2023 (69394)
H1 ISI CPS Noise Spectra Check - Weekly

Closes FAMIS 19657

BSC CPS:

ETMY_ST2_CPSINF_V2 looks slightly elevated

ITMX_ST1/2_CPSINF_V1 looks slightly elevated

HAM CPS:

All looks nominal

Images attached to this report
H1 General
austin.jennings@LIGO.ORG - posted 07:58, Monday 08 May 2023 (69392)
Ops Day Shift Start

TITLE: 05/08 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Observing at 131Mpc
CURRENT ENVIRONMENT:
    SEI_ENV state: CALM
    Wind: 2mph Gusts, 1mph 5min avg
    Primary useism: 0.02 μm/s
    Secondary useism: 0.10 μm/s 
QUICK SUMMARY:

- IFO has been locked and in OBSERVING (17 hours!)

- CDS/FOMs/DMs ok

- Slight spike in ground motion at EY (prob ongoing wind fence work) - I have disabled sensor correction at EY

H1 General
austin.jennings@LIGO.ORG - posted 16:01, Sunday 07 May 2023 (69387)
Sunday Operator Summary

TITLE: 05/07 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Observing at 127 Mpc
SHIFT SUMMARY:

Main Notes:

Lock #1:

- Acquired NLN @ 15:33

- PI 31 ringup @ 15:39, PI 24 @ 17:25/17:44/18:17 - no intervention needed

- IN OBSERVING: 15:57, taken out @ 17:03 by VAC CP1 channel - Screenshot attached, back to OBSERVING @ 17:08

- EX saturations @ 17:16/20:03

- LOCKLOSS @ 20:34 UTC - Unsure what caused this one, could have been an ASC ringup, but could have also been due to an uptick in local ground motion at the end stations - ASC plot attached

Lock #2:

- Issues w/ DRMI - moved PRM P from -1517.1 to -1506.1, Y from 354.0 to 329.4, BS P from 90.95 to 90.62, Y from -381.78 to -381.32

- Acquired NLN @ 21:56, in OBSERVING @ 22:20

- EX saturation @ 22:07
LOG:

No log for this shift.

Images attached to this report
H1 ISC
austin.jennings@LIGO.ORG - posted 15:38, Sunday 07 May 2023 (69379)
ADS Signals Diverging in Early Acquisitions of NLN

Over the past couple of acquisitions of NLN, we noticed that the observe SDF diffs took ~25 minutes to read 0 and allow us to flip to the OBSERVING bit. The node in question that is the culprit of this, falls under ASC, screenshot attached below. After further investigation, we noticed that ~7/8 minutes into what seems to be every aquisition of NLN, something causes the ADS signals to drift (when it was originally converging), and therefore causing the ADS to take longer to converge and preventing us from going into OBSERVING. Also noticed that this drift in question, seems to be most apparent on ADS_YAW_3, but this mystery drift does seem to cause a divergence of all degrees of freedom (including the ADS P signals). However, the magnitude of this change does seem to vary depending on the lock. I have attached screenshots of some examples for visual representation.

So far I have checked a couple things and still have yet to find a definitive reason. I have looked at both ISC_LOCK and the CAMERA_SERVO guardian logs during first acquisitions of NLN to see if they are doing any actuation, but both nodes seem to be dormant during this period. Jenne suggested that I also look into the OMC_LOCK guardian to see if the OMC is doing any whitening, as that time period falls around the same time as the drifts take place. I have attached a screenshot shows the guardian state of OMC_LOCK alongside an ADS drift - no state change. After checking some of the OMC_LOCK logs during these drifts, the node seemed to also be dormant during this time (but it does seem to be adding whitening ~10 minutes before the drift in question, not sure if this time frame is suspicious or not - screenshot attached - the drift time occured ~14:45 UTC for that particular occurance).  After doing some alog digging, I saw that this issue has happened before fairly recently - alog here. The issue then was that the REFL signals were way off alignment. After looking into some of ASC signals that feed into the ADS, some ASC signals - particularly PRC 2 Y (and maybe INP1?) seem to also behave in a similar fashion - but I'm not entirely convinced that this is the culprit, since the ASC signals don't seem to move a whole lot. It could also be the case where the ADS drifts are causing PRC 2 to act up as well. But, it might be worth looking into the sensors that feed into these signals (REFL A/B 9I for PRC2 and REFL A/B 45I for INP1). I have also started looking into some LSC signals - particularly the PRCL signals, but have not found anything really suspicious in that initial search.

I do think that this is something we should look at. This issue is preventing us from going into OBSERVING mode for ~15 minutes for every NLN aquisition, which in the long run, is quite substantial when considering total uptime. I will continue to look into this issue and post any updates/findings accordingly.

Images attached to this report
H1 General
austin.jennings@LIGO.ORG - posted 14:17, Sunday 07 May 2023 (69391)
Mid Shift Report

IFO has lost lock after ~5 hours and is in the process of relocking.

LHO VE
david.barker@LIGO.ORG - posted 10:11, Sunday 07 May 2023 (69390)
Sun CP1 Fill

Sun May 07 10:08:05 2023 INFO: Fill completed in 5min 4secs

Note fill did not start at the scheduled time of 10:00:00, most probably a timing glitch on the IOC. I started it at 10:03:00 by resetting the yaml's start time.

Images attached to this report
H1 General (PEM)
austin.jennings@LIGO.ORG - posted 08:03, Sunday 07 May 2023 - last comment - 08:10, Sunday 07 May 2023(69386)
Ops Day Shift Start

TITLE: 05/07 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Lock Acquisition
CURRENT ENVIRONMENT:
    SEI_ENV state: CALM
    Wind: 15mph Gusts, 11mph 5min avg
    Primary useism: 0.04 μm/s
    Secondary useism: 0.14 μm/s 
QUICK SUMMARY:

- IFO is relocking, currently @ RESONANCE

- CDS/FOMs

- High seismic motion at EX, possibly due to movement of recent EQ (5.2 from Philippines)?

- IFO has been struggling overnight trying to lock - after there was a NLN lockloss it took 2.5 hours for it to get past DRMI acquisition

- Few DMs are showing up WHITE on alert handler - All EX DM1, all EY DM1s, Tagging PEM

Comments related to this report
austin.jennings@LIGO.ORG - 08:10, Sunday 07 May 2023 (69388)
Link

Update: The DMs are no longer reading white.

H1 General (SUS, VE)
austin.jennings@LIGO.ORG - posted 16:02, Saturday 06 May 2023 - last comment - 11:44, Monday 08 May 2023(69378)
Saturday Operator Summary

TITLE: 05/06 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Commissioning at 130 Mpc
SHIFT SUMMARY:

Main Notes:

- Second ER15 candidate! @ 17:09:35

- ACQUIRE_XARM_IR during IA is still failing, and therefore I cannot run a full initial alignment

- ACQUIRE DRMI 1F has been quite cumbersome recently trying to acquire

- IFO is at OBSERVING @ NLN @23:02
Lock #1:

- Came in with IFO locked, switched to OBSERVING @ 14:56 UTC, out of OBSERVING @ 17:00 UTC due to difference from VAC SDF (was only temporary, but maybe this channel should be unmonitored), screenshot attached - Tagging Vac

- PI 24 ring ups at 15:26/15:37 UTC, but the SUS_PI guardian was able to damp it - no intervention needed

- Starting a calibration suite @ 17:01 UTC in NLN CAL MEAS

- EX saturation @ 18:02 UTC and again @ 20:20, huge jump in ADS signals and OMC DCPD output went off the rails - glitch?

- Back to NLN and OBSERVING @ 18:53 UTC

- LOCKLOSS @ 2026 UTC - this looks like an ASC ringup, potentially from SRC1/2 and/or MICH? - however ground motion looks a bit elevated so that could be a culprit as well

Lock #2:

- IMC had issues acquiring, but resetting ISC_LOCK to INIT and having it tried again worked fine

- DRMI issues 3x times trying to lock, gave up and trying an initial alignment - just like yesterday I again cannot ACQUIRE_XARM_IR. Skipping IA and trying again.

- Acquired NLN @ 22:38

- Some OBSERVE snaps are preventing me from going to OBSERVING, SUSITMY - Screenshots attached, accepting and went into OBSERVING, Tagging SUS

LOG:

No log for this shift.

Images attached to this report
Comments related to this report
austin.jennings@LIGO.ORG - 08:40, Sunday 07 May 2023 (69389)
Link

For ITMY diffs, the changes I accepted showed up as differences on SDF observe. Accepting them and setting them back to what they were before.

Images attached to this comment
elenna.capote@LIGO.ORG - 11:44, Monday 08 May 2023 (69408)
Link

This doesn't appear to be an ASC instability lockloss. Looks like something kicked all the ASC signals right before lockloss- potentially the large ground motion you mention? I attached a screenshot of the DRMI signals for example.

Images attached to this comment
H1 CAL
austin.jennings@LIGO.ORG - posted 15:30, Saturday 06 May 2023 (69380)
Calibration Suite taken using pydarm measure at 17:03UTC to 18:45UTC

Referencing Camilla's alog 69333, I ran a calib suite.

Attached is a screenshot of the sitemap > CAL CS > Calibration Monitor screen shortly after starting the measurement.

'pydarm report' command gave attached report: 

Images attached to this report
Non-image files attached to this report
H1_calibration_report_20230506T182203Z.pdf
LHO VE
david.barker@LIGO.ORG - posted 10:09, Saturday 06 May 2023 (69381)
Sat CP1 Fill

Sat May 06 10:05:20 2023 INFO: Fill completed in 5min 19secs

Images attached to this report
H1 General
austin.jennings@LIGO.ORG - posted 07:56, Saturday 06 May 2023 - last comment - 09:55, Monday 08 May 2023(69377)
Ops Day Shift Start

TITLE: 05/06 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Calibration
CURRENT ENVIRONMENT:
    SEI_ENV state: CALM
    Wind: 3mph Gusts, 1mph 5min avg
    Primary useism: 0.01 μm/s
    Secondary useism: 0.12 μm/s 
QUICK SUMMARY:

- IFO was locked at NLN for 1:13 (acquired at 13:41 UTC)

- All observe SDFs are cleared, so I flipped the intention bit to OBSERVING @ 14:46 UTC

- CDS/SEI/FOM/DMs all look good

Comments related to this report
camilla.compton@LIGO.ORG - 09:55, Monday 08 May 2023 (69397)OpsInfo
Link

I was mistakenly under the impression that changing the IFO_MODE from Managed (1) to Auto (0) was enough for the IFO to change to OBSERVING (H1:GRD-IFO_OK = 1) once the IFO was READY. As the Guardian Overview shows, this is not enough, I also should have taken the Intention bit (H1:GRD-IFO_REQUEST) to OBSERVE for this to work. I didn't do that when I left Friday night as we were doing a calibration suite (69362) but I could have done as the IFO_READY flag would not have actually taken us to Observing until calibration was finished.

Images attached to this comment
H1 DetChar (DetChar, ISC)
andrew.lundgren@LIGO.ORG - posted 04:49, Saturday 06 May 2023 - last comment - 11:54, Monday 08 May 2023(69376)
DCPD overflows after DARM offset change 
Tito Dal Canton from PyCBC reported that there are periods with many ADC overflows of the DCPD in the last few days. This is probably due to the increased DARM offset alog 69358. See attached plots of the DQ vector. There's no obvious effect on the spectrum or the glitchiness, so it must not be a severe overflow. But at least some of the searches are using this as a veto and aren't analysing the data.
Images attached to this report
Comments related to this report
daniel.sigg@LIGO.ORG - 12:47, Saturday 06 May 2023 (69382)
Link

What is this triggered on? The attached plots show the DCPD min/max monitors for the same time. There are saturation events at the beginning (violin modes) and at the end of locks (lock loss), or during lock attempts, there are no saturations visible once the instruments have stabilized. One can also see some larger values during the most recent lock, which are still below saturation and which might be violin modes or parametric instabilities.

The plotted channels are 16 Hz EPICS channels that are caluclated in the front-end using data at the full converter rate (219 Hz). For each DCPD channel 4 18-bit ADC channels are getting summed to form a 20-bit word. Saturation happens once the sum channels reach ±219 counts. Since individual ADC offsets and gains could be slighlty different, a saturation monitor should propbably trigger around ±500,000.

Images attached to this comment
andrew.lundgren@LIGO.ORG - 14:37, Saturday 06 May 2023 (69384)DetChar
Link 

It will just be using the overflow channel calculated in the front end, H1:FEC-8_ADC_OVERFLOW_0_12 and 0_13 for DCPD A and B. The Simulink models that are exported show those as the only ADCs for the DCPDs, but they haven't been updated since 2022-07. Looking at the traces closely, the overflows do seem to be reported whenever STAT_MAX goes over ~2^17.
daniel.sigg@LIGO.ORG - 15:00, Saturday 06 May 2023 (69385)
Link

No sure what this indicates, seems wrong. Use H1:FEC-179_ADC_OVERFLOW_0_x with x = 0, 1, 4, 5, 8, 9, 12, and 13 instead.

jeffrey.kissel@LIGO.ORG - 10:05, Monday 08 May 2023 (69398)CAL, CDS, DetChar, ISC
Link 

Indeed, filling out Daniel's description:
This is one of the major changes to the detector for O4 -- the OMC DCPD signal processing chain is entirely different.

At LHO, we're reading out the same two in-vacuum OMC DCPDs with
    - A brand new low-noise transimpedance amplifier
    - A brand new whitening chassis
    - *no* analog anti-aliasing filter, but making 4 analog copies of the two input voltages
    - digitized with a brand new, lower noise, 18- (instead of 16 bit), 32 channel (instead of 16) ADC, sampled at 524 kHz, in an entirely new "segregated" IO chassis,
    - whose digital signal is created with a brand new real-time code Input-Output processing architecture -- whose DCUID is 179, hence H1:FEC-179_ADC_OVERFLOW_0_x with x = 0, 1, 4, 5, 8, 9, 12, and 13  -- that is "sampling" the 524 kHz voltage from the ADC at 65 kHz (i.e. grabbing 8 samples each 65 kHz clock cycle)
    - a new time-stamping system correlated with the DAQ, such that the data is *time stamped* as though its 5x 524 kHz clock cycles (i.e. 9.6 usec) behind all other data in the CDS architecture

After digitization, each channel's 4 copies are coherently summed, then divided by 4.0 in order to average the signal coherently across the 4 ADC channels (and thus improving the noise further by reducing the coherent digitization noise by sqrt(4)). Then, each averaged channel is
    - signal conditioned in new filter banks (running at 524 kHz)
        H1:OMC-DCPD_A0
        H1:OMC-DCPD_B0
    where the signal conditioning includes conversion from ADC counts back into milli-amps of current from the DCPDs via
        i. inversion of the measured transimpedance amplifier response
        ii. inversion of measured analog whitening response
        iii. inversion of the ADC "gain", now 2^18 ct / 40 V
        iv. two digital anti-aliasing filters, one for 524 kHz to 65 kHz, and one for 65 kHz to 16 kHz
and then that down-sampled, conditioned 524 kHz is shipped over inter-process communication to the old h1omc front-end model (with DCUID 8  -- i.e. H1:FEC-8_ADC_OVERFLOW), where it marries with identical 16 kHz digital infrastructure from O3 (save the *lack* of signal conditioning in 16 kHz the H1:OMC-DCPD_A and H1:OMC-DCPD_B filter banks because their function is now in the 524 kHz IOP model).

Please check out LHO:67644 for all the current digital version of the OMC DCPD channels.

Apologies that we've not yet had time to present this formally to all members of the collaboration repeatedly, as it was not until ~2 months ago that all this was settled. Information is stil scattered across aLOGs, to be gathered for review documentation.
And, naturally, only a portion of this is true at LLO. (They're still using a 65 kHZ analog AA filter, not using segregated IO chassis, and they're shipping from the 524 kHz IOP model to 16 kHz OMC model directly, rather than over IPC so their 524 kHz data is stamped as though it is 3 524 kHz clock cycles = 5.7 usec *ahead* of all the other CDS channels).

Note, all of these changes are "calibrated out" of the data stream, so down stream data analysis need not know *anything* about this, nor treat the calibrated data stream any different. 
But -- *that* why the number to look at for ADC saturation has changed.
ezekiel.dohmen@LIGO.ORG - 11:54, Monday 08 May 2023 (69409)CDS
Link 

We have found an issue, with the calculation of the ADC limit in user models. User models are not accounting for 18 bit ADC values, and are erroneously reporting ADC overflows.

IOP models do have the proper logic. Can you switch the channel you use to check for overflows to the one provided by h1iopomc0?

Example:  H1:FEC-179_ADC_OVERFLOW_0_12 instead of  H1:FEC-8_ADC_OVERFLOW_0_12
H1 ISC
camilla.compton@LIGO.ORG - posted 21:50, Friday 05 May 2023 - last comment - 13:05, Saturday 06 May 2023(69371)
OMC Whitening Checker in ISC_libary.check_for_violins_saturation Adjusted

Elenna, Camilla, Vicky,

In our relock with OMC whitening changes (69350) PLUS a higher DCPD sum (69358), ICS lock kept us at OMC_WHITENING with the message "Violins are too high for whitening" but comparing violin levels (500Hz Violins on DARM were 4 x 10-16 m/Hz1/2 ) and DCPD levels in the last lock we realized we would be fine.

Elenna manually turned on the whitening and manualled us to NLN. Elenna updated the ISC_libary.check_for_violins_saturation(power='high') checker from her guesstimated factor of 6 (worked with higher whitening before higher DCPD offset) to a factor of 3.

DCPD signals with T-cursor when whitening turned on attached.

Images attached to this report
Comments related to this report
daniel.sigg@LIGO.ORG - 13:05, Saturday 06 May 2023 (69383)
Link

This one is cutting it pretty close!

Images attached to this comment
Displaying reports 20381-20400 of 87736.Go to page Start 1016 1017 1018 1019 1020 1021 1022 1023 1024 End