Lockloss 04/16 23:39UTC - IX saturation at lockloss

The HASH_INT channel was updated at 16:00 and its new value was accepted into CDS SDF. These two channels were removed from h1calcs SDF by hand editing the snap file.

As a test that the cal_hash_ioc restores it values on startup, we restarted it at 16:30. CDS SDF didn't notice it had gone, the EDC connected channel counter dropped by 5 for about 15 seconds.

TITLE: 04/16 Eve Shift: 23:00-07:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Commissioning
OUTGOING OPERATOR: Ibrahim
CURRENT ENVIRONMENT:
    SEI_ENV state: CALM
    Wind: 25mph Gusts, 19mph 5min avg
    Primary useism: 0.15 μm/s
    Secondary useism: 0.14 μm/s
QUICK SUMMARY:

Locked for almost 2 hours, Jim is doing some last minute tests and sqz team is trying to last minute troubleshoot the sqzer before we go into observing without sqz

J. Oberling, T. Guidry, R. Crouch

Progress since our last update.

When we ended last, we had large Z axis deviations for our PSI monuments that we speculated was due to the fit being unable to pick up on the tilt of the X axis in our global coordinate system, since the height marks we were using for Z axis alignment were all in a line along the Y axis.  To test this, we decided to try aligning the FARO in the local LVEA coordinate system, then translating that alignment to our global coordinate system.  We first tried this on 4/9/2024, then again with a slightly different method today, 4/16/2124.

4/9/2024

We first tried assuming the LVEA is perfectly flat; in other words, we used the same Z axis coordinate for all PSI monuments.  With a new Z axis coordinate for BTVE-1 (-1060.3 mm) that we tied back to BSC2 Z=0, and our differential height measurement from alog 75669, we calculated a new Z axis coordinate for PSI-1 of -1885.4 mm.  We then used this Z axis coordinate for PSI-2 and PSI-6 to give us a very rough Z axis alignment.  We then followed the same method we used in alog 76889 to align the FARO, except we did not convert the Z axis coordinates to our global coordinate system, we left them in LVEA local.  In the end, we were using BTVE-1, PSI-1, PSI-2, and PSI-6 for X and Y axis alignment, and BTVE-1, 901, 902, and 903 for Z axis alignment.  At first we thought this looked pretty good, but then we noticed that we had some significant deviations in X axis coordinates for 901, 902, and 903 that were not there before we finalized the alignment.  Turns out that excluding PSI-2 from the Z axis alignment drags everything off in X by several mm, and the alignment is still incapable of accurately resolving a Z coordinate for PSI-2 (16.3 mm deviation!).  It's almost like the PolyWorks alignment algorithm thinks PSI-2 is rotated/tilted somehow w.r.t. the other monuments, we really don't have an explanation for it.  This can be seen in the 1st 2 attached pictures.  Regardless, it seems like this isn't the route to go down, so we ended the day at this point.  Also, the maintenance window was almost over so we had to shut things down anyway.

Before completely shutting down, we did try converting from the local coordinate system to the global one.  PolyWorks allows the user to set a new coordinate system by several methods, most importantly to us by translation and rotation about known axes.  In other words, we can set a new Cartesian coordinate system, and translate and rotate it however we need.  This is good for our purposes, as we can enter the rotation of our X and Y axes that converts from LVEA local to site global.  Recall that, if you are sitting at the origin of the IFO's coordinate system, the global X axis is tilted down by 619.5 µrad w.r.t. the local LVEA X axis and the global Y axis is tilted up by 12.5 µrad w.r.t. the local LVEA Y axis.  PolyWorks uses axis tilt as a roll of the axis, so we enter the Y axis tilt as an X axis roll and the X axis tilt as a Y axis roll (we don't translate the axes at all, the origins are supposed to be the same spot).  This all seemed to go smoothly, but left a few things that we didn't have time to dig into: It looked like the rotation of the X and Y axes also induced a rotation of the Z axis, even though that is supposed to be held constant, and it looked like it was rotating too much (our Z axis coordinates changed more than we expected).  More to look into for next time.

Speaking of next time...

4/16/2024

Today we essentially did a repeat of last week, except we did not assume the LVEA was perfectly level.  Since we had the previous differential height measurements and the new BTVE-1 local Z axis coordinate, we used the differential height measurement data to calculate new LVEA local Z axis coordinates for PSI-1, PSI-2, and PSI-6.  It should be noted that we weren't really in love with way we had to do the differential height measurement for PSI-6, due to line of sight issues, so we entered that as a placeholder only, it was not included in any alignments.  To keep things consistent we used sphere features for BTVE-1, PSI-1, PSI-2, and PSI-6; this means we also included the correction for the sphere-fit rod, which measures the bottom of the monument punch and not the monument surface that the coordinate is registered to.  The new LVEA local Z axis coordiantes we used today, properly corrected for punch depth, are:

• BTVE-1: -1061.1 mm
• PSI-1: -1886.5 mm
• PSI-2: -1887.3 mm
• PSI-6: -1881.9 mm (placeholder only!)

Since these Z axis coordinates for PSI-1 and PSI-2 are ones we directly measured with the FARO, via differential height measurement w.r.t. BTVE-1, and have tied back to BSC2 Z=0, via our water tube level survey and differential height measurements, we also included them in the alignment routine; as stated previously PSI-6 was excluded from the alignment routine as we do not trust that number.  Following the same method previously used, we measured the X and Y coordinates for height marks 901, 902, and 903, and used our previously-measured Z axis coordinates from our water level survey.  The results of the final alignment routine are shown in the 3rd attachment; this is in the LVEA local coordinate system.  This is the closest we've yet been to something that looks like a good alignment to one of our coordinate systems (LVEA local in this case).  One thing we immediately notice, the measured X axis coordinate for BTVE-1 is reported as off by -1.2 mm.  At this point in time we really have no way to know if this is a true error in the placement of BTVE-1 or an error in our alignment routine.  It could be a result of the sphere fit rod, as we don't really trust its repeatability; probing method and who operates the rod has a large impact in the measurement, but we have no other way to probe BTVE-1 due to its dome shape.

We also took another shot at converting from the LVEA local coordinate system to our site global coordinate system.  We did this in the same was previously, setting up a new coordinate system and rotating the X and Y axes by our known rotations.  We immediately saw that, again, it appeared to be over-rotating.  I didn't get a picture, but using height mark 901 as an example.  It has an LVEA local Z axis coordinate of -55.2 mm, which, when using the listed X and Y coordinates for this mark, translates to a site global Z axis coordinate of -55.7 mm.¹  The new coordinate system kept putting the gloabl Z axis coordinate for 901 at -56.0 mm, 0.3mm below where it should be.  We were a little puzzled by this, and were initially thinking that PolyWorks was rounding up to the displayed number of digits after the decimal point, and not using the axis tilts as we entered them (for example, Y axis tilt in degrees is 0.000716, which the PolyWorks display rounds up to 0.001).  We set the display to 7 digits after the decimal and our entered angles were display properly, so this was not the cause.  Turns out it was a sign issue.  We had been told by PolyWorks support, back in 2022, that we needed to enter the opposite sign of our actual axis tilt due to the way PolyWorks references tilts w.r.t. the FARO.  We checked the Properties of the new coordinate system and noticed the IJK unit vectors for the Z axis did not have the correct signs.  In fact, it had both X and Y rotations listed as negative, when they should be opposite.  Correcting the signs the displayed the correct global Z axis coordinates; shown in the final attachment.  We still have the issue of the new coordinate system also rotating about the Z axis when it shouldn't.  This is best seen by looking at the nominal X axis coordinate for BTVE-1.  In the 3rd attachment, in LVEA local, BTVE-1 X is properly listed as 0.0.  In the final attachment, in site global, the nominal X for BTVE-1 is now -0.657mm.  In looking at the direction cosines for the LHO Corner Station as listed in T980044, there is a small rotation from global Z to local Z, which we did not implement.  Will test and see if implementing this small rotation corrects this.

We like this alignment, it's the best we've seen to date.  So, next steps are to start moving the FARO through the LVEA and measuring known points while setting up a volume of alignment monuments, in hopes that we can get enough monuments so we can set the FARO anywhere in the LVEA and be able to align it.  This will continue on Tuesdays as our schedules allow, until we are done.

1) Keeping this here for posterity, as it was my thinking as we were on the floor, but this is incorrect.  In writing this alog I realized that I got my direction cosine signs flipped when we were setting up the coordinate system transform in the PolyWorks software this morning.  *facepalm*  The proper global Z for 901 based on the X and Y axis coordinates listed in the referenced picture is -54.7 mm, not -55.7 mm.  This also means that we then flipped the signs for both direction cosines when we "corrected" the incorrect signs we initially saw with the new coordinate system.  I'm going to take a look tomorrow and see about correcting this, pretty sure we can do that "offline" without a FARO connected.  Will post any update as a comment to this alog.

TITLE: 04/16 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Commissioning
INCOMING OPERATOR: Oli
SHIFT SUMMARY:

IFO is in NLN and COMISSONING - the squeezer is having lock issues and a team of comissioners, fellows and other staff are in the control room and working on fixing it.

19:29 UTC: Began Locking (No initial alignment)

20:15 UTC: Began Initial Alignment (after 3 IR_COMM based locklosses)

IR COMM Weirdness:

Once we started locking, ALS locked fine but IR_COMM was not found, prompting a troubleshoot of this part. There was extremely noise behavior in the relevant channel (H1:LSC-TR_X_NORM_INMON) where the usually normalized (to 1) values were showing up bizarrely high. (Screenshot 1). After initial alignment, this issue was resolved.

20:36 UTC: Initial Alignment Complete

21:21 UTC: Reached NLN

21:24 UTC: H1 is OBSERVING

21:36 UTC: Dropped out of observing due to squeezer lockloss - a team of commissioners are on the case.

22:55 UTC: Trying to get into OBSERVING without squeezing while we troubleshoot the squeezer.

LOG:

Shiela, Naoki, Rahul, Kar Meng, Terry

Same as yesterday (77188), we are again today having trouble locking the FC. Wind is again constant at 0-20mph. Can get green flashes but no locking, even with FC feedback turned off, VCO locking also fails. 

Rahul changed the M1 coil driver state on FC1 and FC2 from state 1 (usually only used in TFs) to state 2: IFO nominal for other triples. State 2 contains a low pass filter. 

Rahul took FC2 P and L transfer functions, look healthy and same as 66414.

Rahul took FC2 OSEMINF spectrums and checked ther health, as previously had issues with FC1 BOSEMS  (72563). Can see the 0.3 to 0.4Hz peak, Rahul's not worried about it but we could check old data for this peak.

I pulled EX Accelerometer Power Conditoiner S2300070 per WP11813 to investigate a bad channel and the restult was that the input resistor had blown on that channel.  The symptom was that the channel was completely off, no LED activity on the front panel for that channel.  The current draw per channel at 24V is < 63mA, the 10 ohm resistor is rated at 1/4 watt which gives us an upper limit of 160mA current.  It could have been inrush current that blew this resistor as they have only been tested on bench and not with kepkos' higher current rating.  This is part of the low pass filter for these accelerometer channels.  I replaced the 10 ohm 1/4 watt resistor with a 5W resistor.  I will keep an eye on these, if this becomes an issue, we might consider replacing the resistors with higher wattage versions as there is space on the board for it.

Channel Order
CH1 - Cable 7 - AA7 - OPLEV-Y
CH2 - Cable 8 - AA8 - BSC9_X
CH3 - BLK_NL - U2_9 - Manifold1
CH4 - Cable 10 - AA10 - BSC9_Z
CH5 - Cable 11 - AA11 - TableX
CH6 - Cable 12 - AA12 - BSC_Floor
CH7 - Cable 13 - AA13 - Ebay_Floor
CH8 - BLK-NL - U2_11 - Manifold2
CH9 - empty - empty (ready to replace the Meggit single channel device)

Jennie W, Sheila,

We turned off the 9 and 45 MHZ sideband EOMs and unplugged the 118 MHz modulation at the PSL racks. See this alog for how to do this.

We locked the IMC and mis-aligned ITMX.

DC centering loops 3 and 4 were turned on.

Sheila took the OMC lock guardian to PREP OMC scan and turned on the OMC ASC.

We waited for this to converge then turned it off.

We turned input power up to 10W from 2W and then locked the OMC in length manually by using the PZT offset slider to search for a high peak ~ 15mA.

After finding it we turned on the locking with a gain of 24 and the boost and int filters on (we checked the settings we needed in the guardian as we couldn't get the OMC guardian to lock it for us).

Then we turned off the OMC ASC.

We tuned up the OM3 and OMC alignment slightly to maximise the power on OMC-DCPD_SUM_OUTPUT and minimise it on OMC-REFL_A_LF_OUT16.

OM3 was moved down in yaw only from -108 to -117 and OMC was moved down in yaw from 210.5 to 195.5 and down in pitch from 350.4 to 340.4. The final values for the sliders are here.

Quiet time locked after aligning 16:21:03 UTC - 16:24:13 UTC

OMC-REFL_A_LF_OUT16 = 0.652644 mW

OMC-DCPD_SUM_OUTPUT = 15.6162 mA

Quiet time unlocked 16:25:31 UTC -16:27:35 UTC

OMC-REFL_A_LF_OUT16 = 30.106 mW

OMC-DCPD_SUM_OUTPUT = 0.000456763 mA

We took the IMC guardian offline and shuttered the green light going into both arms at the ISC tables, and with ISC_LOCK guardian in idle so it wouldn't keep trying to relock IMC.

Quiet time dark measurement 16:43:38 - 14:46:19 UTC

OMC-REFL_A_LF_OUT16 = -0.0132979 mW

OMC-DCPD_SUM_OUTPUT = -0.00106226 mA

The scan template is saved as /ligo/home/jennifer.wright/Documents/OMC_scan/2024_04_16_OMC_scan.xml

With the references for the time series of the three pertinent channels as:

Ref 0 OMC-DCPD_SUM_OUT_DQ

Ref 1 OMC-PZT2_EXC

Ref 2 OMC-PZT2_MON_DC_OUT_DQ

J. Kissel

EXECUTIVE SUMMARY

After having visited the calibration of the HAUX alignment sliders since 2016 (see LHO:30010), it's time to re-measure and recalculate what these gains should be, as we've lost confidence in their accuracy. 

I've recalibrated the IM1, IM2, IM3, IM4 alignment sliders today, such that one count offset in [pitch, yaw] produces 1 microradian in [pitch, yaw] according to the OSEMs. To correct from the previous incorrect calibration, slider values would have needed to be multiplied by factors of 20 to 30 [urad/"urad"].

METHODS AND RESULTS

In order get something in play quickly, I've used the OSEMs as my absolute angle reference, having been calibrated into microns [um] for years and cast into PITCH and YAW through trusted OSEM2EUL matrices to produce pitch and yaw in microradians [urad]. 

Recall the goal is to calibrate the output of the OPTICALIGN banks, which are nominally in Euler basis DAC counts [DAC ct] such that when you request the slider to move in your desired physical unit (in this case, [urad]) then the output of the bank requests the appropriate amount from the DAC. Thus, we need a gain, H where
    slider offset [urad] * H [DAC ct / urad] = DAC output [DAC ct]        (1)


In short, the method is:
    (i) Set the OPTICALIGN gain to 1.0 so there's no confusion during the measurement.
    (ii) Request the slider offset to be a range of values (I arbitrarily chose [-10000, -5000, -1000, 1000, 5000, 10000]; six data points such that a linear regression had plenty to grab on to)
    (iii) At each requested value, record the OSEM values.
    (iv) Fit a line to a plot of OSEM value vs. slider value, i.e. (slope * slider value + offset). The slope will be in units of [urad / DAC ct].
    (V) Invert the slope to obtain the slider calibration gain in units of [DAC ct / urad].
I used an ipython session interactively to do so for all IMs at the same time via for loops, while the IMC was unlocked and misaligned.

Here're the results in units of [DAC ct / urad] (rounded to the nearest 4th decimal place):
         PIT       YAW
IM1     9.9093    5.3457
IM2    10.1124    5.4868
IM3    12.5623    6.8758
IM4    10.9005    5.6289

Note that the results are not insignificantly different from suspension to suspension, even though every HAUX actuation chain should be the same. So, I've elected to install these suspension specific gains (like we have with other suspensions that have had recent attention), rather than a generic gain for suspension type based on a model of the actuation strength.

The first two pages of 2024-04-16_H1SUSIM_AlignmentSlider_Recalibration_GainFitPlots.pdf shows the plots of data, and their fits. As an aside, because I drove the slider requests programmatically via ipython, it was easy to measure both the pitch and yaw OSEM values during the other DOF's slider changes. As such, I also gathered the amount of cross-coupling between pitch drive and yaw response and vice versa, yaw drive and pitch response, in units of [urad / DAC ct]. This is shown on the last two pages. The good news is that the off-diagonal cross-coupling is an order of magnitude or two less than the expected diagonal coupling (I think auto-y-axis-scaling dataviewer or ndscope sessions have scared us for years). That being said, IM3 is the worst offender.

After I installed the new calibration, I've made sure to set the *actually* [urad] value to reproduce the same [DAC ct] output that had been requested with the previous incorrect ["urad"] values (generated with the incorrect [DAC ct / "urad"] gain, G). The method for doing so is revealed through a quick algebra exercise,
    slider offset [urad] * H [DAC ct / urad] = DAC output [DAC ct]                 (1)
    bad slider offset ["urad"] * G [DAC ct / "urad"] = DAC output' [DAC ct']       (2)

    want
    DAC output [DAC ct] = DAC output' [DAC ct'],

    so
    slider offset [urad] * H [DAC ct / urad] = bad slider offser ["urad"] * G [DAC ct / "urad"]
    
    slider offset [urad] = bad slider offset ["urad"] * G [DAC ct / "urad"] / H [DAC ct / urad]     (3)
or, in words, multiply the old bad offsets by the ratio of (old gain / new gain) to get the new well-calibrated offsets.

In addition to calibrating the offsets, I also updated the visual representation of the available range on the MEDM screen. In this case, given that the HAUX are driven with 18-bit DACs, then each OSEM can drive 2^17 - counts. The EUL2OSEM matrix elements for conversion of pitch and yaw to each of the four OSEMs is +/-8.594 [um / urad], so that means that a Euler basis pitch and yaw [DAC ct] drive cannot exceed 2^17 / 8.594 = 15251 [Eul DAC ct]. After dividing by the above calibration for each, that takes the slider limits to (roughly, taking the lowest range -- IM3 -- as the limiter) +/- 1200 [urad] for pitch and +/- 2200 [urad] for yaw.

2024-04-16_H1SUSIM_AlignmentSlider_Recalibation_OPTICALIGN_Screens_after_correctunits.png shows the final product after all of these changes,
    - the new slider gain values,
    - the new slider offset values to reproduce the same DAC output, and
    - the updated range of values.

Operators be aware: the amount of nudging you need to do on IM4 during initial alignment will need to change as a result of this re-calibration. Using the same math as Eq. (3) above, [1, 10, 100, 1000] "urad" nudges now are actually, roughly [0.05, 0.5, 5. , 50] "urad" nudges.

2024-04-16_H1SUSIM_AlignmentSlider_Recalibration_Notes.txt are my very rough notes from the morning's work, including all mistakes, as well as ipython code snippets to take actions, store information, and make plots. Could be cleaner for future users, but alas.

Swept the LVEA at the end of maintenance. All of the usual checks on  T1500386 were done, the only thing I noticed was that there were some cables plugged into the SQZT0 table that were not connect to anything else. These are for the hymodyne delta DC, PD1 DC, and PD2 DC according to their labels.

The items mentioned from last week's sweep (alog77058) are still there - pem setups, FARO, etc. as expected for the near future.

IFO is in IDLE for TUESDAY MAINTENANCE

Maintenance work is wrapping up, Camilla is detransitioning our LVEA Hazard state and we will being locking as soon as maintenance activities end.

New channels have been added to the CDS SDF, and h1cdssdf was restarted on h1ecatmon0. Current values have been accepted and are being monitored.

WP11812

Jamie, Jonathan, Erik, Dave:

The two CAL HASH channels (H1:CAL-CALIB_REPORT_HASH_INT and H1:CAL-CALIB_REPORT_ID_INT) were moved from the h1calcs model, where they were being acquired as single-precision floats, to a new custom EPICS IOC called cal_hash_ioc, where they are acquired as signed 32bit integers.

The h1calcs.mdl model was modified to remove the EPICS-INPUT parts for these channels. The model was then restarted.

The IOC, which was being tested with H3 channels, was modifed and rstarted to serve the H1 channels as integers.

The EDC was modified to add H1EPICS_CALHASH.ini to its master list. This added 5 channels the the DAQ, the two hash channels and three IOC status channels.

The EDC was restarted, followed in short order by a DAQ restart.

Lately it's been very difficult to fit an efficient SRCL feedforward filter, as reported many times by Camilla et al. (76993). Here I'm trying to figure out why. Spoiler alert: I don't have an answer yet.

The main problem with the SRCL FF filter (see first plot), is that the transfer function to fit has a large phase rotation, that looks basically like a phase advance (the opposite of a delay) of about 2 ms. This is very large, and being an advance, it can't be realized in a precise and simple wasy digitally.

First observation: we can fit a pretty good SRCL FF if we allow for unstable poles, i.e. poles with positive real parts (see second plot) Of course this is not something that can be implemented in the real time system. The fit ends up having a unstable complex pole at about 420 Hz and about 5 Hz. I have no intepretation for the origin of those poles, and they might very well be only a way to reproduce a phase advance (think of the Padé approximation for phase delays).

So the question is: what is the origin of this large phase rotation? It's not seen at LLO, for example see 70548

Second observation: this phase advance appeared after we switched the LSC FF from ETMX (full chain) to ETMY PUM. The third plot compares the MICH and SRCL feedforward to be fit in two cases: an old measurement when the FF was going to ETMX, and a more recent measurement with the FF going to ETMY PUM only. For both MICH and SRCL the orange traces (FF to ETMY) show a phase advanced with respect to the blue traces (FF to ETMX). For some reasons I don't fully understand, this rotation is more problematic for SRCL than for MICH, although fitting MICH has also been more difficult and the MICH FF is relevant at lower frequencies than SRCL, so maybe the phase advanced isn't that problematic.

Looking at the measurement of the MICHFF to DARM and SRCLFF to DARM, one can see that there seems to be an additional phase delay in the FF path through ETMY PUM with respect to the FF path through ETMX full chain. Since this transfer function is at the denominator when computing the ratio SRCLtoDARM/SRCLFFtoDARM that gives use the LSC FF to fit, this seems to explain the additional phase advance we observe.

The ETMY PUM L2 lock filter bank contains a "QPrime" filter module that compensates partially the additional 1/f^2 due to the actuation from L2 instead of L3. This filter however doesn't seem able to explain this additonal phase delay.

I'm now suspicious that there might be something wrong or mistuned in the ETMY L2 drive, maybe a whitening filter missing or not functioning properly or not properly compensated?

It would be worth doing a quick test in the next commissioning time with full IFO locked: inject some white noise on ETMX L2 L and ETMY L2 L and comapre the two transfer functions to DARM. In theory they should be equal, except for a sign difference. If they're not, then there must be something wrong with ETMY, since we're using ETMX L2 to lock without issues.