This morning, B&K hammer measurements were carried out on SR2 suspension cage located in HAM4. Attached plots are comparing SR2 force to acceleration transfer functions in two degrees of freedom X and Y (respectively corresponding to Longitudinal and Transverse suspension's euler basis -not global coordinates-), with similar measurements carried out on MC1 MC3 PRM.

The main/lowest structural resonance below 150Hz (97Hz in both directions) is well damped, and the Q is comparable with the one from other suspension cages.

[Method]

HEPI and ISI were unlocked during the measurement. A tri-axis accelerometer was mounted on the top corner of the structure (as shown in the picture). Data acquisition was made with the Pulse software on the windows laptop, and the template used was copied from T1000697, using a hammer trigger threshold of 10N.

Data collected lives under the svn in

/ligo/svncommon/SusSVN/sus/trunk/HSTS/H1/SR2/BandK

[Sheila Lisa Jeff Arnaud]

Sheila and Lisa were having alignment issues in the X arm this afternoon, in the sense that some fine alignment was always needed after recovering from a lock loss. A good example is shown in the attached screenshot. In that case, diff lost lock, and kicked both etmx and etmy suspensions (top right plot showing the diff error signal in purple). At that point the green transmitted power in the X arm (bottom left plot) dropped to 0, and came back to 0.5 after the test mass settled down (instead of expected 1). This was due to the alignment : comparing the pitch of the test mass with the oplev before and after the event, we see that etmx pitched down by ~1.5urad (left middle plot in green). Neither ISIs HEPIs SUS were tripped during those times, the Ry DC values (corresponding to sus pitch) from ST1 ST2 ISI and HEPI were identical, etmx yaw was identical, itmx pitch and yaw were similar.
Sheila then used itmx (bottom right plot in yellow) to realign the cavity, pitching it by ~1.5urad which was just enough to bring the transmitted power back to 1 (bottom left plot in red).
It would be interesting to know if this is what happens all the time.

- Keita to EY to work on WFS
- Jeff and Andres to HAM5
- Betsy in the LVEA for INS work
- Water delivery
- Ln2 delivery to CP2
- Restart guardian nodes, the memory is getting full.  Jamie is working on a long term solution to this
- Tour through the LVEA at 4:30 PM PT

stage 1 actuators, this happened when diff lost lock. 

As Dave reported earlier (alog 12078) we again experienced memory issues on the h1guardian0 machine over the weekend, this time associated with the BSC ISI guardian nodes.  The symptoms are the same as what we saw previously with the so-called "CSD" nodes: node worker process memory usage and thread count shoots through the roof.   The problem seems to have in fact started Thursday evening, roughly around the time I restarted all the ISI nodes to incorporate a seemingly inocuous improvement to the handling of the ISI "MASTERSWITCH" (alog 12036).  This turns out to be the key to the whole problem.

Later, on Friday, Arnaud reported that some of the ISI nodes went into error because of channel access errors associated with the masterswitch (alog 12058).  Associating this with the other issues was the eureka! moment for me.  Here's a summary of the issue:

The problem has to do with so-called "IFO rooted" channels.  Guardian modules can specify a channel access prefix, which is prepended to all further channel access calls (e.g. the prefix 'ISI-ETMX_ST1' will cause calls to "ezca['FOO']" to be converted to the channel 'H1:ISI-ETMX_ST1_FOO').  This has a couple of benefits, most notably that we can use the exact same code for systems that differ only in the prefix.  However, sometimes it is necessary for a prefix-specified module to monitor the state of a channel that is not in it's domain (e.g. ISI_ETMX needs to look at the state of the ISI masterswitch at 'H1:ISI-ETMX_MASTERSWITCH').  This is done by specifying the full channel starting with a colon (ezca[':ISI-ETMX_MASTERSWITCH']), which tells the ezca module to ignore the channel prefix for this channel.  We've been noticing problems with these IFO-rooted channels for a while now, but it now appears that those problems and the memory/thread usage problems are in fact related.  Each IFO-rooted channel call is somehow creating a new PV object that is not being managed efficiently.

Last Thursday I modified the ISI code to start monitoring the ISI masterswitch, which is an IFO-rooted channel for those nodes, after which the ISI nodes started having memory/thread issues.  Similarly, the CSD nodes connect to only IFO-rooted channels (and thousands of them) thus all the problems we've been seeing with those nodes.  (Note: the CSD problem can be easily solved by properly stripping prefixes from the channels it accesses).

The solution for the ISI nodes was a targetted change to the function that checks the current value of the masterswitch so that the switch state is not checked for the ISI stages.  This is a temporary change until we can get the core ezca handling of IFO-rooted channels fixed.

To illustrate the change, here is an ISI node running with IFO-rooted masterswitch checking.  After just a couple of minutes, the memory usage (SZ, RSS) and thread count (C) are both creeping up:

UID        PID  PPID  C    SZ   RSS PSR STIME TTY          TIME CMD
controls 19314  1488  1 91613 27120   8 15:47 ?        00:00:01       guardian ISI_ITMY_ST1
controls 19342 19314 13 112312 34300  8 15:47 ?        00:00:13         guardian ISI_ITMY_ST1 (worker)

In contrast, here is the same node after the masterswitch check was removed, and after more time has ellapsed than the example above:

UID        PID  PPID  C    SZ   RSS PSR STIME TTY          TIME CMD
controls 20932  1488  0 91621 27004   7 15:49 ?        00:00:05       guardian ISI_ITMY_ST1
controls 20967 20932  9 109968 24760  0 15:49 ?        00:00:48         guardian ISI_ITMY_ST1 (worker)

Node the memory usage and thread count are lower and stable after a longer time.  This is pretty conclusive to me that these IFO-rooted channels are indeed the cause of the problem.

I have committed the patch (userapps svn r8075) and restarted all nodes for BSC ISI stages (ISI_{BS,{I,E}TM{X,Y}}_ST{1,2}).  This should eliminate both the resource problems with the ISI nodes, and the EzcaErrors associated with the masterswitch channels.

Now it's off to figure out how to fix the problem upstream in the Ezca object...

Actuation limit reached.

Actuation limit reached.

Both were T240 trips.  There isn't a clear reason, the tour hasn't started yet, the door work is finished.

This could have been due to signals from the LSC which were msitakenly sent to the BS suspension, I'm not sure. 

This morning, I:

- added the SR3 and SRM baffles to the HAM5 table and all of their dog clamps - I don't know why I didn't anticipate that the AOS cookie cutters for these were incorrect since I just went through this at HAM4, but they are.  This time, I set all of the baffles in place according to the hole pattern on the table and the maps.  I will get the cookie cutters modified to be correct, but at least the weight is on the table and roughly in the correct place.

- switched out the TFE stops on the SR3 - this was more involved than I anticipated since the 2" all metal/TFE stops were trapped by their own EQ stop bracket (chalk that one up to another #facepalm #poordesign).  I ended up having to dissassemble the brackets somewhat in order to rotate them to get the old and new EQ stops to clear.  We shoulda done this before putting it i the chamber - put the TFE was so nice during the move...

- added a vertical 3" wafer holder and a 1" horizontal witness plate holder to the table

- added the temperature cable and it's 8 brackets to the table - I did not lace the cable since there is still an open question on how I can do this better this time.  It's bundled out of the way and is of negligable weight for SEI rebalancing.

- finished securing the cable routing.

- removed all tools/pans.

So, all payload is on the table and I've turned it over to SEI for balanceing/testing.

10:08 PDT, rebooted h1guardian0 due to memory swapping. All guardian processes started automatically.

HughR, HugoP,

We installed the rencently-designed HAM-HEPI generic position loops on HAM4 last Friday. We had tried in the past but there was an issue with the way they were saved as a .mat file, before being loaded by the commissioning scripts. This has been fixed, and the updated controllers, and scripts are available under the SVN:

Generic Position Loops:
/ligo/svncommon/SeiSVN/seismic/HEPI/Common/Generic_Pos_Loops/Position_Loops/HAM_HEPI_Generic_Position_Loops_23_May_2014.mat

Plots:
/ligo/svncommon/SeiSVN/seismic/HEPI/Common/Generic_Pos_Loops/Plots/

This Sled is trivial in that it only has one lens & the WFS are not on the Sled (already on table).  So basically, one just lays the optics out according to the drawing; this was done months ago.  Unfortunately, I didn't take photos of the layout or measure distances between optics; I opened up the Sled and did this the other day.

Photos for the Sled are here.

For the positions of the optics, I used a ruler and measured from the edge of the Breadboard to each individual optic.  So, we have:

• 11mm:  Edge of Sled -> L101 (center of lens mount)
• 57mm:  L101 -> M101 (front surface)
• 50mm:  M101 (front surface) -> Edge of Sled (toward WFS)
• 280mm:  M101 (front surface) -> M102 (front surface)
• 50mm:  M102 (front surface) -> Edge of Sled (toward WFS)

All of these measurements are +/- 1mm.  (see attached photo for rough drawing of layout).

Found the DTS frame writer had a kernel panic, and the ldas gw was frozen.  It's still down, and likely will be until this afternoon.

(Corey, Gerardo, Patrick)

Two days ago (about 4am Thurs morning) the ISS diffracted power started to drift down from about 6.7% down to 2.0%.  Once at 2%, it started to go into oscillations (roughly around 3am this morning); see attached 2-day trend.

Without instructions on how to address this (and no recent alogs noting this behavior), we scratched our heads a bit, moved some gains and hit buttons to no effect.  We returned back to the original state.  Then Patrick suggested we try switching from PD A to PD B, and this appeared to stop the oscillations.  It took us to a value of about 5%.  Unfortunately, it looks like the power is still sloping down however.  (Curious to see if the ISS goes back into oscillations at 9pm tonight).

J. Kissel

The Messages:
(1) The effective bias voltage charge on the test mass is -28 [V_pk], which quite small compared to the BIAS range of 400 [V_pk] (and in conflict with Keita's assessment from the optical levers [see LHO aLOG 11905]).
(3) The force actuation strength is asymmetric with respect to bias voltage, confirming Keita's result (see LHO aLOG 11872)
(2) The force coefficient at 11 [Hz], measured as a function of bias voltage, is roughly 0.5e-10 [N/V^2], a factor of 8 below the expected 4.2e-10 [N/V^2] (expected from pg 7 of G0900956)

DETAILS
-------------
Brett: please proof read:
-------------
Note, no linearization algorithm was used during this experiment.

(1) Using the time-honored MIT method of measuring test-mass charge (e.g. from Brett, John, and Lisa), I drove a single frequency line into the control voltage H1 SUS ETMX ESD drive, looking at the X arm cavity length (as measured by green), while stepping the bias voltage through its entire range. As one steps down the bias voltage, we expect the linear response to drop to zero. Further, one can take the slope of the response as the bias decreases from both the positive and negative side and predict an effective bias voltage created by residual charge on the test mass. I attach the results. Using a linear regression, weighted by the coherence as follows,

unc = sqrt( (1 - coh) / 2*nAvgs*coh )
weight = 1 / unc^2;

I calculated the intersection of the two slopes to be -27.99 [V_pk].
I took an excessive amount of data points, because I didn't know what I was doing when I started and wasn't sure how the results would turn out. The template is here for excitation:
/ligo/svncommon/SusSVN/sus/trunk/QUAD/H1/ETMX/SAGL3/Data/2014-05-15_H1SUSETMX_L3_ChargeTest.xml
For the record, I tried using the script recommended by Brett (LHO aLOG 11914), but after updating all the necessary channel names and input variables. it failed deep with in its subfunctions because of some xmlconv library that's now gone missing since 2009. I didn't have the time nor expertise to debug, so I just changed the bias by hand and measured the ten averages with the GUI DTT session, capturing references as I went, and exported at the end for processing.

(2) No further details here, it's obvious that the drive strength is weaker with positive bias voltage than with negative. I have no good explanation for this.

(3) From these displacement response results, one can also obtain the force coefficient for each bias step. The calculation is as follows:

disp_mpk = 1e-12 * sqrt(2) * sqrt(binWidth) * disp_pmrtHz;
force_Npk = disp_mpk * 1./compliance_11Hz.mpN;

V_CTRL.Vpk = nActs * sqrt(2) * sqrt(binWidth) * esdGain * excChannel.amplitude.V_DAC.VrtHz;
V_BIAS.Vpk = bias.V_ESD;

forceCoefficient = abs(force_Npk ./ (2*V_CTRL.Vpk * V_BIAS.Vpk));
where

disp_pmrtHz = displacement response of the arm cavity length at 11 [Hz] during each bias voltage setting, in [um] (which I subsequently turned into [pm] for plotting clarity) as measured from the calibrated ALS control signal, H1:ALS-X_REFL_CTRL_OUT_DQ
sqrt(2)*sqrt(binWidth) = sqrt(2)*sqrt(0.09375 [Hz]), calculation needed to change from noise units ?_{rms}/rtHz to amplitude units of ?_pk
1e-12 = 1e-12 [m/pm]

compliance_11Hz.mpN = 5.3e-06 [m/N], TST to TST, L to L transfer function at 11 [Hz], obtained from the QUAD model.

excChannel.amplitude.V_DAC.VrtHz = 12.45 [V/rtHz], measured output request voltage from the DAC (calibrated from counts out of the digital last output to ESD, i.e. H1:SUS-ETMX_L3_MASTER_OUT_LL_DQ using the 180bit DAC gain, 20/2^18 [V/ct])
esdGain = 40 [V/V], gain of the ESD driver
sqrt(2)*sqrt(binWidth) = sqrt(2)*sqrt(0.09375 [Hz]), calculation needed to change from noise units ?_{rms}/rtHz to amplitude units of ?_pk
nActs = 4, since the calibrated output requested voltage is only from one channel.

and I've computed the force coefficient, a, using only the linear term, since I used the amplitude of the linear term alone

F = a (V_CTRL sin(wt) - V_BIAS)^2
  = a V_CTRL^2 sin^2(wt) - 2 a V_CTRL V_BIAS sin(wt) + a V_BIAS^2
      [ sin^2(wt) = 1/2 - 1/2 cos(2wt) + O(4w) ] 
  = a(V_BIAS^2 + 1/2 V_CTRL^2) - 2 a V_CTRL V_BIAS sin(wt) + - 1/2 a V_CTRL^2 cos(2wt)
         DC Term                       Linear Term                 Bi-linear Term
=>> Linear Term,
  a = F (2 V_CTRL V_BIAS)


Confusingly, this resulting coefficient, a = 0.5e-10 [N/V^2] is a factor of 8 weaker than expected, which is *different* than the factor of 4 weaker that was needed to explain the linear transfer functions taken last week (see the second attachment of LHO aLOG 11676).

The script that processes the measurements is here:
/ligo/svncommon/SusSVN/sus/trunk/QUAD/H1/ETMX/SAGL3/Scripts/analyzeesdcharge_20140515.m