To complete the autoburt alog: how to restore from a snapshot file.

Just 'cd' into the directory containing the snapshot files to be used
for the restoration. Then use the 'burtwb' command, it takes the snapshot
file as an argument.

For example, burt restore g1isihamepics to 14:01 snapshot of jul 20th 2010:

[controls@seiteststand 14:01]$ cd /data/autoburt/2010/07/20/14:01/
[controls@seiteststand 14:01]$ burtwb -f ./g1isihamepics2010072014:01.snap 

I have created an autoburt for seiteststand. It works by scanning the target
area for any autoBurt.req files. For each file it finds, it performs a burtrb
and saves the snapshot file in the /data/autoburt area.

The autoburt runs every hour at one minute past the hour (as a cronjob
as user controls on seiteststand). If we need to run at a different rate
this is just a crontab change.

Currently the script (/opt/rtcds/geo/g1/scripts/autoburt) uses the geo/g1
targets. When we move to X1 this will have to be edited.

Here is an example the data directories and snapshot files names:

[controls@seiteststand 14:01]$ pwd
/data/autoburt/2010/07/20/14:01
[controls@seiteststand 14:01]$ ls
g1isihamepics2010072014:01.snap  g1x01epics2010072014:01.snap

Richard took another look at the EE QUAD test stand and fixed some current issues, such that signals are now back to expected values of ~32k counts.  He also did some work on the satellite box for the AOSEMs, as he determined that jumpers were needed there.

The logbook will be unavailable from 10-noon tomorrow.  If you are working on an entry during that period, please save it as a draft before 10am Pacific time.

The logbook will undergo some maintenance tomorrow morning.  This is to fix some layout issues with the quick search area for webkit based browsers (ie Safari and Chrome).

Also it appears that there still is an issue with log session times. It appears that simply extending the Shibboleth SP Session time to a large value is insufficient.  I will attempt another short term fix tomorrow.  Ultimately it must move to a system that will require re-authentication when the Shibboleth session times out.  However it is not quite a quick fix as we must ensure that no data can be lost when handling this.  Likely the user will be given a notice that they must login again, but that their post (if there were any pending) have been saved to a draft.

My script which generated the G1ECU.ini file had a bug
and built a file with syntax errors in it. This is the
reason for the "daqd respawning too fast" error as the
daqd process was being autostarted by inittab and crashing
out. I fixed the script (/opt/rtcds/geo/g1/scripts/create_edcu_ini.pl)
and also changed it to generate a G0EDCU.ini file (not G1).
At this point daqd started. However the EDCU complained about
another IOC at 10.12.0.13 which had all the channels 10.11.0.24
was serving. Found out that eth1 was active with the 10.12.0.13 IP
address, and the IOC was being found on both IP ports. I
disabled eth1.

we should watch that eth1 is not reinstated on the next reboot

Later Corey was having nds issues, so I restarted daqd and nds
together since they had a split-start earlier in the day (see
above).

(Richard, Dave, Corey)
Came in this morning to find INVALID/WHITE medm screens on the iMac near our Test Stand rack in the Staging Bldg.  I tried to ssh into the frontend to no avail.  Since we were in this state and flying blind (diagnostics-wise), we tried several things to get the frontend re-started:

* Hit RESET button on front
* Hit REBOOT button on front
* Pulled out (both) power cords from back

None of these actions did anything (some were tried several times).  We had a monitor directly hooked up to the front end, and after various steps, we'd get the following error (right after a login prompt):

INIT:  Id "fb" respawning too fast:  disabled for 5 minutes"

Richard logged into the frontend via the login prompt, and was finally able to sorta get the frontend started.  We still had issues though.  Richard then tracked this down to a network issue (can't remember how he discovered this).  He then unplugged the power to the Network router on top of our rack.  After this, we were able to get back mostly.  There was still a daq/framebuilder issue, and Dave resolved this.

Another get_data Issue
Towards the end of the day, I tried running a matlab measurement, but there were issues with get_data (our matlab script made 25 attempts at getting data to no avail).  Called up Dave, and he ultimately restarted our nds and daq (I believe he said one of them was old).

This seemed to have done the trick, since I was able to get_data on my next attempt.

After doing another set of yaw adjustments to get the side BOSEMs in range at the Tablecloth and jacking the TC up by a mm or 2, we continued work to install BOSEMs.  So far, we have MO FACE 1,2,3 aligned to 50% OLV, with the rest cabled and mounted to the Tablecloth.  This also required the addition of washers behind the FACE magnet flag assemblies in order to get the flag into the BOSEM range.  (I recall seeing this at LASTI, but had hoped it had been reconciled in the later drawing revisions.  Guess not.)
DV and the "InputSignals.adl" medm both proved healthy and useful for this work.

Richard verified that the uncalibrated signals which go from 0 to ~ -4000 counts indeed correspond to 0-2V, so we can make some calibration of the channels eventually.

Mark Barton came over and began working through input matrices and appropriate signs in various fields.  

AOSEM installation into Q1 is back on hold, while John re-thinks the RGA scans which had a pretty good air leak.

More of the same tomorrow.

I have begun a low-frequency (down to 0.001Hz) measurement on the LHO X1 TestStand. This should take approx. 12hrs, barring any earthquake or major seismic event triggering our watchdogs.  It was begun tonight at approx. 18:45 (local). Please no changes to filter banks, medm screens, or use of awgstream. Thanks.
 

So, I jumped the gun a little on the newly-calculated matrices from Jeff K. & Celine R.  We decided to revert the matrices to the previous values for unit_1 measurement consistency.  The new MATLAB versions of the matrix calculations I ran yesterday will be implemented for unit_2 and beyond...

Previous filters via the bash scripts located in directory: 
'/opt/svncommon/seisvn/seismic/HAM-ISI/X1/Scripts' on the X1 TestStand.  They are titled 'setgeo2cenmtrx', setdisp2cenmtrx', and 'setcont2actmtrx'. 

I wrote a script to generate the G0EDCU.ini file from the
model db files (every epics record is put into the ini file).
I ran the script and noticed that we are only recording
2066 of the 4320 epics chans now in the system.

On the next daqd restart, the number of chans should go
up to 4320 and the full frame should get larger than the current
7.1MB. Since ISI measurements are ongoing, we'll delay the daq
restart till next week.

The script to generate the EDCU.ini is:
/opt/rtcds/geo/g1/scripts/create_edcu_ini.pl

At the moment we should run this script by hand each time the hamisi
model is changed.

Working on the damping loops. The controller will be made of:

- a compensation filter to account for the difference in the electronics between HAM 6 and X1.
- the original damping loops from eLIGO.

The plant, controller, open loop, closed loop and sensitivity are shown on the figure attached.

I have updated the G1ISIHAM.txt filter file and loaded the filters in the MEDM. It should be ready to go.

 

The scripts used to calculate the coordinate transformation matrices for the HAM-ISI have been updated by Jeff Kissel to include the Stage 0 L4Cs and include new measurements and calculations by Celine Ramet at Livingston and documented here:  https://dcc.ligo.org/cgi-bin/private/DocDB/ShowDocument?docid=13330

The scripts used to calculate and write the new matrices are located in Seismic SVN repository in '~/HAM-ISI/X1/SensorDefinitions/'.  The script to calculate the matrices is 'make_HAMX1_projections_100709.m'.  Since we have the "ezca" library installed on the TestStand, the matrices can be written directly to EPICS channels via the script 'fill_matrix_values.m' in the repository directory ~/HAM-ISI/X1/Common/MatlabTools/'.

I implemented the new matrices this afternoon and confirmed the changes in the medm screens from the overview screen 'G1ISIHAM_HAM_OVERVIEW.adl'. The new matrix values are attached below. 

Note: the L4C2CEN matrix was NOT written.


Some notes by Jeff Kissel on the new matrix calculation scripts:

- The GS13 coordinate definitions now use the center of the proof mass as measured by the model in SolidWorks instead of the bottom of the flange + 15 cm (Thanks Celine!). The L4Cs are also defined from the proof mass. 

- I've defined the displacement sensor directions to be the opposite of the direction of the rest of the sensors in the cluster. E.g. the H1 GS-13 and H1 actuator point -60 deg from the +X axis, and the displacement sensor points in +120. This is to account for the fact that an increase in displacement sensor gap *decreases* the voltage. I've confirmed that with this addition, the signs of every applicable matrix produced by make_HAMX1_projections_100709.m match those currently in place at on the L1 HAM6 ISI and H1 HAM6 ISI.

- Even with the above GS13 coordinate change, and a couple of other minor differences in coordinates between Celine's measurements of the aLIGO system and HPD's measurements of the eLIGO system, the absolute value of each matrix element in the GEO2CEN matrix changed less than 1e-5, and the DISP2CEN, and CONT2ACT matrices changed less that 1e-12 compared with  L1 HAM6 ISI and H1 HAM6 ISI.

- The colocated basis is defined with respect to the cartesian basis as though it were an eLIGO HAM6 ISI with V1 on the +X-axis. Therefore the matrices will be valid only for for all test stand HAMs (assuming we keep the V1-on-the-(+X)-axis convention) and aLIGO HAMs 4,5,6 (10,11,12) as per G1000125 -- for the remaining tank's we'll have to rotate everything by 90 deg. I'll create a separate "Coordinates" script to do this some time closer to when we need it. 

BOSEMs were hooked to the EE test stand today.  Signals look live but need calibration before interpretation.  Medm looks to have a nice set of appropriate filter banks for the QUAD controls, but interestingly, does not have a main.adl.  We're also looking to load or rebuild the medm's Brett created for LASTI which have easy to read voltage meters to aid in alignment.  AOSEMs are fresh from the oven and will be installed in the PenRe tomorrow.

The Sleeve was installed onto the QUAD for fit-check today.  It is cumbersome to man handle into place, so we should fiure out a better what to install it...  Clearance holes in the structures could be opened up a bit.

Attached are spectra of the Assy #1 GS-13's when the table was Locked & Unlocked.

Ran three different spectra for our Displacement Sensors.  Since we currently have an electronics issue with the operation of our Sensors (there's "electronic cross talk" between our mini-racks when they're both powered up---resulting in a ~0.35Hz oscillation seen by all sensors) we ran three different measurements in the following running states:

1)  Both mini-racks powered ON*
2)  Only Rack#1 (for H1V1H2V2) powered ON*
3)  Only Rack#2 (for H3V3) powered ON


* DTT BUG:  Although the plots for these measurements say there was only "1 Avg", they in fact were run at the required 10 Averages.  (When one saves References from a previous measurement, and makes a new plot of these References, the new plot posts the averages as "Avg1".  (If one already has the Reference plots made and one re-runs their "Now" measurement, the average value will change to whatever your "Now" measurement Average # is.

Tonight, I had hopes of starting one of Fabrice's transfer functions overnight.  Unfortunately, I was not able to because I was never able to clear the Watchdog.  As for what seemed to be causing the trips, for the most part, the H1&V1 Actuators would immediately rail to 32k.  This would cause two things:  an Actuator WD trip & an H1/V1 Geophone WD trip.  Additionally the Actuator would remain in RED in the "First Trig" state.  I tried various tricks with gains, and turning off input/outputs to no avail.  Looked at the rack and nothing was amiss.

Reboot of seiteststand
At this point, I performed a reboot of the frontend.  It was straightforward and there were no issues, but it didn't change the situation.

Matrices Re-filled
All of our filter bank paths are fairly simple (no filters engaged and gains of 1 all around).  Oddities I found were in some of the matrices.  In the CONT2ACT matrix there was a 1 in a place there wasn't supposed to be a 1 and one of the matrix elements had a sign different from what it should be as well (see attached image of cont2act matrix).  Because of this I decided I might as well run the scripts to fill the matrices.

One more note:  the DispAlign matrix was also empty.  So, I put 1's down the diagonal of this guy as well.

Matrix Set-Up Scripts
Jeff Kissel has some bash scripts which fill the HAM-ISI matrices (they are the ones used to fill the HAM6 ISI for both sites).  Since Dave copied over the epics bin area to the seiteststand, we were now able to run bash scripts (as well as use commands such as "caput" & "caget").  Nic helped me edit the scripts to make them work (among some location-dependent changes, commented out the --noprofile --norc values at the beginning of the script.  I ran the scripts for the following matrices:  Disp2Cen, Geo2Cen, & Cont2Act.

After all of this, I was still not able to clear the Watchdog (or run our measurement).  I've attached a snapshot of the Watchdog.  Notice the "First Trig" of the Actuators (and how the Actuator Monitors are all 0).  As soon as I click RESET, these monitors would show H1/V1 rail to 32k.