Michael, Krishna
We made a lot of progress on the BRSX upgrades today.
The new belt driven turntable was installed which is designed to have less vibrations during damping. The previous turntable used gears and a loud stepper motor which caused vibrations which in turn drove up modes of the balance. (see 26265) The new turntable uses a belt and a much quieter Bekchoff stepper motor which has noticably less vibrations. Also, the Mightex camera, a 2048 pixel CCD read over USB, was replaced with a Basler camera which has 4096 pixels and is read over ethernet. This new camera will remove the USB communication error which was causing the system to crash every couple weeks.
Additionally, we replaced the laptop and NI DAQ card with a Beckhoff computer and electronics. Both the C# code, which turns images into angles, and the PLC code, which controls the damper and sends data out, are working and data is being sent out to the seismic front ends. However, the EPICS communication has yet to be set up so many of the diagnostic channels are not being read.
The balance is rung up to a very large amplitude due to the work today but we've begun to slowly damp it down with the new turntable.
During the power outage recovery,(06/06/2016) took the opportunity to replace the PSL in-line coolant filters. The first set of filters are located in the Chiller Room on the discharge side of the chiller. There was some debris in the bottom of the Crystal Chiller filter. There was significantly less debris in the bottom of the Diode Chiller filter. There was little difference in the discoloration between the two filters in the Chiller room. After 31K hours of operations these filters are in very good condition. The second set of filters are located in the PSL enclosure on the intake side of the cooling system manifold block. There was very small amount of debris in the bottom of the Crystal chiller filter. There was no noticeable debris in the Diode chiller filter. There was no noticeable difference in discoloration between these two filters. As expected, these filters were not as discolored as the filters in the Chiller Room. In general, the PSL filtration appears to be working very well. In the future, these filter will be replaced at the same time the primary and backup chillers are swapped (planned for once a year). I am working on getting a local lab to analyze the debris from the Cristal chiller. When we know what all this material is, we can make a better informed decision as to the internals of the PSL cooling system.
ETMX Mode3: FM1, FM2, FM4, +10 gain (BP, -60deg, 100dB) seems to have worked quite well. No need to turn on a damping for 505.707 (which should be checked again with 0.0001 BW to see if it's really exists). 505.707 should be watched while damping 505.710 just in case.
The new violin mode table is being uplated slowly. The modes with Damp Setting column filled out are the one that's been sucessfully damped by the filter settings indicated in the table.
Jenne, Sheila
With the long lock tonight we got to do some more violin damping. I haven't updated the wiki or guardian.
At first we thought the largest mode was the 505.71 mode that Nutsinee logged about, and tried to damp it without sucsess using the settings she described. After getting a 0.002 Hz spectrum we could see that it was really 505.706Hz. (on the wiki there is a note asking if this mode really exists, looks like it does) I tried to damp this with ETMX MODE 2, which has a narrower filter, and in the last 5 minutes of the lock it looked like a phase of -60 and gain of 30 was working, with the 505.71 notch engaged, but we unlocked before we could be really sure it was working.
We damped the mode at 506.922 (ETMX mode5) using a phase of 0 degrees and a gain of 30. FM1 and FM4. (This is different from what is on the old wiki) the new wiki has this mode not damped.
We also damped a 500.062Hz mode (ITMX mode 1, 0 degrees gain +30, also different from the wiki) With higher gain and the same filter settings we rang the mode up.
The top panel shows the fundamental violin mode forest at the begining of the lock in blue, and the end in red. (There is a tiny improvement in 505.707Hz) The bottom panel shows the 0.002 Hz spectrum with cursors at 505.71 and 505.707Hz.
A negative gain for the 505.707 mode worked for me (although it's not clear whether it damped 505.707 or 505.710).
The setting I had seems to work on both 505.707 and 505.71.
Morning Meeting
- ER9 in a month
- Extended maintenance is cutting in commissioning time. Try to not do it on Mon,Wed, Fri
SEI: No issue on recivering
CDS: "Fine"
Software: Don't assume things are perfect (due to the recovery from scratch).
FAC: Facility assessment tomorrow
Activities (All time in UTC)
~15:30 Fil to EX
16:00 Kristina to MX, Karen to MY
16:48: Robert setting up stuff by PSL
17:00 Robert out
18:04 Karen leaving MY
18:20 Betsy to LVEA
18:30 Betsy out
19:20 Fil back
~19:30 dave to Mid stations
20:00 Dave back
20:05 Betsy to optics lab
20:20 Fil to EX outside VEA (pulling cables)
21:01 Kyle to CP3 overfill
Patrick, Krishna, Michael, Dave, Jim to EX to install BRS Beckhoff comp.
21:29 Kyle back
21:55 Dave+Jim back
22:07 Patric+Fil back
22:10 Fill dropping stuff at EX
Drift over 3+ hours:
1410 - 1425 hrs. local -> To and from Y-mid Opened exhaust check valve bypass, opened LLCV bypass valve 1/2 turn -> LN2 at exhaust after 4 minutes and 40 seconds -> Restored valves to as found -> Increase CP3 LLCV manual %open from 19% to 20%. Next manual overfill to be Friday, June 10th.
Terra, Carl B., Rich Terra and Carl had noticed a ~20mVp-p 1.3MHz oscillation on the ETMX ESD Driver at viewed with an oscilloscope at the output (SHV connectors on the front panel) leading to the ETM. After confirming that the oscillation was really generated by the ESD Driver, Carl and I opened the spare to see if we could find a set of conditions that might produce such an oscillation. By examining the spare, we were able to trace the potential instability to the first stage of the two stage pole-zero dewhitening filter. Poking at this stage with my finger will occasionally get it to latch into the 1.3MHz oscillation state. We drove the output of the opamp in question with a 1vp-p, 1kHz square wave through 400 ohms to get a better feel for the dampling and phase margin under normal operating conditions. This stage is completely stable with the pole-zero stage engaged, but becomes marginally stable with the pole-zero stage bypassed. The problem can be cured by the addition of a small (~30pf) capacitor across the feedback 21kohm resistor implying it is worsened by input capacitance. We opted not to pull all the drivers and fix this problem. The reasoning is that this stage is in its stable state (pole-zero engaged) while the driver is used as a low noise driver for length control of the ETM. There is really no driving need to fix this at this point, but we wanted to document its existence for optional repair at a later date. A fault report will be written to further document the issue.
Added to FRS https://services.ligo-la.caltech.edu/FRS/show_bug.cgi?id=5675
As I reported in 27622, the elevated noise is still present in the Stage1 V1 CPS of the ITMX. We should try a power cycle of the chassis at earliest opportunity. I don't have access to DetChar at the moment to see if this noise is impacting the ISI performance.
It is very likely that a noisy vertical sensor will create excess noise in the horizontal direction, particularly if the 45mhz blends are being used
I'm going to guess that a power cycle isn't going to help, I would consider swapping the electronics
But Rich, If you look at the history in the plots and words, you see that this type of noise has come and gone. The noise on this channel wasn't there before the power outage cycle and the others that were present have gone away.
I have analyzed the IM OSEM signals from the May 19th ISI trip, while both the ISI and the IM optics were tripped, and have injected a step funtion into TEST PIT and TEST YAW to measure Q values.
Power Spectrums:
Length signals for IM1-4, and the signal for IM4 shows clean peaks for yaw and it's 1st harmonic and length and it's first harmonic. All other IMs have peaks that are lower than the peaks in IM4.
Q values:
I measured the length and yaw Q values for each IM, and my original post is alog 27327. In that post, I list pitch and yaw, however the resonant frequency I measured that's labeled as pitch is actually length.
IM4 shows the highest Q values for length and yaw, and comparing the Q values for IM1-3, they are all between 54% and 95% of the IM4 Qs.
Time Series:
The time series of the individual OSEM signals reveals a difference in amplitude in the oscillations of OSEMs, when comparing the four OSEMs on each optic.
I measured the period of oscillations for each OSEM on each IM, and in reality the period was not very enlightening, however, I also collected the oscillation amplitude of each OSEM near the end of the time the optic was tripped. The oscillation amplitudes proved to be very revealing, and show that the OSEMs on IM4, the IM that has never shown an alignment shift , have very different behavior than the OSEMs on IM1-3, that have all had significant alignment shifts.
Details / Summary of power spectrums for IM1-4: attachment 1
IM1 length | 1.21 |
IM2 length | 1.43 |
IM3 length | 1.36 |
IM4 length | 1.50 |
IM1 yaw | 1424 |
IM2 yaw | 1087 |
IM3 yaw | 1270 |
IM4 yaw | 2009 |
Details / Summary of oscillation amplitude data for IM1-4: attachment 2
IM4: no alignment shifts observed in my IM investigation, highest average percent of max oscillation at 94%
IM2: largest and most consistent alignment shifts observed, has the OSEM with the lowest percent of max OSEM oscillation at 70%
IM3: second largest alignment shifts, has the lowest average percent of max oscillation at 81%
IM1: smallest alignment shifts, one OSEM at 92%of max oscillation, others at 76% and 82%
TITLE: 06/08 Eve Shift: 23:00-07:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Corrective Maintenance
INCOMING OPERATOR: None
SHIFT SUMMARY: Spent the entire shift getting through Initial Alignment. Major by-hand alignment required at every step of IA. Things are still drifting as a result of temperature excursion during power outage. But we made it . . through IA that is.
LOG: 8 hours of aligning
We were able to engage vertex angular loops and hard arm loops without issue. The soft arm loops pulled the recycling gain up but made the sideband powers drop. This may indicate the PRM pointing loop is fighting the soft loops.
Violin modes are still high (about where they were on Friday) and will need attention if we want to engage DCPD whitening.
One of the problems we encountered was that the X arm alignment was so far off when we started that the ITM camera image was clipped on the digital aperature, and the servo appeared to be running but didn't bring the spot to the reference position. Travis moved the ITM by hand to fix this.
DAQ frame writers
The frame writers instability cleared up overnight without us doing anything. Today fw1 was more unstable than fw0 (one fw0 restart, three fw1 restarts). Still a far cry from fw0 restarting every few minutes yesterday afternoon.
DAQ EDCU connection to digital video channels
After the power outage the DAQ EDCU would not connect to any digital video channels. Today I remembered that the EDCU only uses EPICS gateways to access IOCs on other subnets. The gateway between the H1AUX lan and the H1FE lan was not in the start script. I started the gateway and corrected the script. The only channels not connecting to the EDCU are now the end station HWS channels.
HWS code at end stations
Nutsinee, Jim, Dave:
When we tried to start the HWS EY code ('python run_HWS.py') on h1hwsey, the EPICS IOC started with both ETMY and ETMX channels (though only the ETMY channels had valid data). This appears only a problem with the new python based system at EY, EX only runs ETMX channels. We'll continue this tomorrow.
SUS PI work
Terra, Rich, Jeff, Carl, Betsy, Tega, Ross, Dave
model changes were made to h1susitmx, h1susitmy, h1susbs, h1susitmpi, h1omcpi. DAC allocations were changed, SHMEM IPC added for binary data exchange, new quad master, new C code. There were several rounds of model and DAQ changes.
ETMY HWWD
Dave, Jim
We are unsure if the hardware watchdog unit at ETMY is working after restoration of power. One sure way is to power cycle the unit and monitor the LEDs for the startup sequence, but that would power down the ISI coil drivers. We can check if the code is running by disconnecting one of the monitor cables and checking for the LED error. We'll test some more on the test stand before risking ETMY.
DTS
Jim
The DTS was powered up and made fully functional. Interestingly like H1 some front ends wound up with the incorrect time because the boot server had not synced up. We'll remember to check on that in future (Tega got compile errors on x1lsc0 because of this).
Attached are the usual long trends of the charge on ETMx and ETMy with the latest data appended. The ETMx charge accumulation is still decaying towards 0, but not there yet. The ETMy charge is also near zero in almost all 4 quadrants. Kissel and I decided we will leave the signs of the ESD bias as they are for another week. It doesn't look like the ETMx data from today was affected too much with the ISI tripped this morning (the SUS was still damped).
J. Kissel, R. Abbott, T. Hardwick, C. Blair, D. Barker, and J. Betzwieser After a good bit of chasing our tails and simulink model juggling, we've confirmed that the new infrastructure necessary for the new ITM ESD drivers (see E1600064) is ready to go. I'll post more details and screens later, but below are a summary of the things we'd touched during the day's installation: - Several top-level models are affected by this change, /opt/rtcds/userapps/release/sus/h1/models h1susauxb123.mdl <-- for the new voltage monitor channels h1susbs.mdl <-- in charge of distributing the BIO signals h1susitmx.mdl <-- integration into the control system h1susitmy.mdl <-- integration into the control system - We also, sadly needed to create a new QUAD library part, because the ITM ESD driver infrastructure is that much different from the ETM ESD driver system; that now lives here: /opt/rtcds/userapps/release/sus/common/models/QUAD_ITM_MASTER.mdl and in addition, we added a new function to the /opt/rtcds/userapps/release/sus/common/src/ CD_STATE_MACHINE.c called "ESD_ITM" which is called in the BIO block of the new QUAD library part. - Also, in the clean up of models, we also re-installed all DAC cards directly from the CDS_PARTS library, which means that a few of the MEDM macro descriptions needed updating -- plus the new macros for the ITM ESD drivers themselves. Those are in /opt/rtcds/userapps/release/sus/common/medm susitmx_overview_macro.txt susitmy_overview_macro.txt - Both the new BIO block and QUAD Library parts needed associated entirely new MEDM screens which are now separate offshoots of the ETMs, /opt/rtcds/userapps/release/sus/common/medm/quad/ SUS_CUST_QUAD_ITM_OVERVIEW.adl SUS_CUST_QUAD_ITM_BIO.adl - I updated the sitemap to call these new SUS_CUST_QUAD_ITM_OVERVIEW screens, passing the appropriate macro files for each optic. The sitemap has been committed here: /opt/rtcds/userapps/release/cds/h1/medm/ SITEMAP.adl
More details about this install can be found in G1601304.
During the power outage the controller at Mid-Y failed its transition from facility power to generator power, Kyle borrowed the controller from Y-End to power the ion pump at Y-Mid (IP9). Today I attempted to swap the DUAL type controller (borrowed controller) with a Gamma type, but the Gamma controller failed, brand new controller, so by mistake I installed the original controller that had been removed by Kyle, such controller decided to work now without problems, so I left installed. I returned the borrowed controller back to Y-End station to power IP11, but it turns out that the signal wire connector is falling apart and needs to be fixed, which will be done on the next opportunity.
The new Gamma controller was bench tested and fuses checked, it continues to fail, it only outputs -50 VDC, not good, it will get send out for warranty repair.
Pulled cable from the annulus ion pump controller on BSC5 all to way to the rack on the south wall of X-End VEA. Next opportunity we'll have the cable landed and terminated.
Michael, Krishna
We are investigating the electrostatic/capacitive actuator on BRS-Y by driving it with a small 50 mV modulation at 20 mHz and at different Bias voltages. BRS sensor correction is turned off and should not be turned ON during the test (winds are low). Avoid going near the instrument, if possible.
We saw the noise at XEnd ESD LL again at ~1.28 MHz, 15mV pk2pk. Rich A. came out to have a look; believe the noise be real (a real oscillation in the hardware).
On LR drive, we also saw the spectrum glitching (bouncing up and down) when driving at 15.5kHz; goes away with no drive signal (note we did not try other drive frequencies yet). Some beat making a low frequency component?
The source of the 1.2Mhz oscillation was identified by opening the spare chasis and looking for marginally stable opamp stages.
The stage that is marginally stable is U6 page 7 of D1500016. It can be made to oscillate at ~600khz or ~1.2Mhz. The stage is a configurable with the pole/zero bypass bit.
When for example the H1:SUS-ETMX_BIO_L3_UL_STATEREQ control word is set to 2 the stage has a pole at 2.2Hz. This is the normal low noise configuration. In this configuration there is no 1.2MHz oscillation.
When this control word is set to 1 the stage is nominally a unitly gain stage. In this configuration some channels (like UL UR LL and LR) have a Q of >5 at 1.2MHz and can be induced to freely oscillate. This oscillation may be damped with a 30pF capacitor across R21.
As this oscillation is not a problem in the low noise configuration no changes will be made. Testing PI channels should be performed with the H1:SUS-ETMX_BIO_L3_UL_STATEREQ control word set to 2.
I have copied Evan's new actuation function to the h1hwinj1 directory currently used for CW injections: ~hinj/Details/pulsar/O2test/. I used the one that corrects for the actuation delay: H1PCALXactuationfunction_withDelay.txt. For reference, the uncorrected version (no "_withDelay") sits in the same directory, along with the one we first tried last week: H1PCALXactuationfunction.txt.25may2016. The perl script that generates the command files in.N (N=0-14) has been updated to use "_withDelay" and the command files regenerated. The CW injections have been killed and automatically restarted by monit. Attached are second trends before and after the gap showing that things look about the same, as expected, but there is a small increase in injection amplitude (~5%).
Evan wondered if the ~5% increase in total injection amplitude was dominated by the highest frequency injection or one at lower frequencies. I took a look for this time interval and found that the total amplitude is dominated by the injection at ~1220.5 Hz. Simply comparing spectral line strengths before and after the changeover turned out not to be a robust way to estimate the frequency-dependent ratio of the new to the old inverse actuation function, because some pulsar injections (especially the highest frequency one) are going through rapid antenna pattern modulations during this period. But comparing the new to the old spectral line strengths at the same sidereal time several days later (after power outage recovery) gives robust measures for a sampling of low-, medium- and high-frequency injections:
Freq (Hz) | "Old" amplitude (before switchover) | New amplitude (4 sidereal days later) | Ratio (new/old)
190.90 | 0.32292 | 0.32211 | 1.00
| 849.00 | 60.502 | 62.344 | 1.03
| 1220.5 | 299.37 | 318.70 | 1.06
| 1393.1 | 207.50 | 224.37 | 1.08
| 1991.2 | 28.565 | 32.788 | 1.15
| |
Here are some pictures of the damper before and after the upgrade. The first one shows the old turn table in the bottom left under the arm and the Mightex camera wrapped in foil on the very top of the autocollimator. The second one show the old stepper motor and gears. The third shows the new turn table on the left, the new electronics box in front of the chamber, and the new Basler camera and mount on the top. The third is more detailed view of the turntable with the new stepper motor in the back and the drive belt running below the damping masses.
Also attached is a video showing the smooth motion of the damper. This was while the balance was rung up so the damper is moving from the counter-clockwise limit in beginning of the video to almost the clockwise limit about two thirds through.