Latest curve. 11 days shown.
loglog plot also attached - note that the BSC6 annulus was probably vented until day 8. There may be a slope change after this date.
Here is a check list for setting up/working with the ground STS2s (or T240)
Before touching a STS2 the THREE test masses should be locked using the special non-magnetic screw driver, the T240s do not have locks and are more rugged
After you are done mucking with it
Make sure that it is aligned using the swizzle stick
Make sure that the instrument is levelled (bubble level on the sensor)
Make sure that the feet are locked (unlocked feet are not stable)
For STS2s unlock the masses using the special screw driver
Restore the thermal insulation (exactly what we need is not clear, but we know that at least an enclosure is important)
Zero the mass positions (this happens at the electronics racks) and read mass positions to make sure that it was successful
------------------------------------'
Eventually it is nice to take a transfer function from the ground instrument to a set of ISI platform instrument to check the alignment
Jim B, Filiburto, Jim W, Dave
we powered down h1seih45 computer and its IO Chassis. Filiburto installed a fourth binary cable to the output port of the second Contec card (HAM5's card). He disconnected HAM4's CN-B from the upper 32 channel input of the binary-output chassis and connected HAM5's CN-A to this port.
We powered up the system, verified that the new logging script worked.
Jim W verified binary output switching for both HAM5 and HAM4 is working correctly.
reboot log shows the startup as:
We have insulated ~100 feet of beam tube as a trial. In the near future vibration measurements will be taken in order to understand the damping performance of this configuration.
Most recent attempt at this had no effect as I had made the programming changes to the 7000V-5000V step instead of the 5000V-3000V step
So we ran overnight at ETMY with the HEPI position loops on. I've revisited the data I posted yesterday morning. In the attached plot I've added some of last night's overnight running with the loops on. The dashed lines continue to be the off loop reference from ~2AM Tuesday, The solid line references(REF17+) are with the loops on but starting about 8am Tuesday, possibly a noisier time. The current traces are from 2AM Wednesday with the loops on. It is involved but my simple analysis says that some places are worse with it on and at another time it is better suggesting it is just dependent on the ground motion at the time.
When I started the loops yesterday I just put the current positions into the Target. From my log I put in 12 March, the Target positions should zero. The differences are 57 to 30 ums in position, 11urads tilt, and <7 urads Rz. The HEPI Position at ETMY is ready for controlling as needed; so if the current position is not close enough, go for it.
Partway through our accumulation on Y2 - strange CC gauge behavior on one gauge and different rate of rise on another.
We might have a small leak at our RGA setup - will check at the end of accumulation.
The default version of Matlab for the control room is now 2012b, as it was previous to Tuesday's internet outage.
So this actuator got 17+hours of bleeding, hopefully sufficient. Shoot, I forgot to grab the serial number again. Replacement SN is 22952
no restarts reported.
Report from Richard McCarthy:
The site contractor was able to make repairs to the fiber and our
network services and phones have been restored as of ~3 AM today.
Before we understood that the noise we were seeing was the bounce and roll modes of ETMX (see alexa's alog) we were looking around at Osem signals, and we saw that there is a large peak at around 25 Hz in L1_VOLTMON_LR_DQ. It was there before the bounce and roll modes were rung up, an it is there with damping on and damping off.
(Sheila, Alexa)
We have been trying to search for the 1/f feature present at 1-50Hz in the noise plots. We tried a few different things as described below. The noise spectrum under nominal conditions is represented in REF0-2.
All these plots can be found under /ligo/home/sheila.dwyer/ALS/HIFOX/COMM/COMM_COherences_April8.xml. In the set of three, the first reference is always the power spec, the second the RMS, and the third a coherence plot. I have attached an image which shows the nominal power spectrum and the cases with the HEPA fan on and one with the power reduced (NoiseSearchPic1).
For future improvement of the QPD servos, we adjusted the input beam onto the QPDs with the servo off, such that the beam was fairly centered onto both QPDS. We did not get a chance to work on the actual servos though.
We proceeded to do some more noise hunting, but halted the effort when we saw two new spikes in the noise spectra. One spike was at 9.8Hz and the other at 13.8Hz (see second png). Turns out these are the bounce and roll modes of the suspension that must have got rung-up during an ISI trip. (Refer to Sheila's alog).
(Sheila, Chris, Alexa)
Since we had seen a difference in the 1/f noise by decreasing the laser power at EX, we decided to try the following -- we placed an ND filter in the path of the x-arm green transmission on ISCT1. With the ND filter the ALS-C_TRX counts dropped from 0.94 to 0.471. We saw an increase by a factor of 2 in the 1/f noise. We would expect a factor of Sqrt(2). We should repeat this measurement by decreasing the power further. The red line in the attached figure is the nominal spectra; while the light blue line (REF 27) is with the ND filter in place. (Overall these spectra look higher then those posted earlier in the attached alog; it's possible that this change came from alignment drift).
Subsequently, we decided to look into the noise at the COMM BBPD.
Signal from the PD w/ no RF pre-amp @ 100Hz BW:
With PD blocked and no RF pre-amp @ 100Hz BW:
With PD blocked and RF pre-amp (+13dBm) @ 100Hz BW:
Instead of using the beatnote signal from the PD, we used a signal generator (-31dBm at 79.8Mhz) to lock the PLL and measured the in loop noise spectrum (with the input gain of the PLL at -25dB w/ the two compensation filters off). We will posts plots tomorrow. At first glance; however, this does not appear to be the 1/f noise source we have been hunting.
I have attached the plots and data we collected last night. The noise spectrum was measured out of IMON of the PFD with the PLL locked to a signal generator instead of the PD signal. This noise spectra is loop suppressed by the COMM PLL loop. The COMM PLL had an input of -25dB with the two common filters off (unlike the nominal settings). We took a OLTF of this modified loop.
The corner station controller medm was found white boxed. The computer (h1hpipumpctrll0) was running--we did not have to restart it. However, the servo program was not running. It has been down since early Sunday morning. The VFD was running at 31.xx Hz, the last value it received. System was recovered in the best way possible:
Restarted servo--see wiki. This database does not process any records when started and begins in manual mode, so even though its output is zero, it actually hasn't written anything to the VFD at this point and it continues driving the pump motors at 31hz.
The Servo motor speed slider was then rapidly dragged to the nominal running output for 80 psi. You can look in the trends at the CONTROL_VOUT channel, for the LVEA system it is ~1080. If you are dexterous enough, this will only slightly glitch the system if at all. I got lucky and saw no pressure drops.
Once pressure is steady, tweek the output til 80psi. Put 80 psi in the Setpoint (it is zero'd with the database restart.) Engage the Servo (push the On button.)
(Rich M, Sheila, Alexa)
HEPI and ISI ETMX tripped because we were playing with the suspension watchdogs.
We had a hard time recovering from this trip which caused several more trips, the wd plotting script is not working this afternoon, possibly beause of our network problems so we don't have any plots. Once HEPI was isolated, we tried letting guardian isolate the ISI. It tripped immediately on CPSs (stage1) after we cleared this and tried again it tripped on GS13 (stage2).
We then paused the guardian, moved all the blends to start, and gave the gaurdian another try. (note, the ground motion is fairly quiet today). This tripped again, GS13s.
We then puased the guardian, and tried the command script, stage 1 level 3, tripped on the actuator limit. I then stopped taking notes.
The rough procedure that Rich used to bring it back was damping both stages, turning on 1 DOF at a time RX, RY, Z, RZ, X, Y (stage 1 first, then stage 2). Then Alexa moved the blends over to Tbetter without anything tripping.
We don't know why the things that used to work don't anymore, but now ETMX ISI is harder to untrip and neither the guardian nor the command script seem to be able to do it anymore.
Rich may not have mentioned in the alog, but after our trip the other day while trying to switch blends, he changed the filters from zero crossing to ramp, which should make it easier to switch blends without tripping the ISIs.
One thing about guardian, its not clear to me how we should unpause if we need to pause it for some reason. TO pause it we paused all three, stage 1, stage2 and the manager. Wen unpausing the subpodinates they don't come back in managed mode. I was able to get them back to managed mode by going to init on all three, but in the meantime they were doing their own thing, but happily didn't trip the ISI. What is the best way to do this? Also, maybe the right way to pause and unpause a manager with subordinates is something that needs to be made more obvious to a user.
Alexa also tried pausing, she only paused the manager, bringing that back was not a problem but it doesn't pause the subordinates.
quoth Sheila:
We don't know why the things that used to work don't anymore, but now ETMX ISI is harder to untrip and neither the guardian nor the command script seem to be able to do it anymore.
But you did mention one likely important thing that changed: new blend filters (Tbetter?). What happens if we go back to Tcrappy? Do we have the same problems?
also quoth Sheila:
One thing about guardian, its not clear to me how we should unpause if we need to pause it for some reason. TO pause it we paused all three, stage 1, stage2 and the manager. Wen unpausing the subpodinates they don't come back in managed mode. I was able to get them back to managed mode by going to init on all three, but in the meantime they were doing their own thing, but happily didn't trip the ISI. What is the best way to do this? Also, maybe the right way to pause and unpause a manager with subordinates is something that needs to be made more obvious to a user.
Alexa also tried pausing, she only paused the manager, bringing that back was not a problem but it doesn't pause the subordinates.
This is definitely sounds like something that could be improved. It is true that all nodes, manager and subordinates, need to independenty be paused and unpaused. I'll try to think about how to make that smoother. In the mean time, here's the procedure that should be used:
The manager INIT state resets all the works, restarts them, and puts them into managed mode. That should be the most straightforward way to do things for the moment.
Thanks Jamie.
We did try both the command script and guardian with all the blends on start, which used to work. I don't think we tried turning sensor correction off, which is another change.
I heard that rumors were spread that this alog was written during an earthquake that we hadn't noticed, so there is no reason to worry about our inability to use guardian or the command scripts to reisolate ETMX. That is false, there was no earthquake. PS, this is sheila, still loged in as Stefan.
Sheila, Chris, Alexa, Stefan
This morning we looked for coherences with many signals with our COMM noise. There weren't any surprises, we see the same coherence with the end station PDH control signal around 1kHz that we are used to, ISCT1 accelerometers around 70-100Hz, and a little coherence (-0.7-0.9) with the oplevs at low frequencies (pitch at 0.5Hz, yaw around 0.1). We don't see much coherence at all with the TMS QPDs, just a small bit at low frequencies. We lose coherence with the in loop sensor below around 100Hz.
Chris and I went out to End X to look into the beam quality problems. We didn't find much that was promising. We checked rather carefully for clipping on the way into the chamber again. We also set up the nanoscan again, and we see that there really isn't anything coming back to the WFS when the ITM and the ETM are both misaligned (so it isn't a second beam that is directly reflecting a lens or some optic on transmon.)
We also tried Daniel's suggestion of closing down an iris in the path. For WFS B this only made things worse. WFS A scans will be attached to this alog soon, but we didn't get any improvement there either.
We came back and tried out some of Rich's blends. We were able to get relatively low noise once without using WFS, 30z down to 0.02 Hz. The attached plot shows the COMM noise (measured by the IR) with different blends on. We are curently seeing a lot of noise on ETMX T240s at around 3Hz, (100 times more) Rich is looking into this.
