The results from today's diagnostic breadboard scan. The relative power noise measurement doesn't look too bad. Better than the reference measurement at low frequencies below about 20 Hz but worse elsewhere but not by much. The first loop power stabilisation servo performance is better than the reference signal at frequencies below about 40 Hz but worse for frequencies above about 1 kHz. The frequency noise measurement is out of whack. It is known that perhaps the gains on the servo are not adjusted to their correct values. Currently the common gain is set to 27 dB and the fast gain to 17 dB. This will have to be tuned up when time permits. The beam pointing measurement looks good, or at least okay. The mode scan looks almost identical to last week. The percentage higher order mode content is fractionally lower this week but not significant I think.
model restarts logged for Mon 11/Aug/2014
2014_08_11 09:36 h1hpiitmy
2014_08_11 09:36 h1iopseib1
2014_08_11 09:36 h1isiitmy
2014_08_11 10:42 h1iopseih23
2014_08_11 10:42 h1isiham2
2014_08_11 10:42 h1isiham3
2014_08_11 10:44 h1hpiham2
2014_08_11 10:44 h1hpiham3
2014_08_11 13:58 h1iopsusb123
2014_08_11 13:58 h1susbs
2014_08_11 13:58 h1susitmx
2014_08_11 13:58 h1susitmy
2014_08_11 14:34 h1iopsusb123
2014_08_11 14:36 h1susbs
2014_08_11 14:38 h1susitmy
2014_08_11 14:39 h1susitmx
recovery restarts due to several problems with cpu freeze and non-driving DACs
[Alexa, Dan, Masayuki, Kiwamu, Keita, and Koji]
(updated 8:15AM, correction of the window deflection angle)
HAM6 alignment work is in progress. The beam reached to the OMC but not aligned to the OMC QPDs yet.
Here is more details of the work done on Aug 11.
= Guiding the beam to HAM6 =
The beam was guided to HAM6 by centering the beam on the SRM. By shaking the SRM alignment in pitch and yaw, we could confirm
how the spot was centered. In reality, we initially suffered from not noticing intentional misalignment of PR2. Once the PR2 alignment was
restored, it was not so difficult to find the spot around the SRM aluminum holder.
= Locating the beam axis at the center of the clear aperture =
Once the beam was found in HAM6, we again shook the SR2 to find the aperture free from clipping by the SRM or OFI.
The beam was settled at the center of the clear aperture.
Note: this does not necesarily mean that the beam is still at the center of the SRM.
The beam coming in the HAM6 chamber has in deed a different angle from the design layout (D1000342).
The beam entrace point is located almost as designed. At the edge of the table the beam height was 97.5mm (3.84"). This already suggests the
beam is downward. The beam is angled not only in vertical, but also in horizontal. At the position of the OM1 tip-tilt, the beam is 2" to South
(towards OMC) and 0.5" too low. This angle is, in deed, 32mrad and 8mrad, respectively. This couldn't only be caused by the spot location on SR2
as it requires tens of cm shift on SR2. Therefore we suspected any wedged optics around HAM5, which means the septem window and
wedged prisms in the OFI.
Keita called LLO and confirmed they had similar horizontal beam shift but observed no vertical shift. This may mean that the horizontal angle
is a systematic effect. We confirmed the wedged window is installed so that it wedges almost horizontally (see attached photo).
Keita calculated the wedge effect of the septem to be 6mrad. This is not enough to explain the angle we actually saw.
We also wondered how the wedged prisms are installed in the OFI.
We checked if the beam power does make sense or not. The detected power right after the spetem was 3.2mW. This is reasonable as it is
not so far from the expected value of 4mW. [1.9W (Pin) x 0.8 (IMC) x 0.03 (PRM) x 0.35 (SRM) x 1/4 (Michelson) = 4mW]
= OM1 Tip-tilt / OM1 transmission path (AS_AIR and AS_C QPD) =
For now, we decided to continue to align the other optics. Firstly, we moved the OM1 to South (towards OMC) by 2".
The transmission path of OM1 had to be moved by 2" as well. This actually causes the counter mass moved towards East door by 2".
Therefore the table balance has to be reviewed eventually. The other steering mirrors were also moved as well. The first two steering mirrors
after OM1 had to be largely angled in pitch to absorb 8mrad beam tilt at OM1. The beam was centered on the last steering and the beam is directed
to AS_C QPD. We used CDS to finely align the beam on the QPD. The beamsplitter before the last steering was aligned such that the beam exits
from the enter of the window emulator.
The beam diverter and its beam dump were aligned. The beam diverter is open when the mirror is at the west side (HAM5 side)
and is closed when the mirror is at the east side (East door side).
= OM1 Tip-tilt alignment =
OM1 was moved so that the beam hit the center of OM2. This requred OM1 facing upward to compensate the vertical angle of the incoming beam.
This was done by the counter balancing screw on OM1. The range was not enough only by moving the screw, the screw was inserted from the back
to give more weight at the back side of OM1.
= Fast shutter / beam dump =
The fast shutter could not be placed at the planned place as the body structure of the shutter blocks the incoming beam.
The shutter was placed west side (HAM5 side) of OM3 rather than the east side. The upwarding beam in this path caused the fast shutter reflection
going up. This caused the beam on the beam dump too high by 1". This needs to be fixed before closing the door.
= OM2 alignment =
OM2 was rotated in yaw in order to accommofate the spot on OM3. The natural misalignment of OM2 in pitch made the spot on OM3 almost at the center.
= OM3 alignment =
We rotated OM3 to hit the input mirror of the OMC cavity. The OMC REFL beam was found. We did not hit any of the OMC QPDs yet.
= Beam clearance =
In the end, the beam is enough away from the tip-tilt cages desppite the move of the incoming beam.
The distance between OM2 and the incoming beam is about 2". The one between OM3 and the incoming beam is about 2.5".
The distance between OM2 and the relfected beam from OM1 is about 0.5" while the beam size there looked less than a mm.
= Adding dog clamps =
The many of the optics on the table was fixed only with one screw or a dog clamp. We started to add more clamps to them. It's not complete yet.
Hopefully the people who actually did the work will fill in the details, but here's my brief summary of HAM6.
Horizontal 2" shift is also here at LHO, in the same direction.
Koji and his crews found that the beam was about 2" off horizontally at OM1 toward OMC. This is the same direction and the same amount as what LLO saw. Koji had to move OM1 by about 2" to the South.
They also confirmed that, horizontally, the beam position at the edge of the ISI table is about right, and the beam angle relative to the screw hole rows is much smaller than it should be (the beam is almost parallel to the screw hole rows).
The beam is too low on OM1.
In addition to horizontal problem, the beam is also too low on OM1 when Koji set the beam at the middle point in the OFI by scanning SR2.
(I thought Koji said that it was an inch too low on OM1, but it cannot be as OM1 is a 2" optic. Maybe 1cm?)
Koji pointed the two steering mirrors behind OM1 to make the AS beam height right. He also needs to point OM1 up and OM2 down to make the OMC path right.
There are some speculations about why (horizontally and vertically) but we need to check the numbers first. Anyway, this should not stop us from aligning the OMC path for now.
Jeff K, Krishna V. After scrounging for the right connecting links between the vacuum chamber and the ion pump, we finally started pulling vacuum on the chamber in the afternoon, around 4 PM, with a pumping station (Turbo pump backed by a diaphragm pump). By the 7:30 PM, the pressure at the Turbo was down to about 6E-6 torr. The balance was centered on the autocollimator and we are taking data overnight. One side of the foam box is open for the vacuum hose so there is a lot of low-frequency drift due to temperature noise. Plots to follow soon. Tomorrow, we will repeat the 'transfer function' measurement to confirm the small distance between CoM and the suspension point/pivot.
Day's Activities
SR2 : top mass damping on / off
SRM : top middle bottom damping on / off
MC2 : top damping on
MC2 guardian was changed to "DAMPED"
SRM and SR2 are still "ALIGNED"
If mode cleaner/ SRC is needed tonight, feel free to kill the matlab sessions on opsws1.
Gains to decouple bottom mass (pitch and yaw) angular motion from top mass longitudinal drive at DC were implemented in L2P and L2Y components of the top mass drivealign matrix of MC2 using the dc magnitude of the top to bottom L2P (urad/Force cts) and P2P (urad/Torque cts) for length to pitch decoupling and L2Y, Y2Y for length to yaw decoupling. This should be tested.
alpha [urad/Force cts] |
beta [urad/Torque cts] |
alpha/beta [Torque cts/Force cts] | |
in pitch | -3.90E-03 | 5.37E-01 | -7.26E-03 |
in yaw | 2.20E-03 | 4.11E-01 | 5.35E-03 |
Attached are the plots of the undamped top to bottom TFs with cross couplings terms. The data cursor shows where I get the numbers from the table above from. The minus sign for Presp/Ldrive comes from the -180 degrees phase.
(Borja)
We have finally manage to solve the mistery regarding the unresponsive LL quadrant of the ETMY ESD (see this and this). So from now on we can add the 4 quadrants in our measurements.
Another measurement was take today (before seeing the effect of green light on the charging of the ETMY) to see repeatibility of measurements or stationarity of the charge.
I attach here the document including the new measurements as well as a comparison plot with the results of Veff and slopes between the data taken 2 days ago (this is 3 days after the 2nd discharge run) and the data taken here.
A summary table is given next with the current values in green (for each quadrant), and we compare them with the measurements before the discharge runs (in blue) and the measurements taken 2 days ago after discharge run2 (in red):
|
UL before |
UL after2 |
UL after22 |
UR before |
UR after2 |
UR after22 |
LL before |
LL after2 |
LL after22 |
LR before |
LR after2 |
LR after22 |
Veff PITCH [V] |
112 |
43 |
123 |
52 |
11 |
65 |
- |
- |
61 |
123 |
31 |
85 |
PITCH slope [10-7 µrad/V] |
-2.585 |
-2.63 |
-2.565 |
1.89 |
2.35 |
2.1605 |
- |
- |
2.154 |
-2.65 |
-2.655 |
-2.612 |
Veff YAW [V] |
125 |
77 |
177 |
103 |
1 |
70 |
- |
- |
54 |
144 |
54 |
100 |
YAW slope [10-7 µrad/V] |
-2.21 |
-2.25 |
-2.178 |
2.34 |
2.54 |
2.428 |
- |
- |
-2.271 |
2.32 |
2.365 |
2.374 |
Conclusions: During the last two days the charge of all quadrants has changed considerably bringing it back for some quadrants to same level (or worse) than before the ionized gas discharge runs took place. Look particularly to quadrant UR in pitch where we manage to do a full discharge after the 2nd discharge run but the current measurement shows again a high charge. The only think related to the ESD that has been done today before the measurements is solving the issue with LL quadrant which required turning off the ESD drive and also moving around some cables at the ESD BIAS low pass filter box. In principle this should not have caused a change in the ETMY charge.
Wow! An alarming result. Is it possible to return the wiring to the pre LL fix result and measure the charge once again? Other possibilities to make changes in the charging: Are the illuminators on the chamber on? When was the ion pump on the chamber turned on?
Jim, Dave
With so much going on this weekend and today, I thought it would be nice to provide a top level summary of events.
The LDAS raid issue is thought to be completely unrelated to the various FE issues, and was purely coincidental.
There is no obvious connection between the various FE issues, but we cannot imaging they are coincidental.
All times are PDT.
LDAS disk9 failure in RAID
Fri 8/8 21:57 | Warning message disk9 |
Fri 8/8 22:37 | Error Disk9, continuous |
Sat 8/9 01:52 | h1fw0 single restart |
Sat 8/9 04:02 | h1fw0 regularly restarting |
Sat 8/9 15:57 | problem resolved, disk9 removed from raid |
h1sush2a CPU freeze
Sat 8/9 00:06 | cpu freeze, DAQ data bad for many other FE |
Sat 8/9 11:48 | pwr CPU, large IRIGB error on IOP |
Sat 8/9 11:54 | restart IOP, still large IRIGB errors |
Sat 8/9 12:11 | pwr CPU and IOChassis, all is good |
h1seib1 CPU freeze
Sun 8/10 18:46 | cpu freeze, DAQ data on others good |
Mon 8/11 08:35 | pwr cpu , IRIGB drifted bad |
Mon 8/11 08:39 | IRIGB drifted good, all is good |
h1susb123 CPU freeze
Mon 8/11 13:46 | cpu freeze, DAQ data good on other FE. SWWD causes SEI trip 5 mins later |
Mon 8/11 13:58 | pwr cpu. All looks good but DAC is undrivable (discovered later) |
Mon 8/11 14:34 | DAC problem discovered, fixed with restart of IOP |
h1seih23 DACs not driving
Sat 8/9 15:30 | h1seih23 DACs not being driven |
Mon 8/11 10:42 | fixed by restarting IOP |
Disconnecting, moving and reconnecting aux. pump cart to annulus systems
Kyle, Gerardo We found the bottom center viewport on HAM4 South door to be leaking at around 2-3 x 10-2 torr*L/sec! -> Tomorrow we will attempt to avoid this leak and resume leak hunting All annulus volumes in the Vertex, YBM and XBM had been evacuated before helium leak testing began this afternoon. GV1 was closed then opened and GV2 was closed to demonstrate that the leak was on the Vertex volume and not the YBM or XBM volumes.
Jim, Arnaud, Dave
at 13:46 PDT this afternoon h1susb123 (SUS ITMX,ITMY,BS) cpu froze up in the same way as the recent problems with h1sush2b and h1seib1.
we removed h1susb123 from the Dolphin remotely and power cycled the CPU. The IOP model did not show any IRIG-B error so we thought this was a clean recovery. Unfortunately the DACs were not driveable (similar to h1seih23 problem this weekend) and when the models were activated no DAC channels could be driven. We then stopped the user models, restarted the iop model, restarted the user models and now all is well.
The reason for this crash (the 3rd in 3 days) is unknown, investigations are continuing.
Added the ESD Bias Path Filter Box (D1400192) to both ETMY and ETMX. Units filter voltage noise present on the ETM bias path of the ESD HV drive. Due to wiring differences inside the chamber, the filter units were connected as follows: ETMY connected to pin 1 (Flange F2-3) ETMX connected to pin 3 (Flange F2-3) Filiberto Clara
EX Current Limit Box D1201288 SN S1400223 ESD Bias Path Filter D1400192 SN S1400403 EY Current Limit Box D1201288 SN S1400224 ESD Bias Path Filter D1400192 SN S1400405
Haven't been messing with this system since doing pumping stuff etc. so this idled along like this.
Overvoltage 3 is our old friend on these VFDs of the Pump Station. They do seem to be occurring more frequently now.
Corey restarted in the documented manner.
The Filter for the DC bias had been placed on connection 3 as it is for all other ESD drives but the person who installed it did not know ETMy is different the DC bias in on pin1. This swap was done and the configuration is correct. The next problem introduced was the lock up of the DAC channels. I had to cycle the ESD off pull the DAC cable reinstall the DAC cable and power the ESD controller back up. The system has been turned over for more testing.
Here are the two vacuum gauges during the two gas injection events last week.
Text files also attached.