Elli, Evan, Dave:
cdsfs0 (the /ligo file server) locked up this evening sometime after 18:43 PST. It was completely unresponsive, so Elli and Evan (with myself on the phone) power cycled the machine using the front panel button. It came back up with no problems and I was able to log into a control room workstation remotely. The system logs dont show any indication of the problem, it locked up silently. The attached logs show the last good log at 18:43 and the start of the reboot following power restoration at 19:17
Dec 29 18:43:58 cdsfs0 mountd[1623]: authenticated mount request from 10.22.0.141:1022 for /ligo (/ligo)
Dec 29 19:17:20 cdsfs0 kernel: imklog 4.2.0, log source = /proc/kmsg started.
no restarts reported. No conlog report.
0810 - 0815 hrs. local -> In and out of Y-end VEA 0825 - 0830 hrs. local -> In and out of X-end VEA 0840 - 0845 hrs. local -> In and out of LVEA -> De-energized XBM turbo controller
Elli, Evan
For the past week or so I've noticed that initial alignment of the corner optics has become much more painful:
The second of these problems was traced to the LSC_CONFIGS guardian: the threshold for LSC-ASAIR_A_LF_NORM_MON was too high (300 ct), so that even if the lock was broken, the guardian would not register it. This would leave the MICH loop trying to acquire lock with two integrators on and no limiter on BS-M3_ISCINF_L. I've turned the threshold down to 30 ct. The MICH locking seems more sluggish than it did a few weeks ago, but now it locks instead of wandering off.
Elli and I tried for a while to fix the first problem, but nothing we tried seem to work. Turning up the gain only made the PRX lock oscillate. Turning off the slow pitch/yaw bleed to the M1 stage seemed to have no effect. Different combinations of REFL_A and REFL_B WFS seemed to have no effect. So this will require some more attention.
Today I was able to get good buildup in PRX (ASAIR_LF of about 4500 ct). In order to do so I had to adjust both PRM (by tens of counts) and PR3 (by a few counts). Since the WFS loop only touches PRM, it is not so surprising that it cannot always achieve good PRX buildup. In the past we usually haven't had to touch PR3 for day-to-day alignment, but since it's now used for DRMI/PRMI ASC loops, its pointing is now more variable.
Once PR3 has been touched up by hand, the WFS loop can take care of the rest. However, since it involves a very slow bleed-off from PRM M2 to PRM M1, we may want to find something faster for initial alignment.
One point I am not clear on is why PRM M1 P/Y and SRM M1 P/Y now have different filter settings. PRM M1 has a 1/f shape everywhere; SRM M1 has a flat gain of order unity. This makes SRM M1 more prone to getting kicked when M1 feedback is turned on or receives a sudden change in input; at one point it caused the HAM5 ISI to trip.
Evan, Elli
A 1611 photodiode has been added to ISCT1 in preparation for a schnupp asymmetry measurment.
The new 1611 photodiode sits between the Refl Air A and Refl Air B photodiodes (see table layout D1201103). An AR optic is placed before Refl Air B as a pickoff to divert a small amount of power to the 1611PD. 29mW of power was going toward Refl Air B, of which 0.4mW of power is now diverted to the 1611PD. The beam is slightly larger that the PD area- we could focus the beam to get more power if needed.
The PD is powered by a +/-15V power supply which we placed on top of ISCT1. The cables for the PD are coming out through a patch panel on the left hand side of the table. Evan has made a longer power cable in order to reach the power supply.
Tomorrow we plan to lock PRMI to the sideband and take a preliminary set of measurements for the schnupp asymetry.
Hey, you probably already accounted for this, but you may need to realign the beam onto REFL AIR B since the transmission through the AR window will have translated the beam somewhat. I guess REFL AIR B DC out is acquired somewhere and can be used to optimize the alignment.
We didn't think of that before, but today we touched up the alignment.
We haven't had much success locking the auxiliary laser with the PLL today. It looked like it would lock to the VCO, but the error signal didn't make sense when viewed with an oscilloscope. The beat note also looked very noisy when the PLL was engaged. The PLL didn't lock at all when using the network analyser to sweep the Aux laser offset. We checked the alignment of the beam on the 1611 PD, and did a 64-67MHz sweep using 801 points.
Looks like h1conlog3 (the web server and replica) suffered a hardware failure on its disk yesterday afternoon at 15:00 PST, Saturday 27 December 2014. I have rebooted this machine, but it still only reports changes to 15:00 12/27 and the hourly reports are failing. We are investigating.
It is running again.
model restarts logged for Sat 27/Dec/2014
2014_12_27 06:27 h1fw1
2014_12_27 12:47 h1fw0
both unexpected restarts. Conlog reporting was offline yesterday.
I have just noticed that the corner station Hartmann Wavefront Sensor EPICS IOC server (h1hwsmsr) is not responding. It failed between 8pm and 9pm Wednesday night (12/24). If anyone needs the HWS for commissioning over the next week, please call me on my cell phone and we can try rebooting.
no restarts reported. Conlog frequently changing channels report attached.
model restarts logged for Wed 24/Dec/2014
2014_12_24 11:21 h1ioplsc0
2014_12_24 11:23 h1lscaux
2014_12_24 11:23 h1lsc
2014_12_24 11:23 h1omc
2014_12_24 11:25 h1iopiscey
2014_12_24 11:27 h1caley
2014_12_24 11:27 h1iscey
2014_12_24 11:27 h1pemey
2014_12_24 11:29 h1odcy
Recovery of LSC and ISC-EY from power glitch
model restarts logged for Thu 25/Dec/2014
2014_12_25 19:27 h1fw1
one unexpected restart over the two days. Conlog frequently changing channels list attached
Yend pumpdown.
Moving the right way now.
Tested the new off-loading feature for the integrator. The attached plot shows two changes in the bias value: the first near 4.5 sec is with the offload button off, the second transition near 8.4 sec is with the offload button on. In both cases the change is reflected on the integrator output value (OUT channel). In the first case, the bias change introduces a transient in the output drive (DRIVE). In the second case, no transient is observed and the integrator value is off-loaded into the bias value smoothly and instantaneously.
Had to restart the EY laser.
Looking at the EY SUS alignment it seems to have more or less recovered from the vent. The green return beam still hits the WFS, even so it somewhat high. Using bias sliders of 93.4/-50 for pitch and yaw for ETMY (75.6/-60.5 old) and 144.2/5.8 (no change) for TMSY steers the beam back onto the center of WFS_A.
The EX SUS alignment did not recover. I did not find the return beam. I only searched around the nominal good alignment, so. Trying to steer back by looking at the plots of SUS bias and oplev positions doesn't help. The EX VEA decided to increase its ambient temperature by 1°C.
Probably from the power glitch, the DAQ EPICS shows garbage data for iscey. EPICS readbacks are fine.
I’ve done some work on IOT2R in preparation for the Schnupp asymmetry measurement:
For the time being, I’ve turned off the PD and dumped the beam immediately after the periscope.
Next steps:
With regards to turning down the aux laser power rather than using a filter in front of the PLL photodiode: the reason we didn't do this before is because we still wanted to maintain maximum aux laser power injected to the PRC (which is unaffected by the filter in front of the PLL PD). If you get plenty of signal on ISCT1 still, then maybe this isn't a problem and turning down the aux laser power is fine.
I was not getting data at home from the external epics gateway, so I restarted this process. Also, I noticed a timing error on h1asc state word, which I cleared with a DIAG_RESET so it most probably was just a glitch on CPU_MAX.