As a potential solution to the BS oplev glitching problem that we were having earlier (before the oplev laser power was increased a bit), we had thought about creating a PRMI ASC state, much like the DRMI ASC state. The idea was that we would try to engage ASC for MICH so that we could stop using the BS oplev for feedback.
However, since the BS oplev seems to no longer be glitching, this is no longer a high priority. So, the state still exists, but both the main and run parts have "if False" loops around all the code in them, so that this state is just a placeholder for now. If we decide to go forward with ASC for PRMI, we've already got a healthy start on it.
The message is: There is no practical change to the guardian code, although there is a new state if you open up the "all" screen on the ISC_DRMI guardian.
Sheila, Vern, Jenne, Jason, Jeff B
We have seen that BS oplev glitching is making it difficult for us to keep PRMI locked, and probably is adding to our DRMI locking difficulties. Vern called Jason, who suggested changing the power a bit. We went out on the floor, turned the knob a wee bit and increased the power by about 5%. For now at least the glitching seems a bit better, which you can see in Jenne's plot which is attached.
Locking update coming soon...
Title: 12/11/2015, Evening Shift 00:00 – 08:00 (16:00 – 00:00) All times in UTC (PT) State of H1: 00:00 (16:00), The IFO is unlocked. High microseism and problems with PRMI. Outgoing Operator: Ed Quick Summary: IFO is down. Shelia and Jenne are working on problems with PRMI locking and a glitching BS OpLev. Wind is a gentle breeze at 8-12mph. Seismic activity is around 0.07um/s, microseism is high, above 1.0um/s.
On Wednesday TJ and I tripped the ISI several times executing the HAM3 ISI GS13 gain switch in various ways. I have several examples to look at but here is a comparison of one trip to a successfull switch of the gains using the SEI COMMAND perl script.
The first attachment has 20 seconds of the trip example next to the switch that does not trip. It looks like there isn't anything different going on before the two cases: There are larger swings in X & Y in both and the other DOFs all look fairly similar too. There is a larger glitch 5+ seconds before the trip in Z but that seems to be settled by the switch/trip time; these would be related to Reference Position ISO servoing.
In the second attachment, again side by side are the two cases zoomed in a bit closer (10 sec.) The two stage switch by the perl script is evident with the gain switch doing more glitching and the whitening stage doing nothing. There is a delay of the sensors wrt to the switching by the guardian which seems weird but I'm not looking at outputs from the filter bank as we have no DQ channels here... These channels are the inputs to the blend bank after transforming to cartesian basis. Again, nothing stands out as to why there is a problem with the guardian script. We (TJ & I) did adjust the guardian code to separate the FMs switching like the perl script does but it still tripped the ISI.
The third attachment is a close zoom in on the ISI trip switch. It is what it is... Given that the Y and RZ are the dofs that ultimately go big, I'll suspect the horizontals but I could be fooled.
4 trips of the ISI with Guardian. Three are H3 and one is V3. Problem with corner 3?
I thought I had a trend going seeing the only excursion hitting the trip level on H3 first (see first three attachments.) Watch the channel colors as I added a channel but they are all H3. The fourth plot shows that the V3 sensor goes to the rail and causes the trip. If I saw a bunch more instances and they all hapened on corner3 I'd be very suspicious of bio switching or something; as it is I'm just suspicious.
The fifth plot compares the switching sequence on the six sensors between the COMMAND perl (left) and the Guardian (right.) Yes, the perl script does manage to hit the switches all at once whereas the guardian is clearly spaced in time. Of course, guardian manages to switch these on all the other platforms (except HAM2) without problem. Hmmmm. I'd like to work HAM2 over too and see wwhat that tells us.
Actually, we, TJ & I, are getting confused as to which of the above is guardian or perl as we mucked with the guardian code.
We think the one on the right is guardian with sleeps between the switches trying to mimic the perl Command script shown on the left.
TITLE: Dec11 DAY Shift 16:00-00:00UTC (08:00-16:00 PDT), all times posted in UTC
STATE Of H1: Environmental
SUPPORT: Sheila, Jenne, Hugh, Evan
INCOMING OPERATOR: Nutsinee
SHIFT SUMMARY:
Another day of record high microseism.
Sheila and Jenne doing some workarounds for the glitchy beam splitter oplev
no locking past PRMI
ACTIVITY LOG:
17:24 Kyle and Gerardo are going to EX and EY mechanical rooms.
17:54 Carlos to MY
18:08 Carlos back
18:10 Kyle returning to corner station towing trailer.
18:39 Kyle and Gerardo back to control room
22:18 Reset timing error on H1SUSETMY
22:27 LASER diode chiller indicator was RED on the Status screen. I added 125mL of water.
Kyle, Gerardo New ion pump had shut down with "excess arcing" error shortly after its initial start yesterday -> Restarted this morning and seems normal now -OK
RichM pointed me to look at the RZ drive as a way to confirm that the Z 45mHz blend is not coupling T240 RZ into the platform.
The attached 24 hour trend at ETMX suggests the nice reduction of inertial Z motion on Stage2 GS13s does not come at the expense of increase drive to the RZ of Stage1. Our z-RZ subtraction is doing well. The Blend switch points are shown too.
TITLE: Dec 11 DAY Shift 08:00-16:00UTC (00:00-08:00 PDT), all times posted in UTC
STATE Of H1: Environment - µSeism extremely high
OUTGOING OPERATOR: Travis - short shift
QUICK SUMMARY: …..it’s Friday.
Title: 12/11 Owl Shift 8:00-16:00 UTC (0:00-8:00 PST). All times in UTC.
State of H1: Unlocked due to microseism
Shift Summary: After Nutsinee's frustrating shift, I have had similar luck. I made it as far as ENGAGE_ASC_PART3 twice, but while waiting there for the ASC loops to do their deeds, we lost lock. Talked to Mike and he gave the OK to throw in the towel for the night. Microseism is still above 1 um/s and winds are still 5-10 mph.
Incoming operator: Ed
Activity log: Attempting to lock all night.
No luck with locking so far. Made it as far as DRMI_ON_POP once in several attempts. Microseisem continues its upward trend above 1 um/s. Winds have calmed from earlier but still ~10 mph.
TITLE: 12/10 EVE Shift 00:00-08:00UTC (16:00-00:00 PST), all times posted in UTC
STATE Of H1: Lock Acquisition/Environment
SUPPORT: Sheila, Jenne
Incoming Ops: Travis (failed to reach him so he's already on site)
END-OF-SHIFT SUMMARY: I've been trying to lock most of the evening. The shift started off with high wind (20-40 mph) then died down towards the end of shift (now ~10mph and below). Useism has been increasing (now at ~1.5 um/s). Locking was unsuccessful even with wind below 10mph.
Activity:
00:19 Kyle driving back from Y28
Mike suggested I should try again when the wind dies down to 0 mph. Wind speed is now ~10 mph ar EY. Without this kind of useism the ifo should have been locked without a problem.
Wind has been below 10 mph so I tried again. Lockloss at SWITCH_TO_QPDS. The furthest I've been all evening.
The wind has died below 20mph but useism is getting higher and higher. ALS has become more stable that the wind is low but I still couldn't get pass DRMI locking.
I just had an uncessful PRMI to DRMI transition. ISC_DRMI Guardian stalled at DRMI_LOCK_WAIT for a really long time. I just went ahead and INIT the ISC_LOCK Guardian and going to DRMI from there.
Wind has already died down below 20 mph. I don't think this is wind problem anymore.......... Redoing initial alignment.
Just for fun, since the microseism is very high for LHO, I looked at the ocean wave activity near the Pacific Northwest coast. The map is a nice bright red, close to the coast, meaning 12+ m wave height.
RichM suggested we get the 45mHz Blends running on the Z axis for the ETMs.
After checking that our current version is in SVN, I saved a backup copy and then copied the blends from the X or Y dof with foton. Attached are Trends with the transition from Quite_90s to the 45mHz Z blends fairly obvious. Also see that there is pretty much no effect in RZ from the switch as evidenced on the optical lever. If our Z to Rz subtraction was not too good, the lower blends might reinject tilt and show up on the oplev or some other IFO signal. So far so good.
It probably should happen on the CS platforms as well.
I committed the ETM filter txt files to the SVN revision 12264.
I stopped the bake this morning -> The pump is still noticeably warm and will be for hours but I went ahead and removed the turbo hardware and "fired it up" this afternoon -> It is now valved-in to the BT as is the X2-8 ion pump but both of their controllers still need to be mounted in the end stations and their HV cables will need to be cut-to-length as both have a few hundred feet excess length!