travis.sadecki@LIGO.ORG - posted 12:15, Wednesday 21 October 2015 (22725)
GRB Alert
Received a verbal GRB alert at 19:00 UTC. Unfortunately, LLO was down at the time.
Received a verbal GRB alert at 19:00 UTC. Unfortunately, LLO was down at the time.
Keita, Evan, Hugh, Kiwamu,
When clearing the SDF differences, we noticed that the pitch bias offset of IM1, 2 and 3 are very different from the past (by ~ 1000 urad). These new offsets are the results of an alignment recovery effort by Cheryl at around 2 pm PT yesterday.
We checked various trends to understand why IMs needed so much change in the bias, but no clue was found. In the end, we accepted the new IM alignments even though we do not understand why they had moved. See the attached for the SDF difference.
The recovery effort
It looks that yesterday Cheryl changed the bias of IM1, 2 and 3 in order to bring them back to a place where their OSEMs read the same alignment values. As written below, after the interferomter was fully locked in this morning, the relevant QPD signals indicated that the laser light hits the same postion in the HAM2 and HAM3 area. This means her alignment recovery really aligned the beam back to where it should be. Or, in other words, IMs really had moved during the power outage for some reason. We still do not know why IMs had moved.
QPDs do not indicate a significant change
We checked the IM4 trans and POP_B QPDs in full lock to see if there is a significant difference in the actual laser beam. It seems that they are back to where they were in the past. This means that the interferomter alignment is not significantly different from the past with the new IM alignment. Good.
Also we looked at the HAM2 ISI oplev. Even though the oplev beam was not centered, we did not see a major change in PIT and YAW signals. They are about the same as before the power outage within a 1 urad.
Note that since POP_A is a part of the ASC control loop, it stayed centered.
HEPI and ISI
With help from Hugh, we checked HAM2 HEPI and ISI to see if they are back to nominal. And they are the same as before except for two HEPI prignle modes. See the attached for a trend of HEPI position locations. The HP mode moved by 5e5 (nm?) and VP mode changed by 3.5e3 (nm?). We are not sure how significant these numbers are in terms of the ISI table alginement.
Note that since the pringle modes are not DC-controlled and therefore it is sort of natural that they did not come back to where they used to be.
The power outage of yesterday caused the EPICS IOC to not exit completely, so the instructions to restart the process did not work. I logged in to the computer running the process and manually killed the dewpoint process. At this point the instructions to restart (in the CDS wiki) worked. Data is now being collected from the dewpoint sensors.
We have made it back to Observing Mode at 16:59 UTC. The violin modes are a bit rung up, but they are coming down in their characteristically slow fashion.
I ran the A2L script before going to Observing. It gave the following error:
IOError: [Errno 13] Permission denied: 'LinFit.png'
The script creates a temporary file (LinFit.png) that I had forgotten to chmod, so the Ops account couldn't write to it.
Travis just successfully ran the a2l from the Ops account, so I *think* all the bugs are fixed. Note that the script won't return the command line prompt to you until you hit Enter on the keyboard, so it's a bit confusing as to when it's finished, but if all the SDF diffs are gone, it's over and you can go to Observe.
Activity Log: All Times in UTC (PT)
07:00 (00:00) Take over from TJ
08:18 (01:18) Jenne & Nutsinee – Leaving the site
08:23 (01:23) Guardian error – Trying to select INIT. ISC_LOCK reload fixed problem.
08:30 (01:30) redoing initial alignment
12:51 (05:51) Finished initial alignment – Trying to relock
12:57 (05:57) Lockloss at LOCK_DRMI_1F
12:58 (05:58) Peter – Going into LVEA to take some photos
13:03 (06:03) Peter – Out of LVEA
13:10 (06:10) Continuing working on relocking
15:00 (08:00) Turn over to Travis
End of Shift Summary:
Title: 10/21/2015, Owl Shift 07:00 – 15:00 (00:00 – 08:00) All times in UTC (PT)
Support: Jenne, Nutsinee, Jeff K., Sheila, Kiwamu
Incoming Operator: Travis
Shift Summary:
No progress in relocking. Redoing initial alignment
No luck with INPUT_ALIGN. Power jumps to 1, then immediately breaks lock. Spoke with Sheila. No progress. Sheila coming to site.
Completed Initial Alignment and start relocking. Made it to LOCK_DRMI_1F before lockloss
Making progress at relocking, but still having difficulties. Sheila and Kiwamu are working through the problems.
Things that Jeff B and I have done:
Jeff was having difficulty locking the X arm in IR for inital alignment. Jeff moved SR3 pit to bring witness and oplev back to where they were before the power outage (both indicated that it lost 1-2urad of pit in the outage) (Still could not lock X arm in IR) Jeff re-ran the green WFS alignment, and moved PR3 by 0.5 urad in yaw.
checked all shutters to retore them to the way they were before the outage. (opened ISCT1 spare, IOT2L spare, and X end fiber.) This uncovered an interesting bug, which is that if theX green beam shutter is open, and the fiber shutter is closed, opening the fiber shutter will close the green beam shutter. In most other situations both of these shutters seem to work fine.
After this the X arm locked. Jeff went through the rest of inital alingment without incident until SRC align. Since SR3 was closer to the alignment from before the outage, we re-engaged the cage servo, which was turned off earlier probably because SR3 had moved. the SRC alignment was way off when we started the SRC align step and Jeff aligned by hand to get us close.
We then were able to lock DRMI several times, but had difficulty engaging DRMI ASC as people described last night. We tried doing things by hand, it seemed like we were OK engaging the MICH and INP1 loops, but PRC2 was a problem. In august, Evan and I changed the input matrix for this loop to no longer use refl 45 (alog 20811.) I found the old input matrix in the svn and tried it. This worked once so I've reverted the matrix in the guardian.
Input matrix for PRC2 since august 24th: refl91A-refl9IB
input matrix before august 24th and now:
# PRC2 REFLA45I - REFLA9I
asc_intrix_pit['PRC2', 'REFL_A_RF45_I'] = asc_intrix_yaw['PRC2', 'REFL_A_RF45_I'] = 0.83
asc_intrix_pit['PRC2', 'REFL_A_RF9_I'] = asc_intrix_yaw['PRC2', 'REFL_A_RF9_I'] = 0.5
asc_intrix_pit['PRC2', 'REFL_B_RF9_I'] = asc_intrix_yaw['PRC2', 'REFL_B_RF9_I'] = 0.5
asc_intrix_pit['PRC2', 'REFL_B_RF45_I'] = asc_intrix_yaw['PRC2', 'REFL_B_RF45_I'] = 0.83
This matrix gets reset in the full lock ASC engage states, so this is not a change to the full lock configuration.
After this change we made it past DRMI, to the point where DHARD WFS are engaged durring the CARM offset reduction. The bounce and roll modes are rung up, we spent a few minutes damping them but probably moved on too soon and lost lock probably due to roll modes in the final stages of CARM offset reduction.
Kiwamu and I spent about 15 minutes locked at CARM 10 picometers to damp bounce and roll. We lost lock after that, possibly because the IFO got misaligned as we were sitting at 10pm damping.
In the next lock, we saw that ETMX violin modes are also rung up, Kiwamu lowered the damping gains to stop PUM saturations.
We made it through engaging the ASC in full lock, and found that we couldn't lock the OMC because the Kepco power supply was off.
After we made it to low noise, I cleared 82 diffs in the ASC SDF. Most of these were due to the dark offset script that Jenne and Jeff ran last night. I accidentally accepted all of these with one button click (I hit accept all assuming that was only the first page which I could read, but accept all really means accept all.) Betsy pointed me to the last version to be accepted in SDF, so I was ble to check on the things I had inadvertently accepted. There were a few oddball things, like ADS SIG DEMOD TRAMPS, (I accepted the new 3 second TRAMPs), and offsets in the SRC1 loops (now set to 0).
Difficult morning for locking. After many lock failures, and at Jenne's suggestion started to run another an Initial Alignment. One had been done earlier during the day. No problem with ALS green. Could not get Input_Align to complete. It would grab lock with good power for a second or two and then drop. Ran through a couple of things with Sheila on the phone, but made no real progress. Sheila is coming in to work on resolving this problem.
Title: 10/21/2015, Owl Shift 07:00 – 15:00 (00:00 – 08:00) All times in UTC (PT)
State of H1: 07:00 (00:00) Relocking after yesterday’s power outage
Outgoing Operator: TJ
Quick Summary: Wind is a light to gentle breeze; seismic activity is low. Working on IFO recovery.
TITLE: "10/20 [EVE Shift]: 23:00-07:00UTC (16:00-00:00 PDT), all times posted in UTC"
STATE Of H1: Unlocked, struggling
SUPPORT: Jenne, Jeff K, Nutsinee, Sheila (by phone)
SHIFT SUMMARY: Problems from the beginning of my shift. Initial alignment presented two issues alog22705 and alog22707. Then trouble with DRMI has taken up the rest of my shift, alog22709 and alog22711. Let's hope for some good luck from here.
INCOMING OPERATOR: Jeff B
J. Kissel, J. Driggers, T. Shaffer After several lock losses of DRMI that were obviously caused by DRMI WFS running away with the optics, we've begun the arduous journey of going through each step of the DRMI WFS turn on process. We're trying the usual -- identify which step is problematic, slow things down, try turning on with low gains and no boosts, etc. We've run through tuning dark offsets suspecting we might have problems there and our earlier experience with the TransMon QPDs, but they didn't change too much. We've checked whitening filters and gains, all looks OK. Jenne's gunna stick it out with Bartlett for a few more attempts of finagling the WFS loops in hopes of working through them enough that they offload goodness to the SUS, but TJ and I are going to call it a night.
J. Kissel, J. Driggers, T. Schaffer After a very quick transition through ALS (nice!) we were stuck trying to get through DRMI 1f lock acquisition. After all of the usual tricks of taking SRM out of the equation, touching up alignment here and there, we began to be suspicous of beckhoff settings of our error signal photodiodes, as just about all of our problems have been this evening. Gains, whitening filters, all checked out. However, finally just *looking* at the error signal in dataviewer, we saw little to nothing. Jenne caught it -- the REFL shutter on ISCT was closed (similar to the earlier shutter gotcha LHO aLOG 22696). 40 mins ... *flush*
J. Kissel, J. Driggers, T. Schaffer [E. Hall and S. Dwyer remotely] Right after finally getting through XARM initial alignment, we advanced to the PRM initial alignment state, and found the ALIGN_IFO guardian manager erroring out. It was relatively easy to trace the problem to /opt/rtcds/userapps/release/isc/h1/guardian/lscparams.py which ALIGN_IFO was expecting to define a variable "prm_m2_cross_over". We found that this definition had been commented out of the code (on line 27), yet (a) there were no svn diffs, (b) the last time the file was touched was Oct 13 (by Evan), and (c) we've done initial alignment several times since Oct 13th. A call to Evan and Shiela revealed that they were just as baffled as to how this could have possibly worked for 7 days as we were. At their advice, we uncommented the variable definition, reloaded the guardian code, and the state succeeded admirably, without error. #facepalm We could try to come up with some boogey-man, malicious theories involving power outtages and gremlins as to how this could possibly be true, but instead we move on with our day. The functional lscparams.py with the uncommented, well-defined prm_m2_cross_over has now been committed to the userapps repo. 30 minutes ... *flush*
J. Kissel, J. Driggers, T. Schaffer [S. Dwyer remotely] While working our way through initial alignment, we were having great difficulties keeping the XARM locked on Red. Looking in every drawer and under every rug we could find, we found three small problems: (1) The TransMon X QPD B dark offset sum compensation needed to be changed +0.8 to -1.2. Not crazy that a site-wide power outtage changed the QPD dark offsets. (2) Digging around the analog signal chain, we found that the TransMon X QPD A and B whitening gains were +18 [db], when they had been +9 [dB] 24 hours ago before the power outtage. Not crazy that some Beckhoff settings did not get restored properly. I'm worried there are more, but I guess we'll find those later... (3) The ALIGN_IFO manager had been continuously requesting PRM and SRM to be misaligned for the XARM state, but PRM and SRM were not misaligned. After re-requesting misaligned (on their respective individual SUS guardian nodes), PRM and SRM actually misaligned, which cleaned up AS AIR RF45 PDH error signal nicely, and the XARM locked right up. Two hours ... *flush*
J. Kissel, for R. McCarthy, J. Worden, G. Moreno, J. Hanks, R. Bork, C. Perez, R. Blair, K. Kawabe, P. King, J. Oberling, J. Warner, H. Radkins, N. Kijbunchoo, E. King, B. Weaver, T. Sadecki, E. Hall, P. Thomas, S. Karki, D. Moraru, G. Mendell
Well, it turns out the IFO is a complicated beast with a lot of underlying infrastructure that we rely on to even begin recovering the IFO. Since LHO so infrequently loses power, I summarize the IFO systems that are necessary before we can begin the alignment / recovery process with pointers to aLOGs and/or names of people who did the work, so that we have a global perspective on all of the worlds that need attention when all power dies. One could consider this a sort of check-list, so I've roughly prioritized the items into stages, where the items within each stage can be done in parallel if the man-power exists and/or is on-site.
Stage 1
--------------------
Facilities - Richard
Vacuum - John / Kyle / Gerardo
Stage 2
--------------------
CDS -
Work Stations - Richard
Control Room FOMs - Operators / Carlos
DC Power Supplies - Richard
Stage 3
--------------------
CDS continued
Front-Ends and I/O Chassis - Dave (LHO aLOG 22694, LHO aLOG 22704)
Timing System
Guardian Machine (comes up OK with a simple power cycle)
Beckhoff PLCs - Patrick (LHO aLOG 22671)
PSL - Peter / Jason / Keita (LHO aLOG 22667, LHO aLOG 22674, LHO aLOG 22693)
Laser
Chillers
Front Ends
TwinCAT Beckhoff (separate from the rest of the IFO's Beckhoff)
IO Rotation Stage
TCS Nutsinee / Elli (LHO aLOG 22675)
Laser
Chillers
TCS Rotation Stage (run on same Beckhoff chassis as IO Rotation Stage, and some PSL PEM stuff too)
ALS Green Lasers - Keita
The interlock for these lasers are on-top of the ISCT-Ends, and need a key turn as well as a "start" button push, so it's a definite trip to the end-station
PCAL Lasers - Sudarshan
These either survived the power outtage, don't have an interlock, or can be reset remotely. I asked Sudarshan about the health of the PCAL lasers, and he was able to confirm goodness without leaving the control room.
High-Voltage - Richard McCarthy
ESD Drivers, PZTs
HEPI Pumps and Pump Servos - Hugh (LHO aLOG 22679)
Stage 4
------------------
Cameras - Carlos
PCAL Spot-position Cameras
Green and IR cameras
SDF System - Betsy / Hugh
Changing the default start-up SAFE.snap tables to OBSERVE.snap tables (LHO aLOG 22702)
Hardware Injections - Chris Biwer / Keith Riles / Dave Barker
These have not yet been restarted
DMT / LDAS - Greg Mendell / Dan Moraru (LHO aLOG 22701)
May we have excercise this list very infrequently if at all in the future!
C. Vorvick should be added to the list of participants! Apologies for anyone else that slipped from my mind late in the evening.
[Basically everyone in the control room]
We spent a lot of time working on pre-initial alignment, since we weren't seeing any light at the ALS transmission cameras or PDs.
All of the test mass optics and PR3 were restored to where their oplevs thought they were before the power outage (since the oplevs are independent of ISI pointing). The TMS suspensions were aligned using the baffle PD script.
Eventually, it was discovered that a shutter on ISCT1 was closed, and blocking the transmitted green beams.
Keita and Daniel are looking into why it was closed at all when the power went out and why it wasn't opened after the burt restore that Patrick did, but I have also added it to the ISC_LOCK DOWN state, so that we don't get bitten by this again.
We're getting good flashing in the arm cavities, both for green and IR, so as soon as this earthquake finishes ringing down, we can do initial alignment and finally relock.
[Jenne, TJ]
After finishing the green initial alignment step, we noticed that the COMM beatnote power was tiny - something like -32dBm. No good. After trying some PR3 alignment with no effect, we looked at some more shutters. As it turns out, the PSL green beam also has a shutter, so we weren't getting any PSL green to the beat PD. Opening that immediately fixed the problem. TJ finished off hand-aligning PR3 to maximize the beatnote power, and we went off to our next initial alignment step. (See aLog 22705 for that "fun"....)
This shutter was also added to the ISC_LOCK down state, so we don't get bitten by it either.
Two hours ... *flush*
Rolf, Richard, Jeff, Betsy, Travis, Filiburto, Carlos, Jonathan, Greg, Sudarshan, Dave
Front Ends
All IOCs and front end computers were power cycled. Recovery of Dolphin systems was delayed due to fault in h1seiex, which needed a second reboot to clear. (Jeff requests the split of the single Dolphin master into three be bumped up in priority).
Some IOP models went into "negative" IRIG range for a few minutes. Some corner station systems went into the high positive range and took several hours to come down to operational range. At time of this report, only SEIH45 has an IRIG error.
Yesterday I checked that there were no partial filter module loads or modified files, so the restart should not have loaded any new filters.
Hartman Wavefront Sensors
Elli restarted the HWS code at both EX and EY. The EDCU alerted us that this was not running, EDCU is now GREEN.
DAQ
The DAQ rode through the outage and recovery with no problems. I have cleared the accumulated CRC errors between the FECS and the DAQ concentrator due to the restarts.
DMT
Greg reports all DMT systems are fully recovered.
PCAL camera
Sudarshan and Carlos brought the PCAL camera systems back online.
SDF
All systems are now using their OBSERVE.snap files for their SDF reference.
Restart log
The full restart log is attached. The filtermodule DAQ restart strings shows the last start times for each model alphabetically
After clearing problems with the dolphin network, and untripping all watchdogs, there remained incosequential IRIG-B Timing Errors present on the CDS State Word on several front-ends, h1sush56 h1seih45 h1seib2 h1psl0 These errors are a result of the IRIG-B system not starting up in sync with the 1 PPS timing signal. They eventually go away after some time as the IRIG-B slowly begins to synchronize, as it did in this case. I document it just for future reference that these errors are not-surprising, and have little-to-no impact on recovery.
J. Kissel, B. Weaver, J. Driggers, H. Radkins When all front-ends die and restart, they come back pointing to their SAFE.snap SDF file. Once front end computers up, running, and mostly happy and we began to *use* the front-ends to recover the IFO, we began to change all of the SAFE.snaps to the nominal OBSERVE.snaps to help us continue to figure out what settings were out of place. This is a pretty tedious task, but once through, we reverted everything and requested the ISC guardians to run their DOWN states. This worked out quite well, but we really could use a script that switches all FE's SDF tables from SAFE.snap to OBSERVE.snap.
J. Kissel, K. Thorne, D. Barker, R. Bork, J. Hanks, R. Blair Since it was unclear where to find the recovery process for the front-ends given how interwoven they are, I outline the process that Keith had suggested and we ended up following once Dave got in: Depending on the length of the power outtage some front-end computers may or may not survive the outtage. However, again, given the interwoven systems, we've found it best to perform a systematic shut-down such that all computers and their interactions can be brought up in a controlled fashion. With the current setup of the dolphin network, the power-down and power-up procedure should be performed at the end-stations first, because the corner-station computers won't start until the end-station dolphin network is up and functional. As Dave mentions, I've requested that LHO adopt LLO's splitting of the dolphin fabrics, such that one can truely exercise the end stations independently of the corner. Power-down and power-up procedure - Power down all front end computers in the MSR (for corner station) or Entrance Lobby (for end stations). This is done by holding down the power button for ~5 [sec]. - Power down all I/O Chassis in the CDS Highbays. The rocker switches on the front panels don't always work, so you may have to use the rocker switch on the back of the chassis above where the +/- 24 [V] comes in. - Power cycle DC power supplies (recommended by LLO, unclear whether Richard did this before we got in in the morning. We did *not* do this systematically when we ran this power-down power-up procedure today) - Power up I/O chassis - Wait for timing slaves to be happy (relatively quick, but be sure to check) - Turn on front-end computers Once you turn on front-end computers, they will automatically start turning on the front-end processes. Recall that for SUS front-ends it may look (from the GDS-TP screens and the CDS overview) that the SUS computers are not coming back, but it's merely because they're running through the 18-bit DAC auto calibration, which takes ~3 to 5 minutes. This happens once the IOP model is started up, so from the CDS overview screen, it'll look like the IOP model came up dead, and the user models didn't start. Give it a few minutes before you get sad and go to restart the front-end processes by hand.
Executive summary:
A matlab file (37 MB) containing the averaged inverse-noise-weighted spectrum from the first week can be found here: https://ldas-jobs.ligo.caltech.edu/~keithr/spectra/O1/H1_O1_week1_0-2000_Hz.mat Because of the way multiple epochs are handled, the matlab variable structure is non-obvious. Here is how to plot the full spectrum after loading the file: semilogy(freqcommon,amppsdwt{1,1})
Keith has found: "There is a sporadic comb-on-comb with 0.088425-Hz fine spacing that appears with limited spans in three places near harmonics of 77, 154 and 231 Hz (ambiguity in precise fundamental frequency)" Using the coherence tool, we have seen coherence between h(t) and a number of auxiliary channels that shows this comb around 77 Hz. Seems to be around the input optics, in channels: H1:PEM-CS_MAG_LVEA_INPUTOPTICS_Z_DQ H1_SUS-ITMY_L1_WIT_L_DQ H1:SUS-BS_M1_DAMP_L_IN1_DQ H1_SUS-ITMY_L1_WIT_P_DQ H1:SUS-BS_M1_DAMP_T_IN1_DQ H1_SUS-ITMY_L1_WIT_Y_DQ H1:SUS-BS_M1_DAMP_V_IN1_DQ H1_SUS-ITMY_L2_WIT_L_DQ H1:SUS-BS_M1_DAMP_Y_IN1_DQ H1_SUS-ITMY_L2_WIT_Y_DQ See the attached figures. Nelson, Soren Schlassa, Nathaniel Strauss, Michael Coughlin, Eric Coughlin, Pat Meyers
The structure at 76.4Hz Nelson listed some channels for above shows up in at least 50 other channels. Greatest coherence is consistently at 76.766 Hz, second greatest is (mostly) consistently at 76.854Hz. Spacing between the two combs is about 0.0013Hz. The epicenter seems to be the INPUTOPTICS/the SUS-BS and SUS-ITM* channels, like Nelson said (see below for fuller list). The plots above are pretty typical, but I have plots for all channels listed and can post any more that are useful. Most or all channels showing the comb with max coherence greater than 0.1 are listed below. Max coherences over 0.2 are marked below as strong, and max coherences under 0.15 as weak. Those marked strongest are around 0.22. I haven't included anything of max coherence <0.1 but I'm sure there are many. H1:ASC-AS_A_RF36_I_PIT_OUT_DQ (weak) H1:ASC-AS_A_RF36_I_YAW_OUT_DQ H1:ASC-AS_A_RF36_Q_PIT_OUT_DQ H1:ASC-AS_A_RF36_Q_YAW_OUT_DQ (weak) H1:ASC-AS_B_RF36_I_YAW_OUT_DQ H1:ASC-AS_B_RF36_Q_YAW_OUT_DQ (strong) H1:ISI-BS_ST2_BLND_RZ_GS13_CUR_IN1_DQ (strong) H1:ISI-BS_ST2_BLND_Z_GS13_CUR_IN1_DQ (strong) H1:ISI-HAM2_BLND_GS13RZ_IN1_DQ H1:ISI-HAM2_BLND_GS13Z_IN1_DQ H1:ISI-HAM3_BLND_GS13Z_IN1_DQ (strong) H1:ISI-HAM5_BLND_GS13RZ_IN1_DQ H1:ISI-HAM5_BLND_GS13Z_IN1_DQ H1:ISI-HAM6_BLND_GS13RZ_IN1_DQ H1:ISI-ITMX_ST2_BLND_RX_GS13_CUR_IN1_DQ (weak) H1:ISI-ITMX_ST2_BLND_Z_GS13_CUR_IN1_DQ (strong) H1:ISI-ITMY_ST1_BLND_RZ_T240_CUR_IN1_DQ (weak) H1:ISI-ITMY_ST1_BLND_Y_T240_CUR_IN1_DQ (weak) H1:ISI-ITMY_ST2_BLND_RZ_GS13_CUR_IN1_DQ (strong) H1:ISI-ITMY_ST2_BLND_Z_GS13_CUR_IN1_DQ (strong) H1:LSC-PRCL_IN1_DQ H1:PEM-CS_LOWFMIC_LVEA_VERTEX_DQ (strong) H1:PEM-CS_MAG_LVEA_INPUTOPTICS_Y_DQ (strongest) H1:PEM-CS_MAG_LVEA_INPUTOPTICS_Z_DQ (strong) H1:SUS-BS_M1_DAMP_L_IN1_DQ (strongest) H1:SUS-BS_M1_DAMP_T_IN1_DQ (strong) H1:SUS-BS_M1_DAMP_V_IN1_DQ (strong) H1:SUS-BS_M1_DAMP_Y_IN1_DQ (strong) H1:SUS-ITMX_M0_DAMP_R_IN1_DQ (strong) H1:SUS-ITMX_M0_DAMP_V_IN1_DQ (strong) H1:SUS-ITMY_L1_WIT_L_DQ (strong) H1:SUS-ITMY_L1_WIT_P_DQ (strong) H1:SUS-ITMY_L1_WIT_Y_DQ (strong) H1:SUS-ITMY_L2_WIT_L_DQ (strong) H1:SUS-ITMY_L2_WIT_P_DQ (strong) H1:SUS-ITMY_L2_WIT_Y_DQ (strong) H1:SUS-MC1_M3_WIT_L_DQ H1:SUS-MC1_M3_WIT_P_DQ (weak) H1:SUS-MC2_M1_DAMP_L_IN1_DQ H1:SUS-MC2_M1_DAMP_T_IN1_DQ H1:SUS-MC2_M1_DAMP_Y_IN1_DQ H1:SUS-PR2_M1_DAMP_P_IN1_DQ H1:SUS-PR2_M1_DAMP_R_IN1_DQ H1:SUS-PR2_M1_DAMP_V_IN1_DQ H1:SUS-PR2_M3_WIT_L_DQ H1:SUS-PR2_M3_WIT_P_DQ (weak) H1:SUS-PR2_M3_WIT_Y_DQ (weak) H1:SUS-PR3_M1_DAMP_P_IN1_DQ H1:SUS-PR3_M1_DAMP_V_IN1_DQ H1:SUS-PRM_M1_DAMP_L_IN1_DQ (strongest) H1:SUS-PRM_M1_DAMP_T_IN1_DQ H1:SUS-PRM_M1_DAMP_Y_IN1_DQ (strong)
The 99.9989Hz comb Keith found (designated H) appears in 109 channels (list is attached). Coherence is uniformly greatest at the ~500Hz harmonic, with many channels approaching .7 and greater, drops off sharply at the ~600Hz and ~700Hz, and is invisible after 700. (See spreadsheet titled "comb_H_sigcohs_wk1.xslx" for a list of cohering channels by line, with coherence value.) At all harmonics except the ~300Hz, the structure manifests in the signal and the coherences as two lines .001Hz apart, but if I recall correctly .001Hz is the resolution of the frequency series, so it's safer to say that this is a bulge with .001Hz < width < .002Hz. At ~300Hz, almost all the cohering channels with data in that range show a bulge of width about 0.5Hz (see attached "disjoint_plots" for a comparison of typical channels by harmonic). This bulge, and the fact that it appears in all the same channels associated with the rest of the comb, makes me think that the fundamental may be the bulge at ~300Hz and not the line at 99.9989Hz. An interesting feature of the bulge is that in many cases, it has a prominent upward or downward spike at 299.96Hz, which is just the place the line would be if it were there (see "bulge_w_spike.jpg"). More to come re: changes in week 4 data, patterns in cohering channels, and the spike.
