This afternoon I ramped CP4's LLCV in 5% increments every 2 minutes, from 39% to 88% open, from 88% full to 100% full, for more data from exhaust flow meter. Fill level SP has been reset to 92% and level is slowly coming back down at 20% open on LLCV. Kudos to Patrick for writing an effective PI-code. It works very well for overfill scenarios!
Gerardo, Chandra On Tuesday, Aug. 23rd, we adjusted potentiometer on PT-140a (pirani) again - this time CCW 11 turns. Since Gerardo terminated cables for AIPs, gauge voltage has changed again and needs to be adjusted again so CC does not keep tripping due to set point interlock.
TITLE: 08/24 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Commissioning
INCOMING OPERATOR: Corey
SHIFT SUMMARY: Commissiong work continues.
LOG:
1425 - 1440 hrs. local -> To and from Y-mid Opened exhaust check-valve bypass-valve, opened LLCV bypass valve 1/2 turn -> LN2 @ exhaust in 1 minute 20 seconds -> Restored valves to as found configuration. Next CP3 overfill to be Friday, August 26th.
Daniel and Vern asked for a list of H1 models which are using the cdsEzcaRead and cdsEzcaWrite parts to transfer data to remote IOCs. This is following my discovery that the h1psliss model is attempting to send data to LLO EPICS channels (which also do not exist at LLO, presumably obsolete channels).
To make the list, I created a list of front end model core mdl files and grep within each file (grep -B 2 cdsEzCaWrite */h1/models/${model}|grep ":")
cdsEzCaRead:
h1ioppemmx.mdl *
Name "H1:DAQ-DC0_GPS"
h1ioppemmy.mdl *
Name "H1:DAQ-DC0_GPS"
h1sushtts.mdl
Name "H1:LSC-REFL_A_LF_OUTPUT"
h1pslpmc.mdl
Name "H1:PSL-OSC_LOCKED"
h1tcscs.mdl
Name "H1:ASC-X_TR_B_SUM_OUTPUT"
Name "H1:ASC-Y_TR_B_SUM_OUTPUT"
Name "H1:TCS-ETMX_RH_LOWERPOWER"
Name "H1:TCS-ETMX_RH_UPPERPOWER"
Name "H1:TCS-ETMY_RH_LOWERPOWER"
Name "H1:TCS-ETMY_RH_UPPERPOWER"
Name "H1:TCS-ITMX_CO2_LASERPOWER_ANGLE_CALC"
Name "H1:TCS-ITMX_CO2_LASERPOWER_ANGLE_REQUEST"
Name "H1:TCS-ITMX_CO2_LASERPOWER_POWER_REQUEST"
Name "H1:TCS-ITMX_CO2_LSRPWR_MTR_OUTPUT"
Name "H1:TCS-ITMX_RH_LOWERPOWER"
Name "H1:TCS-ITMX_RH_UPPERPOWER"
Name "H1:TCS-ITMY_CO2_LASERPOWER_ANGLE_CALC"
Name "H1:TCS-ITMY_CO2_LASERPOWER_ANGLE_REQUEST"
Name "H1:TCS-ITMY_CO2_LASERPOWER_POWER_REQUEST"
Name "H1:TCS-ITMY_CO2_LSRPWR_MTR_OUTPUT"
Name "H1:TCS-ITMY_RH_LOWERPOWER"
Name "H1:TCS-ITMY_RH_UPPERPOWER"
h1odcmaster.mdl
Name "H1:GRD-IFO_OK"
Name "H1:GRD-IMC_LOCK_OK"
Name "H1:GRD-ISC_LOCK_OK"
Name "H1:GRD-OMC_LOCK_OK"
Name "H1:PSL-ODC_CHANNEL_LATCH"
* mid station pem systems do not have IRIG-B timing, cdsEzCaRead is used to remotely obtain starting GPS time.
EzCaWrite:
h1psliss.mdl *
Name "L1:IMC-SL_QPD_WHITEN_SEG_1_GAINSTEP"
Name "L1:IMC-SL_QPD_WHITEN_SEG_1_SET_1"
Name "L1:IMC-SL_QPD_WHITEN_SEG_1_SET_2"
Name "L1:IMC-SL_QPD_WHITEN_SEG_1_SET_3"
Name "L1:IMC-SL_QPD_WHITEN_SEG_2_GAINSTEP"
Name "L1:IMC-SL_QPD_WHITEN_SEG_2_SET_1"
Name "L1:IMC-SL_QPD_WHITEN_SEG_2_SET_2"
Name "L1:IMC-SL_QPD_WHITEN_SEG_2_SET_3"
Name "L1:IMC-SL_QPD_WHITEN_SEG_3_GAINSTEP"
Name "L1:IMC-SL_QPD_WHITEN_SEG_3_SET_1"
Name "L1:IMC-SL_QPD_WHITEN_SEG_3_SET_2"
Name "L1:IMC-SL_QPD_WHITEN_SEG_3_SET_3"
Name "L1:IMC-SL_QPD_WHITEN_SEG_4_GAINSTEP"
Name "L1:IMC-SL_QPD_WHITEN_SEG_4_SET_1"
Name "L1:IMC-SL_QPD_WHITEN_SEG_4_SET_2"
Name "L1:IMC-SL_QPD_WHITEN_SEG_4_SET_3"
h1pslpmc.mdl
Name "H1:PSL-EPICSALARM"
* all writes to L1 channels will be removed on next restart of PSL ISS model.
Tagging all subsystems that are nominal responsible for these models.
Interestingly h1tcscs is cdsEzCaRead'ing some of its own EPICS channels. Looks like a copy-paste issue, I'll work with Nutsinee when she gets back.
See yesterday's alogs for a review of yesterday's activities. Please close all FRSs.
SEI - All good. ITMX did not trip from the two big EQs last night.
SUS - Charge is growing, needs a sign flip soon.
CDS - Next tues pulling demod board and putting the common mode board back in.
PSL - After recovering from a trip yesterday, things look good.
Vac - Hoping for more experiments on CP4, it will cause alarms. Kyle still baking vertex RGA, noise will continue for the week.
Facilities - [[Edit]] Safety meeting today.
I was asked to summarize the SWWD (software watchdog) timing sequence as a reminder.
t=0: SUS IOP detects top OSEM RMS exceeds trip level, starts its 1st countdown (5 mins)
t=5mins: SUS IOP 1st countdown expired, its IPC output goes to BAD and it starts its 2nd countdown (15 mins). SEI IOP receives the BAD IPC, and starts its 1st countdown (4 mins)
t=9mins: SEI IOP 1st countdown expired, its IPC output goes to BAD and it starts its 60 second 2nd countdown. SEI user models get the 60 second warning IPC so they can cleanly shutdown before the DACs are killed
t=10mins: SEI IOP 2nd countdown expired, DAC cards associated with the chamber the SUS is located in are killed
t=20mins: SUS IOP 2nd countdown expired, SUS DAC cards are killed
For the hardware watchdog (HWWD) the times are doubled. The power to the ISI Coil Driver chassis is removed after 20 mins of continuous SUS shaking.
Tagging SEI and SUS.
| Work Permit | Date | Description | alog | |
| 6119.html | 2016-08-23 22:37 | Activity: GC core router will be removed and replaced. Wireless and Internet connectivity will be disrupted during the short outage. CDS and LDAS internal networks will be unaffected, along with the phone system. * New equipment will be racked ahead of time (non-disruptive) * New cabling will be run ahead of time as needed * Cisco 6504 will be left on (for ease of rollback) * External optical taps will be removed (too lossy at 10Gb) * External and internal connections from the Cisco will be moved to the new routers * Internal links will be brought up and internal traffic will be checked (flows between wireless/wired) * External link will be brought up and routing tables will be checked. Area of Activity: GC core router and Internet connection | ||
| 6118.html | 2016-08-23 19:46 | Activity: Move code from separate device support module and libraries directly into FMCS EPICS IOC. Add relative humidity channels to FMCS EPICS IOC. Requires DAQ restart to add humidity channels to frames. | ||
| 6117.html | 2016-08-23 16:49 | Activity: Recompilation of the SUSETMY front-end model to add the second synchronized oscillators (LHO alog 29231). | 29245, 29231, 29245 | |
| 6116.html | 2016-08-23 15:35 | Activity: Removal and inspection of the OMC DCPD AA chassis. Repair and testing of channel 14. | 29257 | |
| 6115.html | 2016-08-23 15:13 | Activity: Improve the lock loss monitor by adding information that the arms are locked in red. Requires TwinCAT corner reboot. [h1ecac1plc1 code changes.] | 29251, 29252 | |
| 6114.html | 2016-08-22 21:43 | Activity: Perform annual maintenance at Mid-Y and LSB. [Hanford Fire Department inspection.] | ||
| 6113.html | 2016-08-22 21:14 | Activity: Collect data from CP4 manual fills to simulate CP3 in an effort to write code for CP3 manual overfill work-around. Experiment: 1) set CP4 LLCV in manual mode to nominal setting 2) increase LLCV in 10% increments while monitoring flow meter, exhaust pressure, and CP level 3) Repeat | 29179, 29248 | |
| 6112.html | 2016-08-22 16:11 | Activity: Replace broken water cooled power meter in HPO. The power meter did not survive the June power outage. This is the power meter in the HPO's external shutter assembly; power meter can be swapped in situ, external shutter assembly removal is not required. The PSL and its chillers will have to be shut down for this work. | 29270 | |
| 6111.html | 2016-08-22 16:02 | Activity: Energize RGA and take post-bake scans to assess effectiveness of recent bake * Combine RGA volume to Vertex Vacuum Volume if applicable * Decouple locally mounted turbo pump from RGA and install 1 1/2" O-ring valve in place of turbo (requires venting one side of metal valve which isolates turbo from Vertex Vacuum Volume * Pump to rough vacuum volume between newly installed O-ring valve and existing metal valve * Shut down pump cart | 29194, 29196 | |
| 6110.html | 2016-08-22 15:53 | Activity: Take Asahi Shinbun staff to the site tour. We'll enter LVEA in the morning and will stay there for less than an hour. | ||
| 6109.html | 2016-08-22 15:46 | Activity: Tour PNNL and visiting physicists through site and LVEA. We expect to be in the LVEA for 30min starting approximately 10am, on Tue 23rd. | ||
| 6108.html | 2016-08-22 15:22 | Activity: Change pre-filters on AHU-2. | ||
| 6107.html | 2016-08-22 15:17 | Activity: Flow test and zero count all dust monitors in LVEA, Diode Room, Optics Lab and both end stations. Have already tested the PSL dust monitors. Do Not need to get into the PSL enclosure. | ||
| 6106.html | 2016-08-22 14:18 | Activity: Install conduit from mechanical room to air handler rooms, pull control wiring and install electric actuators at AHU 1,2,& 3. | ||
| 6105.html | 2016-08-19 21:44 | Activity: Pull EtherCAT End Station Chassis 3 at EY and EX. Will inspect cabling for the Demod readbacks for the green WFS. Fault Report 6024 | ||
| 6104.html | 2016-08-19 20:12 | Activity: Install 200 kHz pole optional daughter board into IMC PDH Common Mode Servo Board (D040180). See LHO aLOG 28363. Should not take more than an hour. | 29250 | |
| 6103.html | 2016-08-19 18:05 | Activity: Valve in 500 L/s ion pump and valve out turbo pump. Pressure is currently at 1.1e-6 Torr (new gauge). Pressure will initially rise; after pressure reduction is observed, we will power down turbo and backing scroll pump. | 29206 | |
| 6102.html | 2016-08-19 16:28 | Activity: upgrade h1dmt0 to SL7, as requested by John Zweizig | ||
| 6101.html | 2016-08-19 13:55 | Activity: Replace Beckhoff temperature module 3202. One channel has been failed since installation. This is the link chassis so all of the Beckhoff will be disconnected. | 29200 | |
| 6100.html | 2016-08-19 12:03 | Activity: Make corrections to the FSS front end model. The corrections are related to the temperature readouts for the reference cavity. Currently these do not reflect the hardware in place. This work will require a restart of the FSS front end model and a DAQ reboot. | 29229, 29197 | |
| 6099.html | 2016-08-18 21:09 | Activity: Measure transfer functions of OMC DCPD AA | 29189 | |
| 6098.html | 2016-08-18 17:13 | Activity: Tour for family member which will include LVEA (as long as Operator gives OK). | ||
| 6097.html | 2016-08-18 15:01 | Activity: VIP tour a group from the US House Science Subcommittee on Research & Technology | ||
| 6096.html | 2016-08-18 00:15 | Activity: Add dust monitor Temp and R/H channels to EPICs | 29177 | |
| 6095.html | 2016-08-17 17:43 | Activity: Check suspected water leak in the high power oscillator. | 29203, 29190, 29186 | |
| 6094.html | 2016-08-17 16:55 | Activity: Replace faulty check valve on instrument air compressor tank at Mid X. | ||
| 6093.html | 2016-08-17 15:45 | Activity: Swap CO2 laser RF driver on TCSx table and measure the output power of the TCSx CO2 laser. We want to test if we have a bad RF driver from the TCSy laser, and the best way to do that is install the RF driver into a working system and see if the output power changes. Once complete we will reinstall the original TCSx RF driver to return the TCSx CO2 laser to normal operation. Risk to the TCSx CO2 laser is low. | 29174 |
The power meter in the high power oscillator's external shutter was replaced. The existing unit ceased to function some time around the planned power outage a couple of months ago for reasons best known to itself. old S/N = 589365 new S/N = 627203 Jason/Peter
Forgot to mention that the hoses for the power meter and front end cooling circuits were swapped over at the water manifold under the table. Given that the hoses were labelled numerically, this seems to have been a remnant from the laser installation.
After the hose swap yesterday the flow of the PSL_AMP_FLOW dropped by 0.3bar and the PWRMETERFLOW increases by 0.4bar.
The hoses were swapped because we found that they were hooked up backwards. I.E. the MOPA cooling hose was plugged into the Power Meter water circuit and vice versa (most likely during the recent water manifold swap). This means that in the trend data the flow for the front end (H1:PSL-AMP_FLOW) was actually reading the flow through the power meter circuit; the same applies to the power meter circuit flow (H1:PSL-OSC_PWRMETERFLOW), it was actually reading the flow for the front end. This was fixed yesterday and the flow data is now reading from the correct water circuits.
Sheila, Terra
Tonight we implemented the decoupled ASC SOFT matrix (first tried here but reverted a few days later). In past locks, the test mass op levs show common soft drift during power up; diagonalizing SOFT and HARD should help reduce this.
Powering up with this new matrix rang up a CSOFT power-to-angle instabiltiy ~0.5 Hz, causing us to break lock a few times near the end of power up. Next lock, we stopped at 20 W and adjusted SOFT offsets to get the best alignment (PRG ~32) and then powered up to 50 W successfully with no instability.
We still lost PRG during the remaining power increase, so adjusted POP_A offsets then SOFT offsets again. POP_A offsets only slightly improved; far less dramatic than with old matrix and SOFT offsets. In the end we recovered quite a bit of PRG but at the cost of sideband loss. Final offsets shown in attachment.
Tomorrow we'll try applying offsets at 2 W and/or putting offsets on QPDs instead. We're leaving the new matrix implemented in the guardian (but not offsets yet).
We also had trouble with bounce modes tonight, mostly in ETMX. This was solved by turning on the EVANBR filter (which was the only test mass that hadn't had this filter turned on). We did not see the 9.4 Hz DHARD_P instability.
We leave the interferometer at 50 W with DOWN requested.
First few hours of the shift were spent with recovery from the issues at the electronics racks out on the floor near the PSL. As Jeff K, noted, when we tried walking through locking we found issues with the MC Common Mode Servo Board (it was swapped out per his alog).
The rest of the evening was spent with commissioning.
There was a 6+ magnitude earthquake which shook things around (while we were still in recovery mode)---hope Virgo rode through it OK.
S. Dwyer, J. Kissel, C. Gray After successfully recovering the IMC's VCO and recovering the IMC (29264), we were able to get up through LOCKING_ARMS_GREEN in the lock acquisition sequence. However, we found that ALS COMM failed caused lock losses during the next step (LOCKING_ALS), when the input for IMC length control in its Common Mode Chassis was switched from the IMC's PDH output to the ALS COMM PLL output. The ALS COMM PLL output is connected to IN2 of the IMC chassis that had a new daughter board installed in the star-crossed ISC rack H1-ISC-R1 today (LHO aLOG 29250). After fighting through MEDM screen confusion* at the racks, we found that OUT2 (an analog pickoff pick-off just after the input gain circuit) indicated a ~2.5 [V] offset, even with IN2 terminated with 50 [Ohms]. Suspecting that this symptom was indicative that the input gain circuit (circled in red in MEDM screen capture) was yet another causality of the unfortunate rack power mishap today (LHO aLOG 29253), we've replaced the entire chassis (which lives in U14 of H1-ISC-R1) with a spare we found in the EE shop -- S/N S1102627 (or Board S/N S1102627MC). Notably, this spare does not have one of the new daughter boards on which Chris has worked so hard. We're not suggesting this swap be permanent, but we make the swap for tonight at least, so we can hopefully make forward progress. We suggest that IN2 and/or the input gain stage of SN S1102626 be fixed tomorrow, and the chassis restored so we can employ the new daughter board. Other Details: - Before removing the chassis, we powered down the entire rack using the voltage sequencer around the back at the top of the rack. - After installing the rack, we were sure to have all cables connected appropriately before turning the rack power on again (via the sequencer again). - We added a few labels to the IMC's PDH output and the ALS COMM PLL output cables such that they're easier to follow and reconnect in the future. *MEDM Screen Confusion -- whether IN1 or IN2 is fed into OUT2 of all common mode chassis is selectable on their MEDM screens. For the IMC's common mode board (at least for SN S1102626), the MEDM screen's indication of the status of that switch is exactly backwards. When the screen indicates that IN1 is feeding OUT2, IN2 is feeding OUT2, and vice versa. #facepalm
With Sheila's help, the OUT 2 switch should now be correct for the MC Common Mode Servo medm (H1IMC_SERVO.adl). This change was committed to the svn.
M. Pirello (reported by J. Kissel from verbal discussion with F. Clara) Marc has inspected the Common Mode Board chassis we've removed (SN S1102626), and indeed found several blown transistors and opamps -- and is not even through the chassis test procedure. Unfortunately, the EE shop needs a restocking of surface mount components before we can make the repairs, but the plan is to shoot for a re-install of this board by next Tuesday (Aug 30th).
Repairs to S1102626 are complete and the chassis has been tested with the 200kHz low pass filter. The chassis performance is similar to the previous test performed September 2011.
When the 200kHz low pass filter is activated we detected a 3mV dc offset which should be noted. The low pass filter works as designed with -3dB gain at 200kHz and rolls off nicely. I have attached files from the testing. File details can be found in the readme.txt included in the zip.
I've made a script to somewhat automate the weekly oplev trends FAMIS task. It makes 3 plots like the attached image of the oplev pit, yaw and sum channels for the test-masses, BS, PR3 and SR3. It still requires a little fiddling with the plot, you have to zoom in manually on any plots that have 1e9-like spikes, but this should still be easier than running dataviewer templates. It uses h1nds for data and a pre-release version of the python nds2 client that has gap handling, so updates in the future could break this. I'll try to maintain this script, so any changes or improvements should come to me. The script lives in the userapps/sys/h1/scripts folder.
The script is run by going to the sys/h1/scripts folder:
jim.warner@opsws0:~ 0$ cd /opt/rtcds/userapps/release/sys/h1/scripts
And running the oplev_trends.py script with python:
jim.warner@opsws0:scripts 0$ python oplev_trends.py
You will then need to do the usual zooming in on useful data, saving screen shots and posting to the alog. I'll look into automating more of this, but it works well enough for now. It would also be very easy to add this to a "Weeklies" tab on the sitemap, which I believe LLO has done with some similar tasks.
I've now added the HEPI monthly pressure trends to the same folder. Admittedly, there's little difference here between running my python script and running the dataviewer template, as the HEPI trends all fit on one dataviewer window easily. But this was pretty easy to throw together, and may allow us to automate these tasks more in the future, say if we could couple this with something like TJ's shift summary alog script.
Running it is similar to the oplev script:
jim.warner@opsws0:~ 0$ cd /opt/rtcds/userapps/release/sys/h1/scripts
jim.warner@opsws0:scripts 0$ python hepi_trends.py
For the oplev trends, they look good. I'll update the FAMIS procedure to run this script instead of using dataviewer.
Can you add the HAM2 oplev to this as well? While its usefulness is debated, it is an active optical lever so we should be trending it as well.
Thanks Jim!
After decoupling the pumping components used during the recent bake out of the Y-end RGA, I exposed the RGA to the Y-end vacuum volume, energized the filament and let it come into equilibrium for an hour or more. I then let the RGA scan continuously with the multiplier (SEM) on for an additional hour or so while I gathered up my mess(es). I periodically checked the scanning as I walked past the screen. At one point, I noticed that the spectrum was changing rapidly towards the "dirty". I monitored the scanning and noted that after reaching a temporary maximum, the amus which had increased then returned to near their original values. After consulting with the Jeff B. (the operator on shift), I feel that the observed changes in partial pressures were likely the result of IFO locking attempts as they coincide closely in time. Perhaps something gets hot when the IFO is locked or when mirrors are steered? See attached scans
If true that could be kind of scary (!) Can we set an RGA in MID (stripchart) mode and run time series following the main peaks through a locking attempt?
I could imagine baking the adsorbed water off the ETM and perhaps nearby baffles. But this should not persist (or repeat) after the first good cavity buildup.
Mike - Chandra's stated goal is to eventually continuously trend 7 AMUs (max allowed by software) at each building. The observance cited in this aLOG obviously would have been missed while in Faraday. Too bad that the RGAs don't live long with their SEMs on 24/7. As we install/commission the RGAs and as she works out the issues with the CDS and/or GC folks this trending will eventually be happening. J. Smith - The partial pressures that are changing are too small and not expected to show up on the total pressure gauges. From the graphic scans and knowing that the total pressure at the Y-end is 2 x 10-9 torr, we see that the partial pressures that changed are small (10-12 torr) - but still interesting because they are measurable and even more interesting if the changes can be shown to be tied to some IFO locking activity. (Science interesting? Who knew?)
Doh!!! Here are the .txt versions of the ASCII data
The indicated currents for these scans are typical of the SEM @ 1300 volts (which is the factory default). I have noticed in the past that setting the SEM voltage value in the EDIT tab does not change the value displayed in the device status screen or vice versa - so, though I set this to 1500 volts in one of those two fields, it may not have taken effect.
One more adjustment to the potentiometer since the CC interlock tripped a couple of times since the last change. 6 more turns CCW.