Jim, Evan Does this behavior of the ETM ESD DAC outputs make sense? Here a large offset has been applied to the UL coil output filter. However, the screen suggests that it's being applied to the UR DAC channels. This behavior is seen on both ETM ESD DACs, but not the ITM ESD DACs.
Hi Stefan,
It seems that the medm screen is not up-to-date. According to page 6 of D1002741-v10, the DAC order should be in BIAS, UR, LR, UL and LL starting from smaller channel number for the ESDs. The front end models (i.e. h1susetmx and h1susetmy) follow exactly what the document says, but the medm screen does not.
J. Kissel, S. Aston The MEDMs were suffering from poorly updated macro substitution files, and other issues that was a result of the last minute addition of the LVLN driver before ER8. Stuart had fixed this problem at LLO months ago (see LLO aLOGs 18819 and 19356), but in the heat of battle at the start of the run we never imported the updates here. A simple svn up of the /opt/rtcds/userapps/release/sus/common/medm/ has solved the problem. Indeed, I've taken it one step further, and modified /opt/rtcds/userapps/release/sus/common/medm/quad/SUS_CUST_QUAD_L3_ESDOUTF.adl to reflect the same order as what is shown on the overview screen. Attached is a replica of the above screenshot, but with the fixed MEDM structure.
The output of the NPRO was measured in front of the phase-correcting EOM to be 1.46 W. This was measured with the 10-W power head. The diode currents for the front end laser were increased to bring the diode powers back to near the 100% level. Jason, Peter
As part of the prep work for firing up the high power oscillator, the optical surfaces were
inspected. Some time back in 2014 there was a water leak over head 3 resulting in some of
the surfaces getting wet.
The high power oscillator was opened. A fragment of lens tissue was found - presumably
from the last wipe down of the laser that was done to accommodate a request from LLO - near
the lid switch close to head 4. Some metal shards were in the location. Small blue coloured
metal shards were also found near the Faraday rotator. These we know to be from the blue
anodising of the lid of the oscillator. There were signs of possible galvanic corrosion after
removing the side of the laser, and at the base of an internal support member.
Signs of water drops on all four 4f lenses. These were drag wiped off successfully:
Drag wiped 4f lens #1. Mostly clean, small spot around 2 and 4 near the edge of the optic.
Drag wiped 4f lens #2. Clean.
Drag wipes 4f lens #3, mostly clean, small spot around 2-2:30 near the edge of the optic.
Drag wiped 4f lens #4. Possible dust spot between 11 and 12 o'clock, ~two-thirds from centre
to edge.
After performing the drag wipe, the inside of the oscillator was wiped down again so as
to not kick up any dust during the remaining dust removal steps.
The two quartz rotators looked fine. All the laser rod surfaces appeared okay. The dust
on the 45 degree dichroic mirrors was successfully blown off.
There was a possible dust spot on output coupler at 3 o'clock as looking towards exit port.
This was blown off.
The pump light lenses looked okay with the possible exception of head 3, which looks like it
has some scratches. The scratches were there before drag wiping, and it would see that the
laser operated previously with them. O. Puncken was consulted.
Jason, Peter
Even though one of the pump lenses at head 3 looks hideous, it is apparently okay to use according to O. Puncken.
Ran new cables at EX for CPS sync fanout install. Cables are run from SUS-R1 rack to CPS units on chamber. Fanout chassis and RF cable (71MHz) still need to be installed/pulled. Filiberto C. & Ed M.
New lab dust monitors
Jeff B, Jim, Dave [WP5697]
As part of the install of the new OSB lab dust monitors:
generated new autoBurt.req file. Generated new DAQ EDCU INI file. Recanned conlog against autoBurt.req file.
New GDS version
Jim [WP 5696]:
completed this morning.
Clearing partially loaded filter modules
Was able to do full loads on susbs, susitmx, susitmy, susprm, sussrm. Still have partial loads on lsc, omc, suspr2 and sussr2 which are waiting for the system experts to clear.
DAQ reconfigure
DAQ EDCU was reconfigured for the missing h1tw0 and new DUST channels. Was applied during today's two DAQ restarts.
DAQ Restarts
In both DAQ restarts the dataconcentrator, tw1 and both framewriters started with no issues. In the first restart the broadcaster started by both NDS daqd process were no longer controlled by monit. A simple monit restart of the daqd processes cleared the error. On the second restart the broadcaster started, stopped and later restarted. Again both NDS processes did not start, and when they were restarted they failed. The error was that high dcuid nodes for the Beckhoff SDF system has been mistakenly added to the testpoint.par file, and only the NDS machines objected to this. The file was repaired and monit was able to restart the NDS daqd processes.
TCS model changes
Aidan and Dave [WP5688]:
Installed new h1tcscs model onto h1oaf0. DAQ was restarted.
New Beckhoff SDF code release
Jonathan [WP5695]:
Jonathan tested the new code on EY Beckhoff. He found an issue with enumerated binaries and backed this version out.
RACCESS monitoring/control of h1hwinj1
Jonathan, Jim, Dave:
Jonathan added the hardware injection machine h1hwinj1 to his RACCESS control system. MEDMs were updated accordingly.
H1OAF change to drive audio in control room
Richard, Evan, Dave [WP5704]:
h1oaf and h1pemcs models were changed to permit oaf to drive the 8th DAC channel of PEM's 18-bit DAC card. This analog signal is then routed from the CER to the control room using the audio channel in the digital video fiber link between the rooms.
New operator table install in the control room
Richard, Carlos, Jim, Dave:
The new powered, adjustable-height operator table was installed in the control room at the operator station. The access control PC, gate cam and alarm machines were shuffled to the left and a new workstation was installed on the new table.
HPI and ISI safe.snap work
Hugh, Dave:
Hugh worked on the safe.snap files for HPI and ISI. I was able to offer some help by creating a new script to convert sub-systems to safe SDF mode and converted all HPI and ISI to safe. Later I converted them back to OBSERVE mode.
New ECAT EPICS channels for corner station
Daniel, Dave:
Daniel made Beckhoff changes in the corner station. I updated the ECAT DAQ INI files and the target autoBurt.req files.
Server patching and rebooting
Carlos, Dave:
Carlos applied security patches and rebooted cdsssh, cdslogin and cdsadminctrl. We restarted the vacuum alarms system on cdslogin.
Note that at the time the DAQ was restarted during Tuesday's maintenance the DAQ dataconcentrator and both frame writers had been running continuously for 55 days 20 hours with no problems.
Installed Whitening Board (D1500389) to ETM Low Voltage Driver Chassis (D1500129) per ECR E1500417. ETMY LV Driver S1500066 - Whitening board installed S1600032 ETMX LV Driver S1500073 - Whitening board installed S1600035 High voltage was disabled prior to disconnecting of any cables. Verified outputs were not railing when units were reinstalled. One thing worth noting, is that the wiring for the binary cables is not consistent at both end stations. For now, we left cables in the same configuration we found them this morning. Filiberto C. & Ed M.
Day Shift: 16:00-23:59UTC (08:00-16:00PT)
State of H1: down all day, down more than 12 hours, currently JimW at EY to fix ESD driver
Shift Summary:
Activities logged in alogs 25155, 25163, and 25171
Other activities:
Two items:
I started updating the slow controls SDF models to the latest version in trunk. This was to fix a bug Daniel saw after the power outage. Unfortunately there were problems with the channel type detection code so we fell back to running the old code again. I have left the work permit open, I will debug, test, and deploy later.
Crashed earlier in the day, I'm guessing with the Beckhoff restart: Jan 26 13:07:00 h1conlog1-master conlog: ../conlog.cpp: 301: process_cac_messages: MySQL Exception: Error: Out of range value for column 'value' at row 1: Error code: 1264: SQLState: 22003: Exiting. Dave updated the channel list to remove the dust monitor channels.
Richard, Dave, Evan
We added infrastructure in the OAF model to take LSC error signals, filter them, and output them to an 18-bit DAC channel in the PEM frontend. This signal is then piped into the control room and can be played on a speaker.
MEDM screeens still need to be made.
1 h1psl PSL.team HPO optic exam x113 00:02:19
2 ISC/IO/ALS Richard add sensors to all tables 202 00:01:19
3 TCSY Alistair startup TCSY laser 202 00:01:19
4 LVEA crane Bubba shim rails - requires lift 202 00:00:48
5 TCS Aidan/Alastair install/modify 202 00:01:21
6 optics lab Betsy work 212 00:02:19
7 ESD drives filiberto/ed EX and EY 212 00:02:38
8 mid-y john/bubba check cryo pumps 202 00:02:43
9 EY Kyle check pumps 202 00:02:48
10 OAF Dave+Evan new oaf and pem, audio testing 202 00:03:44
List of reservation system - some times are longer than the task is expected to take 1 h1psl PSL.team HPO optic exam x113 00:03:40 2 ISC/IO/ALS Richard add sensors to all tables 202 00:02:40 3 TCSY Alistair startup TCSY laser 202 00:02:40 4 LVEA crane Bubba shim rails - requires lift 202 00:02:09 5 LN2 company? delivery on site 202 00:01:03 6 TCS Aidan/Alastair install/modify 202 00:02:42 7 optics lab Betsy work 212 00:03:40 8 ESD drives filiberto/ed EX and EY 212 00:03:59
Before bringing the SEI platforms back up. Checked the diffs with the safe.snaps in SDF. There were a few chages related to GS13 gains and FF ON/OFFs. All updated and committed to svn.
Brought all the SEI Platforms down and spun down the HEPI Pumps to check the Accumulator charge. Most accumulators showed a few PSI drop--likely from pressure gauge loss. No charge dropped to action level. It was mid July last time this was done. This suggests none of the accumulators are leaking. I'll check again in a couple months--checking the pressure sometimes loosens the valve and it can start leaking.
In anticipation of going to Observe last night (which didn't happen due to a lockloss sometime just before 1am), I had accepted some ISI SDF diffs while JimW was still here to approve them. I'm sure he'll look at his data more, and make a final decision on whether we should keep the sensor correction changes or not, but for now the Guardian will probably put the EY ISI back in its old nominal state, so we'll see a bunch of SDF diffs when we get to NomLowNoise. Anyhow, I attach a screenshot of what I accepted, and we can re-accept the old values later today, until Team SEI decides to change the Guardian.
Updated the corner TwinCAT system to the most recent svn. This includes the changes needed for adding the ASC-AS_A/B.RF90 channels.
The new AS_B.RF90 demod channels won't be active until the corner 4 chassis is upgraded. The AS_A.RF90 demod channels have been taken over from the old (now deleted) AS_D. Whitening channels for both AS_A and AS_B are working.
Added whitening screens for AS_A/B_RF90 to ISC-CUST_WHITENING_OVERVIEW.adl
Added related screen buttons for AS_A/B_RF90 to ASC_OVERVIEW.adl
There are small glitches very close to each second boundary in the ETMY drive signal and the ETMY SUS ad SEI rack magnetometers. In order to investigate the half-Hz combs in DARM (see alog 20790), I took an hour of data and folded it with a four-second period. If there is a repeated glitch at any multiple of this period, it should become far more visible. The result is that in several channels, there are glitches very near the boundary of each GPS second. The peak time of these glitches seems to be about 10 milliseconds after the start of the second. The glitch does not repeat identically every second. There is one shape in the first second, then one with an opposite polarity in the next second. The first two attached plots are for the SUS and SEI racks, which are shown with a 40-Hz zero-phase lowpass. The SUS has a narrow spike, and the negative spike is larger than the positive one. The SEI signal is more complicated. The ETMY L3 MASTER signal, which is the DARM output to the ESD, is shown with a 10 to 50 Hz bandpass. These glitches are more like sine-Gaussians, but the even and odd seconds still seem to have opposite polarities. There are more channels with similar glitches. We can make a more thorough investigation, and use more data and more times, to try to track down the origin of these glitches. Hopefully these glitches are responsible for the 0.5 Hz combs such that removing them will improve those.
J. Kissel
Tagging CDS in this entry.
I'd recently taken a look at the requested output of the ETMY L1 stage,
${IFO}:SUS-ETMY_L1_MASTER_OUT_*_DQ
and was interested to find ~1 [Hz] combs in the requested output. Though this isn't the 0.5 [Hz] combs that Andy mentions above, I think it's an excellent place to take the investigation further in a more focused manner -- with the point being that even the SUS's *requested* signal has a comb.
Attached is a 100-sec FFT ASD, of a typical, 1000 [sec] stretch of observation-ready data during the run (2016-01-04 04:00 UTC). Here, to give a feel for the physical amplitude of these signals: at 30 [Hz] the noise amplitude of one of these comb peaks is roughly 1e-3 [ct/rtHz] of requested DAC output, which corresponds to
1e-3 [ct/rtHz] * (20.0 [V] /2**18 [ct]) = 7.6e-8 [V/rtHz] @ 30 Hz ( * sqrt(2 * 0.01) = 1.1e-08 [V_pk])
Potentially verifiable / refutable Crack-pot Theories / Wild guesses:
- Perhaps there is some of this glitching in the inter-process communication (IPC) on the reflected memory (RFM) data transfer from the corner to the end station, that's only exposed for requested drives that have such a huge dynamic range? For whatever channels in the signal chain that are stored, can you reproduce the same combs by filtering those channels offline?
- Recall that the power supplies for the Hartmann Wavefront Sensor (HWS) were replaced some time ago, see Integration Issue 1062. Has anyone made a before-and-after comparison on this searched other sources for such combs in auxiliary channels? Perhaps forming a BruCo-like search where this UIM / L1 stage control signal is the response instead of DARM?
- Keith has already done a long-term study of the analog-to-digital converters (ADCs), looking for combs: see, eg. pg 44-49 (yeah!) of G1300997. He found no-such combs. Perhaps we should do a similar study on the digital-to-analog convert (DAC) side of things? I could also imagine a similar set up for a set of RFM channels that make the 4km journey along the arms.
In response to Jeff: The SUS-ETMY MASTER signal is just a filtered version of DARM, so if DARM has the comb so do the drive signals. I don't think that tells us where in the loop they originate. But you're right, this could be a digital problem or an electronics one involving something synched to GPS. Keith T. and Annamaria both suggested that the power supply of the timing fanout might be involved. That of course can be perfectly synched to the GPS second. Annamaria showed me an ADC in the L1 corner station being used as a temporary monitor of one of these power supplies. That signal (attached) jumps downward one second and upward the next, matching what we see in the magnetometers and DARM. Could we check if there's such an effect at the H1 Y-end?
Note that the small glitches in Andy's post are exactly synchronized to GPS; this makes coupling to many power supply glitches (HWS, or trickle chargers for magnetometers, etc.) an unlikely source.