Conlog crash on attempted insert of invalid JSON text

conlog-master.log:
2018-01-09T19:52:31.208080Z	    4 Execute	INSERT INTO events (pv_name, time_stamp, event_type, has_data, data) VALUES('H1:SQZ-SPARE_FLIPPER_1_NAME', '1515527550857386596', 'update', 1, '{"type":"DBR_STS_STRING","count":1,"value":["��'�"],"alarm_status":"NO_ALARM","alarm_severity":"NO_ALARM"}')
2018-01-09T19:52:31.208301Z	    4 Query	rollback

syslog:
Jan  9 11:52:31 conlog-master systemd[1]: Unit conlog.service entered failed state.

conlog.log:
Jan  9 11:52:31 conlog-master conlogd[10598]: terminate called after throwing an instance of 'sql::SQLException'
Jan  9 11:52:31 conlog-master conlogd[10598]: what():  Invalid JSON text: "Invalid escape character in string." at position 44 in value for column 'events.data'.

Suspect that it occurred with a Beckhoff restart.

Restarted and updated channel list. 59 channels added. 25 channels removed. List attached.

Found it crashed again, same issue, different channel:

2018-01-10T00:32:45.744823Z	    5 Execute	INSERT INTO events (pv_name, time_stamp, event_type, has_data, data) VALUES('H1:SQZ-LO_FLIPPER_NAME', '1515544365629095108', 'update', 1, '{"type":"DBR_STS_STRING","count":1,"value":["@@@e?@@@j�"],"alarm_status":"NO_ALARM","alarm_severity":"NO_ALARM"}')

SQZ Automation update

A new TwinCAT code for the corner h1ecatc1 was loaded. It includes the following fixes and features:

1. Fixed the communication between PLC1/2 and 4. This includes adding the new frequency counters for the squeezer VCO, beat note and VCXO, as well as the whitening that is controlled by PLC2. The communication structures needed to be "rewired" in the system manager.
2. The demodulator channels were hooked up backwards. Two of the demod chassis do not have a power ok readback, and this was fixed.
3. The order of PZTs was not identical between the wiring diagram and the TwinCAT configuration. Furthermore, the second OPO PZT is no longer driven and has been replaced by the seed PZT.
4. The whitening channels assignment for individual PDs was fixed and the whitening eliminated for 2 PDs that have none.
5. The TTFSS v4 uses the same PFD/demod boards as the ISC demodulators. The software now reflects this and uses the same demod interface for this sub part.
6. The ALS laser autolocker has been abstracted, so it can work for the squeezer laser as well. The hardware configurations are nearly identical with the main difference that the ALS uses a common mode board for the locking servo, whereas the squeezer laser uses a TTFSS. The fiber delivery, photodetector monitoring, RF and frequency readbacks are all the same. In principle, the line searching and locking should be able to follow the same steps, using the same medm screens and procedures.
7. The startup delay for the EPICS interface has been increase by 2 seconds to make sure that the loading of the TwinCAT kernel driver completes beforehand.
8. The RF power readbacks for the AOM drivers needed to be adjusted for the different ratio of the RF coupler (–30dB instead of –20dB).

Pending: Timing comparator code update for implementing ECR E1700246.

Weekly PSL Status Report (FAMIS #6750)

Laser Status:
SysStat is good
Front End Power is 35.91W (should be around 30 W)
HPO Output Power is -0.03017W
Front End Watch is RED
HPO Watch is RED

PMC:
It has been locked 0 days, 0 hr 0 minutes (should be days/weeks)
Reflected power = 0.03334Watts
Transmitted power = -0.01236Watts
PowerSum = 0.02098Watts.

FSS:
It has been locked for 0 days 0 hr and 0 min (should be days/weeks)
TPD[V] = 0.04611V (min 0.9V)

ISS:
The diffracted power is around 3.6% (should be 3-5%)
Last saturation event was 0 days 0 hours and 34 minutes ago (should be days/weeks)

Possible Issues:
FSS TPD is low
Epics alarm, see SYSSTAT.adl

Note: Peter is in the PSL enclosure working on PSL alignment.

EY Clean-Up: Mystery Optics & Optical Hardware Recovered

Many horizontal/working surfaces at EY have/had a hodge podge of tools, parts, cables, etc.  To help peck away at the clean-up needed at EY, we cleared away some of this stuff (with a focus on optics, optical hardware, some cabling, etc.).  Restored a few cables/connectors to the Commissioning rack & Mark/Tyler moved this out of the way into the middle roll-up door room.  

On some tables and a white beam on the West wall, there were optics, empty optics containers, an IR viewer, optical hardware (i.e. non-ISC dog clamps, posts, mounts, etc.), etc.  I put all of this stuff in a tote and will claim it on behalf of ISC.  If someone recognizes any of this stuff, please let me know (attached are photos of most of the parts).  A possible clue for the ownership of some of this stuff is that some of it was accompanied by two copies of the "Test Procedure for Low Noise VCO" T1100083....but perhaps this was just scrap paper.

late entry about Friday's work on the IMC

Having eliminated alignment issues in vacuum, having driven the PZT, and searched with MC1 and MC3, the remaining alignment issue was the bottom periscope mirror in the PSL.  The PZT and optics were restored to the Wednesday night values, when flashing was good, and the beam was on the IOT2L table on the REFL path, and I entered the PSL and adjusted the bottom periscope mirror while Jenne watched MC2 Trans, and the camera on HAM2 West door that I'd repositioned to view the last baffle of the second IMC in-vacuum REFL periscopes, which showed the beam position and when the beam was clipped or not clipped on the baffle aperture.

I aligned to the known good IMC using the bottom periscope mirror, and used both pitch and yaw.  Pitch needed more adjusting, yaw less.  Had pitch been the only alignment change that needed to be recovered, yaw shouldn't have needed any changes, however, in Peter's alog from Saturday (40024), he mentioned a possible alignment change from over the holidays, and on Thursday there was  a temperature excursion in the PSL, which may have combined to shift the beam from it's position from Wednesday night, which may explain why both pitch and yaw on the lower periscope mirror needed to be adjusted to optimize the alignment into the IMC.

After adjusting MC3/MC1 to get good flashing, we checked the power of the REFL beam on IOT2L, and that was 65mW, and when I measured in the PSL, the power incident on the bottom periscope mirror was 71mW, and the reflected power was 65mW (harder to read as the beam heads to the top periscope mirror).

After leaving the LVEA, I worked on the alignment using the PZT and MC1/MC3, and decided to put them all back, and I put the optics back first, and this increased the flashing as seen on MC2 Trans, and then shortly after that the HPO tripped, as I alogged here (40022).

The alignment at the end of Friday was flashing and the beam was on IOT2L, but since I'd moved the HAM2 West door camera, the beam from MC2 was not centered on the HA1 baffle (after IM1), so once the camera is restored to looking at HA1, the MC2 alignment can be evaluated.

Centering the IMC input beam on MC2 Trans and centering the beam from MC2 on HA1 baffle will restore as close as possible the beam path from the in-air IMC alignment.  This is important because this also restores as close as possible the in-air beam position and angle through the IMs, and restores the well centered beam through the IO Faraday, which is a condition we want to have under vacuum. 

I have two more cameras to place on HAM2, to look at the alignment through the IO Faraday.  The camera for the HAM2 East door is in it's camera housing and sitting on the floor between the HEPI peers.

The alignment change I mentioned was only internal to the oscillator and has no effect on alignments after the pre-modecleaner as the temperature change referred to was the operating temperature inside the housing of the oscillator and not the room temperature.

EY vented

EY vent complete (2 hr slow vent). PT-410A should be adjusted to atmospheric pressure once door is off. Purge air is valved out for the night.

I opened the valve to GV18 gate annulus and left a note on LOTO to remember to close before opening GV18.

 

EY vent in progress

Continued EY vent but needed to leave site so will resume tomorrow morning. Pressure currently at 600 Torr.

IMC locked, alignment not yet complete

[TVo, Jenne, Sheila]

TVo and I were struggling to lock the IMC this morning, so we went to go look in the LVEA at the RFPD signals.  We weren't seeing anything at all, despite the beam being well centered on IMC Refl PD.  Sheila helped us flip the whitening switches and then power cycle the PD interface board (slot 11 on the ISC R1 rack).  After the power cycle, we were seeing PDH-like signals at the output of the demod board.  So, we're not happy that we don't know why that box needed power cycling, but things seem okay for now, so we're moving forward.

After some more alignment tweaking (and a long break for the PSL recovery after it tripped), TVo and I were able to get the IMC locked.  We're in process of trying to get the WFS loops on, and get the aligment to a good place - we hope to finish this tomorrow.

PSL Tripped due to HPO Power Watchdog

At ~12:40 PST the PSL HPO shut off due to a trip of the HPO power watchdog; the FE was still running.  The laser restarted without issue, but once again we were having issues engaging the injection locking.  To get a reasonable power out of the HPO we had to increase the operating current for DB1 again, this time up to 59.5 A (from 56.6 A); this unfortunately did not help with the injection locking, so we let the laser warm up for an hour or so to see if this helped.  It didn't.  Therefore, Peter and I went into the PSL enclosure at about 14:30 PST to take a look at the alignment onto the injection locking PD.  The PDH error signal looked to be a little out of phase, but other than that looked fine, however no realignment of the beam into the PD improved the situation with the injection locking.  Just to see what happens, Peter removed the pump light filter from the front of the PD; the injection locking engaged almost immediately.  The lock broke when Peter reinstalled the pump light filter, but relocked almost immediately.  Possibly not enough light on the PD?

Looking into the trip, it appears that a rapid decay of DB1 caused a drop in laser power, which eventually triggered the power watchdog.  This seems to be the cause behind last Saturday's trip as well.  The two attached trends show both laser trips; the 1st is the power out of the HPO and the 2nd is the relative power (in %) of the 4 HPO DBs.  It is clear that DB1 was rapidly decaying, especially compared to the other 3 DBs (DB4 also shows the same decay pattern, but its relative power out is much higher than DB1 and the decay is not as fast so it is unlikely to be part of the issue), and this decay tracked with the drop of the HPO output power, leading me to the conclusion that the decay of DB1 caused both of these trips.  This is somewhat surprising as DB1 was swapped out last June, while DBs 2, 3, and 4 are all the original DBs installed with the PSL in 2011/2012.  DB1 will need to be swapped at the next opportunity if the HPO is to survive the month.

The laser is back up and running to enable commissioning while we prepare for the diode box swap.  If it trips off overnight or after hours we will take a look at it in the morning; given the nature of the trips any restart attempts will have to be done onsite.  However, the odds of recovering from another trip are low; we are driving DB1 pretty hard to keep the laser up and running right now, so should the laser trip off again I would prefer to swap the DB before attempting a restart.  We will check on the laser first thing in the morning.

Ops Day Shift Summary

EndY HEPI is on Stops & Fluid Pump System in filterchange/recirc mode

This means the EPICS alarm is major.  It has been acknowledged and will remain in alarm until back to normal ops in a month or two.  WP 7271

started EY vent

Started EY vent at 11:30 am local. Currently at 1.5 Torr; purge air valved out till after lunch. I needed to reboot local Beckhoff computer because of screen freezing up, which caused CC gauges to trip and CP7 PID settings to default to values of zero. Gerardo did a BURT restore.

Purge air DP measures -46degC.

SQZ Slow Control Testing

Sheila, Daniel, Nutsinee

 

We tested Beckhoff communications to the SQZ chassis on the squeezer rack and ISC rack. Below is the summary:

 

VCO

• 3 cables connecting the RF frequency monitor to the timing comparator were missing: ISC_SQ_370_1, 371_1, and 372_1 all BNC->N. Fil already know about these.
• Temperature calibration was wrong: ~5 V measured out of the chassis gets converted to something like 1C on the medm screen. Probably just the Beckhoff code problem

 

VCXO

• VCXO link screen was missing from the lsc electronics medm. Otherwise everything else seemed fine. 

 

Phase Shifter/Delay Line (U32, U23, U19)

• phase shifter screen didn't show errors when it's supposed to. Every LED on the tester lit up fine.

 

6MHz Demod (U31)

• medm screen complained "power supply out of range" and that power is not okay. The chassis powered just fine. Same complain for the other Demod screens.
   

CLF Common Mode Board Servo

• Filter button in the "Slow Path" didn't engage properly. One time I plugged the tester in the light was on all the time. The other time I plugged it back the light was off all the time. There's no problem on the software end. I have yet to check the back of the Ethercat chassis itself.

 

SHG Common Mode Board Servo

• When engaged Boost in the Slow Path the signal was actually sent to Compensation on the tester, when engaged Compensation the signal was sent to Boost.

 

LO and OPO Common Mode Board Servo

• Both work fine

 

PZT Driver (There's only one PZT driver chassis that controls all the SQZ PZTs)

• OPO PZT screen didn't work during this test (channels white out) and SEED PZT screen didn't seem to be talking to anything.
• "Offset 1" on the tester corresponded to LO PZT remote offset. When asked for 5V on the screen the tester read 0.248V. When asked for 10V it read 0.5V. When asked for -10V it read -5V. Seems like the signal that's actually get sent to the chassis read 5V to low (or high, when asked for negative V) consistently.
• "Offset 4" on the tester corresponded to SHG PZT offset. Also read 5V too low (or high when asked for negative). 
• I didn't get a chance to test what actually came out of the monitor. Not sure what's the best way to do that either.

 

Binary IO (whitening chassis)

• I could only test the binary IO signal sent to the chassis on the ISC rack (the bottom of the far right rack). They're fine. The one of SQZ rack needed a model restart before they can be tested.

 

What's left to do?

• Test the rest of the demod signal (35MHz, 80MHz, and I think the other one is 3MHz?).
• Complete testing PZT driver signal (OPO, SEED) and the monitors if there's anything we could test.
• Complete testing the binary IO signal to the whitening chassis on the SQZ rack.
• Check the CLF common mode board signal at the back of the Ethercat chassis.

 

Since every chassis has been tested to receive the signal properly prior to the installations (according to Daniel) I didn't bother to test that again (except for the VCO chassis, Sheila tested that one).

flashing IMC, then no light

I was mproving the flashing in the IMC, and then something happened, and there was no light into the IMC.

I collected 14 seconds of data on channels I think might help an expert explain, and have attached the plots.

NPRO is running, and OSC_PD_AMP_DC_OUT shows 35W, so that part of the laser in on.

Squeezer Faraday TFP extinction ratios

The afternoon of December 22nd I went back into the optics lab and made a few measurements to try to understand why our Faraday isolation was only 20dB(see alog 39861).  It turned out that one of the TFPs had an extinction ratio that didn't meet the spec, and by switching the positions of the TFPs I was able to measure an isolation of -28dB. 

Thin Film Polarizer extinction ratio:

To measure the extinction ratio of each of the TFPs I used a set up very similar to the image in 39861, with the rotator removed (PD monitoring input power in position A, PD monitoring TFP transmission in position C which is after the Faraday path, used chopper to measure transfer function between PDs).  

After the fiber collimator, there was already in place a PBS mounted to clean up the input polarization by reflecting horizontally polarized light.  I also used whichever TFP I wasn't measuring to further clean up the polarization.  I rotated the half wave plate to measure the maximum and minimum power transmitted by the second TFP to get its extinction ratio.  For the TFP mounted with the backplate labeled SN9, I got a ratio of 2220:1 (coated side facing down in the mount, so that incident beam hit uncoated side first), for the one mounted on the backplate SN08, I got 336:1 with the coated side down and 577:1 with the coated side up (so that the incident beam first hits the coated side).  The spec for these TFPs is greater than 1000:1 (spec here)

Better Faraday Performance

In the original set up, SN08 was the first polarizer in the Faraday (the one closer to the OPO), and both TFPs were mounted coated side down.  Scattered light from the interferometer will be mostly in the polarization to be rejected by the TFP closer to the OPO, so swapped the two positions.  Now SN09 (2220:1) is mounted coated side down closer to the OPO, and SN08 is mounted coated side down further from the OPO.  

With this arrangement I repeated the measurements of isolation, transmission, and backscatter (I also increased the laser power compared to 39861).  For transmission measurements I got 95.6% and 97.6%, for isolation I got -27.9 dB and -27.8dB (0.16%), and for backscatter I got -40dB and -41dB.