During the last 22 hours lock, the camera servo was working except for 4 short periods (~30s each). The attached figure shows the ADS signals, the camera guardian state, and the camera PIT3 error signal during one of the periods. As you can see, the camera PIT3 was frozen for 3s and after that the camera guardian moved to DITHER_ON state (300) and came back to CAMERA_SERVO_ON state (600) in ~30s. During this period, the ADS signals had a glitch.
Because this camera freezing is short, we can wait a bit before turning on the ADS and if the camera comes back after waiting, we can engage the camera servo again without moving to the ADS. I will implement this function in the guardian soon.
Jamie, Vicky, Naoki
The short camera freeze happened again today (Fig 1). The camera was frozen for 4s and 30s. As I have not seen this freeze since two weeks ago, I was lazy that I have not updated the guardian. I will update it soon.
We also noticed the user message in the DITHER_ON.main during the freeze (Fig. 2). This message appears when the guardian turns off the input switch of the camera PIT1.
We are not sure the reason, but Jamie recommended to remove the 'wait=False' in the ramp_gain (line 55) in the camera guardian. We removed it and reloaded the camera guardian. We hope that this message never appears again.
Naoki, Vicky
We updated the camera guardian as shown in the attached figure. We added a new state named WAIT_FOR_CAMERA. In the WAIT_FOR_CAMERA state, the guardian waits for 30s without the camera servo and ADS, and checks again the cameras. If the cameras are OK, it turns on the camera servo without going back to the ADS.
Which camera is the signal coming from that freezes?
In the latest camera freeze, all of ETMX, ETMY, BS cameras were frozen at the same time.
Interesting. ETMX and ETMY should be running different code.