Run Planning Meeting Notes, December 9 - December 15, 2021

From GlueXWiki
Revision as of 09:57, 12 December 2021 by Ssanties (Talk | contribs)

Jump to: navigation, search

MCC whiteboard

Sunday December 12

  1. Last 24h:
    • Compton Run on C at 50 nA (Runs 090505 - 090508) [1] (Sasha)
    • Raw mode run 150 nA (090508-090509) (Sasha)
    • Low intensity run (090510-09512)
    • Continued C production running.
  2. Issues (Mainly the LV4 Sector in the BCAL)
    • Swing shift: An alarm was reported on BCAL: low voltage: downstream: channel 4. Sift crew turn it back on the LV [2].
    • LV4 Sector issue[3]: The shift workers found an empty block corresponding to the LV4 sector. After some investigation we found that the LV was off for that sector but no alarm was activated. They turned it on manually. Mark Dalton checked briefly after turning them back on and found that scalers and also the BCAL RootSpy occupancy and both are within expectations. He called the counting house and agreed with the shift takers that we are good to go. However there are some questions to solve yet. Hovanes looked into the problem and found that LV channel (namely the negative one) tripped at 18:24, and in a few seconds it also turned off the coupled positive channel as well [4]. The LV channels were then turned back on individually and not with the button that turns them on together (as one can see from the GUI in the log entry and in the MYA), which is not the right thing to do for the BCAL coupled LV channels. In such cases the LV channels can get turned off as if the operators deliberately turns them off and no alarm will be generated. The button to turn these coupled channels back on was pushed at around 23:19.
    • During the owl shift alarms and tripping of the LV4 continue [5]
    • Minor issue
  3. Plans
    • Continue running with C

Saturday December 11

  1. Last 24h:
    • Continued C production running.
    • Currently taking Compton run on C.
  2. Issues
    • Downtime Incident: Hall D and Hall D Tagger B chain dropped to restricted access (11:00 - 13:00) . SSG investigated the cause for the drop. According to the PLC diagnostic buffer the Hall D Tagger Sys B network switch and downstream hardware lost communication with MCC for 8 seconds. They tested the fiber connection between Tagger Service Bldg and MCC using fiber tester. Both fibers tested OK at 8 dB loss. Power cycled both network switches at the MCC and Tagger SB. Re-seated fiber interface cards at both network switches. [6] [7]
    • ROCTOF3 connection. Sasha requested rapid access to check the ROC and also re-booted the ROC. It has been working since then [8]
    • DAQ frozen and restarted in both swing and swing shifts.
    • This morning there were power glitches in the accelerator [9][10] starting at about 6:20 am. Simultaneously the sweeping magnet tripped and Hovanes helped us to recover it [11]. Everything looks normal after that.
  3. Plans
    • Take raw data mode
    • Continue C production.

Friday December 10

  1. Last 24h:
    • Took one AMO run (Run 90474)
    • Continued C production running.
  2. Issues
    • Downtime Incidents: RF separators tripped off and simultaneously MBD5C10V Mismatch from 7:30 - 9:30 am [12][13]
    • There were multiple alarms reported on FDC, BCAL, ST, PS. All of them became half green immediately. Clear after acknowledged them and the shift crew checked the corresponding panels, everything seemed normal.
    • One channel in the FDC tripped and was power cycled
    • DAQ frozen and rebooted (Day shift)
    • Coherent peak instable for 45/135 PARA [14]
    • One base lost communication and FCAL bases were reset [15]
  3. Plans
    • Compton run, Low intensity and Trigger studies over the weekend.
    • Continue C production.
    • No RC meeting during the weekend. RC summary will be posted in the logbook with copy to the Hall D mailing list. Next RC meeting: Monday Dec 13 at 8:45 am
    • Reminder: Target Change to LD will happen on Tuesday Dec 14 (Tim, Chris and all people involved have the plan layout). Solenoid ramp down will start during the late afternoon of Monday. People requiring access to the hall please let me know as soon as possible

Thursday December 9

  1. Last 24h:
    • Start counter dark rate test, Run 90454 [16] (Beni)
    • 1 FCAL base changed [17] (Mark)
    • FADC replaced in the tagger crate[18] (Sasha)
    • PWO modules and lead scint glass installed in the PS setup [19] (Sasha)
  2. Issues
    • Hall D beam stopper cause almost four hours of down time [20] We recovered the beam at 15:20 pm. The biggest suspect is an air pressure issue that caused a "beam stopper stuck" fault within PLC. The cause needs further investigation.
    • Solenoid dump due to a lost communication with both VCL flows for over 1 minute [21][22] (Tim & Nick)
    • Solenoid ramping up, from 19:50 pm (Dec 8) to 4:30 am (Dec 9) up to 1360 A (Tim & Beni). Waited for 15 minutes and ramp down to 1350 A (Phoebe, Jackson & Nathaly) [23][24]
    • Current major event, Separators tripped [25]
  3. Plans
    • Continue running with C
    • AMO run
    • Raw mode run
    • Beam Studies next Tuesday (accelerator), Dec 14