Run Planning Meeting Notes, December 9 - December 15, 2021

From GlueXWiki
Jump to: navigation, search

MCC whiteboard

Wednesday December 15

  1. Last 24h:
    • Hall opened at 7 am (Full house)
    • BCAL LV module replaced (Nick) [1] issues restoring BIAS (?)
    • Survey and alignment did an "as-found" on the carbon ladder target, measuring the z position of the up- and down-stream faces of all 8 discs. Changed C by LD2 [2]
    • Parasitic setups were upgraded in the hall [3] [4]
    • Solenoid started ramping up at about 18:00 [5]
    • Beam restoration [6] same than the previous one [7]
    • Trigger studies (Sasha)
    • Started LD production at about 2:30 am (Run 90558)
  2. Issues
    • Ground fault in the solenoid power supply [8]
    • 0/90 PERP [9] difference with respect to 0/90 PARA [10]
  3. Plans
    • LD production 140 nA (0/90 PARA/PERP)
    • Reminder jlab changed to zoom, please check here [11] for new meetings links.

Tuesday December 14

  1. Last 24h:
    • Took AMO run (90532)
    • Continued C production running until 18:00
    • 18:00 started ramping down the magnet [12].
    • Rate scan (Sasha)
    • PS background check (Run 09540 and 09541) Run with only PS trigger, TPOL retracted and 1E-4 Al amorphous radiator (Sasha).
    • 7 hours of straight track runs (00:00 - 07:00) (Lubomir and Sasha)
  2. Issues
    • Early riser: FCal communication lost in one base, and it communication was restarted properly.
    • Swing shift: FCal communication lost in two bases, and it communication was restarted properly.
    • DAQ frozen and restarted.
    • Multiple FDC trips when magnet current was below 300 A and beam current was 50 nA [13]
  3. Plans
    • Fiducialization of the carbon target
    • Change from C to LD2
    • Ramp up the magnet
    • Start LD2 production
    • We will use only the 0/90 PARA/PERP configuration

Monday December 13

  1. Last 24h:
    • Continued C production running.
  2. Issues
    • Several beam downtimes: RF Separators Tripped Off (~5 hours) [14], Macropulse Chasis fault [15] and multiple beam trips.
    • Beam current stayed around 160 nA for around 15 minutes. Called MCC to tune it back to 150 nA in swing shift. (Bo)
    • An alarm was reported on CDC HV: Ring H: channel 23. This channel was internally tripped. This channel was power cycle [16]. (Bo and Beni)
    • One FCAL channel lost communication. FCAL bases were reset (swing shift) (Bo) [17]
  3. Plans
    • Continue running with C during the day and part of the swing shift.
    • Ramp down the solenoid at 7 pm tonight.
    • Tomorrow target change (If need access to the hall and not requested yet, contact (Mark/Tim/RC) )

Sunday December 12

  1. Last 24h:
    • Compton Run on C at 50 nA (Runs 090505 - 090508) [18] (Sasha)
    • Raw mode run 150 nA (090508-090509) (Sasha)
    • Low intensity run (090510-09512)
    • Continued C production running.
  2. Issues (Mainly the LV4 Sector in the BCAL)
    • Swing shift: An alarm was reported on BCAL: low voltage: downstream: channel 4. Sift crew turn it back on the LV [19].
    • LV4 Sector issue[20]: The shift workers found an empty block corresponding to the LV4 sector. After some investigation we found that the LV was off for that sector but no alarm was activated. They turned it on manually. Mark Dalton checked briefly after turning them back on and found that scalers and also the BCAL RootSpy occupancy are within expectations. He called the counting house and agreed with the shift takers that we were good to go. However there are some questions to solve yet. Hovanes looked into the problem and found that LV channel (namely the negative one) tripped at 18:24, and in a few seconds it also turned off the coupled positive channel as well [ttps://]. The LV channels were then turned back on individually and not with the botton that turns them on together (as one can see from the GUI in the log entry and in the MYA), which is not the right thing to do for the BCAL coupled LV channels. In such cases the LV channels can get turned off as if the operators deliberately turns them off and no alarm will be generated. The botton to turn these coupled channels back on was pushed at around 23:19.

During the owl and early riser shift alarms and tripping of the LV4 continue [21]

    • During the owl shift alarms and tripping of the LV4 continue [22]
    • Minor issue. It seems that the logbook is not letting put entries from the CH and apparently that issue is also seen by other halls.
  1. Plans
    • Continue running with C

Saturday December 11

  1. Last 24h:
    • Continued C production running.
    • Currently taking Compton run on C.
  2. Issues
    • Downtime Incident: Hall D and Hall D Tagger B chain dropped to restricted access (11:00 - 13:00) . SSG investigated the cause for the drop. According to the PLC diagnostic buffer the Hall D Tagger Sys B network switch and downstream hardware lost communication with MCC for 8 seconds. They tested the fiber connection between Tagger Service Bldg and MCC using fiber tester. Both fibers tested OK at 8 dB loss. Power cycled both network switches at the MCC and Tagger SB. Re-seated fiber interface cards at both network switches. [23] [24]
    • ROCTOF3 connection. Sasha requested rapid access to check the ROC and also re-booted the ROC. It has been working since then [25]
    • DAQ frozen and restarted in both swing and swing shifts.
    • This morning there were power glitches in the accelerator [26][27] starting at about 6:20 am. Simultaneously the sweeping magnet tripped and Hovanes helped us to recover it [28]. Everything looks normal after that.
  3. Plans
    • Take raw data mode
    • Continue C production.

Friday December 10

  1. Last 24h:
    • Took one AMO run (Run 90474)
    • Continued C production running.
  2. Issues
    • Downtime Incidents: RF separators tripped off and simultaneously MBD5C10V Mismatch from 7:30 - 9:30 am [29][30]
    • There were multiple alarms reported on FDC, BCAL, ST, PS. All of them became half green immediately. Clear after acknowledged them and the shift crew checked the corresponding panels, everything seemed normal.
    • One channel in the FDC tripped and was power cycled
    • DAQ frozen and rebooted (Day shift)
    • Coherent peak instable for 45/135 PARA [31]
    • One base lost communication and FCAL bases were reset [32]
  3. Plans
    • Compton run, Low intensity and Trigger studies over the weekend.
    • Continue C production.
    • No RC meeting during the weekend. RC summary will be posted in the logbook with copy to the Hall D mailing list. Next RC meeting: Monday Dec 13 at 8:45 am
    • Reminder: Target Change to LD will happen on Tuesday Dec 14 (Tim, Chris and all people involved have the plan layout). Solenoid ramp down will start during the late afternoon of Monday. People requiring access to the hall please let me know as soon as possible

Thursday December 9

  1. Last 24h:
    • Start counter dark rate test, Run 90454 [33] (Beni)
    • 1 FCAL base changed [34] (Mark)
    • FADC replaced in the tagger crate[35] (Sasha)
    • PWO modules and lead scint glass installed in the PS setup [36] (Sasha)
  2. Issues
    • Hall D beam stopper cause almost four hours of down time [37] We recovered the beam at 15:20 pm. The biggest suspect is an air pressure issue that caused a "beam stopper stuck" fault within PLC. The cause needs further investigation.
    • Solenoid dump due to a lost communication with both VCL flows for over 1 minute [38][39] (Tim & Nick)
    • Solenoid ramping up, from 19:50 pm (Dec 8) to 4:30 am (Dec 9) up to 1360 A (Tim & Beni). Waited for 15 minutes and ramp down to 1350 A (Phoebe, Jackson & Nathaly) [40][41]
    • Current major event, Separators tripped [42]
  3. Plans
    • Continue running with C
    • AMO run
    • Raw mode run
    • Beam Studies next Tuesday (accelerator), Dec 14