Difference between revisions of "Run Planning Meeting Notes, Mar 21 - Mar 27, 2019"

From GlueXWiki
Jump to: navigation, search
(Created page with "__TOC__ * Run_Coordination_Meetings:Spring2019_Run * [https://halldweb.jlab.org/wiki/index.php/Spring_2019_Daily_Meetings PrimEx daily meeting] * [https://halldweb.jlab....")
 
(→‎Wednesday, Mar. 27 at 08:45)
 
(43 intermediate revisions by the same user not shown)
Line 1: Line 1:
 
__TOC__
 
__TOC__
−
 
  
 
* [[Run_Coordination_Meetings:Spring2019_Run]]
 
* [[Run_Coordination_Meetings:Spring2019_Run]]
Line 9: Line 8:
 
* To join via a Web Browser, go to the page [https://bluejeans.com/660743227 https://bluejeans.com/660743227]
 
* To join via a Web Browser, go to the page [https://bluejeans.com/660743227 https://bluejeans.com/660743227]
  
−
== Wednesday, Mar. 20 at 08:45 ==
+
== Thursday, Mar. 21 at 08:45 ==
 
# Past 24 hours:
 
# Past 24 hours:
−
#* TAC runs finished at 12:20
+
#* RF recovery during the day shift
−
#* Empty target running proceeded through the evening
+
#* Beam recovery delayed due to (1) issue in the injector, resolved by evening, then (2) UPS battery needs replacement in the north linac.
−
# Issues  
+
# Issues
−
#* [https://logbooks.jlab.org/entry/3669963 Low TDC occupancies in BCAL upstream sector 1.][https://logbooks.jlab.org/entry/3670093 Follow up from Elton]
+
#* low counts seen in bcal sector ([https://logbooks.jlab.org/entry/3670196 logbook entry 3670196])
 
# Reports
 
# Reports
−
#*  
+
#* helium target refilled, ready
−
#Today
+
#* [https://logbooks.jlab.org/entry/3670533 trigger rate and beam flux history for Spring 2019 run]
−
#* RF recovery today. Beam likely back back early swing shift.
+
# Today:
−
#* Refilling target today.
+
#* Production: 200nA, Al 10<sup>-4</sup>
 
#Run Plan
 
#Run Plan
−
#* Tentative plan: run with target full until Sunday
+
#* Production running: when beam returns, check beam conditions as follows:
−
#* Empty target Sunday afternoon
+
#*# do harp scans, both accelerator and Hall D radiator
−
#* Return to regular schedule of beam studies on Monday and RF recovery on Tuesday.
+
#*# check radiation monitor rates in tagger hall and Hall D
−
#* Richard takes over as RC this afternoon. Thanks to everyone for the help, and thanks to Beni for the pastries!
+
#*# verify transmission in PSC and PS coincidence rates @ 200nA, 1e-4 radiator
 +
#* Beam studies planned for day shift on Monday
 +
#* RF Recovery scheduled for Tuesday
  
−
== Tuesday, Mar. 19 at 08:45 ==
+
== Friday, Mar. 22 at 08:45 ==
 
# Past 24 hours:
 
# Past 24 hours:
−
#* Accelerator ops calibrated BPMs in the morning
+
#* beam recovery, conditions checked, running resumed.
−
#* Preparation for TAC run began after lunch
+
−
#* TAC runs started around 6pm last night and continue (2 hours long, 10M events each)
+
 
# Issues
 
# Issues
−
#* Hovanes checked the performance of the Active Collimator at small current using Beam profiler. Richard was called in to help. It appeared that pedestal subtraction was not disabled when the active collimator gains were changed to 10^-11.
+
#* HV tripped on two inner FCal blocks ([https://logbooks.jlab.org/entry/3670196 logbook entry 3670196])
 +
#* HV in CDC tripped, without alarm! ([https://logbooks.jlab.org/entry/3670885 logbook entry 3670885])
 
# Reports
 
# Reports
−
#* [https://logbooks.jlab.org/entry/3669708 Beam energy monitoring] (Alexandre)
+
#* collimator transmission drops again ([https://logbooks.jlab.org/entry/3670862 logbook entry 3670862])
−
#* [https://logbooks.jlab.org/entry/3669644 TAC run conditions] (Sasha)
+
# Today:
−
#Today
+
#* Production: 200nA, Al 10<sup>-4</sup>
−
#* The accelerator will go down for a couple hours to recover some magnets. This will begin no later than 2pm. The most likely time is after Hall A completes a Compton run around 10-11am, but they do not want to interrupt our TAC runs. This may be a good time to switch from TAC run conditions to production conditions.
+
 
#Run Plan
 
#Run Plan
−
#* RF Recovery tomorrow. Who needs an access?
+
#* Production running continues
−
#** Do we want to refill target? Or are more statistics needed on empty target?
+
#* Beam studies planned for day shift on Monday
−
#* Hall A goes down tomorrow so we will move to their slit for the rest of the run.
+
#* RF Recovery scheduled for Tuesday
  
−
== Monday, Mar. 18 at 08:45 ==
+
== Monday, Mar. 25 at 08:45 ==
−
# Past 48 hours:
+
# Past 72 hours:
−
#* Production running
+
#* production running on He target until Sunday 5:30pm
−
#* [https://logbooks.jlab.org/entry/3669217 Emptied target at 3:30pm Sunday]
+
#* continued running on empty target through Monday 8:30am
−
#* [https://logbooks.jlab.org/entry/3669252 Attempted TAC run but beam position was too unstable][https://logbooks.jlab.org/entry/3669283].
+
#* now in controlled access, RF recovery underway
 
# Issues
 
# Issues
−
#* [https://logbooks.jlab.org/entry/3668424 Short access Friday evening due to PS/ST problems. Vlad saw 2 of these alarms yesterday evening.]  
+
#* Ongoing issue with CDC HV readback ([https://logbooks.jlab.org/entry/3670885 logbook entry 3670885], [https://logbooks.jlab.org/entry/3671598 3671598], [https://logbooks.jlab.org/entry/3671943 3671943])
−
#* [https://logbooks.jlab.org/entry/3668691 Bubbler in gas shed needs to be filled]
+
#* PS scalers communication issue ([https://logbooks.jlab.org/entry/3670950 logbook entry 3670950])
−
#* [https://logbooks.jlab.org/entry/3669341 Target temperature continues to rise over 100K.]
+
#* DAQ interruptions ~1 per shift, recovery is straight-forward ([https://logbooks.jlab.org/entry/3671151], [https://logbooks.jlab.org/entry/3671542])
 +
#* Compcal monitoring plots, timing shows double peak, considered ok ([https://logbooks.jlab.org/entry/3671757 logbook entry 3671757])
 +
#* Compcal HV communication failure ([https://logbooks.jlab.org/entry/3671849 logbook entry 3671849 around 1:00am])
 +
#* FCal HV communication failure ([https://logbooks.jlab.org/entry/3671351 logbook entry 3671351])
 +
#* Individual FCal bases need reset ([https://logbooks.jlab.org/entry/3671386 logbook entry 3671386], [https://logbooks.jlab.org/entry/3671474 resolved here])
 +
#* FCal fadc scalers stop updating ([https://logbooks.jlab.org/entry/3671880 logbook entry 3671880])
 +
#* Accelerator tightens AC lock constraints ([https://logbooks.jlab.org/entry/3671930 logbook entry 3671930])
 
# Reports
 
# Reports
−
#* Daily FCAL update
+
#* Several reference plots for monitoring histograms out of date ([https://logbooks.jlab.org/entry/3671340 logbook entry 3671340])
−
#* Daily compcal RF timing discussion
+
#* Active collimator lock is mostly working, sometimes fights to keep up ([https://logbooks.jlab.org/entry/3671214 logbook entry 3671214])
−
#Today
+
#* Accelerator using machine development time to investigate this effect in Hall D
−
#* Halls B and D only halls at 5 pass. We're both at low current and MCC can not see the beam in the machine, only when it enters our halls. Beam trips may take a little longer to recover, so they asked for us to be patient.
+
# Today:
−
#* MCC looking into noisy BPMs (calibration issue?). They're looking into this now.
+
#* work on solenoid during 9:00am - 12:00pm, warm-up to begin ([https://logbooks.jlab.org/entry/3670889 logbook entry 3670889])
−
 
+
#* accelerator plans to work on Hall D line optics during second half of day shift
 
#Run Plan
 
#Run Plan
−
#* Empty target running with standard production setup.
+
#* Production running continues
−
#* What else can be done about the beam position to ensure a successful TAC run?
+
#* Beam studies planned for day shift on Monday
 +
#* RF Recovery scheduled for Tuesday
  
−
== Friday, Mar. 15 at 08:45 ==
+
== Tuesday, Mar. 26 at 08:45 ==
 
# Past 24 hours:
 
# Past 24 hours:
−
#* Short access after meeting
+
#* access day shift on Monday, out at 3:00pm
−
##* [https://logbooks.jlab.org/entry/3667744 Beni/Tim: adjust rotometer flow to 1 L/hr]
+
#* beam restored by 18:30
−
##* Sergey and Hovanes: DAQ
+
#* empty-target running continues
−
##* [https://logbooks.jlab.org/entry/3667703 logbook entry Chris, Colin, Nick, Sasha: FCAL VME crate 11 power supply/ fan issue]
+
−
#* gluon01 is back with a new kernel and graphics driver
+
−
#* Production running
+
 
# Issues
 
# Issues
−
#* Problematic FCAL channels near beamline
+
#* Ongoing problems with dropped communication to CDC HV crate ([https://logbooks.jlab.org/entry/3671943 logbook entry 3671943], [https://logbooks.jlab.org/entry/3671980 crate rebooted])
−
##* One base lost connection and was recovered by power cycling the power supply. This happens 1-2 times per week. Right now I (Colin) am the only one capable of doing this. Need to train more people how to spot and fix these errors. It is a relatively simple process, but one that should not be left to the shift crew.
+
#* Reboot of CompCal HV crate failed to restore comms, crate is replaced ([https://logbooks.jlab.org/entry/3671980 logbook entry 3671980])
−
##* [https://logbooks.jlab.org/entry/3667944 One base lost HV near beam line.][https://logbooks.jlab.org/entry/3668146 Follow up.] Access to replace after morning meeting. Friendly reminder: please do not call Colin at night about base problems. Emails and texts are fine.  
+
#* Accelerator switched to manual current adjustment overnight, attenuator problems ([https://logbooks.jlab.org/entry/3672307 logbook entry 3672307])
−
#* [https://logbooks.jlab.org/entry/3668020 Monitoring problem with file/DB management. Usually does not happen]
+
#* Ongoing DAQ issues ([https://logbooks.jlab.org/entry/3672178 logbook entry 3672178])
−
#* David and Thomas looking into CCAL pedestal Rootspy issue. Appears to be a Rootspy display issue and Dave will investigate more next week.
+
#* Advice to shift crew
 +
#*# shift summaries are great for overview of what happened during a shift
 +
#*# this summary is NOT a replacement for separate logbook entries related to exceptional conditions or events
 +
#*# PLEASE make separate logbook entries to report ANY SIGNIFICANT ISSUES that come up!
 +
#*# without these, it is difficult to search the record, create follow-up entries, post plots, etc.
 
# Reports
 
# Reports
−
#* [https://logbooks.jlab.org/entry/3667917 FCAL scalers since LHe target switchover (follow up from disconnected crates)]
+
#* solenoid warm-up underway ([https://logbooks.jlab.org/entry/3671978 logbook entry 3671978])
−
#* [https://logbooks.jlab.org/entry/3667806 Topic of the week: CCAL shower time - RF time] (Beni)
+
#* Updated normalized trigger rates history ([https://logbooks.jlab.org/entry/3672059 logbook entry 3672059])
 +
#* Near-line FCAL gains calibration ([https://logbooks.jlab.org/entry/3672065 logbook entry 3672065])
 +
#* Tagger accidentals fraction ([https://logbooks.jlab.org/entry/3672169 logbook entry 3672169])
 
# Today:
 
# Today:
−
#* Production: 200nA, Al 10<sup>-4</sup>
+
#* machine rf recovery during day shift
−
#Run Plan
+
#* did we get enough empty target data, are we ready to fill the He target?
−
#* Production running  
+
# Run Plan
−
#* Empty target run starts during the day on Sunday, 3/24.
+
#* Production running continues
  
−
== Thursday, Mar. 21 at 08:45 ==
+
== Wednesday, Mar. 27 at 08:45 ==
 
# Past 24 hours:
 
# Past 24 hours:
−
#* RF recovery during the day shift
+
#* access day shift on Tuesday, filled the He target, out by 11:40pm
−
#* Beam recovery delayed due to (1) issue in the injector, resolved by evening, then (2) UPS battery needs replacement in the north linac.  
+
#* beam restored by 13:10, harp scans, running resumes at 13:30
 +
#* He-target filled ([https://logbooks.jlab.org/entry/3672522])
 +
#* running continues
 
# Issues
 
# Issues
−
#* getting beam back
+
#* big changes in beam x position at dump during early morning ([https://logbooks.jlab.org/entry/3672308 logentry 3672308, see IPMAD00C.XPOS])
 +
#* Alexandre explains this as a change in tagger dipole field ([https://logbooks.jlab.org/entry/3672359 logentry 3672359])
 +
#* tagger NMR probe reading was not updating, restored after ioc reset ([https://logbooks.jlab.org/entry/3672433 logentry 3672433])
 +
#* older-model HV crate moved from compcal/dirc to CDC, newer-model HV crate swapped from CDC to compcal/dirc (WHY NO LOG ENTRY?)
 +
#* after startup, discovered TOF HV was not fully on, recovered after ~30min lost ([https://logbooks.jlab.org/entry/3672310 logentry 3672310])
 +
#* AC beam positions seem more stable with improved slow-lock ([https://logbooks.jlab.org/entry/3672599 logbook entry 3672599])
 +
#* tof FADC scalers stopped updating, fixed by ioc reboot ([https://logbooks.jlab.org/entry/3672491 logentry 3672491])
 +
#* occasional HV alarms, reset themselves without intervention ([https://logbooks.jlab.org/entry/3672491 logentry 3672491])
 +
#* dirc alarm, sensor reports water flow stopped, temperature 300 deg! ([https://logbooks.jlab.org/entry/3672654])
 
# Reports
 
# Reports
−
#* any?
+
#* beam energy drifts by ~10 MeV over several hours ([https://logbooks.jlab.org/entry/3672471 logentry 3672471])
 +
#* ccdb translation table is messed up -- recovery underway today
 
# Today:
 
# Today:
−
#* Production: 200nA, Al 10<sup>-4</sup>
+
#* beam going away at 10:00 for hall C extraction change from 3-pass to 4-pass, expected to take ~3 hours
−
#Run Plan
+
#* will take access at 10:00 to check on the DIRC water temperature sensor
−
#* Production running  
+
#* production continues when beam is restored
−
#* Beam studies planned for day shift on Monday
+
# Run Plan
−
#* RF Recovery scheduled for Tuesday
+
#* Production running continues
 +
#* RC change today at 1:30 - new RC is Stuart Fegan

Latest revision as of 08:44, 27 March 2019

Thursday, Mar. 21 at 08:45

  1. Past 24 hours:
    • RF recovery during the day shift
    • Beam recovery delayed due to (1) issue in the injector, resolved by evening, then (2) UPS battery needs replacement in the north linac.
  2. Issues
  3. Reports
  4. Today:
    • Production: 200nA, Al 10-4
  5. Run Plan
    • Production running: when beam returns, check beam conditions as follows:
      1. do harp scans, both accelerator and Hall D radiator
      2. check radiation monitor rates in tagger hall and Hall D
      3. verify transmission in PSC and PS coincidence rates @ 200nA, 1e-4 radiator
    • Beam studies planned for day shift on Monday
    • RF Recovery scheduled for Tuesday

Friday, Mar. 22 at 08:45

  1. Past 24 hours:
    • beam recovery, conditions checked, running resumed.
  2. Issues
  3. Reports
  4. Today:
    • Production: 200nA, Al 10-4
  5. Run Plan
    • Production running continues
    • Beam studies planned for day shift on Monday
    • RF Recovery scheduled for Tuesday

Monday, Mar. 25 at 08:45

  1. Past 72 hours:
    • production running on He target until Sunday 5:30pm
    • continued running on empty target through Monday 8:30am
    • now in controlled access, RF recovery underway
  2. Issues
  3. Reports
    • Several reference plots for monitoring histograms out of date (logbook entry 3671340)
    • Active collimator lock is mostly working, sometimes fights to keep up (logbook entry 3671214)
    • Accelerator using machine development time to investigate this effect in Hall D
  4. Today:
    • work on solenoid during 9:00am - 12:00pm, warm-up to begin (logbook entry 3670889)
    • accelerator plans to work on Hall D line optics during second half of day shift
  5. Run Plan
    • Production running continues
    • Beam studies planned for day shift on Monday
    • RF Recovery scheduled for Tuesday

Tuesday, Mar. 26 at 08:45

  1. Past 24 hours:
    • access day shift on Monday, out at 3:00pm
    • beam restored by 18:30
    • empty-target running continues
  2. Issues
    • Ongoing problems with dropped communication to CDC HV crate (logbook entry 3671943, crate rebooted)
    • Reboot of CompCal HV crate failed to restore comms, crate is replaced (logbook entry 3671980)
    • Accelerator switched to manual current adjustment overnight, attenuator problems (logbook entry 3672307)
    • Ongoing DAQ issues (logbook entry 3672178)
    • Advice to shift crew
      1. shift summaries are great for overview of what happened during a shift
      2. this summary is NOT a replacement for separate logbook entries related to exceptional conditions or events
      3. PLEASE make separate logbook entries to report ANY SIGNIFICANT ISSUES that come up!
      4. without these, it is difficult to search the record, create follow-up entries, post plots, etc.
  3. Reports
  4. Today:
    • machine rf recovery during day shift
    • did we get enough empty target data, are we ready to fill the He target?
  5. Run Plan
    • Production running continues

Wednesday, Mar. 27 at 08:45

  1. Past 24 hours:
    • access day shift on Tuesday, filled the He target, out by 11:40pm
    • beam restored by 13:10, harp scans, running resumes at 13:30
    • He-target filled ([3])
    • running continues
  2. Issues
    • big changes in beam x position at dump during early morning (logentry 3672308, see IPMAD00C.XPOS)
    • Alexandre explains this as a change in tagger dipole field (logentry 3672359)
    • tagger NMR probe reading was not updating, restored after ioc reset (logentry 3672433)
    • older-model HV crate moved from compcal/dirc to CDC, newer-model HV crate swapped from CDC to compcal/dirc (WHY NO LOG ENTRY?)
    • after startup, discovered TOF HV was not fully on, recovered after ~30min lost (logentry 3672310)
    • AC beam positions seem more stable with improved slow-lock (logbook entry 3672599)
    • tof FADC scalers stopped updating, fixed by ioc reboot (logentry 3672491)
    • occasional HV alarms, reset themselves without intervention (logentry 3672491)
    • dirc alarm, sensor reports water flow stopped, temperature 300 deg! ([4])
  3. Reports
    • beam energy drifts by ~10 MeV over several hours (logentry 3672471)
    • ccdb translation table is messed up -- recovery underway today
  4. Today:
    • beam going away at 10:00 for hall C extraction change from 3-pass to 4-pass, expected to take ~3 hours
    • will take access at 10:00 to check on the DIRC water temperature sensor
    • production continues when beam is restored
  5. Run Plan
    • Production running continues
    • RC change today at 1:30 - new RC is Stuart Fegan
Retrieved from "https://halldweb.jlab.org/wiki/index.php?title=Run_Planning_Meeting_Notes,_Mar_21_-_Mar_27,_2019&oldid=92002"