Difference between revisions of "Run Planning Meeting Notes, October 28 - November 3, 2021"
From GlueXWiki
(15 intermediate revisions by the same user not shown) | |||
Line 1: | Line 1: | ||
[https://cebaf.jlab.org/files/ops/accboard/ MCC whiteboard] | [https://cebaf.jlab.org/files/ops/accboard/ MCC whiteboard] | ||
− | [https://bluejeans.com/660743227 Meeting Bluejeans 660743227] | + | [https://bluejeans.com/660743227 Meeting Bluejeans 660743227] |
− | = | + | =Wednesday November 03= |
#Last 24h: | #Last 24h: | ||
− | #* Recorded | + | #* Recorded 1029M+ events with full LHe4 target |
+ | #* [https://logbooks.jlab.org/entry/3936040 U1-9-TOP reboot] | ||
+ | #* [https://logbooks.jlab.org/entry/3936068 runai on gluon47 hangs] | ||
+ | #* Some repeated trouble with RootSpy [https://logbooks.jlab.org/entry/3936227] and DAQ comments box [https://logbooks.jlab.org/entry/3936204] | ||
+ | #Issues: | ||
+ | #* Bleedthrough | ||
+ | # [https://halldweb.jlab.org/primexd/run_plan/Run_Plan_Fall_2021.pdf Plan:] | ||
+ | #* '''TODAY: Maintenance day''' | ||
+ | #* Continue production at 200nA with the solenoid on after that. | ||
+ | #* MCC (Jay) discussed possibility to move up the energy shift to Thu, swing | ||
+ | #** Controlled access planned for maintenance work (e.g. FCAL [https://logbooks.jlab.org/entry/3935581]) | ||
+ | #* Beam energy change planned for 11/05-11/07 | ||
+ | #** RadCon survey for friday morning is requested | ||
+ | #* Next RC will be Alex Austregesilo | ||
+ | =Tuesday November 02= | ||
+ | #Last 24h: | ||
+ | #* Recorded 83M events with empty target, one run in raw-mode, then 726M+ events with full LHe4 target | ||
+ | #* gluonraid1 was degraded -> one disk was replaced (RAID5, no hot spare; rebuild started automatically) | ||
+ | #* [https://logbooks.jlab.org/entry/3935583 ADC-TDC Timing Shift in TAGH] | ||
+ | #* [https://logbooks.jlab.org/entry/3935590 BCAL ADC-TDC Timing Shifts] | ||
+ | #* [https://logbooks.jlab.org/entry/3935599 Thermocouple locations] | ||
+ | #* [https://logbooks.jlab.org/entry/3935642 NMR signal and tagger rates] | ||
+ | #* Day shift crew updated RootSpy reference plots for full target run [https://logbooks.jlab.org/entry/3935704] | ||
+ | #* CDC Voltages IOC lost communication, rebooted [https://logbooks.jlab.org/entry/3935781] | ||
+ | #Issues: | ||
+ | #* Bleedthrough | ||
+ | # [https://halldweb.jlab.org/primexd/run_plan/Run_Plan_Fall_2021.pdf Plan:] | ||
+ | #* Continue production at 200nA with the solenoid on. | ||
+ | #* Since Hall A cannot run at present, we are the priority hall. | ||
+ | #* '''Maintenance day tomorrow, Wed 11/03''' | ||
+ | #** Controlled access planned for maintenance work (e.g. FCAL [https://logbooks.jlab.org/entry/3935581]) | ||
+ | #* Beam energy change planned for 11/05-11/07 | ||
+ | =Monday November 01= | ||
+ | #Over the weekend: | ||
+ | #* Recorded 331M events from Fri, 8am till target was emptied. Then 1400M+ events with empty target until now | ||
+ | #* Comment box on DAQ machine did not show up -> restarting DAQ brought it back. Missing comments for some runs recorded in logbook [https://logbooks.jlab.org/entry/3934452] | ||
+ | #* [https://logbooks.jlab.org/entry/3934565 Follow-up Re: Follow-up Re: TAGH scalers, roctagh2] | ||
+ | #* Target was emptied at 6:10pm Firday [https://logbooks.jlab.org/entry/3934588] | ||
+ | #* [https://logbooks.jlab.org/entry/3934591 BCAL noise at 10 o'clock] | ||
+ | #* [https://logbooks.jlab.org/entry/3934854 Note on neutron background in Hall D] | ||
+ | #* Over the weekend, reference plots have been updated to reflect status with empty target [https://logbooks.jlab.org/entry/3935037] | ||
+ | #* When we run with empty target, dont run AI for CDC HV. Stay with 2125V fixed value [https://logbooks.jlab.org/entry/3935003] | ||
+ | #* [https://logbooks.jlab.org/entry/3935332 power cycle of rocTAGH1, rocTAGH2, restart DAQ] due to frozen TAGH scalers | ||
+ | #* Spooky halloween shift - accelerator became unstable (lots of RF trips) sunday night [https://logbooks.jlab.org/entry/3935421] | ||
+ | #* Target will be filled this morning (after the meeting). Chris C requested short access during filling to watch target interlock panel | ||
+ | #Issues: | ||
+ | #* <s>BCAL noise at 10 o'clock</s> | ||
+ | #* Bleedthrough | ||
+ | # [https://halldweb.jlab.org/primexd/run_plan/Run_Plan_Fall_2021.pdf Plan:] | ||
+ | #* Continue production at 200nA with the solenoid on. | ||
+ | #* Since Hall A cannot run at present, we are the priority hall. | ||
+ | #* Maintenance day on Wed 11/03 '''beam will be shut off at ~6am''' | ||
+ | #** Hall D will go into controlled access for maintenance work (FCAL, ...?) | ||
+ | #* Beam energy change planned for 11/05-11/07 | ||
+ | #* Hall A scheduled to come back online on Monday, 11/08 | ||
+ | =Friday October 29= | ||
+ | #Last 24h: | ||
+ | #* Recorded 1090M+ events | ||
#* [https://logbooks.jlab.org/entry/3933753 Change out of C02] | #* [https://logbooks.jlab.org/entry/3933753 Change out of C02] | ||
− | #* RadCon replaced light bulb (really broken) in our rapid access beacon with an LED bulb | + | #* RadCon replaced light bulb (was really broken) in our rapid access beacon with an LED bulb |
#** Access was necessary, but machine was actually down during that time | #** Access was necessary, but machine was actually down during that time | ||
#* RCDB web interface was unreachable, shift crew contacted administrator and the service was restarted | #* RCDB web interface was unreachable, shift crew contacted administrator and the service was restarted | ||
#* HallC BCM calibration should not have affected us, but they use varying beam currents for that procedure, which may have changed bleedthrough during run 81620 (see [https://logbooks.jlab.org/entry/3933796]) | #* HallC BCM calibration should not have affected us, but they use varying beam currents for that procedure, which may have changed bleedthrough during run 81620 (see [https://logbooks.jlab.org/entry/3933796]) | ||
#* New wiki page with short-term guidance for shifters [https://halldweb.jlab.org/hdops/wiki/index.php/Short_Term_Run_Plan] | #* New wiki page with short-term guidance for shifters [https://halldweb.jlab.org/hdops/wiki/index.php/Short_Term_Run_Plan] | ||
+ | #* [https://logbooks.jlab.org/entry/3934095 Beam - RF Timing differences with different trigger types] | ||
+ | #* [https://logbooks.jlab.org/entry/3934271 RAID beeping in DAQ room] | ||
#Issues: | #Issues: | ||
#* BCAL noise at 10 o'clock | #* BCAL noise at 10 o'clock | ||
Line 18: | Line 77: | ||
#* Maintenance day '''shifted to Wed 11/03''' | #* Maintenance day '''shifted to Wed 11/03''' | ||
#* Beam energy change planned for 11/05-11/07; decision pending due to availability of Hall A | #* Beam energy change planned for 11/05-11/07; decision pending due to availability of Hall A | ||
− | =Thursday | + | =Thursday October 28= |
#Last 24h: | #Last 24h: | ||
#* Recorded 949M+ events | #* Recorded 949M+ events |
Latest revision as of 08:23, 3 November 2021
MCC whiteboard Meeting Bluejeans 660743227
Contents
Wednesday November 03
- Last 24h:
- Recorded 1029M+ events with full LHe4 target
- U1-9-TOP reboot
- runai on gluon47 hangs
- Some repeated trouble with RootSpy [1] and DAQ comments box [2]
- Issues:
- Bleedthrough
- Plan:
- TODAY: Maintenance day
- Continue production at 200nA with the solenoid on after that.
- MCC (Jay) discussed possibility to move up the energy shift to Thu, swing
- Controlled access planned for maintenance work (e.g. FCAL [3])
- Beam energy change planned for 11/05-11/07
- RadCon survey for friday morning is requested
- Next RC will be Alex Austregesilo
Tuesday November 02
- Last 24h:
- Recorded 83M events with empty target, one run in raw-mode, then 726M+ events with full LHe4 target
- gluonraid1 was degraded -> one disk was replaced (RAID5, no hot spare; rebuild started automatically)
- ADC-TDC Timing Shift in TAGH
- BCAL ADC-TDC Timing Shifts
- Thermocouple locations
- NMR signal and tagger rates
- Day shift crew updated RootSpy reference plots for full target run [4]
- CDC Voltages IOC lost communication, rebooted [5]
- Issues:
- Bleedthrough
- Plan:
- Continue production at 200nA with the solenoid on.
- Since Hall A cannot run at present, we are the priority hall.
- Maintenance day tomorrow, Wed 11/03
- Controlled access planned for maintenance work (e.g. FCAL [6])
- Beam energy change planned for 11/05-11/07
Monday November 01
- Over the weekend:
- Recorded 331M events from Fri, 8am till target was emptied. Then 1400M+ events with empty target until now
- Comment box on DAQ machine did not show up -> restarting DAQ brought it back. Missing comments for some runs recorded in logbook [7]
- Follow-up Re: Follow-up Re: TAGH scalers, roctagh2
- Target was emptied at 6:10pm Firday [8]
- BCAL noise at 10 o'clock
- Note on neutron background in Hall D
- Over the weekend, reference plots have been updated to reflect status with empty target [9]
- When we run with empty target, dont run AI for CDC HV. Stay with 2125V fixed value [10]
- power cycle of rocTAGH1, rocTAGH2, restart DAQ due to frozen TAGH scalers
- Spooky halloween shift - accelerator became unstable (lots of RF trips) sunday night [11]
- Target will be filled this morning (after the meeting). Chris C requested short access during filling to watch target interlock panel
- Issues:
-
BCAL noise at 10 o'clock - Bleedthrough
-
- Plan:
- Continue production at 200nA with the solenoid on.
- Since Hall A cannot run at present, we are the priority hall.
- Maintenance day on Wed 11/03 beam will be shut off at ~6am
- Hall D will go into controlled access for maintenance work (FCAL, ...?)
- Beam energy change planned for 11/05-11/07
- Hall A scheduled to come back online on Monday, 11/08
Friday October 29
- Last 24h:
- Recorded 1090M+ events
- Change out of C02
- RadCon replaced light bulb (was really broken) in our rapid access beacon with an LED bulb
- Access was necessary, but machine was actually down during that time
- RCDB web interface was unreachable, shift crew contacted administrator and the service was restarted
- HallC BCM calibration should not have affected us, but they use varying beam currents for that procedure, which may have changed bleedthrough during run 81620 (see [12])
- New wiki page with short-term guidance for shifters [13]
- Beam - RF Timing differences with different trigger types
- RAID beeping in DAQ room
- Issues:
- BCAL noise at 10 o'clock
- Bleedthrough
- Plan:
- Continue production at 200nA with the solenoid on.
- Since Hall A cannot run at present, we are the priority hall.
- Maintenance day shifted to Wed 11/03
- Beam energy change planned for 11/05-11/07; decision pending due to availability of Hall A
Thursday October 28
- Last 24h:
- Recorded 949M+ events
- Emergency Control box intermittent alarms
- TAGH scalers froze (roctagh2); TAGH1 and TAGH2 crates power cycled
- Some issues with starting root spy (see day shift summary)
- 2.5hrs downtime due to issue with magnet shunt module (has been replaced)
- Chris C (target) requested access to check heater power supply of the condenser [14].
- Our rapid access beacon is not working again (turning, but no light!)
- RadCon escorted Chris in and out of the hall.
- Rebinning issue in compton monitoring macro
- Issues:
- Rapid access beacon gives no light. The same failure happened 2 weeks ago, light bulb was replaced [15]
- Low energy tagger counters?
- BCAL noise at 10 o'clock
- Bleedthrough
- Plan:
- Continue production at 200nA with the solenoid on.
- Since Hall A cannot run at present, we are the priority hall.
- Hall C BCM calibration moved to today (Thursday) 2:30pm.
- Maintenance day planned for Tue 11/02
- Beam energy change planned for 11/05-11/07; decision pending due to availability of Hall A