Difference between revisions of "Run Planning Meeting Notes, February 16 - February 22, 2023"

From GlueXWiki
Jump to: navigation, search
(→‎Reports)
 
(119 intermediate revisions by 2 users not shown)
Line 8: Line 8:
  
 
[https://halldweb.jlab.org/data_monitoring/run_statistics/RunPeriod-2023-01/index.html Integrated Trigger Count]
 
[https://halldweb.jlab.org/data_monitoring/run_statistics/RunPeriod-2023-01/index.html Integrated Trigger Count]
 +
 +
= Wednesday February 22 =
 +
 +
=== Past 24 hours ===
 +
* Production runs. Several cosmic runs were taken during downtimes.
 +
 +
=== Shift Reports ===
 +
* [https://logbooks.jlab.org/entry/4136448 0400-1000]
 +
* [https://logbooks.jlab.org/entry/4136217 2200-0400]
 +
* [https://logbooks.jlab.org/entry/4135998 1600-2200]
 +
* [https://logbooks.jlab.org/entry/4135812 1000-1600]
 +
* [https://logbooks.jlab.org/entry/4135672 0600-1000]
 +
 +
=== Reports ===
 +
* [https://logbooks.jlab.org/entry/4135840 Follow-up Re: IBC0L02 calibration to FC2] (Alexandre)
 +
* [https://logbooks.jlab.org/entry/4135883 Hall D Target compressor cooling water] (Tim)
 +
* [https://logbooks.jlab.org/entry/4136018 ISEG modules swapped] (Mark)
 +
* [https://logbooks.jlab.org/entry/4136001 TAGM wave forms] (Beni)
 +
* [https://logbooks.jlab.org/entry/4136129 MPOD installed on CCAL test setup] (Sasha)
 +
* [https://logbooks.jlab.org/entry/4136455 brief grafana error] (Simon)
 +
* [https://logbooks.jlab.org/entry/4136518 Tagger NMR] (Scot)
 +
* [https://logbooks.jlab.org/entry/4136530 FDC CDC gas pressure alarm PT501]
 +
* Statistics balance
 +
**PARA 0/90: 21.0%
 +
**PERP 0/90: 20.5%
 +
**PARA 45/135: 21.4%
 +
**PERP 45/135: 20.6%
 +
**AMO : 16.5%
 +
 +
=== Issues ===
 +
Pending:
 +
* [https://logbooks.jlab.org/entry/4117764 Start Counter (Beni)] - We do not understand why there is a mismatch in the software between start counter and tracking.
 +
* [https://logbooks.jlab.org/entry/4133378 TAGH hot channels w/ no beam]
 +
** Hovanes rebooted the crate but it didn't help.
 +
** Sasha explains the problematic channel(s) are deteriorated and this is not an issue which can be easily fixed.
 +
* [https://logbooks.jlab.org/entry/4133666 Noise in FDC2] (Lubomir)
 +
* [https://logbooks.jlab.org/entry/4133998 FCAL11 crate problems] (Sergey)
 +
** Network port on the fan tray for FCAL11 crate is not working. When we have an expected long downtime, this tray should be replaced (Chris Stanislav).
 +
 +
=== Plans ===
 +
* Install CCAL whenever there is an extended accelerator downtime
 +
** Restricted Access is needed to be able to use the ramp <-done
 +
** Cabling
 +
** Survey is needed
 +
* Production (300 nA, 2h runs) with cycle:
 +
**2 runs PARA (0/90)
 +
**2 runs PERP (0/90)
 +
**2 runs PARA (45/135)
 +
**2 runs PERP (45/135)
 +
**1 run AMO (use it to rejuvenate AMO reference in CBREM EPICS GUI)
 +
 +
= Tuesday February 21 =
 +
 +
=== Past 24 hours ===
 +
* BCM Calibration executed. MCC decided no recalibration was needed.
 +
* Production runs resumed after beam came back.
 +
 +
=== Shift Reports ===
 +
* [https://logbooks.jlab.org/entry/4135672 0600-1000]
 +
* [https://logbooks.jlab.org/entry/4135612 0400-0600]
 +
* [https://logbooks.jlab.org/entry/4135462 2200-0400]
 +
* [https://logbooks.jlab.org/entry/4135276 1600-2200]
 +
* [https://logbooks.jlab.org/entry/4135189 1000-1600]
 +
* [https://logbooks.jlab.org/entry/4134981 0400-1000]
 +
 +
=== Reports ===
 +
* [https://logbooks.jlab.org/entry/4135208 Hall D NMR Meter - Installed] (Scot)
 +
** MCC stopped beam delivery after NMR IOC communication loss (just a monitor device) was detected.
 +
** (In RC meeting) They say they are not expert in Hall equipment and they will stop the beam delivery when whatever crew chief does not know happens.
 +
* [https://logbooks.jlab.org/entry/4135236 BCM calibration]
 +
** MCC called shift crews to ask what the maximum calibration current should be.
 +
** (In RC meeting) They say this is on us (Hall D), so we should prepare this value beforehand in the future BCM calibration work.
 +
* [https://logbooks.jlab.org/entry/4135353 Harp scan after BCM work]
 +
* [https://logbooks.jlab.org/entry/4135733 Harp scan after the long midnight downtime]
 +
* [https://logbooks.jlab.org/entry/4135463 BCAL ISEG modules swapped during a long downtime] (Mark)
 +
** [https://logbooks.jlab.org/entry/4135758 BCAL voltage instability gone]
 +
* Took BCAL cosmic run (R120783 and more) during the calibration work (and during midnight downtime).
 +
* Early-riser shift leader felt unwell and Keigo (RC) took over. We need a volunteer for tomorrow morning shift.
 +
* Statistics balance
 +
**PARA 0/90: 21.0%
 +
**PERP 0/90: 20.5%
 +
**PARA 45/135: 21.4%
 +
**PERP 45/135: 20.6%
 +
**AMO : 16.5%
 +
 +
=== Issues ===
 +
Pending:
 +
* [https://logbooks.jlab.org/entry/4117764 Start Counter (Beni)] - We do not understand why there is a mismatch in the software between start counter and tracking.
 +
* [https://logbooks.jlab.org/entry/4133378 TAGH hot channels w/ no beam]
 +
* [https://logbooks.jlab.org/entry/4133666 Noise in FDC2] (Lubomir)
 +
* [https://logbooks.jlab.org/entry/4133998 FCAL11 crate problems] (Sergey)
 +
* [https://logbooks.jlab.org/entry/4133226 Temperature rise of target compressor cooling water] (Tim)
 +
 +
=== Plans ===
 +
* Install CCAL whenever there is an extended accelerator downtime
 +
** Restricted Access is needed to be able to use the ramp <-done
 +
** Cabling
 +
** Survey is needed
 +
* Production (300 nA, 2h runs) with cycle:
 +
**2 runs PARA (0/90)
 +
**2 runs PERP (0/90)
 +
**2 runs PARA (45/135)
 +
**2 runs PERP (45/135)
 +
**1 run AMO (use it to rejuvenate AMO reference in CBREM EPICS GUI)
 +
 +
= Monday February 20 =
 +
=== Past 72 hours ===
 +
* Production runs. ABU = 35.5 h for the weekend. (12.7 h Friday, 13.8 h Saturday, 9.0 h Sunday)
 +
 +
=== Shift Reports Friday===
 +
* [https://logbooks.jlab.org/entry/4133204 Day shift summary]
 +
* [https://logbooks.jlab.org/entry/4133443 Swing shift summary]
 +
* [https://logbooks.jlab.org/entry/4133638 Owl shift summary]
 +
* [https://logbooks.jlab.org/entry/4133777 Early Riser shift summary]
 +
 +
=== Shift Reports Saturday===
 +
* [https://logbooks.jlab.org/entry/4133952 Day shift summary]
 +
* [https://logbooks.jlab.org/entry/4134082 Swing shift summary]
 +
* [https://logbooks.jlab.org/entry/4134308 Owl shift summary]
 +
* [https://logbooks.jlab.org/entry/4134418 Early Riser shift summary]
 +
 +
=== Shift Reports Sunday===
 +
* [https://logbooks.jlab.org/entry/4134593 Day shift summary]
 +
* [https://logbooks.jlab.org/entry/4134777 Swing shift summary]
 +
* [https://logbooks.jlab.org/entry/4134888 Owl shift summary]
 +
* [https://logbooks.jlab.org/entry/4134981 Early Riser shift summary]
 +
 +
=== Reports ===
 +
* [https://logbooks.jlab.org/entry/4134037 Hall A fire alarm on Sat.]
 +
* [https://logbooks.jlab.org/entry/4133226 Temperature rise of target compressor cooling water] (Tim)
 +
* [https://logbooks.jlab.org/entry/4133378 TAGH hot channels w/ no beam]
 +
* [https://logbooks.jlab.org/entry/4133568 Harp scan on Friday]
 +
* [https://logbooks.jlab.org/entry/4133666 Noise in FDC2] (Lubomir)
 +
* [https://logbooks.jlab.org/entry/4133998 FCAL11 crate problems] (Sergey)
 +
* [https://logbooks.jlab.org/entry/4134665 For production runs please try to start the DAQ with beam on] (Naomi)
 +
* [https://logbooks.jlab.org/entry/4135073 Tagger cycled] (Alexandre)
 +
* Statistics balance
 +
**PARA 0/90: 21.4%
 +
**PERP 0/90: 20.7%
 +
**PARA 45/135: 20.7%
 +
**PERP 45/135: 20.4%
 +
**AMO : 16.8%
 +
 +
=== Issues ===
 +
* BCAL (Mark)
 +
** [https://logbooks.jlab.org/entry/4133137 ISEG channel measured voltage has ripple]
 +
** [https://logbooks.jlab.org/entry/4133949 ISEG replacement module has bad channel] --> talk to Chris.
 +
** [https://logbooks.jlab.org/entry/4133950 BCAL has been at wrong temperature since Thursday]
 +
Pending:
 +
* [https://logbooks.jlab.org/entry/4117764 Start Counter (Beni)] - We do not understand why there is a mismatch in the software between start counter and tracking.
 +
* BCM calibration. Alexandre recommends that we request a recalibration of AD00C BCM. (~2h) --> Got the green light from PD and RCs. Will be done this morning.
 +
 +
=== Plans ===
 +
* BCM calibration from 10:00am (takes ~2h). Cannot access the hall during this task.
 +
** Good time for BCAL cosmic runs?
 +
** Do Harp scan after this downtime.
 +
* Install CCAL whenever there is an extended accelerator downtime
 +
** Restricted Access is needed to be able to use the ramp <-done
 +
** Cabling
 +
** Survey is needed
 +
* Production (300 nA, 2h runs) with cycle:
 +
**2 runs PARA (0/90)
 +
**2 runs PERP (0/90)
 +
**2 runs PARA (45/135)
 +
**2 runs PERP (45/135)
 +
**1 run AMO (use it to rejuvenate AMO reference in CBREM EPICS GUI)
 +
 +
= Friday February 17 =
 +
 +
=== Past 24 hours ===
 +
* Mark D. replaced the bad BCAL HV board with a new one at 9:40am.
 +
* No beam from 13:00 to 17:30 (0L40 cryo issue). Several people worked in the hall during this downtime.
 +
* Production runs resumed after beam came back.
 +
 +
=== Shift Reports ===
 +
* [https://logbooks.jlab.org/entry/4132987 0400-1000]
 +
* [https://logbooks.jlab.org/entry/4132887 2200-0400]
 +
* [https://logbooks.jlab.org/entry/4132703 1600-2200]
 +
* [https://logbooks.jlab.org/entry/4132557 1000-1600]
 +
* [https://logbooks.jlab.org/entry/4132353 0400-1000]
 +
 +
=== Reports ===
 +
* [https://logbooks.jlab.org/entry/4102104 Please don't prestart very early (for CDC calibration)] (Naomi)
 +
* [https://logbooks.jlab.org/entry/4132522 Broken TAGM scaler issue resolved] (Hovanes)
 +
* [https://logbooks.jlab.org/entry/4132530 BCAL HV board changed] [https://logbooks.jlab.org/entry/4132613] [https://logbooks.jlab.org/entry/4132636] (Mark)
 +
* [https://logbooks.jlab.org/entry/4132557 Several works done in the hall during day-shift downtime]
 +
** IOC for chiller issue --> Fixed by Josh, Scot, and Bobb.
 +
** BCAL chiller alarm --> Fixed by Hovanes.
 +
** DIRC FPGA communication errors, DAQ issues (not communicating with most of ROCs) --> Fixed by Sergey.
 +
** Sasha and Vlad entered the hall for the calorimeter work.
 +
* [https://logbooks.jlab.org/entry/4132736 Higher occupancy in FDC] -> [https://logbooks.jlab.org/entry/4133053 Efficiency looks OK.] (Lubomir)
 +
* Statistics balance
 +
**PARA 0/90: 20.8%
 +
**PERP 0/90: 20.0%
 +
**PARA 45/135: 21.1%
 +
**PERP 45/135: 20.7%
 +
**AMO : 17.3%
 +
 +
=== Issues ===
 +
Pending:
 +
* [https://logbooks.jlab.org/entry/4117764 Start Counter (Beni)] - We do not understand why there is a mismatch in the software between start counter and tracking.
 +
* BCM calibration. Alexandre recommends that we request a recalibration of AD00C BCM. (~30 mins?)
 +
 +
=== Plans ===
 +
* On Monday, Sasha and Chris are planning to work in the hall (?)
 +
* Install CCAL whenever there is an extended accelerator downtime
 +
** Restricted Access is needed to be able to use the ramp <-done
 +
** Cabling
 +
** Survey is needed
 +
* Production (300 nA, 2h runs) with cycle:
 +
**2 runs PARA (0/90)
 +
**2 runs PERP (0/90)
 +
**2 runs PARA (45/135)
 +
**2 runs PERP (45/135)
 +
**1 run AMO (use it to rejuvenate AMO reference in CBREM EPICS GUI)
 +
 
= Thursday February 16 =
 
= Thursday February 16 =
  
 
=== Past 24 hours ===
 
=== Past 24 hours ===
−
* [https://logbooks.jlab.org/entry/4131388 HARP scan results]
+
* Production runs with a stable beam.
−
* No beam between 9am and 12am (beam came back at midnight)
+
* 2 BCAL HVs have been off since midnight due to a bad voltage board.
−
* CCAL transported to experimental hall and fixed to its platform by Scot and his team
+
  
 
=== Shift Reports ===
 
=== Shift Reports ===
 +
* [https://logbooks.jlab.org/entry/4132353 0400-1000]
 +
* [https://logbooks.jlab.org/entry/4132224 2200-0400]
 +
* [https://logbooks.jlab.org/entry/4131994 1600-2200]
 +
* [https://logbooks.jlab.org/entry/4131761 1000-1600]
 
* [https://logbooks.jlab.org/entry/4131531 0400-1000]
 
* [https://logbooks.jlab.org/entry/4131531 0400-1000]
−
* [https://logbooks.jlab.org/entry/4131283 2200-0400]
 
−
* [https://logbooks.jlab.org/entry/4131075 1600-2200]
 
−
* [https://logbooks.jlab.org/entry/4130950 1000-1600]
 
−
* [https://logbooks.jlab.org/entry/4130746 0400-1000]
 
  
 
=== Reports ===
 
=== Reports ===
−
* [https://logbooks.jlab.org/entry/4131000 Hall D survey] (Moser/RadCon)
+
* [https://logbooks.jlab.org/entry/4131747 Rebooting NMR meter and its IOC] (Hovanes)
−
* [https://logbooks.jlab.org/entry/4130972 CDC HVB current vs HV for 250nA HV scan runs on 4.5E-4RL Al on 17 Jan 2023] (Naomi)
+
* [https://logbooks.jlab.org/entry/4131780 target LCW temperature alarm] (Alexandre)
−
* gluon100 rebooted PS/PSC scalers are visible (Sergey)
+
* [https://logbooks.jlab.org/entry/4132086 Noise on some TPOL channels (Data should be good enough).] (Michael Dugger, Beni)
−
** Sasha also power cycled PS/PSC crates
+
 
* Statistics balance  
 
* Statistics balance  
−
**PARA 0/90: 20.7%
+
**PARA 0/90: 20.9%
−
**PERP 0/90: 20.2%
+
**PERP 0/90: 19.9%
−
**PARA 45/135: 21.1%
+
**PARA 45/135: 20.8%
−
**PERP 45/135: 19.7%
+
**PERP 45/135: 20.4%
−
**AMO : 18.3%
+
**AMO : 18.1%
  
 
=== Issues ===
 
=== Issues ===
−
* [https://logbooks.jlab.org/entry/4131558 TAGM discriminator TRG scaler gone wild]
+
* [https://logbooks.jlab.org/entry/4132216 A bad voltage board in the BCAL. Needs to be changed.] (Mark)
 +
* [https://logbooks.jlab.org/entry/4132129 HallD Run #120726: LED ADC reference SiPM time anomaly]
 
Pending:
 
Pending:
 +
* [https://logbooks.jlab.org/entry/4131558 TAGM discriminator TRG scaler gone wild]
 
* [https://logbooks.jlab.org/entry/4117764 Start Counter (Beni)] - We do not understand why there is a mismatch in the software between start counter and tracking.
 
* [https://logbooks.jlab.org/entry/4117764 Start Counter (Beni)] - We do not understand why there is a mismatch in the software between start counter and tracking.
 
* BCM calibration. Alexandre recommends that we request a recalibration of AD00C BCM.
 
* BCM calibration. Alexandre recommends that we request a recalibration of AD00C BCM.
  
 
=== Plans ===
 
=== Plans ===
 +
* A bad BCAL HV board should be changed. Chris Stanislav is out today, and Mark D. is trying to find someone to change the BCAL board.
 
* Install CCAL whenever there is an extended accelerator downtime
 
* Install CCAL whenever there is an extended accelerator downtime
 
** Restricted Access is needed to be able to use the ramp <-done
 
** Restricted Access is needed to be able to use the ramp <-done

Latest revision as of 09:40, 22 February 2023

Zoom connect Meeting ID: 160 199 0060 / Passcode 8394

MCC whiteboard

Hall D counting room white board

Spreadsheet with list of runs

Integrated Trigger Count

Wednesday February 22

Past 24 hours

  • Production runs. Several cosmic runs were taken during downtimes.

Shift Reports

Reports

Issues

Pending:

  • Start Counter (Beni) - We do not understand why there is a mismatch in the software between start counter and tracking.
  • TAGH hot channels w/ no beam
    • Hovanes rebooted the crate but it didn't help.
    • Sasha explains the problematic channel(s) are deteriorated and this is not an issue which can be easily fixed.
  • Noise in FDC2 (Lubomir)
  • FCAL11 crate problems (Sergey)
    • Network port on the fan tray for FCAL11 crate is not working. When we have an expected long downtime, this tray should be replaced (Chris Stanislav).

Plans

  • Install CCAL whenever there is an extended accelerator downtime
    • Restricted Access is needed to be able to use the ramp <-done
    • Cabling
    • Survey is needed
  • Production (300 nA, 2h runs) with cycle:
    • 2 runs PARA (0/90)
    • 2 runs PERP (0/90)
    • 2 runs PARA (45/135)
    • 2 runs PERP (45/135)
    • 1 run AMO (use it to rejuvenate AMO reference in CBREM EPICS GUI)

Tuesday February 21

Past 24 hours

  • BCM Calibration executed. MCC decided no recalibration was needed.
  • Production runs resumed after beam came back.

Shift Reports

Reports

  • Hall D NMR Meter - Installed (Scot)
    • MCC stopped beam delivery after NMR IOC communication loss (just a monitor device) was detected.
    • (In RC meeting) They say they are not expert in Hall equipment and they will stop the beam delivery when whatever crew chief does not know happens.
  • BCM calibration
    • MCC called shift crews to ask what the maximum calibration current should be.
    • (In RC meeting) They say this is on us (Hall D), so we should prepare this value beforehand in the future BCM calibration work.
  • Harp scan after BCM work
  • Harp scan after the long midnight downtime
  • BCAL ISEG modules swapped during a long downtime (Mark)
  • Took BCAL cosmic run (R120783 and more) during the calibration work (and during midnight downtime).
  • Early-riser shift leader felt unwell and Keigo (RC) took over. We need a volunteer for tomorrow morning shift.
  • Statistics balance
    • PARA 0/90: 21.0%
    • PERP 0/90: 20.5%
    • PARA 45/135: 21.4%
    • PERP 45/135: 20.6%
    • AMO : 16.5%

Issues

Pending:

Plans

  • Install CCAL whenever there is an extended accelerator downtime
    • Restricted Access is needed to be able to use the ramp <-done
    • Cabling
    • Survey is needed
  • Production (300 nA, 2h runs) with cycle:
    • 2 runs PARA (0/90)
    • 2 runs PERP (0/90)
    • 2 runs PARA (45/135)
    • 2 runs PERP (45/135)
    • 1 run AMO (use it to rejuvenate AMO reference in CBREM EPICS GUI)

Monday February 20

Past 72 hours

  • Production runs. ABU = 35.5 h for the weekend. (12.7 h Friday, 13.8 h Saturday, 9.0 h Sunday)

Shift Reports Friday

Shift Reports Saturday

Shift Reports Sunday

Reports

Issues

Pending:

  • Start Counter (Beni) - We do not understand why there is a mismatch in the software between start counter and tracking.
  • BCM calibration. Alexandre recommends that we request a recalibration of AD00C BCM. (~2h) --> Got the green light from PD and RCs. Will be done this morning.

Plans

  • BCM calibration from 10:00am (takes ~2h). Cannot access the hall during this task.
    • Good time for BCAL cosmic runs?
    • Do Harp scan after this downtime.
  • Install CCAL whenever there is an extended accelerator downtime
    • Restricted Access is needed to be able to use the ramp <-done
    • Cabling
    • Survey is needed
  • Production (300 nA, 2h runs) with cycle:
    • 2 runs PARA (0/90)
    • 2 runs PERP (0/90)
    • 2 runs PARA (45/135)
    • 2 runs PERP (45/135)
    • 1 run AMO (use it to rejuvenate AMO reference in CBREM EPICS GUI)

Friday February 17

Past 24 hours

  • Mark D. replaced the bad BCAL HV board with a new one at 9:40am.
  • No beam from 13:00 to 17:30 (0L40 cryo issue). Several people worked in the hall during this downtime.
  • Production runs resumed after beam came back.

Shift Reports

Reports

Issues

Pending:

  • Start Counter (Beni) - We do not understand why there is a mismatch in the software between start counter and tracking.
  • BCM calibration. Alexandre recommends that we request a recalibration of AD00C BCM. (~30 mins?)

Plans

  • On Monday, Sasha and Chris are planning to work in the hall (?)
  • Install CCAL whenever there is an extended accelerator downtime
    • Restricted Access is needed to be able to use the ramp <-done
    • Cabling
    • Survey is needed
  • Production (300 nA, 2h runs) with cycle:
    • 2 runs PARA (0/90)
    • 2 runs PERP (0/90)
    • 2 runs PARA (45/135)
    • 2 runs PERP (45/135)
    • 1 run AMO (use it to rejuvenate AMO reference in CBREM EPICS GUI)

Thursday February 16

Past 24 hours

  • Production runs with a stable beam.
  • 2 BCAL HVs have been off since midnight due to a bad voltage board.

Shift Reports

Reports

Issues

Pending:

Plans

  • A bad BCAL HV board should be changed. Chris Stanislav is out today, and Mark D. is trying to find someone to change the BCAL board.
  • Install CCAL whenever there is an extended accelerator downtime
    • Restricted Access is needed to be able to use the ramp <-done
    • Cabling
    • Survey is needed
  • Production (300 nA, 2h runs) with cycle:
    • 2 runs PARA (0/90)
    • 2 runs PERP (0/90)
    • 2 runs PARA (45/135)
    • 2 runs PERP (45/135)
    • 1 run AMO (use it to rejuvenate AMO reference in CBREM EPICS GUI)
Retrieved from "https://halldweb.jlab.org/wiki/index.php?title=Run_Planning_Meeting_Notes,_February_16_-_February_22,_2023&oldid=119316"