Difference between revisions of "Run Planning Meeting Notes, November 30 - December 7, 2022"
From GlueXWiki
(→Reports) |
(→Reports) |
||
| Line 21: | Line 21: | ||
**The first diagnostic from the Hodoscope main expert was that the lost information was not critical for the experiment, so we continued running for about 30min until Igal informed us that the data was not usable in these conditions. | **The first diagnostic from the Hodoscope main expert was that the lost information was not critical for the experiment, so we continued running for about 30min until Igal informed us that the data was not usable in these conditions. | ||
**Counting room and RC called Boris, but without success at first. Then we called Hovanes and Sasha for help. [https://logbooks.jlab.org/entry/4094882 Sasha was able to come in around noon and accessed the tagger area. However, Sasha could not reset the crate and diagnosed that it was dead and needed to be swapped.] However, the knowledge of good crate spares and expertise to replace crates is with the electronics group, which has no on-call contact. RC called many colleagues (Beni, Eugene, Tim, Hovanes, Lubomir...) to see if they had a useful phone number (Chris Stanislav, Fernando or Chris Cuevas) to call, but none had one. RC emailed Chris Stanislav in case he would check his emails and be able to help/instruct. (Chris replied Sunday morning.) | **Counting room and RC called Boris, but without success at first. Then we called Hovanes and Sasha for help. [https://logbooks.jlab.org/entry/4094882 Sasha was able to come in around noon and accessed the tagger area. However, Sasha could not reset the crate and diagnosed that it was dead and needed to be swapped.] However, the knowledge of good crate spares and expertise to replace crates is with the electronics group, which has no on-call contact. RC called many colleagues (Beni, Eugene, Tim, Hovanes, Lubomir...) to see if they had a useful phone number (Chris Stanislav, Fernando or Chris Cuevas) to call, but none had one. RC emailed Chris Stanislav in case he would check his emails and be able to help/instruct. (Chris replied Sunday morning.) | ||
| − | **After | + | **After discussion between Eugene and RC, it was decided that without the problem fixed, data taking would be worthless and we should stop running over the week end (standby shifts) unless we could fix the problem. At 3pm, RC asked for the beam to be stopped and for [https://logbooks.jlab.org/entry/4094908 the Tagger Hall to be surveyed] in case a spare is found. |
**[https://logbooks.jlab.org/entry/4095025 Meanwhile, Beni had come to JLab and looked for a spare in the EEL building, found some spare power supplies and at 15:45 went in the tagger hall to replace the broken crate.] Boris could be reached and went in to help Beni do the replacement. | **[https://logbooks.jlab.org/entry/4095025 Meanwhile, Beni had come to JLab and looked for a spare in the EEL building, found some spare power supplies and at 15:45 went in the tagger hall to replace the broken crate.] Boris could be reached and went in to help Beni do the replacement. | ||
**After Beni saved the day (the week-end), we could restart data taking around 17:00. | **After Beni saved the day (the week-end), we could restart data taking around 17:00. | ||
Revision as of 08:26, 5 December 2022
Meetings are via ZoomGov: connect (Meeting ID: 160 199 0060 / Passcode 8394)
Hall D counting room white board
Back to Run Coordination Meetings:Fall2022
Contents
Monday Dec. 5th
Last 72h
- Friday, Saturday and Sunday were spent mostly with production on full target. 49 of ABU (Sat.7am - Monday 7am).
- Biggest ABU loss occurred Saturday due to a dead hodoscope crate. Fixing it took a long time because we have no off-work-hours electronic support (see report below).
- Hall C re-started running Friday around 8pm. Now Hall A, C and D are running concurrently.
Reports
- Friday:~1h Hall Rapid Access between 11am and 12:10pm to change a dying FCal basis (Malte/Mark Dalton) and check the possibly faulty FDC fADC that had been installed Wednesday. It was eventually not replaced (Lubomir). Also, RadCon surveyed some modules that needed to be taken out of the Hall.
- Friday 18:20: Harp scans (first ones since 11 days). Beam profiles looks good. (Hovanes)
- Saturday: At 10:14am, a crate for the hodoscope HV died, compromising 120 chanels on the photon high energy side.
- The first diagnostic from the Hodoscope main expert was that the lost information was not critical for the experiment, so we continued running for about 30min until Igal informed us that the data was not usable in these conditions.
- Counting room and RC called Boris, but without success at first. Then we called Hovanes and Sasha for help. Sasha was able to come in around noon and accessed the tagger area. However, Sasha could not reset the crate and diagnosed that it was dead and needed to be swapped. However, the knowledge of good crate spares and expertise to replace crates is with the electronics group, which has no on-call contact. RC called many colleagues (Beni, Eugene, Tim, Hovanes, Lubomir...) to see if they had a useful phone number (Chris Stanislav, Fernando or Chris Cuevas) to call, but none had one. RC emailed Chris Stanislav in case he would check his emails and be able to help/instruct. (Chris replied Sunday morning.)
- After discussion between Eugene and RC, it was decided that without the problem fixed, data taking would be worthless and we should stop running over the week end (standby shifts) unless we could fix the problem. At 3pm, RC asked for the beam to be stopped and for the Tagger Hall to be surveyed in case a spare is found.
- Meanwhile, Beni had come to JLab and looked for a spare in the EEL building, found some spare power supplies and at 15:45 went in the tagger hall to replace the broken crate. Boris could be reached and went in to help Beni do the replacement.
- After Beni saved the day (the week-end), we could restart data taking around 17:00.
- [https://logbooks.jlab.org/entry/4095474 The power for Tagger halo monitors are on the crate that died. The halo monitors thus went off around 10:15am and were restarted only Sunday at 8:30am (this issue was noted by the early riser shift and acted upon after RC was informed around 8:30).
- Sunday morning: RF timing shift after DAQ restart.
Issues
- Friday:timing shift, also Sunday. Sean and Alex Somov looked at it.
- Friday/Saturday: Beam locks sometime do not work and the beam transmission drops. (Plot for Saturday,~6am. It also happened Friday around 11:50pm) MCC explains that when Hall C is off or ramping up. (I also saw this instance Friday at 1pm when Hall A was ramping up, although at the time, I wondered if it was just a coincidence). => Watch carefully the beam position.
- Sunday: gluon03 logged out for some reason and usual hdops pwd did not allowed to log back.
- Sunday: Remote Connection from newer Macs OS
- Sunday: Twice, (around midnight and 7:50am) the trigger rate froze/stopped being sent to EPCIS database. The second time, it was frozen to a typical value so it was hard to spot without a monitoring chart.
- Monday: Beam current instable. MCC was notified by shift leader.
Short term run plan
- Empty target runs?
- If beam is off, we have opportunistic work:
- Beam off for more than 1h: rapid access for A. Somov, Vlad and Harshak for oportunistic work in the PS test area.
Tentative longer term run plan
- Possible RF recovery Wednesday Dec. 7th.
- TAC run?
- Luminosity study?
Friday Dec. 2nd
Last 24h
- Thursday was spent mostly with production on full target. It was an efficient day, with 17.8h of ABU (7am-7am).
- A. Somov did a low beam current test of the Active Colim. high gain in anticipation of a future TAC run. It looks good.
- Hall C was supposed to get back to running in mid-swing shift, which would be followed by us doing Harp scans (last ones done: Oct. 21st), but it did not happen (Hall C vacuum issue).
Reports
Issues
- The FDC fadc installed Wednesday by Naomi and Lubomir needed baseline calibration. Done during swing shift (Sergei/Lubomir).
- cdc p_diff alarms are becoming rarer: 1 (day shift) + 1 (swing shift) + 0 reported in owl shift + 1 early riser.
Short term run plan
- Hall C is expected to be back to running ??at some point??.
- Harp scan after Hall C beam is tuned (last harps done on Nov. 21st, but Transmission is fine.)
- FDC fadc: 20 min raw mode run with beam. Beni will analyse it and determine if the fadc needs to be replaced
- If beam is off, we have opportunistic work:
- Beam off for more than 1h: rapid access for A. Somov and Vlad for ??
- Beam off for more than 2h: rapid access for Lubomir or Beni IF FDC fadc needs to be changed
- Beam off for more than 2h and RadCon is available, need to survey fadc to removed them from the Hall (Lubomir/Codi)
Tentative longer term run plan
- Possible RF recovery Wednesday Dec. 7th.
- TAC run
- Luminosity study
Thursday Dec. 1st
Last 24h
- Wednesday started with production on empty target. It was an efficient day, with 17.1h of ABU (7am-7am).
- 9:10am-11:30 Rapid access to change fadc for FCal and FDC, and fill the target.
- Filled the target, from 10am-1pm. It took 3h to reach nominal density, and the first production runs are done while it was still filling. Target status indicated that it was full&ready by 11:30am.
- Hall A re-started running at 17:48. A consequence is that the Hall D beam energy was quite instable from 6pm (Wed.) to 1am (Thursd.), with total drift of about 14 MeV.
Reports
- PrimEx Analysis progress (Drew)
Issues
- The new FDC fadc is worst than the one replaced (Naomi/Lubomir). The problem is only for raw mode run.
- Many cdc p_diff alarms (e.g. 0.8 alarm/hour during swing and owl shifts) leading to runs often shorter than 2h (this has already been discussed several time. This is just a report).
Run plan
- Hall C is expected to be back to running during swing shift today.
- 20 min test at low beam current/high Active Colim. gain to see if adjustment are needed (A. Somov)?
- Last harp scan was done on Nov. 21st. Should we do a new one? (Transmission is fine.)
- Luminosity study?
