Online-raw-data-nov-2012
Contents
Energies/Charges and Times
Cuts: E>0, 0<t<32 usec
Problems encountered:
- out-of-range errors, mainly for CDC and BCAL.
- too many hits in one channel (e.g. 26 for one FDC channel!)
q too large for CDC: 1790779 q too large for CDC: 1998616 q too large for CDC: 1318402 E too large for BCAL: 6214239 E too large for BCAL: 11915921 E too large for BCAL: 6686497 E too large for BCAL: 12308178 E too large for BCAL: 5799682 E too large for BCAL: 8164122
TOF
FCAL
BCAL
FDC
CDC
TOF
TAG
Event Sizes and Compression
Compression
The data format for N=1 block mode readout is very verbose and redundant. Much of this is needed to allow the DAQ system to verify all crates/slots are responding properly. But once data integrity is confirmed the redundant data can be removed (perhaps at the ROC and EB levels). Note that for initial detector checkout and perhaps for early engineering runs we will write data in this format.
I propose the following strategy for removing redundant information:
- ADC: replace block header, event header, trigger time and block trailer with single "slot header" word.
- TDC: same as for ADC, additionally remove chip header and chip trailer words.
- Store trigger time in one place only.
- Remove all filler words.
- Remove slots with no hits.
- Remove crates with no hits.
Initial studies show a factor of 4 reduction in data size following this strategy.
To zip or not to zip:
Initial test shows gzip reduces the compressed file size by factor 0.60 (bzip2 by 0.44, but slower). But note that the JLab tape drives compress data upon writing/reading, so gzipping data files should NOT save any tape space. It could save disk staging space and bandwidth, but I believe we'll have plenty of both. Thus it is not clear whether gzipping data files is useful.
Tagger ADC and TDC before:
<bank content="bank" data_type="0x10" tag="58" num="1" nchildren="1"> <bank content="uint32" data_type="0x1" tag="0" num="1" nwords="52"> 0x808c0801 0x90800001 0x98001f40 0x88000004 0x80cc0801 0x90c00001 0x98001f40 0x88000004 0x810c0801 0x91000001 0x98001f40 0xbb001234 0xc3000006 0x88000006 0xf8000000 0xf8000000 0x814c0801 0x91400001 0x98001f40 0xbd801234 0xc5800006 0x88000006 0xf8000000 0xf8000000 0x818c0801 0x91800001 0x98001f40 0xbf801234 0xc7800006 0x88000006 0xf8000000 0xf8000000 0x81cc0801 0x91c00001 0x98001f40 0xba801234 0xc2800006 0x88000006 0xf8000000 0xf8000000 0x820c0801 0x92000001 0x98001f40 0xbd001234 0xc5000006 0x88000006 0xf8000000 0xf8000000 0x824c0801 0x92400001 0x98001f40 0x88000004 </bank> </bank> <bank content="bank" data_type="0x10" tag="59" num="1" nchildren="1"> <bank content="uint32" data_type="0x1" tag="0" num="1" nwords="32"> 0x80940801 0x90800001 0x980003e8 0xa4010000 0xac01001f 0x88000006 0xf8000000 0xf8000000 0x80d40801 0x90c00001 0x980003e8 0xa4010000 0xb4861234 0xb49b1234 0xac01001f 0x88000008 0x81140801 0x91000001 0x980003e8 0xa4010000 0xb48f1234 0xb4951234 0xac01001f 0x88000008 0x81540801 0x91400001 0x980003e8 0xa4010000 0xb48a1234 0xac01001f 0xf8000000 0x88000008 </bank> </bank>
Tagger ADC and TDC after:
<bank content="bank" data_type="0x10" tag="58" num="1" nchildren="1"> <bank content="uint32" data_type="0x1" tag="0" num="1" nwords="15"> 0xe10c0002 0xbb001234 0xc3000006 0xe14c0002 0xbd801234 0xc5800006 0xe18c0002 0xbf801234 0xc7800006 0xe1cc0002 0xba801234 0xc2800006 0xe20c0002 0xbd001234 0xc5000006 </bank> </bank> <bank content="bank" data_type="0x10" tag="59" num="1" nchildren="1"> <bank content="uint32" data_type="0x1" tag="0" num="1" nwords="8"> 0xe0d40002 0xb4861234 0xb49b1234 0xe1140002 0xb48f1234 0xb4951234 0xe1540001 0xb48a1234 </bank> </bank>