Difference between revisions of "Online farm testing 5-dec-2012"
From GlueXWiki
m |
m |
||
| Line 7: | Line 7: | ||
* gluon02-05 have both GBit and IB interfaces. | * gluon02-05 have both GBit and IB interfaces. | ||
* DAQ software installed and working (EVIO, cMsg, ET). | * DAQ software installed and working (EVIO, cMsg, ET). | ||
| − | * Used rawevent plugin to convert hddm file | + | * Used rawevent plugin to convert Dave L's hddm file to EVIO format using mc2coda package from Dave A. |
| − | * Generated compressed EVIO file using mc2coda_compress (8 GB/evt vs 24 GB/evt). | + | * Generated compressed EVIO file using my mc2coda_compress program (to 8 GB/evt vs 24 GB/evt). |
| − | + | * Created ET system on gluon02. | |
| − | + | * Fake event builder (ET producer) montest injects EVIO events from a file into ET system. | |
| − | + | * Local injection rate about 110 kHz per producer (5 producers gives 550 kHz), each producer takes 100% of one core. | |
| − | * Dave L updated JANA to read uncompressed EVIO files. | + | * Minimal ET system overhead, maybe 5% of a core. |
| − | * | + | * Local consumer can keep up with producer rate. |
| + | * Remote consumer saturates GBit Ethernet at 95% level! | ||
| + | * Further tests when InfiniBand working, expect factor 10 increase, not factor 40. | ||
| + | ** DAQ tests show you get 25% of 40 GBit using TCP over IB. | ||
| + | ** we will purchase 60 GBit IB. | ||
| + | * Remote event rate 5 kHz for uncompressed 25 kB events, expect 20 kHz for 8 kB compressed events. | ||
| + | * Another step is to introduce JANA analysis process. Dave L: | ||
| + | ** is waiting for development of evioETchannel class (me, soon). | ||
| + | ** has updated JANA to read uncompressed EVIO files. | ||
| + | ** is working on reading compressed EVIO data. | ||
Revision as of 14:32, 4 December 2012
Current status of online farm tests in the counting house
- Main GBit Ethernet switch working (has 10 GBit capability).
- Big UPS working.
- Borrowed nodes gluon02-gluon05 working (16 cores each, lots of memory).
- Dave A will install borrowed 40 GBit/sec InfiniBand switch (will eventually purchase 60 GBit).
- gluon02-05 have both GBit and IB interfaces.
- DAQ software installed and working (EVIO, cMsg, ET).
- Used rawevent plugin to convert Dave L's hddm file to EVIO format using mc2coda package from Dave A.
- Generated compressed EVIO file using my mc2coda_compress program (to 8 GB/evt vs 24 GB/evt).
- Created ET system on gluon02.
- Fake event builder (ET producer) montest injects EVIO events from a file into ET system.
- Local injection rate about 110 kHz per producer (5 producers gives 550 kHz), each producer takes 100% of one core.
- Minimal ET system overhead, maybe 5% of a core.
- Local consumer can keep up with producer rate.
- Remote consumer saturates GBit Ethernet at 95% level!
- Further tests when InfiniBand working, expect factor 10 increase, not factor 40.
- DAQ tests show you get 25% of 40 GBit using TCP over IB.
- we will purchase 60 GBit IB.
- Remote event rate 5 kHz for uncompressed 25 kB events, expect 20 kHz for 8 kB compressed events.
- Another step is to introduce JANA analysis process. Dave L:
- is waiting for development of evioETchannel class (me, soon).
- has updated JANA to read uncompressed EVIO files.
- is working on reading compressed EVIO data.
