Difference between revisions of "Online farm testing 5-dec-2012"
From GlueXWiki
m |
m |
||
Line 1: | Line 1: | ||
'''Current status of online farm tests in the counting house''' | '''Current status of online farm tests in the counting house''' | ||
− | * Main GBit Ethernet switch working (has 10 GBit capability). | + | * Infrastructure |
− | * Big UPS working. | + | ** Main GBit Ethernet switch working (has 10 GBit capability). |
− | * Borrowed nodes gluon02-gluon05 working (64-bit RHEL6, 16 cores each, lots of memory). | + | ** Big UPS working. |
− | * Dave A will install borrowed 40 GBit/sec InfiniBand switch (will eventually purchase 60 GBit). | + | ** Borrowed nodes gluon02-gluon05 working (64-bit RHEL6, 16 cores each, lots of memory). |
− | * gluon02-05 have both GBit and IB interfaces. | + | ** Dave A will install borrowed 40 GBit/sec InfiniBand switch (will eventually purchase 60 GBit). |
− | * DAQ software installed and working (EVIO, cMsg, ET). | + | ** gluon02-05 have both GBit and IB interfaces. |
− | * | + | |
− | * | + | * Software |
− | * | + | ** DAQ software installed and working (EVIO, cMsg, ET). |
− | * | + | ** rawevent plugin converts hddm file to EVIO format using mc2coda package from Dave A, 24 kB/event. |
− | * | + | ** mc2coda_compress compresses raw EVIO file 8 GB/evt. |
− | ** | + | ** montest reads EVIO files and inserts events into ET system |
− | ** | + | |
− | + | ||
− | * | + | * Results for local transfers |
− | * Local consumer can keep up with producer rate. | + | ** Injection rate about 110 kHz per producer for uncompressed events, 350 kHz for compressed events. |
− | * Remote consumer saturates GBit Ethernet at 95% level | + | ** Injection rate scales with the number of producers. |
+ | ** Each producer takes 100% of one core. | ||
+ | ** ET system overhead about 5% of a core. | ||
+ | ** Local consumer can keep up with producer rate. | ||
+ | |||
+ | |||
+ | * Results for network transfers | ||
+ | ** Remote consumer saturates GBit Ethernet at 95% level | ||
** 5 kHz for uncompressed events, 15 kHz for compressed events. | ** 5 kHz for uncompressed events, 15 kHz for compressed events. | ||
− | * | + | ** Consumers use about 25% of a core. |
+ | ** ET system about 20% of a core. | ||
+ | |||
+ | |||
+ | * InfiniBand testing | ||
+ | ** Expect factor 10 increase, not factor 40. | ||
** DAQ tests show you get 25% of 40 GBit using TCP over IB. | ** DAQ tests show you get 25% of 40 GBit using TCP over IB. | ||
** we will purchase 60 GBit IB. | ** we will purchase 60 GBit IB. | ||
− | + | ||
− | * | + | |
− | ** | + | * JANA analysis processes |
− | ** has updated JANA to read uncompressed EVIO files. | + | ** Waiting for development of evioETchannel class (me, soon). |
− | ** is working on reading compressed EVIO data. | + | ** Dave L has updated JANA to read uncompressed EVIO files. |
− | * Vardan and I will develop | + | ** Dave L is working on reading compressed EVIO data. |
+ | |||
+ | |||
+ | * Farm manager | ||
+ | ** Vardan and I will develop this. |
Revision as of 14:55, 4 December 2012
Current status of online farm tests in the counting house
- Infrastructure
- Main GBit Ethernet switch working (has 10 GBit capability).
- Big UPS working.
- Borrowed nodes gluon02-gluon05 working (64-bit RHEL6, 16 cores each, lots of memory).
- Dave A will install borrowed 40 GBit/sec InfiniBand switch (will eventually purchase 60 GBit).
- gluon02-05 have both GBit and IB interfaces.
- Software
- DAQ software installed and working (EVIO, cMsg, ET).
- rawevent plugin converts hddm file to EVIO format using mc2coda package from Dave A, 24 kB/event.
- mc2coda_compress compresses raw EVIO file 8 GB/evt.
- montest reads EVIO files and inserts events into ET system
- Results for local transfers
- Injection rate about 110 kHz per producer for uncompressed events, 350 kHz for compressed events.
- Injection rate scales with the number of producers.
- Each producer takes 100% of one core.
- ET system overhead about 5% of a core.
- Local consumer can keep up with producer rate.
- Results for network transfers
- Remote consumer saturates GBit Ethernet at 95% level
- 5 kHz for uncompressed events, 15 kHz for compressed events.
- Consumers use about 25% of a core.
- ET system about 20% of a core.
- InfiniBand testing
- Expect factor 10 increase, not factor 40.
- DAQ tests show you get 25% of 40 GBit using TCP over IB.
- we will purchase 60 GBit IB.
- JANA analysis processes
- Waiting for development of evioETchannel class (me, soon).
- Dave L has updated JANA to read uncompressed EVIO files.
- Dave L is working on reading compressed EVIO data.
- Farm manager
- Vardan and I will develop this.