Online farm testing 5-dec-2012

From GlueXWiki
Jump to: navigation, search

Current status of online farm tests in the counting house

  • Infrastructure
    • Main GBit Ethernet switch working (has 10 GBit capability).
    • Big UPS working.
    • Borrowed nodes gluon02-gluon05 working (64-bit RHEL6, 16 cores each, lots of memory).
    • Dave A will install borrowed 40 GBit/sec InfiniBand switch (will eventually purchase 60 GBit).
    • gluon02-05 have both GBit and IB interfaces.


  • Software
    • DAQ software installed and working (EVIO, cMsg, ET).
    • rawevent plugin converts hddm file to EVIO format using mc2coda package from Dave A.
    • Event size is 24 kB/event out of mc2coda.
    • mc2coda_compress compresses raw EVIO file, 8 GB/evt.
    • montest reads EVIO files and inserts events into ET system (fake EB).
    • Online monitoring plugins ready, make occupancy plots for all supported detectors.
    • Waiting for JANA improvements.


  • Results for local transfers
    • montest injection rate about 110 kHz per instance for uncompressed events, 350 kHz for compressed events.
    • Injection rate scales with the number of instances.
    • Each producer instance takes 100% of one core.
    • ET system overhead about 5% of a core.
    • Local consumers can keep up with producer rate.


  • Results for one-way network transfers
    • Remote consumer saturates GBit Ethernet at 95% level
    • 5 kHz for uncompressed events, 15 kHz for compressed events.
    • Consumers use about 25% of a core.
    • ET system about 20% of a core.
    • Results mostly independent of the number of consumer nodes.


  • InfiniBand testing
    • Expect factor 10 increase, not factor 40.
    • DAQ tests show you get 25% of 40 GBit using TCP over IB.
    • Can trunk IB ports.
    • We expect to purchase 60 GBit IB.


  • JANA analysis processes
    • Waiting for development of evioETchannel class (me, soon).
    • Dave L has updated JANA to read uncompressed EVIO files.
    • Dave L is working on reading compressed EVIO data.
    • Full test requires implementing translation table in JANA in a standard way.


  • Farm manager
    • Vardan and I will develop this.
    • CODA component manages farm processes, which are NOT CODA components.
    • CODA components must be running and partake in run control state machine.


  • Additional testing
    • RootSpy when JANA online processes working.
    • Test different ET architectures, more nodes, etc.
    • Two-way network transfers.
    • Simulated event recorder?
Retrieved from "https://halldweb.jlab.org/wiki/index.php?title=Online_farm_testing_5-dec-2012&oldid=42742"