Hall Network Specifications
From GlueXWiki
Below we propose a design for fiber connections between the Rack Room and the Hall, and network switch and terminal server placement. The plan below now includes multi-mode fibers for interoperation with the accelerator.
- Elliott Wolin, Mark Dalton
- 15-Oct-2013
Notes
- Based on 10**8 photons/sec, 200 kHz trigger rate, existing L1 rejection algorithm, block=1 mode, no compression.
- Ethernet port counts from Fernando's rack layouts and information from Dave Butler concerning other controls requirements.
- We own about 15 24-port managed switches with 10 GBit/s fiber uplinks, some being used in EEL building and other places.
- Switches can uplink 1.25 GBytes/sec.
- 8-port managed switches with 1 GBit/s fiber uplinks exist.
- Only managed switches (no hubs!) allowed in hall.
- Fibers come in bundles of 12 or 24, 2 fibers needed for one uplink.
- Only single-mode fibers are used in JLab networking systems, multi-mode fibers are used for triggering and fast DAQ applications.
- 24 multi-mode fibers are run from the Rack Room to the Telecom Room, we will extend them downstairs to the Accelerator rack area.
- Standard JLab practice is to terminate all fibers in patch panel when bundles installed.
- Bundle/patch panel installation is expensive, can run much cheaper patch fibers from switch/ROC in one rack area to patch panel in nearby rack area.
- DAQ crates require two Ethernet connections each, one for the ROC and one for the fan tray.
- Some ROC's are only used for control, not DAQ (e.g. BCAL LED Pulser).
- ROC's each need an RS232 connection to a terminal server, terminal servers require an Ethernet connection.
- Each terminal server has 32 ports.
- Can run copper RS232 patch cable from one rack area to terminal server in nearby rack area.
- Will eventually need direct fiber link to ROC if max bandwidth > 80 MB/s, assuming 10**8 photons/sec.
Important Notes
- We have no information about US1!
- A somewhat extreme minimal-cost option exists whereby we eliminate some 12-strand fiber bundles and patch additional switches into nearby rack areas using fiber patch cables. We will explore this option if the plan below costs too much.
Rack | DAQ Connections | Controls Connections | TS Connections | Max Single ROC Bandwidth | Total Uplink Bandwidth | Fiber Bundle | Switches | Terminal Servers | Possible ROC Fiber Uplinks | Comment |
---|---|---|---|---|---|---|---|---|---|---|
T1 | 6 | 12 | 6 | <80 MB/s | 151 MB/s | 12 strand single-mode (Tagger area, already installed) |
1 | 1 | 0 | Also installed 24 multi-mode fibers |
U1A | 0 | 7 | 1 | 0 | low | none | 1 | 0 | 0 | Fiber patch switch to U1B, TS port patch to U1B |
U1B | 5 | 8 | 5 | <80 MB/s | 93 MB/s | 12 strand single-mode | 1 | 1 | 0 | Fiber uplinks from U1A, U1C, UN1 |
U1C | 2 | 2 | 2 | low | low | none | 1 | 0 | 0 | Fiber patch switch to U1B, TS port patch to U1B |
U1D | 0 | 1 | 0 | 0 | 0 | none | 0 | 0 | 0 | Copper Ethernet patch to switch in U1C |
N1 | 8 | 13 | 8 | < 80 MB/s | 450 MB/s | 12 strand single-mode | 1 | 1 | 0 | |
S1 | 8 | 13 | 8 | < 80 MB/s | 450 MB/s | 12 strand single-mode | 1 | 1 | 0 | |
US1 | 0 | 0 | 0 | 0 | 0 | none | 0 | 0 | 0 | Not sure what's going on here |
U2A | 9 | 12 | 9 | 100 MB/s | 770 MB/s | 12 strand single-mode | 1 | 1 | 5 | 2 ROC fiber patches to D2A if needed at 10**8 |
U2B | 9 | 12 | 9 | 136 MB/s | 916 MB/s | 12 strand single-mode | 1 | 1 | 5 | 2 ROC fiber patches to D2B if needed at 10**8 |
D2A | 9 | 19 | 10 | < 80 MB/s | 370 MB/s | 12 strand single-mode | 2 | 1 | 0 | includes 2 gas shed controls connections |
D2B | 6 | 8 | 6 | < 80 Mb/s | 300 MB/s | 12 strand single-mode | 1 | 1 | 0 | |
UN1 | 0 | 20 | 0 | 0 | low | none | 1 | 0 | 0 | Fiber patch switch to U1B |
ACC | 0 | 0 | 0 | 0 | 0 | 24 strand multi-mode | 0 | 0 | 0 | For interoperation with accelerator |
Summary
- Need 7 x 12 = 84 single-mode fibers plus 24 multi-mode fibers (12 single-mode and 12 multi-mode fibers already installed in Tagger building under previous contract).
- Need 12 switches total.
- Need 8 terminal servers.
- 10 ROC's may need fiber uplinks at 10**8 photons/sec, none needed at 5 10**7 photons/sec.
- Inexpensive fiber patch cables used to minimize number of fiber bundles needed.
- Inexpensive RS232 patch cables used to minimize number of terminal servers needed.
- Important: these recommendations do not follow Fernando's note exactly:
- Removed fiber patch panel and TS from U1A
- Removed TS from U1C
- Removed fiber patch panel and switch from US1
- Removed switch from U2A
- Added TS to U2B
- Added TS to D2B