VME/VXS Crate control Applications
We will be using Wiener UEP6000/PL500 VME crates and UEP6023/PL500 VXS crates. We need to control and monitor the power supply/fan base of these crates. The variables include:
- Power supply status
- Fan tray status
- Fan speed
- Temperature sensors
- Various voltage and current sensors on the crate
The power supply for the crates does not seem to be identical for different crates. One old VME crate in F117 does not stay on the network for long, it drops out after a few SNMP requests. The VXS and VME crates have different fan speed limits and different temperature sensor configurations.
These crate have three interface which we could use to communicate remotely:
- Serial RS-232
- Ethernet SNMP
Our choice would be SNMP since it will not require us to create any new piece of infrastructure to control crates.
Some Related Documents
- A presentation downloaded from the web which can be used as a service manual for Wiener VME crates.
- A presentation downloaded from the web about Ethernet control of the Wiener VME crates.
Mike Johnson from the JLab accelerator division had EPICS interface for Wiener VME crates using SNMP protocol. They run softIOC on Linux platforms. Apparently they are using RHEL6 currently utilizing the following SNMP packages:
According to Mike they had to upgrade the Linux SNMP support, so it remains to be seen if we can use their software on RHEL5 which we have on our machines now.
RHEL5 distribution does have these packages. To compile on RHEL5 we also needed to install package
For Ubuntu there are the following packages for SNMP:
I first tried the EPICS driver on JLab standard Linux RHEL5 distribution with some SNMP packages from the RHEL5 repository:
net-snmp-126.96.36.199-9.el5_5.1 net-snmp-devel-188.8.131.52-9.el5_5.1 net-snmp-libs-184.108.40.206-9.el5_5.1 net-snmp-utils-220.127.116.11-9.el5_5.1
The first build
Mike Johnson sent us a tar file with the whole structure which contained both the IOC part and the support part. To match to our scheme we had to rearrange the files and directories as well as Makefile.
- Create two separate directories, one for support one for app: vmeWiener and vmeCrates.
- Create a separate IOC command file to run vmeCrates executable.
- Use automatic flag selection for net-snmp instead of setting it by hand in the Makefile
- I had to modify the drvSnmp.c file to add the current directory ./ to the list where the SNMP driver looks for the MIB files.
I booted IOC with the read database file wienerR.db, and the IOC seems to have connected to the halldvxs1 VME crate in the test stand and returned valid values for some temperatures and power status. There are some problems though that I noticed:
- Fan temperature (fanAirTemp) is wrong.
- Fan nominal speed (fanNominalSpeedR) is wrong.
- Number of fans(fanNumberOfFans) is wrong .
These turned out to be due to a mistake in the DB-file. In SNMP requests one needs to specify the index for the scaler, e.g. WIENER-CRATE-MIB::fanNominalSpeed.0 and not WIENER-CRATE-MIB::fanNominalSpeed. When we added scaler indexes, these problems disappeared.
To be able to use set-commands for the VME crate, I had to use guru community for SNMP requests. For the IOC, this can be set in the IOC command file where the corresponding databases are being loaded.
At the end, all functions of the EPICS driver that I was interested at this point seem to be working propoerly.
- Added current directory to the list of the directories where the driver is lookinf for the MIB-file. In general, it might be a good idea to keep the WIENER-CRATE-MIB.txt file in /usr/share/snmp/mibs/ directory where the Linux command-line SNMP tools look for MIB-files.
- Modified the DB file to have indexes for all scalar variables in the SNMP requests.
- Our crates have four outputs, so I added two more (4 and 6) to the existing two in the driver.
- Removed all the set commands except Power On/Off, Reset and Fan Sped set-point.
- Created a set of BOY OPI-files for monitoring and control of the VME crates.
- There seem to be different number of temperature sensors on the power supplies for the crates. The new VME crate has 4 fans which show up on the web screen of the crate, and the SNMP reports decent numbers. The VXS crates do not show any temperature sensor (other than fan air temperature) and the corresponding response from SNMP request is -128.
- This is due to the fact that the crates come without the temperature sensors installed. We (probably the electronics group) will install the sensors on the backplane. The right-most sensor when looking from the front of the crate ends up as Sensor 1.
- The fan speed that can be set from SNMP is 3620 RPM even for fans that supposed to be running at 6000 RPM (on VXS crates). The fans can be set to run at 6000 RPM from the switches on the front panel or through the web server. Will bring this up on a meeting with the Wiener representatives.
- Turns out that this is due to the old version of the MIB-file WIENER-CRATE-MIB.txt. I got a new file from Andreas Ruben, and now we can increase the fan speeds until the VXS refuses to execute the command above 6000 RPMs (3600 for VME crates with slower fans). It works on IOC as well.
- It was not clear what to do with the fan speed. The 6000 RPM is very noise and probably will not be useful if the crate is not well populated. On the other hand, not all crates have temperature sensors to set-up an external control loop. Plus, the maximum fan speed settable from SNMP is only 3620 RPM. Therefore, at this point we just set the fan speed by hand to maximum 6000 RPM when we have a populated crate.
- We just need to install the sensors on the backplane by ourselves.
- The crates have internal binary loop such that when the temperature goes above some threshold (warning threshold) the fans will go to maximum speed independent what we set as the nominal speed. So in principal we can set up an external PID-loop to control the temperatures.
Further Software Developments
- When I got the SNMP-based driver for MPOD crates I decided to unify the SNMP driver and the application for both types of Wiener crates. So, I renamed the support package vmeWiener snmpCEBAF, and the application vmeCrates to wienerCrates. This resulted in name changes in library file libdevSnmpCEBAF.a and DBD-file devSnmpCEBAF.dbd. The reason for CEBAF suffix is because I got the SNMP support from Mike Johnson from accelerator. There is another SNMP driver in the EPICS community which I got from NCSL, but it was making the INP fields of the record too long requiring base modifications.
- We decided to only use the SNMP EPICS driver from NCSL. For that reason Vanik Kakoyan changed the EPICS database for the vmeCrates application to use the NCSL SNMP device support syntax and to link against the NCSL devSnmp library.
- Nerses moved the NCSL driver into $EPICS/drivers directory from where all our SNMP application will get the library.
- The vmeWiener application will need to be remover from the SVN since it is the device support from Mike Johnson which we will not be using in Hall D applications. The snmpCEBAF package has never been checked into SVN. If for some reason we need to revive the SNMP EPICS support that I got from Mike Johnson we would have to resurrect the vmeWiener from SVN.