Revision as of 20:50, 7 March 2012

System installation notes on the first Hall D PowerEdge 710 Dell server

General Notes

• We ordered this unit to learn more about Dell servers and their performers, how to install them in the rack-room, how to configure the software them. It cost us about $4.5K.

• The rails that came with it only worked with big square or bog round holes in the vertical rails of the racks. Sice our racks have small circular threaded holes we had to order adapters (~$45) to be able to install them in the racks. Randy gave us a set of rails for circular holes, but it did not match the holes on the sides of the Dell server. But Randy told us that we should be able to order the rails for threaded holes.

• I connected the PowerEdge to network using its Ethernet interface #1 and requested an IP address from the computer center based on the MAC address of the Ethernet port #1. But when I tried to connect to network the light on the switch would stop blinking indicating that the switch port got disabled. After checking with Brent Morris and Paul Letta we suspected that this must be the IPMI services are interfering with that Ethernet port. So I disabled the IPMI by rebooting it and pressing <CTRL>R just when IPMI stuff appeared. After disabling it the port on the switch did not get blocked. Note that this server does not have a dedicated IPMI interface. We will need to figure out how to enable IPMI services so that Vardan's scheme would work with this.

• I had to install 64-bit RHEL 6 from JLab on it since the 32-bit RHEL 6.2 would only show 16 GB memory instead of 32 GB which was on the system. Besides, everything at the lab was moving to 64-bit. We will need tp compile the 32-bit codes for controllers from the controllers themselves, or use 32-bit compat-libraries to build them on 64-bit machines.

• During installation I had an issues with the partitioning program on the JLab installation disk. The program was probably based on fdisk utility which only could create MBR partition table thus limiting the maximum partition size to 2TB. I had to partition the RAID using gparted utility from my Ubuntu 10.04 LTS flash drive. After partitioning I ran the JLab installation disk with "no-default" partitioning scheme and installed the 64-bit RHEL 6.2 on this server using the pre-existing partition.

Extra Linux Packages Installed

• In the /etc/yum.conf file add "proxy=http://jprox:8082" as the last line.
• Install yum-priorities

  sudo yum install yum-priorities

• Download EPEL RPM and install

  sudo rpm -Uvh epel-release-6-5.noarch.rpm

• Change the EPEL repositories in epel.repo and epel-testing.repo by disabling the mirrors and changing the baseurl to a value that computer center proxy allows us to access:

  baseurl=http://download-i2.fedoraproject.org/pub/epel/....

• We have a suspicion that the repository files are being overwritten by the computer center.
• It turns out that system-config-netboot has been deprecated in RHEL6. Probably will have to configure PXE booting manually.

• sudo yum install tftp-server xinetd syslinux dracut-network dhcp

• Install readline packages needed by EPICS:

   sudo yum install readline-devel.x86_64 compat-readline5.x86_64 readline-static.x86_64 compat-readline5.i686 compat-readline5-devel.x86_64 compat-readline5-devel.i686 compat-readline5-static.x86_64

• Install openmotif

  sudo yum install openmotif openmotif-devel openmotif.i686 openmotif-devel.i686

• sudo yum install libXmu.i686  libXmu-devel.i686

• sudo yum install libXpm libXpm-devel 

• sudo yum install finger.x86_64  re2c

• Install Eclipse

  sudo yum install eclipse-rcp.x86_64 eclipse-cdt.x86_64 eclipse-callgraph.x86_64 eclipse-eclox.noarch eclipse-cdt-sdk.x86_64

Installing backintime

tar -xzf backintime-1.0.8_src.tar.gz
cd backintime-1.0.8
cd common/
./configure
make 
sudo make install
cd ../gnome/
make
sudo make install

Setting up TFTP server

• Edit /etc/xinetd.d/tftp and change "disable = yes" to "disable = no"
• Restart TFTP server

  sudo service xinetd restart

• Copy pxelinux.0 to the tftp root directory :

  sudo cp /usr/share/syslinux/pxelinux.0 /var/lib/tftpboot

• Create a pxelinux.cfg directory inside the tftp root directory:

  sudo mkdir -p /var/lib/tftpboot/pxelinux.cfg/

Setting Up NFS exporting

• Create a directory for NFS exporting and PXE booting:

  sudo mkdir /exported/diskless/i686

• Create a directory for NFS exporting for Online software purposes:

  sudo mkdir /exported/online

• Editing /etc/exports and adding lines

   /exported/online	129.57.36.0/255.255.252.0(rw,sync,no_root_squash) 

  /exported/diskless/	129.57.36.0/255.255.252.0(rw,sync,no_root_squash) 

  to the file.
• Reload NFS

  sudo service nfs reload

• Make sure NFS starts on booting

  /sbin/chkconfig --level 345 nfs on

  .

EPICS Installation notes on the first Hall D PowerEdge 710 Dell server

EPICS Base

• Create the top directory for EPICS for 3-14-12-2

  sudo mkdir /exported/online/controls

  mkdir /exported/online/controls/epics/R3-14-12-2

  cd /exported/online/controls/epics/R3-14-12-2

• Build EPICS base 3-14-12-2 after downloading and unzipping it:
  1. Create a soft link /usr/local/epics/base/ pointing to the directory with the newly created version of EPICS distribution.
  2. Edit base/startup/Site.cshrc file and comment out the line with echo $ADTHOME. Also it is better to change the definition of EPICS_BASE variable to environmental variable setenv EPICS_BASE /usr/local/epics/base.
  3. In order to be able to work with NCSL SNMP EPICS driver I had to modify base/src/dbStatic/dbStaticLib.c to increase the length of the links from 80 to 256. This was done following the recommendation by John Priller who provided me with the driver from NCSL.

  4. source startup/Site.cshrc

  5. Compile EPICS BASE by typing :

     make

EPICS Extensions

• Got the extensionsTop_20070703.tar.gz for the extension top directory and unzipped it in $EPICS directory.
• Get the follwing extensions and unzip them in $EPICS_EXTENSIONS/src directory:

  alh1_2_26.tar.gz  medm3_1_5.tar.gz  msi1-5.tar.gz  StripTool2_5_13_0.tar.gz

• Create soft links on them:

  ln -s alh1_2_26 alh

  ln -s medm3_1_5 medm

   ln -s msi1-5 msi

   ln -s StripTool2_5_13_0 StripTool

EPICS Support

• Get the following packages

  asyn4-14.tar.gz  busy_R1-4.tar.gz  ether_ip-2.23.tar.gz  ipac-2.11.tar.gz  motorR6-7-1.tar.gz  seq-2.1.4.tar.gz

IPAC

•  tar -xzf ipac-2.11.tar.gz

• In th ipac-2.11/configure/RELEASE file add and leave only the follwing lines:

  EPICS_BASE=$(EPICS)/base

  EPICS_EXTENSIONS=$(EPICS)/extensions

  EPICS_SUPPORT=$(EPICS)/support

• Type make

SEQ

• tar -xzf seq-2.1.4.tar.gz

• In th seq-2.1.4/configure/RELEASE file add and leave only the follwing lines:

  EPICS_BASE=$(EPICS)/base

  EPICS_EXTENSIONS=$(EPICS)/extensions

  EPICS_SUPPORT=$(EPICS)/support

• Type make

EtherIP

• tar -xzf ether_ip-2.23.tar.gz

• In th ether_ip-2.23/configure/RELEASE file add and leave only the follwing lines:

  EPICS_BASE=$(EPICS)/base

  EPICS_EXTENSIONS=$(EPICS)/extensions

  EPICS_SUPPORT=$(EPICS)/support

   ETHER_IP=$(EPICS_SUPPORT)/ether_ip-2.23

• Type make

Asyn

• tar -xzf asyn4-14.tar.gz</p>

• In th <i>asyn4-14/configure/RELEASE</i> file add and leave only the follwing lines: <pre>LINUX_GPIB

  EPICS_BASE=$(EPICS)/base

  EPICS_EXTENSIONS=$(EPICS)/extensions

  EPICS_SUPPORT=$(EPICS)/support

  IPAC=$(EPICS_SUPPORT)/ipac-2.11

  SNCSEQ=$(EPICS_SUPPORT)/seq-2.1.4

• Type make

Busy

• tar -xzf busy_R1-4.tar.gz

• In th busy-1-4/configure/RELEASE file add and leave only the follwing lines:

  TEMPLATE_TOP=$(EPICS_BASE)/templates/makeBaseApp/top

  EPICS_BASE=$(EPICS)/base

  EPICS_EXTENSIONS=$(EPICS)/extensions

  EPICS_SUPPORT=$(EPICS)/support

  ASYN=$(EPICS_SUPPORT)/asyn4-14

• Type make

Hall D EPICS tree

• Check out Hall D SVN tree for EPICS:

  svn co https://halldsvn.jlab.org/repos/trunk/controls/epics 

• In controls/epics directory create .epicsrc file and edit it to setup the proper EPICS location.

  setenv EPICS /gluon/online/controls/epics/R${EPICS_VERSION}