Alarm System for EPICS

From GlueXWiki
Revision as of 12:37, 4 November 2013 by Hovanes (Talk | contribs) (Alarm Server)

Jump to: navigation, search

Introduction

One of the most important components of the Hall D online monitoring will be the alarm system. Such a system should be configurable to annunciate in the control room and send e-mail messages in case EPICS variables severity changes. It is also very desirable to have an alarm system which can be integrated with the Controls System Studio (CSS).

ALH

Alarm Handler (ALH) is a standalone program that is part of the EPICS extensions that was developed for monitoring and reacting to the alarm condition of the EPICS records. ALH can sound an audible alarm, displays suggested actions for various situations, and can execute commands for different alarm states and severities. It has been extensively used at Jefferson Lab since mid-nineties, and in principle can be used in the 12 GeV era.

BEAST

EPICS community developed an alarm system called Best Ever Alarm System Toolkit (BEAST) providing integration with CSS. This system consists of multiple components, and each component is built and configured from Eclipse RCP environment. Some of the component should run standalone, some of them are supposed to be built-in into the CSS executable, and some can used in both ways (Annunciator).

Compiling executables

The compilation of the executable is done in Eclipse RCP. I downloaded SNS source from the SNS web site, unzipped. Then I import the plugins from the source directory as existing project. Since we will be using MySQL databases I edited preferences.ini file in org.csstudio.alarm.beast to have "rdb_schema=" .

Alarm Server

  • Since the setting in the plugin_customization.ini overwrite the setting in preferences.ini I changed plugin_customization.ini in /org.csstudio.alarm.beast.server to have
    org.csstudio.alarm.beast/rdb_schema=
    .
  • Also change the default root component to Test
    org.csstudio.alarm.beast/root_component=Test

Then I used standard procedure with AlarmServer.product to create an executable.

  • I created a file css/3.1.0/AlarmServer/alarm_server.ini to add the the IP addresses to the list of the CA servers the Alarm Server will be trying to access:
org.csstudio.platform.libs.epics/addr_list=127.0.0.1 129.57.36.96 
. I could have done this when building the executable for the BEAST Server in the /org.csstudio.alarm.beast.server/plugin_customization.ini file. To run the server one needs to issue a command
./AlarmServer  -pluginCustomization /home/hovanes/css/products/sns_epics_w_alarm_3.1.5/AlarmServer/alarm_server.ini
. The manual suggest to always use the full path to the customization file.

Alarm Configuration Tool

  • In plugin_customization.ini for /org.csstudio.alarm.beast.configtool I changed to
  org.csstudio.alarm.beast/rdb_url=jdbc:mysql://localhost/alarm
  org.csstudio.alarm.beast/rdb_user=alarm
  org.csstudio.alarm.beast/rdb_password=$alarm
the default was some Oracle type setup with different username amd password for SNS.

Alarm Annunciator

  • I compiled this with a two changes in the plugin_customization.ini file:
    <org.csstudio.alarm.beast.annunciator/jms_url=failover:(tcp://129.57.88.55:61616)
    org.csstudio.alarm.beast.annunciator/jms_topic=TALK,Test_TALK


CSS Product with BEAST

  • In order for the Alarm menu to show up as a submenu of the CSS menu one needs to add org.csstudio.alarm.beast.ui.feature in the CSS .product's Dependencies tab and then recompile the product. The locations of the Alarm submenus are defined in org.csstudio.alarm.beast.ui.feature plugin.
  • It turns out that if one wants to be able to modify the alarm configuration of the BEAST through CSS then the CSS executable will need to be able to authenticate and authorise the user. Before doing this I was trying to compile the CSS based on features. But when I looked at the list of the dependencies with the features I did not find how to add a features to include the authentication and authorisation into the build. So I decided to compile the EPICS CSS based plugins. To do this one needs to
    clicks the radio-button plug-ins in the CSS.product file in the Overview tab.
    Add the following plugins in the Dependencies tab in CSS.product :
    Validate the dependencies. When I did it I got a message that
    org.csstudio.opibuilder.rap<pre> plugin had <pre>Missing Constraint
    and that org.csstudio.rap.core was missing. I just removed this plugin from the dependency list and I was able to compile the executable successfully.
  • Go back to the Overview tab, synchronise and create the executable using product wizard.
  • After launching the newly built CSS go to Edit->Preferences->CSSCore->JAAS Authentication and change the Login Configuration Source to "File" and the Configuration File Entry to "Dummy". This will allow to have dummy authentication allowing everyone to login and be authorised to change the alarm configuration.

Creating RDB

BEAST alarm system utilizes RDB for configuration and maintaining the alarm statuses even when the Alarm Server is down. We will be using MySQL database, so we need to create MySQL users and database for the alarm system.

  • Use org.csstudio.alarm.beast/dbd/MYSQL_USER.sql to create usernames and passwords using
 mysql -h halld-sc -u root < MYSQL_USER.sql

command.

  • Manually create a MySQL database alarm on the MySQL server since the SQL-files provided in the CSS source code did not seem to do it.
mysql> CREATE DATABASE alarm ;
  • Use org.csstudio.alarm.beast/dbd/ALARM_MYSQL.sql to create the tables for the alarm system.
mysql -u username2 -p'password' < ALARM_MYSQL.sql

I had to change all instances of "ALARM." to "alarm." in the ALARM_MYSQL.sql prior to issuing the command above to keep the database name consistent. The username2 user with that simple password was defined in the org.csstudio.alarm.beast/dbd/MYSQL_USER.sql file, and obviously, can be named differently in the production configuration of the Hall D alarm system.

Java Message Server

Install Apache ActiveMQ

  • Download ActiveMQ from http://activemq.apache.org, both the tar file and the key file.
  • Copied both files to /usr/local/activemq.
  • Verifying the checksum with PGP did not work on "halld-sc" computer. Ignored it for now.
  • Untared the tar-file in the /usr/local/activemq directory.
  • Go to /usr/local/activemq/apache-activemq-5.5.1 directory and start by issuing
sudo bin/activemq start
  • When checking the port with netstat I get
halld-sc:apache-activemq-5.5.1> netstat -an | fgrep 61616 
tcp        0      0 :::61616                    :::*                        LISTEN      
indicating that ActiveMQ is running.
  • To configure CSS as a proper JMS client, I need to specify the the URL for the JMS server I setup with ActiveMQ
    In /org.csstudio.basic.epics.product/plugin_customization.ini change the URL ti have the following lines:
     org.csstudio.logging/console_level=FINE
     org.csstudio.logging/jms_level=FINE
     org.csstudio.logging/jms_url=failover\:(tcp\://129.57.88.55\:61616)
     org.csstudio.platform/log4j.appender.css_jms.providerURL=failover\:(tcp\://129.57.88.55\:61616)
  • I can check if the client connected to the JMS server by checking http://localhost:8161/admin on the machine that runs the JMS servr (in this case it was halld-sc.jlab.org. Note that this web service was not accessible from some other machines. If asked for credentials, the default is (admin,admin).
  • It appears that the ActiveMQ on halld-sc.jlab.org freezes once in a while. Then Alarm Panel on CSS indicated Server Timeout. Restarting the ActiveMQ server solves the problem, but the alarm GUIs need to be restarted which might be problematic. This will need to be investigated.