Difference between revisions of "Slow Controls Shift"

Revision as of 17:59, 9 October 2014

Operator Interface Overview

The operator interface for Hall D control systems is based on Control System Studio or CSS. It allows the operator to use different important tools for EPICS from a single application. CSS is based on Eclipse RCP, and many features of CSS to the operator in the control room who already used Eclipse IDE will look and feel like Eclipse. For instance, there are Perspectives in CSS for running EPICS controls GUI, for defining and running strip charts for EPICS variables, for running an alarm handler. Most of the time the proper perspectives will already be open when a shift operator arrives at the shift or when an operator starts the EPICS controls or alarm screens.

Multiple control screens can be combined into a single CSS window as different tabs. Clicking on the tab in a view will activate the corresponding control screen. The tabs can be moved within a window or to a different window by clicking and dragging it into the desired location. One can also open a new empty window by selecting WindowNew Window menu item from the menu-bar of any of the CSS windows and drag the tab of one of the control screens into that new window.

To exit the current CSS session one needs to select FileExit from the menu-bar of any of the CSS windows. That will close all windows that belong to the same CSS session. When starting a new CSS instance all the window that were open just before exiting the CSS session will open. If one chooses to exit CSS by closing all CSS windows one-by-one using the window manager then only the window that was closed last will open when restarting CSS.

EPICS Control Screens

Figure1. CSS BOY controls GUI for Hall D show the action bar on the left side and the Hall D tagger magnet power supply control screen on the right side.

The EPICS display management system is based on Controls System Studio (CSS), namely on BOY package. To start the main control screen one needs to be logged into the gluon01 console machines in the Hall D control room and at the Linux prompt type:

|HDOPS> gluex_css 

This will bring up the controls screen configuration as it was just before the last control screen was closed from the previous CSS session.

The Hall D controls screens are organized in a hierarchy of subsystems and components. The examples of subsystems are FDC,CDC, FDC etc. And each subsytem can have components such us Voltages, Gas etc. If the GUI you are looking for is not displayed you can open it using the main action toolbar by clicking on the Top GUI->MainActionBar button on the toolbar of any of the CSS window, see Figure 1. This should open a list of action buttons on the left side of the CSS GUI from which one can select which of the control screens one wants to launch by scrolling up and down the list of buttons. Clicking on any of the button on the main action bar will pop up a new window with the corresponding control screen. You can open other control screens using the widgets on the newly open window for the selected component.

Accelerator Screens

Sometimes it is useful to view EPICS screens of accelerator division. Accelerator division uses a different display management framework called EDM, and they use a system called JTABS to find the control screens. In order to open JTABS on gluon machines, simply type

|HDOPS> jtabs  

in a gluon desktop computer which will open a set of tabs of top of the current desktop. One can navigate to the desired screen by clicking these tabs.

Data Browser / Striptooling

CSS has an integrated feature called Data Browser that allows users to display stripcharts within the same program where the controls screens are. As with the old EPICS StripTool, the CSS Data Browser allows one to create a file describing which EPICS variables need to be plotted in a give stripchart. Then one can open the desired file by a clicking a previously configured Data Browser file.

In addition, CSS Data Browser allows the user to create stripchart on the fly for the EPICS variable that one currently is monitoring on the CSS BOY screen. To start a stripchart, one needs to bring the mouse pointer to the desired EPICS variable and using righ-click select Process Variable Data Browser, This will open a stripchart in a separate tab in BOY and the time evolution of the EPICS variable will start being plotted. At this point there should be a visible toolbar on top of the strip chart that will allow the user to configure the properties of the stripchart, such as the axis limits, autoscrolling enable/disable, colors etc.

MYA Archiver

Figure 2. MyaViewer screen with open channel selection menu.

The archiving of Hall D EPICS variables is done using MYA archiver developed and maintained by the controls group of the JLAB accelerator divisions. At present time CSS and MYA are not integrated and the archived data from MYA cannot be displayed on CSS screens. In order to view the history of an EPICS variables one needs to:

1. Determine the name(s) of the EPICS variables that one wants to view,
2. Open the graphical viewer for MYA archiver called MyaViewer by typing at Linux prompt

   |HDOPS> MyaViewer

3. Select the EPICS variables to be plotted by clicking the Add Signal button. The Hall D specific variables are typically kept in HD_* MYA groups, for instance HD_MAGNETS, HD_BCAL_TEMPS etc.
4. Setup the axes and traces on MyaViewer to view the history of desired EPICS variables. For instructions how to use MyaViewer please refer to the MyaViwer User's Guide.

One can also have the history of a given EPICS variable or time slice tables be printed on the screen using command line tools like myget, myData, mySampler. There is also a command line tool called myStats that allows to compute and printout the statistics on EPICS channel history stored in the MYA archiver.

Alarm Handler

Figure 3. CSS screen in Alarm Perspective showing the Area Panel, Alarm Tree and Alarm Table.

The EPICS alarm system is based on Controls System Studio (CSS), namely BEAST. To start the main alarm screen one needs to be login to the gluon02 console machines in the Hall D control room and at the Linux prompt type:

|HDOPS> gluex_css 

This will bring up the screen configuration as it was just before the last time the alarm screens were closed from the previous CSS session. The program for the alarm handling is the same as for the display management, therefore one can use the action buttons on the alarm browser to open the control GUIs appropriate for a alarm condition.

The alarm perspective of CSS consists of three key views: Area Panel, Alarm Tree and Alarm Table. Area Panel shows which subsystem of GlueX detector is in a normal condition and which subsystems contain alarming EPICS variables. Alarm Tree view allows one to browse the alarm hierarchy tree to find the variables that are in alarming state. Each node on the Alarm Tree has a menu that can be accessed by right-clicking on the node on the tree. The menu may have a selection options for acknowledging the alarm, for information and guidance, and for opening related displays. The shift personnel needs to read all of the information items marked by i on the left side to get more information about the event and to get guidance on the possible actions thay may be required to solve the problem at hand.

Shift personnel should take actions suggested by the information buttons and after that acknowledge the alarm. Acknowledging an alarm should move the corresponding line in the Alarm Table from the Unacknowledged Alarms panel to the Acknowledged Alarms panel. The line will disappear from the Acknowledged Alarms panel in the Alarm Table once the alarming condition goes away. If the same EPICS variable alarms again, the corresponding line in the Acknowledged Alarms panel will move back to the Unacknowledged Alarms panel, in which case the shift personnel should repeat the actions suggested by the guidance unless the information screen explicitly suggest a different set of action for repeated alarms.

Input/Output Controllers (IOC-s)

Figure 4. IOC heartbeat screen.

Figure 5. IOC statistics screen.

All EPICS variables that are visible on the control and monitoring screens in Hall D counting house are provided by server programs running on different Linux or VxWorks machines, including VME/VXS controllers. It is important that all EPICS IOC are running properly. If these process die then the shift operator sees the "Disconnected" over a magenta background instead of usual values of the EPICS variables. There is a special EPICS GUI that allows to monitor the status of all Hall D EPICS IOC-s, where one can see the heartbeat LED for each of the IOC-s. In the normal condition the LED for each LED should be blinking and not be static. If any of the EPICS variables on the controls screens shows "Disconnected" message more than a minute or any of the LED on the IOC heartbeat GUI is static for more than a minute the shift personnel should can the slow controls expert.

In addition, if any of the variables served by a dead IOC is present in the alarm configuration an alarm will be audible and visible in the alarm handler. Usually a failure of a single IOC will lead to a large number of channels triggering "Disconnected" alarm. In case any of the disconnected alarms continues more than a minute the shift operator should page the slow controls expert. GlueX shift personnel is not expected to identify which IOC needs to be rebooted unless the guidance information in the alarm handler,a detector expert, or the operators instructions for one of the other GlueX systems explicitly specify that a particular IOC needs to be rebooted.

In order to reboot an IOC one needs to bring up the IOC heartbeat EPICS GUI, shown in Figure 3. Select the desired IOC In the list of the IOC by left-clicking on it. A drop-down menu will appear where one needs to select the item called View Status of IOC ... . A new window will pop up showing statistics for that IOC. Click Soft Reboot button to reboot the IOC. The fields in the new GUI will become disconnected for a short time (turn pink) until the IOC is booted and is back online. This should take about 10 seconds. If the soft-reboot fails you can try to click Hard Reboot button to reboot the IOC.

Cameras in the halls

In order to be able to see the status of the systems in the halls that are not electronically controlled and monitored we installed a set of video cameras in the halls. All the cameras allow to zoom in and out. Some of the cameras allow the user to pan and manually focus. In order to view a specific camera, click on the Camera button which should open a new browser window with view for the specified camera.

Expert personnel

The individuals responsible for the Slow Controls are shown in following table. Problems with normal operation of the Slow Controls System should be referred to those individuals and any changes to their settings must be approved by them. Additional experts may be trained by the system owner and their name and date added to this table.