Difference between revisions of "Archiver for EPICS"

From GlueXWiki
Jump to: navigation, search
(Performance Test)
(Result)
Line 95: Line 95:
 
* Then to view variables in the archive select <b>CSS->Trends->Data Browser</b>. If the Archive search panel is not visible, then on Data Browser view <b>RightClick->Open Archive Search Panel</b>. This should pop a panel on the left hand side. There one can use regular patterns to search for the PVs store in the archive and display them on the <b>Data Browser</b>.
 
* Then to view variables in the archive select <b>CSS->Trends->Data Browser</b>. If the Archive search panel is not visible, then on Data Browser view <b>RightClick->Open Archive Search Panel</b>. This should pop a panel on the left hand side. There one can use regular patterns to search for the PVs store in the archive and display them on the <b>Data Browser</b>.
  
== Result ==  
+
== First Results ==  
  
 
I was able to configure Archiver Engine to archive "bi" and "bo" records. Addind a single "ai" or "ao" record caused  
 
I was able to configure Archiver Engine to archive "bi" and "bo" records. Addind a single "ai" or "ao" record caused  
Line 101: Line 101:
 
Kay Kasemir and [[Response from Kay Kasemir regarding analog record problem in RDB Channel Archiver | got a response]] that there is an issue with handling <i>NaN</i> in the older version of the archiver (which was I was using) when  
 
Kay Kasemir and [[Response from Kay Kasemir regarding analog record problem in RDB Channel Archiver | got a response]] that there is an issue with handling <i>NaN</i> in the older version of the archiver (which was I was using) when  
 
using MySQL RDB. This highlighted a potentially recurring problem in the future coming from the fact that SNS will be using RDB Channel archiver in Oracle RDB, while we probably will prefer MySQL. Compiling a new version of the compiler was giving other error messages.  
 
using MySQL RDB. This highlighted a potentially recurring problem in the future coming from the fact that SNS will be using RDB Channel archiver in Oracle RDB, while we probably will prefer MySQL. Compiling a new version of the compiler was giving other error messages.  
At this point I gave up fighting with it until we discuss a possibility of using the MyA with CSS DataBrowser.  
+
At this point I gave up fighting with it until we discuss a possibility of using the MyA with CSS DataBrowser.
  
 
== Performance Test ==  
 
== Performance Test ==  

Revision as of 18:59, 2 December 2011

RDB Archiver

RDB Archiver is the new new version of the EPICS Channel Archiver which was widely used by the EPICS community. Channel Archiver kept the history in data file and had some kind of indexing scheme. The new RDB Archiver is a part of the Controls System Studio (CSS) and, as the name suggests, it utilizes databases to keep the archive engine configurations and the archived values. RDB Archiver can be used with both MySQL and Oracle (and probably others as well). Folks at ANL tested the RDB Archiver archiver with MySQL and I followed their example, and below you can find what I observed.

  • To create CSS products one needs to have Eclipse RCP which I downloaded from Eclipse download page (version 3.7.0 Indigo) and installed in /usr/local/eclipse/indigo-3.7.0-rcp on my local machines.
  • As CSS sources I used the SNS snapshot 3.0.2 posted on the SNS products web page. I unzipped the file and put it in ~/sns/source directory on my local computers.
  • One also need MySQL server which is included in Linux distributions.


Archive Engine Installation

  • Start Eclipse RCP
    /usr/local/eclipse/indigo-3.7.0-rcp/eclipse
  • Select /home/hovanes/css/source as Workspace since the CSS source was unzipped there.
  • Import the sources into the Workspace by File->Import->General->ExistingProjectsIntoWorkspace->Next, select the /home/hovanes/css/source where the source is located using Browse button, and click Finish.
  • Open org.csstudio.archive.engine/plugin_customization.ini file and remove the schema assignment, it is only needed for Oracle: "org.csstudio.archive.rdb/schema=" . If this is not done then an error similar to what is described in ArchiveConfigTool section will occur because of SQL syntax error, although it will look like an access problem from the error message.
  • Open /org.csstudio.archive.engine/ArchiveEngine.product file.
  • Synchronize and Launch an Eclipse application in the Overview tab. The launch should fail because it requires parameters, but I see the help message, so it runs properly.
  • Export the product using Eclipse Product export wizard in the Overview tab:
    Root directory = ArchiveEngine
    Destination Directory = /home/hovanes/css
uncheck Generate metadata repository, and click Finish button.
  • This created Archive Engine executable /home/hovanes/css/ArchiveEngine/ArchiveEngine
  • If there is a problem with libreadline.so.4 then soft-linking libreadline.so.5 to libreadline.so.4 works, at least it did for me.


Archiver Configurator Tool Installation

  • Here I assume that we are in Eclipse RCP session and the sources are imported when building the Archive Engine.
  • Open /org.csstudio.archive.config.rdb/ArchiveConfigTool.product file.
  • Synchronize and Launch an Eclipse application in the Overview tab. The launch should fail because it requires parameters, but I see the help message, so it runs properly.
  • Export the product using Eclipse Product export wizard in the Overview tab:
    Root directory = ArchiveConfigTool
    Destination Directory = /home/hovanes/css
uncheck Generate metadata repository, and click Finish button.
  • This created Archive Engine executable /home/hovanes/css/ArchiveConfigTool/ArchiveConfigTool.
  • I think the ArchiveConfigTool is defective. Also ANL test document said that they could not use "import" feature and they configured the database manually.


Configuring the MySQL RDB for this Archiver

  • Run
    halld-sc:source> mysql -u root < /home/hovanes/css/source/org.csstudio.archive.rdb/dbd/mysql_schema.txt
    This creates the users and database structure for the RDB archiver.
  • Check if the RDB is fine :
halld-sc:source> mysql -u archive -h localhost -p'$archive' -D archive -e 'select * from smpl_mode;'
+--------------+---------+-----------------------------+
| smpl_mode_id | name    | descr                       |
+--------------+---------+-----------------------------+
|            1 | Monitor | Store every received update |
|            2 | Scan    | Periodic scan               |
+--------------+---------+-----------------------------+
halld-sc:source> 
  • Try to export an XML file for the Demo engine already in there:
halld-sc:source> ../ArchiveConfigTool/ArchiveConfigTool -engine Demo -config engine_Demo.xml -export -host localhost 
Exporting config for engine Demo to engine_Demo.xml
2011-11-02 09:51:51.328 SEVERE [Thread 10] org.csstudio.archive.config.rdb.ArchiveConfigApplication (start) - Exception
com.mysql.jdbc.exceptions.MySQLSyntaxErrorException: SELECT command denied to user 'archive'@'localhost' for table 'smpl_mode'
        at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:936)
        at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:2870)
        at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:1573)
        at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:1665)
        at com.mysql.jdbc.Connection.execSQL(Connection.java:3170)
        at com.mysql.jdbc.Connection.execSQL(Connection.java:3099)
        at com.mysql.jdbc.Statement.executeQuery(Statement.java:1169)
        at org.csstudio.archive.config.rdb.RDBArchiveConfig.loadSampleModes(RDBArchiveConfig.java:109)
        at org.csstudio.archive.config.rdb.RDBArchiveConfig.<init>(RDBArchiveConfig.java:80)
        at org.csstudio.archive.config.rdb.XMLExport.export(XMLExport.java:38)
        at org.csstudio.archive.config.rdb.ArchiveConfigApplication.start(ArchiveConfigApplication.java:120)
        at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:196)
        at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:110)
        at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:79)
        at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:344)
        at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:179)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:616)
        at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:622)
        at org.eclipse.equinox.launcher.Main.basicRun(Main.java:577)
        at org.eclipse.equinox.launcher.Main.run(Main.java:1410)
  • First I thought that the ArchiveConfigTool is broken. The error message about denial of SELECT command to "archive" user for table "smpl_mode" does not make sense because I tested exactly that in the previous command. I tried to play with it, called Dmitri, but I do not think there is a problem with the database. I tried it on three different machines. And I also got the same result by checking out the sources from the sourceforge repository. I will have to configure archiving engines manually for now.
  • It turns out that in the source code there is a variable schema which is probably used for Oracle databases. We are setting up for MySQL databases. When I went into the file /org.csstudio.archive.rdb/preferences.ini and removed the actual assignment to empty string "schema=" then the code starts to work.

Configuring the MySQL RDB for this Archiver

  • To view the data in the saved in RDB Archiver we can use CSS Data Browser. In fact, the SNS Base EPICS CSS build from the SNS site already contains Data Browser plugin. To read PVs from a desired archive one needs to configure the Data Browser. Follow the steps below.
  • Select CSS->Preferences->CSS Applications->Trends->DataBrowser. Then add the MySQL URL, for instance jdbc:mysql://localhost/archive, in the Archive Data Server URLs and the Default Archive Data Sources sections. In the CSS->Preferences->CSS Applications->Trends->RDB Archive one needs to add the username and password which can read the MySQL database. Note, that this user needs to have MySQL access as username@hostname. For isntance, I used archive@localhost. The Database Schema and Stored procedure should stay blank (at least for MySQL).
  • Then to view variables in the archive select CSS->Trends->Data Browser. If the Archive search panel is not visible, then on Data Browser view RightClick->Open Archive Search Panel. This should pop a panel on the left hand side. There one can use regular patterns to search for the PVs store in the archive and display them on the Data Browser.

First Results

I was able to configure Archiver Engine to archive "bi" and "bo" records. Addind a single "ai" or "ao" record caused a problem. No samples were written to the database. I send a message to the author of the RDB Channel Archiver Kay Kasemir and got a response that there is an issue with handling NaN in the older version of the archiver (which was I was using) when using MySQL RDB. This highlighted a potentially recurring problem in the future coming from the fact that SNS will be using RDB Channel archiver in Oracle RDB, while we probably will prefer MySQL. Compiling a new version of the compiler was giving other error messages. At this point I gave up fighting with it until we discuss a possibility of using the MyA with CSS DataBrowser.

Performance Test

We will need to setup an archiving system that can handle on the order of 10K samples per second. Therefore, it is very important to verify that the RDB Archiver engines using a MySQL database server can handle such a load. I have not done this because of the problems I ran into with analog records.

JLab MYA Archiver