EPICS control of digitizers

EPICS (xperimental Physics and Industrial Control System) is a framework of software that provides a distributed database of process variables (PVs) shared among an arbitrary number of computers, known within EPICS documentation as IOCs (Input-Output Controllers). By itself, EPICS does nothing of use, but it is a communications framework upon which control systems are built. The process variables are bits, multi-bit groups, integer words and floating point values that represent states or values within the data acquisition system. Process variables are connected to actual hardware by developing device driver software that communicates with the hardware in the format the hardware requires and by developing device support software that connects process variables to the device drivers to either read information from the hardware for display to the user (monitoring) or to cause writes to the hardware when the user changes the value of a process variable (control).

User interface to process variables

The user interfaces with process variables either through graphical user interfaces (GUIs) or by command-line operations. Digital Gammasphere uses the EDM (Extensible Display Manager) as its GUI. Everything in the GUI screens is either text, simplistic graphics, monitoring objects that display the value of PVs, control objects that allow users to change the value of PVs or links to other screens or scripts. On the command line, the 'caget' command allows the user to read the value of a PV and the 'caput' command allows the user to set a PV to a new value.

Databases

Process variables are defined using database files that enumerate all the PVs in the system. Each PV is a 'record' that defines the type of PV that it is and has a field (INP or OUT) that provides the linkage information needed to connect the PV to the name of some hardware register. Each PV also has a DTYP field that defines which device support software package will process the PV. "Input", or "read", or "monitoring" PVs will typically have a SCAN field that specifies how often they are read and processed. "Output", "write" or "control" PVs only process when changed.

Building the DGS EPICS system

DGS (Digital Gammasphere) contains multiple VME crates each with some number of digitizer and/or trigger boards under the control of an IOC (MVME5500 processor board running the VxWorks operating system). Each VME IOC runs multiple threads of software of which EPICS is just one. EPICS is a lower priority thread than the readout software, and EPICS is not the readout software.

In addition to the VME IOCs, the four Collector Boxes each contain a Raspberry Pi model 3b+ running Raspbian Linux, and one of the threads running on each Collector Box is EPICS making each of these four Raspberry Pis an IOC.

There is also an embedded VME processor board in the liquid nitrogen system that is an EPICS IOC, and finally Linux machine DGS1 is also an IOC.

All of these IOCs host databases containing PVs related to the hardware that each IOC is directly connected to, with the exception of DGS1. The IOC of DGS1 connects to no data acquisition hardware directly, and is used only to run process variables that connect to other process variables. Examples of this are

• Overall system controls such as Run/Stop or Save/NoSave
• Global values that, once changed, then propagate throughout the system to cause many hardware-connected PVs to all take on the value of the global PV
• Miscellaneous conversion/calculation PVs that take values from hardware-related PVs to generate monitoring values in a more pleasant format for the human to read

Root sources of information

The root source of all EPICS information are the Code Generating Spreadsheets maintained in a Subversion repository by the engineers of the Low Energy Technical Support group. These Excel spreadsheets enumerate all the registers within the firmware of each device (digitizer, SBX, collector box, trigger, etc.) throughout Digital Gammasphere. Code within these spreadsheets (Visual Basic for Applications, or VBA) uses text template files to write all EPICS database files used throughout the system. If a firmware upgrade occurs, the engineer updates the physical register maps in the spreadsheet, hits a button and a few seconds later all the databases are rewritten to match.

Building the IOC device support

The device support for the VME IOCs is a mix of C and C++ code that is maintained on the same Subversion repository (https://svn.inside.anl.gov/repos/dgs) as the code generating spreadsheets. As the MVME5500 VME processors run VxWorks, a cross-compiler is required. An ancient Solaris machine (con6.phy.anl.gov) is the sole and only host of the cross-compiler license. This ancient machine has no disk drives of its own and uses a part of the shared disk structure used by all the Linux boxes in the data room. However, one must log in to this machine to use its software.

• The full path to where the code is stored is /dk/fs2/dgs/global_sandbox/devel/.
• When logged into machine con6 as user dgs, this path now shows as /global/devel
• Beneath this path are two subfolders, dgsDrivers and dgsIoc.
  • dgsDrivers is the device support code.
  • dgsIoc is a "psuedo project" in that there's no code here, it's just used for the cross compilation.

Specific compilation steps on con6

• log in to machine con6 as user dgs.
• cd /global/devel/dgsDrivers
• make clean
• make
  • You should get 0 warnings and 0 errors. Anything else is failure.
• cd ../dgsIoc
• make clean
• make
  • You should get 0 warnings and 0 errors. Anything else is failure.

Updating the code from the code generating spreadsheets

If the firmware has been updated and the spreadsheets modified, running the VBA code for the different spreadsheets will, in addition to writing databases, also write C code that is part of the con6 compile. There are three spreadsheets associated with this:

• The DGS_CSDigitizer spreadsheet models the digitizers and writes two files used by con6.
• The DGS_MTrig spreadsheet models the master trigger and writes two files used by con6.
• The DGS_RTrig spreadsheet models the router triggers and writes two files used by con6.

If any spreadsheet has been changed and re-run, the files generated thereby must be copied to machine con6 before a recompile. This is done by entering the commands

• cd /global/devel/dgsDrivers/dgsDriverApp/src
• ./Export_SVN_ParameterFiles.sh

The Bash script will ask for a username and password for access to the Subversion repository and also ask for the password to ssh into machine DGS1 for the Subversion operations, because machine con6 has no working copy of the Subversion client.