IOC Code Design

Intro

When an IOC (Input/Output Controller) has a pile of buffers from a digitizer or trigger module, each is pointed to by a pointer in one of three VxWorks queues:

• qFree
• qWritten
• qSender

The queues are lists of pointers that are used by the state machines inLoop, outLoop, and minisender to interact with the buffers. Each machine takes a pointer from a queue (if not empty) and uses the pointer to move data in/out of a buffer. When a machine is finished using the pointer to work with the buffer, the machine moves only the pointer from one queue to another. The state machines do not ever copy buffers.

State Machines

1. inLoop: Takes a pointer from qFree, fills the buffer pointed to with VME data, then moves the pointer to qWritten.
2. outLoop: Takes the pointer from qWritten, checks the data for sanity, and (if ok) moves the pointer to qSender.
3. minisender: Waits for gtReceiver to ask for data. When this occurs, minisender takes a pointer from qSender, formats the buffer into packets, then sends it to gtReceiver. The pointer is returned to qFree when finished.

Flow Control Connections and Error Handling

Flow control is accomplished by modulating how inLoop fills buffers

inLoop:

Every time inLoop checks the digitizer, it looks at all the flags associated with the board-wide FIFO and reads the FIFO depth from the firmware.

• Will pause scanning for a variable time if the number of buffers in qFree < 10. The delay time is linearly proportional to the number of free buffers.
• Does not enforce any kind of flow control. Throttle should be used to ensure no events are needlessly dropped if the FIFO becomes too full, and inLoop must reset the FIFO to resynchronize to the event boundaries in the data.
• Requires that the digitizer firmware implements an event-bounded FIFO depth count (for versions newer than ~2019).

outLoop:

Checks buffers filled by inLoop for structure errors (does not start with 0xAAAAAAAA, length errors, timestamp of current event is later than timestamp of last event). There is a compile-time option as to whether or not outLoop will try to realign after a length error before continuing scanning, or just exit checking after the first error. These errors are counted, but they do not stop data from being passed from qWritten to qSender.

• Has no flow control connections.
• The support function CheckAndMoveBuffers will disable the transfer of buffers from qWritten to qSender if a software error results in no free buffers.
• In this state, prevents queue overrun by immediately recycling from qWritten back to qFree, resulting in loss of data in qWritten.

minisender:

Constantly polls for data request packets from gtReceiver. A zero length packet from gtReceiver is not considered an error and minisender will just retry continuously.

• Does not participate and does not respond to flow control.

Interface between State Machines and the control system (EPICS)

inLoop:

inLoop monitors and responds to the PVs Online_CS_StartStop, DAQBx_1_CS_Ena, DAQBx_2_CS_Ena, DAQx_3_CS_Ena, DAQBx_4_CS_Ena. It asserts data to the PVs VMEx:MDIG1_CV_Running, DAQCx_CV_InLoop1, and DAQCx_CV_InLoop2.

1. VMEx:MDIG1_CV_Running is an internal communication PV between inLoop and outLoop. If set, outLoop understands that inLoop is running.
2. DAQCx_CV_InLoop1 provides inLoop's measurement of the total Mbytes/sec it is transferring, for monitoring.
3. DAQCx_CV_InLoop2 provides inLoop's measurement of the number of informational "type F" headers per second it is generating.

outLoop:

outLoop monitors and responds to the PVs Online_CS_SaveData, Online_CS_StartStop, VMEx:MDIG1_CV_Running, DAQCx_CS_TraceBd, DAQCx_CS_TraceChan, DAQCx_CS_TraceHorns. It asserts data to a large number of PVs

1. DAQCV_CV_BuffersAvail and DAQCx_CV_NumSendBuffers are the buffer counts of qFree and qSender.
2. DAQCx_CV_OutLoop1,2,3,4 are the number of lost buffers per digitizer.
3. DAQCx_OL_DataRate0,1,2,3 are the read rates in kBytes/sec per digitizer.
4. DAQCx_OL_Data0,1,2,3 are the total data amounts in Mbytes per digitizer.
5. DAQCV_OL_NumxxBuffers report the number of buffer counts (xx = Written, xx = Lost).
6. DAQCx_OL_BufLostPercent is what you think it is.
7. DAQCx_CV_SendRate is the sender data rate in kbytes/sec.
8. DAQCx_CV_TraceLen is the length of the diagnostic trace sent.

minisender:

minisender monitors and responds to the PVs Online_CS_StartStop and Online_CS_SaveData. It does not assert any monitoring or statistics data to PVs, as the load from outLoop covers everything.

• 

  IOC State Diagram for inLoop