Linking Systems Together - Revision history

Jta: /* Linking Multiple Systems Together */

2024-02-29T22:46:30Z

Linking Multiple Systems Together

@@ Line 1: / Line 1: @@
 == Linking Multiple Systems Together ==
-The trigger logic implemented for the various DAQ systems and the hardware design of the trigger module allows for the possibility to link multiple systems together.  This allows both clock synchronization and trigger sharing to occur.  A number of rules must be obeyed, so read carefully.
+The trigger logic implemented for the various DAQ systems and the hardware design of the trigger module allows for the possibility to link multiple systems together.  This allows both clock synchronization and trigger sharing to occur.  A number of rules must be obeyed, so read '''''carefully'''''.
 === Clock Sharing ===
@@ Line 47: / Line 60: @@
 * Link U is historically used with the MyRIAD module to receive trigger information from an external detector.
 ** Link U can be programmed to receive data from another master trigger.
 == The concept of Propagation Control ==

Jta at 22:24, February 29, 2024

2024-02-29T22:24:35Z

Jta: /* Processing of information from other masters */

2019-09-11T21:33:14Z

Processing of information from other masters

← Older revision		Revision as of 21:33, September 11, 2019
Line 89:		Line 89:
	As is seen in the picture, the algorithm can perform a coincidence calculation of the trigger message ''received from that particular link'' relative to ''any of the other triggers''. The resultant coincidence then creates trigger acceptance messages FOR THAT LINK. This means that, in our example,		As is seen in the picture, the algorithm can perform a coincidence calculation of the trigger message ''received from that particular link'' relative to ''any of the other triggers''. The resultant coincidence then creates trigger acceptance messages FOR THAT LINK. This means that, in our example,

	* If DGS		* DGS is receiving messages from the X-Array over link L. Thus, DGS can calculate a timestamp offset for the messages from the X-Array to compensate for time-of-flight, different integration time, etc., generate an internal pulse at the adjusted timestamp, and then perform a coincidence of that pulse with the other triggers ''at the moment they occur'' to generate an internal DGS trigger (algorithm #6).
			** The Algorithm 6 trigger is sent to DGS, to DFMA and to the MyRIAD, '''but is not sent back to the X-Array to avoid infinite trigger loops'''
			* DGS is receiving messages from DFMA over link R. Thus, DGS can calculate a timestamp offset for the messages from DFMA to compensate for time-of-flight, different integration time, etc., generate an internal pulse at the adjusted timestamp, and then perform a coincidence of that pulse with the other triggers ''at the moment they occur'' to generate an internal DGS trigger (algorithm #7).
			** The Algorithm 7 trigger is sent to DGS, to X-Array and to the MyRIAD, '''but is not sent back to DFMA to avoid infinite trigger loops'''

			The MyRIAD - link U in this example - is treated different.y.

Jta: /* Trigger Algorithm 6 */

2019-09-11T21:26:49Z

Trigger Algorithm 6

← Older revision		Revision as of 21:26, September 11, 2019
Line 81:		Line 81:
	'''''As the firmware is currently written there is no way to perform a three- or four-system coincidence and broadcast that coincidence trigger to all systems'''''		'''''As the firmware is currently written there is no way to perform a three- or four-system coincidence and broadcast that coincidence trigger to all systems'''''

	=== ~~Trigger Algorithm 6~~ ===		=== Processing of information from other masters ===

	~~Algorithm 6, that processes messages from~~ link L, ~~uses~~ a fixed ~~piece~~ of ~~code called REMOTE_MASTER_TRIG_SUPPORT when receiving~~ triggers ~~from another master~~ trigger.		Whenever a SERDES link (L, R or U) is configured to be a 'remote master' data format, the firmware has a fixed way of processing that data, shown here.

			[[File:RemoteTrigAlgorithm.PNG]]

			As is seen in the picture, the algorithm can perform a coincidence calculation of the trigger message ''received from that particular link'' relative to ''any of the other triggers''. The resultant coincidence then creates trigger acceptance messages FOR THAT LINK. This means that, in our example,

			* If DGS

Jta: /* Collection of triggers from one master trigger to another */

2019-09-11T21:02:59Z

Collection of triggers from one master trigger to another

← Older revision		Revision as of 21:02, September 11, 2019
Line 74:		Line 74:

	[[File:RemoteTriggerQueuing.png]]		[[File:RemoteTriggerQueuing.png]]

			== Currently Supported Cross-system Triggers ==

			The important detail of the above picture is that the trigger messages coming in from links L, R and U must pass through ''Trigger Algorithms'' and only those combinations explicitly supported by the specific algorithms can create multi-detector coincidences. There are also limits, as indicated in the picture, of what trigger messages pass through to the other systems.

			'''''As the firmware is currently written there is no way to perform a three- or four-system coincidence and broadcast that coincidence trigger to all systems'''''

			=== Trigger Algorithm 6 ===

			Algorithm 6, that processes messages from link L, uses a fixed piece of code called REMOTE_MASTER_TRIG_SUPPORT when receiving triggers from another master trigger.

Jta: /* Collection of triggers from one master trigger to another */

2019-09-11T20:55:34Z

Collection of triggers from one master trigger to another

← Older revision		Revision as of 20:55, September 11, 2019
Line 73:		Line 73:
	This is a legitimate setup because link L can be configured by writing to DGS's registers to use 'remote master' format, link R is natively 'remote master' format, and link U of DGS can be configured by writing to registers to use 'MyRIAD' format. This setup then means that the DGS master trigger has to somehow attempt to combine information not only from itself but from the other setups to make multi-system trigger accept messages. The overall data flow by which this occurs is shown in the following picture.		This is a legitimate setup because link L can be configured by writing to DGS's registers to use 'remote master' format, link R is natively 'remote master' format, and link U of DGS can be configured by writing to registers to use 'MyRIAD' format. This setup then means that the DGS master trigger has to somehow attempt to combine information not only from itself but from the other setups to make multi-system trigger accept messages. The overall data flow by which this occurs is shown in the following picture.

	[[File:~~RemoteTriggerQueueing~~.~~PNG~~]]		[[File:RemoteTriggerQueuing.png]]

Jta: /* Collection of triggers from one master trigger to another */

2019-09-11T20:54:47Z

Collection of triggers from one master trigger to another

← Older revision		Revision as of 20:54, September 11, 2019
Line 73:		Line 73:
	This is a legitimate setup because link L can be configured by writing to DGS's registers to use 'remote master' format, link R is natively 'remote master' format, and link U of DGS can be configured by writing to registers to use 'MyRIAD' format. This setup then means that the DGS master trigger has to somehow attempt to combine information not only from itself but from the other setups to make multi-system trigger accept messages. The overall data flow by which this occurs is shown in the following picture.		This is a legitimate setup because link L can be configured by writing to DGS's registers to use 'remote master' format, link R is natively 'remote master' format, and link U of DGS can be configured by writing to registers to use 'MyRIAD' format. This setup then means that the DGS master trigger has to somehow attempt to combine information not only from itself but from the other setups to make multi-system trigger accept messages. The overall data flow by which this occurs is shown in the following picture.

	[[File:RemoteTriggerQueueing.~~png~~]]		[[File:RemoteTriggerQueueing.PNG]]

Jta: /* Timestamp sharing and Imperative Sync */

2019-09-11T20:54:32Z

Timestamp sharing and Imperative Sync

← Older revision		Revision as of 20:54, September 11, 2019
Line 62:		Line 62:
	* If the LINK L PROPAGATION CONTROL register, bit 0, is SET, '''and''' the subservient master has set its link L to be receiving commands from another master trigger, '''then''' the subservient master will respond only to Imperative Sync from the "timing monarch" AND cannot issue its own local Imperative Sync.		* If the LINK L PROPAGATION CONTROL register, bit 0, is SET, '''and''' the subservient master has set its link L to be receiving commands from another master trigger, '''then''' the subservient master will respond only to Imperative Sync from the "timing monarch" AND cannot issue its own local Imperative Sync.
	* If either of those two conditions are not met, then the subservient master, ''even if using the clock from the timing monarch'', will IGNORE any Imperative Sync from the timing monarch but CAN issue its own Imperative Sync within its own system.		* If either of those two conditions are not met, then the subservient master, ''even if using the clock from the timing monarch'', will IGNORE any Imperative Sync from the timing monarch but CAN issue its own Imperative Sync within its own system.

			== Collection of triggers from one master trigger to another ==

			Each master trigger receives copies of all the 'local' (within a given detector) trigger accept messages from all the other master triggers it is connected to through links L, R and U. Hypothesize that the setup in play is Digital Gammasphere (DGS) as the "timing monarch" with DFMA and X-Array as the "subservient" master triggers. Just to fill out the example, also assume that there's some other external detector in the setup that is using a MyRIAD. Further assume that the wiring setup is as follows:

			* DGS link R is connected to DFMA, link L.
			* DGS link L is connected to X-Array, link L.
			* DGS link U is connected to the MyRIAD in the external detector.

			This is a legitimate setup because link L can be configured by writing to DGS's registers to use 'remote master' format, link R is natively 'remote master' format, and link U of DGS can be configured by writing to registers to use 'MyRIAD' format. This setup then means that the DGS master trigger has to somehow attempt to combine information not only from itself but from the other setups to make multi-system trigger accept messages. The overall data flow by which this occurs is shown in the following picture.

			[[File:RemoteTriggerQueueing.png]]

Jta: /* The concept of Propagation Control */

2019-09-11T20:21:41Z

The concept of Propagation Control

← Older revision		Revision as of 20:21, September 11, 2019
Line 52:		Line 52:
	Every master trigger is the master of its own detector. However, each master trigger that is receiving data from another master trigger from the SERDES links (L, R or U) can ''selectively propagate information received from another master trigger'' into itself. There is a '''Propagation Control''' register associated with each of the three links that is bit-wise mapped to enumerate what information is allowed to enter. This is described in the Trigger Timing and Control Link Specification document that I know you haven't read, so here's the text again.		Every master trigger is the master of its own detector. However, each master trigger that is receiving data from another master trigger from the SERDES links (L, R or U) can ''selectively propagate information received from another master trigger'' into itself. There is a '''Propagation Control''' register associated with each of the three links that is bit-wise mapped to enumerate what information is allowed to enter. This is described in the Trigger Timing and Control Link Specification document that I know you haven't read, so here's the text again.

	[[File:Ttcl cutout.~~png~~]]		[[File:Ttcl cutout.PNG]]

			These Propagation Control registers are important not only to set up the timestamp, but also in the setting up of shared triggers across systems.

	=== Timestamp sharing and Imperative Sync ===		=== Timestamp sharing and Imperative Sync ===

Jta: /* The concept of Propagation Control */

2019-09-11T20:19:00Z

The concept of Propagation Control

← Older revision		Revision as of 20:19, September 11, 2019
Line 52:		Line 52:
	Every master trigger is the master of its own detector. However, each master trigger that is receiving data from another master trigger from the SERDES links (L, R or U) can ''selectively propagate information received from another master trigger'' into itself. There is a '''Propagation Control''' register associated with each of the three links that is bit-wise mapped to enumerate what information is allowed to enter. This is described in the Trigger Timing and Control Link Specification document that I know you haven't read, so here's the text again.		Every master trigger is the master of its own detector. However, each master trigger that is receiving data from another master trigger from the SERDES links (L, R or U) can ''selectively propagate information received from another master trigger'' into itself. There is a '''Propagation Control''' register associated with each of the three links that is bit-wise mapped to enumerate what information is allowed to enter. This is described in the Trigger Timing and Control Link Specification document that I know you haven't read, so here's the text again.

			[[File:Ttcl cutout.png]]

	=== Timestamp sharing and Imperative Sync ===		=== Timestamp sharing and Imperative Sync ===

← Older revision		Revision as of 22:24, February 29, 2024
Line 1:		Line 1:
	== Linking Multiple Systems Together ==		== Linking Multiple Systems Together ==

	The trigger logic implemented for the various DAQ systems and the hardware design of the trigger module allows for the possibility to link multiple systems together. This allows both clock synchronization and trigger sharing to occur. A number of rules must be obeyed, so read carefully.		The trigger logic implemented for the various DAQ systems and the hardware design of the trigger module allows for the possibility to link multiple systems together. This allows both clock synchronization and trigger sharing to occur. A number of rules must be obeyed, so read carefully.

	=== Clock Sharing ===		=== Clock Sharing ===

	The basic method of clock sharing that most users are familiar with is that the master trigger sends clock to the routers, and the routers re-distribute that clock to the digitizers. Many users also understand that this tree of clock distribution has been extended to include Digital Gammasphere (DGS) sending the clock to DFMA, as shown here.		The basic method of clock sharing that most users are familiar with is that the master trigger sends clock to the routers, and the routers re-distribute that clock to the digitizers. Many users also understand that this tree of clock distribution has been extended to include Digital Gammasphere (DGS) sending the clock to DFMA, as shown here.

	[[File:ClockHierarchy.png\|Standard clock distribution]]		[[File:ClockHierarchy.png\|Standard clock distribution]]

	Those that have been involved from the beginning will recall that the DGS master trigger could also be connected to 'Analog' Gammasphere using the GITMO module - and use the clock from 'Analog' Gammasphere. Those that have been involved with hosting external detectors such as CHICO, Microball, ORRUBA, etc. also understand that there is clock sharing associated with the MyRIAD module. A more full picture of this clock distribution hierarchy is shown here.		Those that have been involved from the beginning will recall that the DGS master trigger could also be connected to 'Analog' Gammasphere using the GITMO module - and use the clock from 'Analog' Gammasphere. Those that have been involved with hosting external detectors such as CHICO, Microball, ORRUBA, etc. also understand that there is clock sharing associated with the MyRIAD module. A more full picture of this clock distribution hierarchy is shown here.

	[[File:FullSystem.png\|full system clocking tree]]		[[File:FullSystem.png\|full system clocking tree]]
Line 15:		Line 15:
	==== The rules of the game as currently understood by users ====		==== The rules of the game as currently understood by users ====

	So people know that you can hook a MyRIAD to DGS, and you can hook DFMA to DGS, and that with appropriate software incantations the clock and timestamp can be distributed from DGS to the other two systems. That is correct, and that is true, but it is not the whole and complete truth of how the firmware works.		So people know that you can hook a MyRIAD to DGS, and you can hook DFMA to DGS, and that with appropriate software incantations the clock and timestamp can be distributed from DGS to the other two systems. That is correct, and that is true, but it is not the whole and complete truth of how the firmware works.

	==== How clock sharing REALLY works ====		==== How clock sharing REALLY works ====

	How the clock sharing works in full detail is as follows.		How the clock sharing works in full detail is as follows.

	* Any master trigger can be declared as the "timing monarch" that is the source of the clock.		* Any master trigger can be declared as the "timing monarch" that is the source of the clock.
Line 40:		Line 40:
	=== Understanding Link usage ===		=== Understanding Link usage ===

	Each master trigger board has three "special" SERDES links named "L", "R" and "U". Originally this was meant to be read as "Left", "Right" and "Up" for drawing complicated link connections, but the convention of direction was abandoned unused. These three links are "special" in the sense that they are intended to be used as ''system interconnect'' links as opposed to the ''internal connections'' of links A through H. More precisely, what this means is that the data formats that these links send and receive are slightly different than the data format used to connect with routers. Each of these three links has a unique purpose and unique characteristics.		Each master trigger board has three "special" SERDES links named "L", "R" and "U". Originally this was meant to be read as "Left", "Right" and "Up" for drawing complicated link connections, but the convention of direction was abandoned unused. These three links are "special" in the sense that they are intended to be used as ''system interconnect'' links as opposed to the ''internal connections'' of links A through H. More precisely, what this means is that the data formats that these links send and receive are slightly different than the data format used to connect with routers. Each of these three links has a unique purpose and unique characteristics.

	* Link L is unique in that it alone can receive a clock from an external source that can be used in place of the board's internal clock.		* Link L is unique in that it alone can receive a clock from an external source that can be used in place of the board's internal clock.
Line 58:		Line 58:
	=== Timestamp sharing and Imperative Sync ===		=== Timestamp sharing and Imperative Sync ===

	There is a difference between sharing the CLOCK and sharing the TIMESTAMP. The timestamp logic of the subservient master trigger has a separate enable bit to allow the subservient master to respond to Imperative Sync commands sent from the "timing monarch". The precise rule here is:		There is a difference between sharing the CLOCK and sharing the TIMESTAMP. The timestamp logic of the subservient master trigger has a separate enable bit to allow the subservient master to respond to Imperative Sync commands sent from the "timing monarch". The precise rule here is:

	* If the LINK L PROPAGATION CONTROL register, bit 0, is SET, '''and''' the subservient master has set its link L to be receiving commands from another master trigger, '''then''' the subservient master will respond only to Imperative Sync from the "timing monarch" AND cannot issue its own local Imperative Sync.		* If the LINK L PROPAGATION CONTROL register, bit 0, is SET, '''and''' the subservient master has set its link L to be receiving commands from another master trigger, '''then''' the subservient master will respond only to Imperative Sync from the "timing monarch" AND cannot issue its own local Imperative Sync.
Line 65:		Line 65:
	== Collection of triggers from one master trigger to another ==		== Collection of triggers from one master trigger to another ==

	Each master trigger receives copies of all the 'local' (within a given detector) trigger accept messages from all the other master triggers it is connected to through links L, R and U. Hypothesize that the setup in play is Digital Gammasphere (DGS) as the "timing monarch" with DFMA and X-Array as the "subservient" master triggers. Just to fill out the example, also assume that there's some other external detector in the setup that is using a MyRIAD. Further assume that the wiring setup is as follows:		Each master trigger receives copies of all the 'local' (within a given detector) trigger accept messages from all the other master triggers it is connected to through links L, R and U. Hypothesize that the setup in play is Digital Gammasphere (DGS) as the "timing monarch" with DFMA and X-Array as the "subservient" master triggers. Just to fill out the example, also assume that there's some other external detector in the setup that is using a MyRIAD. Further assume that the wiring setup is as follows:

	* DGS link R is connected to DFMA, link L.		* DGS link R is connected to DFMA, link L.