REDAC Network Protocol

The REDAC network protocol is a request-response message based protocol. Messages are encoded as JSON strings and delimited by newline characters. The message definition can be found in the messages namespace. Please refer to their documentations and the therein contained sequence diagrams for details about the messaging process.

A usual sequence of messages for running one or multiple analog computations is:

1. Get the current hardware configuration via a GetEntitiesRequest.

2. Start a new user session (if required) via a StartSessionRequest.

3. Configure the analog computation via a series of SetCircuitRequest.

4. Start the analog computation via a StartRunRequest.

5. Monitor the analog computation by listening to incoming RunStateChangeMessage and RunDataMessage notifications.

6. The analog computation is done once you receive a RunStateChangeMessage with new set to DONE.

7. Evaluate the data received and repeat from step 3 if necessary.

8. End your session with a EndSessionRequest.

Shared Datatypes

The following data types are shared between several messages.

class pybrid.redac.run.DAQConfig(*, num_channels: int = <pybrid.redac.run.DAQConfigurationNumChannels object at 0x73597dbaa690>, sample_rate: int = 10000, sample_op: bool = True, sample_op_end: bool = True)

num_channels: int

List of channels that should be sampled. The element corresponding to each channel is implicitly defined by the computer’s configuration.

sample_op: bool = True

Whether to sample during IC

sample_op_end: bool = True

Whether to sample during OP_END

sample_rate: int = 10000

Sample rate to use in samples/second.

class pybrid.redac.run.Run(*, id_: ~uuid.UUID = <factory>, created: ~datetime.datetime = <factory>, config: ~pybrid.redac.run.RunConfig = <factory>, state: ~pybrid.redac.run.RunState = RunState.NEW, flags: ~pybrid.redac.run.RunFlags = <factory>, daq: ~pybrid.redac.run.DAQConfig = <factory>, data: dict[pybrid.redac.entities.Path | str, list[float]] = <factory>, related_to: ~typing.Optional[~uuid.UUID] = None, sync: ~pybrid.redac.sync.SyncConfig = <factory>, partition: ~pybrid.redac.partitioning.PartitionConfig = <factory>)

A run is one computation executed by the REDAC.

config: RunConfig

Defines the duration the run should be executed and similar parameters. Does not contain element configuration.

daq: DAQConfig

The configuration of the data acquisition for this run.

flags: RunFlags

Flags, e.g. overload, the run has triggered. These are persistent once triggered.

classmethod get_persistent_attributes() → set[str]

Get a list of attributes that should usually be persistent between consecutive runs. For example, it’s reasonable to persist the ‘config’ attribute in a series of runs. :return: List of attribute names to persist

id_: UUID

A unique identifier for the run.

partition: PartitionConfig

related_to: Optional[UUID] = None

Possibly the ID of a related run, e.g. one that triggered this one.

state: RunState = 'NEW'

The current state of the run.

sync: SyncConfig

Define how the computation is synchronized, e.g. to other parts of the computer.

class pybrid.redac.run.RunConfig(*, ic_time: int = 100000, op_time: int = 2000000, calibrate: bool = True, halt_on_external_trigger: bool = False, halt_on_overload: bool = False)

Configures parameters related to the execution of one Run.

calibrate: bool = True

Whether to calibrate routes before the computation.

halt_on_external_trigger: bool = False

Whether to halt the computation when the external halt signal is triggered.

halt_on_overload: bool = False

Whether to halt the computation when it enters an overload.

ic_time: int = 100000

Duration of IC (initial condition) mode in nanoseconds.

op_time: int = 2000000

Duration of OP (operating) mode in nanoseconds.

exception pybrid.redac.run.RunError

Base exception class for all errors during a Run.

class pybrid.redac.run.RunFlags(*, externally_halted: bool = False, overloaded: Optional[list[pybrid.redac.entities.Path]] = None)

Flags that can be triggered by a Run. Once triggered, they are persistently True, even when the original condition is lost.

externally_halted: bool = False

Whether the run was halted because of an external halt trigger.

overloaded: Optional[list[pybrid.redac.entities.Path]] = None

Any element that entered an overload during computation.

class pybrid.redac.run.RunState(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)

The state of a Run.

DONE = 'DONE'

Run has successfully finished.

ERROR = 'ERROR'

Run has encountered an error and has been aborted.

IC = 'IC'

Run is in IC (initial condition) mode.

NEW = 'NEW'

Run has just been created.

OP = 'OP'

Run is in OP (operating) mode.

OP_END = 'OP_END'

Run is principally done, pending final data acquisition or other finalizing tasks.

QUEUED = 'QUEUED'

Run is queued for execution.

TAKE_OFF = 'TAKE_OFF'

Run has been selected for execution and is being prepared to start.

TMP_HALT = 'TMP_HALT'

Run is temporarily halted and can be resumed.

classmethod get_possibly_sampled_states()

Return a list of states in which data acquisition is possible.

class pybrid.redac.protocol.types.SuccessInfo(*, success: bool, error: Optional[str] = None)

Data type used in responses indicating whether the associated request was successful.

error: Optional[str]

An optional error message. Always None if success is True.

success: bool

True if response was successful, False otherwise.

Envelope

class pybrid.redac.protocol.envelope.Envelope(*, id: Optional[UUID] = None, type: str, msg: Optional[dict] = None, success: Optional[bool] = True, error: Optional[str] = '')

Envelope containing a Message.

While the relevant data for the communication as part of this protocol is encoded using the various message classes, the most fundamental data package that is sent between two recipients is this envelope.

The envelope contains an id, which is used to identify responses to previous requests (optional for notifications).

The type field contains a unique type identifier, telling the recipient which Message is contained in the msg field. The type identifier is defined to be the python message class name, with any Request, Response or Notification suffix removed and converted to underscore case.

For responses, the success field and error field define whether the request was successfully handled. Only if success is true, msg contains an actual response.

error: Optional[str]

Optional error, in case success is false

id: Optional[UUID]

Optional ID of the request and its response. None for Notifications

json(*args, **kwargs)

Generate a JSON representation of the model, include and exclude arguments as per dict().

encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().

msg: Optional[dict]

The msg, None if success is false for responses

success: Optional[bool]

Whether the request was handled successfully (only in responses)

type: str

Unique string defining the type of the contained message

Base Message Classes

All messages are based on the pybrid.redac.protocol.messages.Message base class. Requests and responses have additional common functionality in the pybrid.redac.protocol.messages.Request and pybrid.redac.protocol.messages.Response base classes respectively.

class pybrid.redac.protocol.messages.Message

Base class for all messages.

Serialization and deserialization is handled with the help of the pydantic package.

json(*, include: Optional[Union['AbstractSetIntStr', 'MappingIntStrAny']] = None, exclude: Optional[Union['AbstractSetIntStr', 'MappingIntStrAny']] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Optional[Callable[[Any], Any]] = None, models_as_dict: bool = True, **dumps_kwargs: Any) → str

Generate a JSON representation of the model, include and exclude arguments as per dict().

encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().

classmethod parse_obj(**data: dict) → Message

Parses data from dict or json-like and returns a message instance.

classmethod register_callback(callback: Callable) → Callable

Register a callback for some Message subclass, which is triggered when the respective message is received.

Parameters:

callback – Function to register as callback

Returns:

The original function

Usage:

Intended to be used as decorator:

# Register callback triggered on e.g. an incoming RunStateChangeMessage
@RunStateChangeMessage.register_callback
def callback(self, msg: RunStateChangeMessage):
    # do something with the message

class pybrid.redac.protocol.messages.Request

Base class for requests sent to the controller.

classmethod get_expected_response_type() → Type[Response]

The Response subclass expected for the answer to this request.

class pybrid.redac.protocol.messages.Response

Base class for responses to a previous request.

property first_error: Optional[str]

Error message of the first error that occurred when the request was handled. None if there was no error.

response_for: ClassVar[Request]

The Request subclass to which this is the response

property successful: bool

Indicates whether the request was handled successfully

Initialization

These messages are mostly used once during the initialization phase of the controller or the client library.

class pybrid.redac.protocol.messages.GetEntitiesRequest

A request for the list of entity types (pybrid.redac.entities.EntityType) by their path (pybrid.redac.entities.Path). The controller responds with a GetEntitiesResponse containing a tree-like representation of all entities.

The GetEntitiesResponse tells you the current assembly structure of the analog computer. Use a series of GetEntityConfiguration messages if you also need to know the current configuration.

class pybrid.redac.protocol.messages.GetEntitiesResponse(*, entities: dict)

A response containing the list of entities (pybrid.redac.entities.EntityType) currently in the analog computer.

entities: dict

A tree-like dictionary structure containing entity type information by path.

response_for

alias of GetEntitiesRequest

Session Management

To facilitate multi-user support and a minimum form of authentication without additional scheduling software (like SLURM), your controller can be configured to require starting a session and reserving resources before any other command using those resources can be executed.

class pybrid.redac.protocol.messages.StartSessionRequest(*, entities: list[pybrid.redac.entities.Path])

Request to start a session for which the requested elements are reserved. No other client can request those elements until the session is ended.

Starting and managing sessions is only necessary if your controller is configured to require it.

entities: list[pybrid.redac.entities.Path]

A list of analog entities to reserve for this session.

class pybrid.redac.protocol.messages.StartSessionResponse(*, id_: Optional[UUID4] = None, success: SuccessInfo)

Response to a prior StartSessionRequest.

If the reservation was successful, a secret ID is returned. This ID is used in subsequent configuration and run requests to authorize their usage of reserved entities.

If not all requested entities could be reserved for the new session, the session is not started.

id_: Optional[UUID4]

Secret session ID or None if the session could not be started.

response_for

alias of StartSessionRequest

success: SuccessInfo

Whether the session could be started and optional error info.

class pybrid.redac.protocol.messages.ResumeSessionRequest(*, id_: UUID4)

Request to resume a session and associate it with the current connection, such that it is automatically used on future requests.

id_: UUID4

Secret session ID.

class pybrid.redac.protocol.messages.ResumeSessionResponse(*, success: SuccessInfo)

Response to a prior ResumeSessionRequest.

response_for

alias of ResumeSessionRequest

success: SuccessInfo

Whether the session could be resumed.

class pybrid.redac.protocol.messages.EndSessionRequest(*, id_: UUID4)

Request to end a session.

If there are any ongoing runs in the session, they are canceled first and any messages related to them are sent first by the controller, before the corresponding EndSessionResponse is sent.

Inactive sessions may be ended automatically depending on controller configuration.

id_: UUID4

The secret session ID to end.

class pybrid.redac.protocol.messages.EndSessionResponse(*, success: SuccessInfo)

Response to a prior EndSessionRequest.

response_for

alias of EndSessionRequest

success: SuccessInfo

Whether the session could be ended and optional error info. Usually True.

class pybrid.redac.protocol.messages.EntityReservationRequest(*, id_: UUID4, entities: list[pybrid.redac.entities.Path])

Request to reserve additional entities for an existing session.

entities: list[pybrid.redac.entities.Path]

A list of analog entities to reserve for this session.

id_: UUID4

Secret session ID

class pybrid.redac.protocol.messages.EntityReservationResponse(*, success: SuccessInfo)

Response to a prior EntityReservationRequest.

response_for

alias of EntityReservationRequest

success: SuccessInfo

Whether the requested entities were reserved and error information if they were not.

Entity Configuration

After potentially reserving certain entities for a session, it is usually necessary to configure them for an upcoming run.

class pybrid.redac.protocol.messages.SetCircuitRequest(*, session: Optional[UUID4] = None, entity: Path, config: dict, reset_before: bool = False, sh_kludge: bool = False, calibrate_routes: bool = False, partition_config: Optional[PartitionConfig] = None)

A request to the controller to set a configuration for an entity. The controller forwards the request to the carrier board on which the entity is located and forwards its SetConfigResponse response back.

Entities are arranged hierarchically in the REDAC. The config dictionary is passed to the entity defined by SetConfigRequest.entity. To allow the configuration of multiple entities, the SetConfigRequest.config dictionary may contain keys starting with a slash (“/”), which are used to denote paths to sub-entities. Such sub-entity path keys must denote a sub-config dictionary, which is again passed on.

The structure and content of the configuration message depend on the entities to be configured. See REDAC Blocks and Configurations for details.

Example of a multi-entity SetConfigRequest message.

{
  "_id": 42, "_type": "set_config", "msg": {
    "entity": ["04-E9-E5-14-74-BF", "0"],
    "config": {
      "/U": {
        "outputs": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15]
        "alt-signals": [3, 8],
      },
      "/C": {
        "elements": {0: 0.42, 3: -0.42}
      }
    }
  }
}

calibrate_routes: bool

Whether to calibrate routes after applying the configuration

config: dict

The configuration to apply. May contain keys denoting paths to sub-entities (starting with a slash) and their config. The data schema of the configuration depends on the type of entity, see REDAC Blocks and Configurations for details.

entity: Path

The entity to configure.

classmethod make(entity)

Factory method to create a config request for some entity.

partition_config: Optional[PartitionConfig]

A :py:class`pybrid.redac.partitioning.PartitionConfig` that should be applied to the run.

reset_before: bool

Whether to reset the existing configuration beforehand. We generally don’t want this, as we already reset when necessary.

session: Optional[UUID4]

The secret session ID for which the entity was reserved. Only required if session management is enabled.

sh_kludge: bool

Whether to cycle the SH-block state once just before configuration. We generally don’t want this, as it’s unnecessary if all else works as expected.

class pybrid.redac.protocol.messages.SetCircuitResponse

A response to SetConfigRequest conveying the success of the latter.

response_for

alias of SetCircuitRequest

class pybrid.redac.protocol.messages.ResetCircuitRequest(*, keep_calibration: Optional[bool] = True, sync: Optional[bool] = True)

A request for the hybrid controller to reset its configuration.

Depending on the flags inside the message, only part of the configuration is reset. The sync flag can be used to prevent actually writing the configuration to the hardware, i.e. it is kept in memory only (useful if subsequent requests change it anyway).

keep_calibration: Optional[bool]

Whether the calibration data should be kept.

sync: Optional[bool]

Whether to immediately sync to hardware.

class pybrid.redac.protocol.messages.ResetCircuitResponse

A response to a previous ResetCircuitRequest.

response_for

alias of ResetCircuitRequest

class pybrid.redac.protocol.messages.GetCircuitRequest(*, entity: Path, recursive: bool = True)

A request to the controller to retrieve the configuration of an entity. The controller responds with a GetCircuitResponse. This configuration includes only the effective analog configuration (e.g. the scalar factor of a digital potentiometer). For any metadata (e.g. calibration) use GetMetadataRequest.

entity: Path

Path to the entity of which the configuration should be returned.

recursive: bool

Whether to retrieve the configuration of sub-entities recursively.

class pybrid.redac.protocol.messages.GetCircuitResponse(*, entity: Path, config: dict)

A response to GetCircuitRequest conveying the configuration of some entity.

config: dict

The configuration of the entity. The data schema of the configuration depends on the type of entity.

entity: Path

Path to the entity of which the configuration is returned.

response_for

alias of GetCircuitRequest

class pybrid.redac.protocol.messages.SetDAQRequest(*, daq: DAQConfig, session: Optional[UUID4] = None)

A request to the controller to set a DAQConfig determining how and when data should be acquired. The controller will respond with a SetDAQResponse

daq: DAQConfig

The DAQ configuration to apply.

session: Optional[UUID4]

The secret session ID for which the entities were reserved. Only required if session management is enabled.

class pybrid.redac.protocol.messages.SetDAQResponse(*, success: SuccessInfo)

A response to SetDAQRequest conveying the success of the latter

response_for

alias of SetDAQRequest

success: SuccessInfo

Whether the request was successful.

class pybrid.redac.protocol.messages.GetMetadataRequest(*, entity: Path)

A request to the controller to retrieve the metadata of an entity. The controller responds with a GetMetadataResponse. The metadata contains entity-specific information (e.g. type identifier, calibration, …).

entity: Path

Path to the entity of which the metadata should be returned.

class pybrid.redac.protocol.messages.GetMetadataResponse(*, entity: Path, config: dict)

A response to GetMetadataRequest conveying the metadata of some entity.

config: dict

The metadata of the entity. The data schema of the metadata depends on the version included in config[‘sp_version’].

entity: Path

Path to the entity of which the metadata is returned.

response_for

alias of GetMetadataRequest

Run Management

To start the analog computer with the previously set configuration, a run is started. A hybrid computation may require multiple runs (e.g. to gather statistical data) that are executed during one session.

class pybrid.redac.protocol.messages.StartRunRequest(*, session: Optional[UUID4] = None, id: UUID4, config: RunConfig, daq_config: Optional[DAQConfig] = None, sync_config: SyncConfig = None, partition_config: PartitionConfig = None)

A request to start a run (computation). After a run is started, the controller sends RunStateChangeMessage notifications about its progress.

config: RunConfig

A pybrid.redac.run.RunConfig that should be applied to the run.

daq_config: Optional[DAQConfig]

A pybrid.redac.daq.DAQConfig that should be applied to the run. If None, the previous configuration is used.

classmethod from_run(run)

Generate a pybrid.redac.protocol.messages.StartRunRequest from a pybrid.redac.run.Run instance.

Parameters:

run – A run

Returns:

A StartRunRequest instance

id: UUID4

An ID that should be applied to the run.

partition_config: PartitionConfig

A :py:class`pybrid.redac.partitioning.PartitionConfig` that should be applied to the run.

session: Optional[UUID4]

The secret session ID in which the run should be started. Only required if session management is enabled.

sync_config: SyncConfig

A :py:class`pybrid.redac.run.SyncConfig` that should be applied to the run.

to_run()

Generate a pybrid.redac.run.Run instance with values from this message.

Returns:

A pybrid.redac.run.Run instance

class pybrid.redac.protocol.messages.StartRunResponse

A response to a StartRunRequest indicating whether the run was accepted.

response_for

alias of StartRunRequest

class pybrid.redac.protocol.messages.CancelRunRequest(*, id_: UUID4)

A request to cancel an ongoing run. Any caused RunStateChangeMessage is sent first, before the CancelRunResponse is sent.

id_: UUID4

The ID of the run to be canceled.

class pybrid.redac.protocol.messages.CancelRunResponse(*, id_: UUID4, success: SuccessInfo)

A response to a prior CancelRunRequest indicating whether the run was successfully canceled.

id_: UUID4

The ID of the run requested to be canceled.

response_for

alias of CancelRunRequest

success: SuccessInfo

Whether the run was successfully canceled and error information if not.

class pybrid.redac.protocol.messages.RunDataMessage(*, id: UUID4, entity: Path, data: list[list[float]])

Notification containing data sampled during a RunState according to the config set with SetDAQRequest. All data corresponding to a RunState is sent out before the state exit is indicated by a respective RunStateChangeMessage.

data: list[list[float]]

Acquired data by entity path, normalized to [-1,+1]

entity: Path

Entity (cluster) that produced the data

id: UUID4

ID of the run

class pybrid.redac.protocol.messages.RunStateChangeMessage(*, id: UUID4, t: int, old: RunState, new: RunState, run_flags: Optional[RunFlags] = None)

Notification that an ongoing Run changed its RunState. A run is done once it enters RunState.DONE or RuntState.ERROR.

id: UUID4

ID of the run

new: RunState

New state

old: RunState

Previous state

run_flags: Optional[RunFlags]

Any RunFlags that the run has triggered (persistent across state changes).

t: int

Current time in microseconds

Internal Messages

Some messages are intended to be used in the internal communication between the hybrid controller and the carrier boards. For testing purposes, you can also generate and send them from the digital control computer.

class pybrid.redac.protocol.messages.GetOverloadRequest(*, entities: Optional[Path] = None)

Request to get the overload status for all or some entities.

Warning

This message is intended to be used internally between the hybrid controller and the carrier boards. As user, refer to the RunStateChangeMessage.run_flags field.

entities: Optional[Path]

Optional path prefix to select only a subset of entities

class pybrid.redac.protocol.messages.GetOverloadResponse(*, entities: list[pybrid.redac.entities.Path])

A response to a prior GetOverloadRequest, containing all overloaded entities matching the requested prefix.

Warning

This message is intended to be used internally between the hybrid controller and the carrier boards. As user, refer to the RunStateChangeMessage.run_flags field.

entities: list[pybrid.redac.entities.Path]

List of overloaded entities.

response_for

alias of GetOverloadRequest

Other Messages

Miscellaneous messages are documented below.

class pybrid.redac.protocol.messages.GetPartitionInformationRequest

Request to get the partition information from a proxy that is virtualizing (parts of) the machine.

class pybrid.redac.protocol.messages.GetPartitionInformationResponse(*, partition_mode: str, entities: Optional[List[List[str]]] = None)

A response to a prior GetPartitionInformationRequest, containing information about the partitioning of the device.

entities: Optional[List[List[str]]]

partition_mode: str

response_for

alias of GetPartitionInformationRequest

class pybrid.redac.protocol.messages.GetStatusRequest(*, recursive: bool = True)

recursive: bool

class pybrid.redac.protocol.messages.GetStatusResponse(*, status: dict)

response_for

alias of GetStatusRequest

status: dict

class pybrid.redac.protocol.messages.HackRequest(*, command: str, data: Any = None)

A message which may contain an arbitrary command and data, intended for development purposes only!

command: str

data: Any

class pybrid.redac.protocol.messages.HackResponse(*, data: Any = None)

data: Any

response_for

alias of HackRequest

class pybrid.redac.protocol.messages.Notification

Base class for notifications.

Notifications are messages that expect no response.

static get_class_for_type_identifier(type_)

class pybrid.redac.protocol.messages.PingRequest(*, now: datetime = None)

A heartbeat request to check for controller status. The controller replies with a PingResponse message.

now: datetime

A timestamp used to synchronize client and controller clocks.

class pybrid.redac.protocol.messages.PingResponse(*, now: datetime = None)

A heartbeat response to an incoming PingRequest message.

now: datetime

A timestamp used to synchronize client and controller clocks. The controller returns its timestamp so the client can check if it was applied correctly.

response_for

alias of PingRequest

class pybrid.redac.protocol.messages.SetStandbyRequest(*, standby: bool = False, hack_pwm_ramp: Optional[bool] = False)

hack_pwm_ramp: Optional[bool]

standby: bool

class pybrid.redac.protocol.messages.SetStandbyResponse

response_for

alias of SetStandbyRequest

class pybrid.redac.protocol.messages.SysRebootRequest

class pybrid.redac.protocol.messages.SysRebootResponse

response_for

alias of SysRebootRequest

class pybrid.redac.protocol.messages.SysTemperaturesRequest

class pybrid.redac.protocol.messages.SysTemperaturesResponse(*, entities: dict)

entities: dict

response_for

alias of SysTemperaturesRequest