LUCIDAC emulation

The LUCIDAC emulation provides python code to mimic the behaviour of a network-enabled LUCIDAC. That is, any kind of LUCIDAC client can connect to the TCP/IP service provided by the class Emulation and the software does best to try to emulate a “mockup”, “virtualized” or “digital twin” version of the real hardware.

Focus is put on the run simulation which of course makes use of the Circuit simulation code. Therefore, in this usage of words, simulation is part of the extended concept of emulation which also takes into account the JSONL network protocol API. This protocol API is another way to ensure the computer model is really constrained to the capabilities of the computer. For instance, while the Simulation class can in principle simulate an unlimited amount of Routes (and therefore a fully connected system matrix), the protocol ensures that the Emulation receives only the sparse system matrix configuration.

How to start the emulator

An easy way to start the server is for instance by making up a script,

#!/usr/bin/env python
from lucipy import Emulation
Emulation().serve_forever()

This can be easily adopted, for instance with this advanced version

#!/usr/bin/env python
import sys
from lucipy import Emulation
Emulation(bind_addr="0.0.0.0", bind_port=int(sys.argv[1])).serve_forever()

This version can be called via ./start-server.py 1234 in order to listen on all interfaces on port 1234

General Features

• Network transparent in the same way as the actual LUCIDAC is. Therefore any kind of client should be able to connect.

• Multiprocessing “non-blocking” forking version is readily available, h owever in this case currently each client sees his own emulated and independent LUCIDAC. By default the server is “blocking” the same way as early LUCIDAC firmware versions used to do.

Known limitations

• Currently emulates only REV0 LUCIDAC hardware with one MInt and one MMul block.

• Very limited support for ACL IN/OUT and ADC IN/OUT. Basically only query based plotting (data aquisition) is supported.

• Only a subset of queries is supported. Call help in order to get a listing. In particular no calls with respect to administration/device configuration, login/logout, etc are supported for obvious reasons.

• The emulator only speaks the JSONL protocol. If you want to do something more advanced such as Websockets and the GUI, you can proxy this service by Lucigo (see there also for alternatives).

API Reference

class lucipy.simulator.Emulation(bind_addr='127.0.0.1', bind_port=5732, emulated_mac='70-79-74-68-6f-6e', debug=False)

A super simple LUCIDAC emulator. This class allows to start up a TCP/IP server which speaks part of the JSONL protocol and emulates the same way a LUCIDAC teensy would behave. It thus is a shim layer ontop of the Simulation class which gets a configuration in and returns numpy data out. The Emulation instead will make sure it behaves as close as possible to a real LUCIDAC over TCP/IP.

In good RPC fashion, methods are exposed via a tiny registry and marked @expose.

The emulation is very superficial. The focus is on getting the configuration in and some run data which allows for easy developing new clients, debugging, etc. without a real LUCIDAC involved.

Please refer to the documentation for a high level introduction.

Note

The error messages and codes returned by this emulator do not (yet) coincide with the error messages and codes from the real device.

Note

Since the overall code does not use asyncio as a philosophy, also this code is written as a very traditional forking server. In our low-volume practice, there should be no noticable performance penalty.

If you choose a forking server, the server can handle multiple clients a time and is not “blocking” (the same way as the early real firmware embedded servers were). However, the “parallel server” in a forking (=multiprocessing) model also means that each client gets its own virtualized LUCIDAC due to the multiprocess nature (no shared address space and thus no shared LUCIDAC memory model) of the server.

There are three usage modes of this class:

• Directly using the emulator methods

• Connection from LUCIDAC over emulated socket

• Connection from any LUCIDAC JSONL client over TCP socket

Direct usage means for instance

>>> emu = Emulation()
>>> print(emu.get_entities()) 
{'entities': {'70-79-74-68-6f-6e': {'/0': {'/M0': ...
>>> emu.get_circuit()         
{'entity': None, 'config': {'adc_channels': [], ...

Given the nature of the JSONL protocol, this usage is somewhat as using LUCIDAC, in particular for setting/getting the circuit and starting a run. Note that this way, no JSON encoding takes place. This mode of operation is primarily useful for unit testing but otherwise does not fulfill the idea of the interface emulation.

The emulated socket usage is like

>>> from lucipy import LUCIDAC
>>> hc = LUCIDAC("emu:/")
>>> hc.get_entities()  
{'70-79-74-68-6f-6e': {'/0': {'/M0': {'class': 2, ...

Note that with this special endpoint URL syntax, even the socket is emulated and no TCP/IP connection is made. Again, this is ideal for unit testing because there is no headache with concurrently testing a server and a client. However, in this mode of operation the control flow remains at the client side and the event loop of the emulator is never triggered. This means that by practice there cannot happen real deviation from a simple query-response principle.

The actual intended TCP/IP server usage is like

>>> emu = Emulation(bind_port=0)
>>> proc = emu.serve_forking()
>>> endpoint = emu.endpoint()
>>> endpoint               
'tcp://127.0.0.1:...'
>>> hc = LUCIDAC(endpoint)
>>> hc.get_entities()  
{'70-79-74-68-6f-6e': {'/0': {'/M0': {'class': 2, ...
>>> hc.close()
>>> proc.terminate() # stops the server process

Note that by the nature of TCP/IP networking, the server can listen also at any other interface and thus can be reached from other computers and processes. In contrast, the previous usage examples were limited to the same python instance, they did not involve real networking.

default_emulated_mac = '70-79-74-68-6f-6e'

The string ‘python’ encoded as Mac address 70-79-74-68-6f-6e just for fun

get_entities()

Just returns the standard LUCIDAC REV0 entities with the custom MAC address.

micros()

Returns microseconds since initialization, mimics microcontroller uptime

ping()

Emulates the ping behaviour (approximatively)

reset()

Resets the circuit configuration. As this models a LUCIDAC, the configuration holds a single cluster and some parts (but currently we don’t emulate front panel and friends).

reset_circuit()

Alias: Reset circuit configuration

get_circuit()

Read out circuit configuration

get_config()

Alias: Read out circuit configuration

set_circuit(entity, config, reset_before=False, sh_kludge=None, calibrate_mblock=None, calibrate_offset=None, calibrate_routes=None)

Set circuit configuration.

Somewhat ironically, as in real LUCIDAC, this function does not comply if you try to set some nonexisting element. In the emulator, it just tries to update the local configuration which does not distinguish between entities and elements, anyway.

Parameters

• entity – A list such as [“AA-BB-CC-DD-EE-FF”, “0”, “/U”], i.e. the path to the entity. As the real LUCIDAC, we reject wrong carrier messages.

• config – The configuration to apply to the entity.

default_run_config = {'halt_on_external_trigger': False, 'halt_on_overload': False, 'ic_time': 123456, 'op_time': 123456}

default_daq_config = {'num_channels': 0, 'sample_op': True, 'sample_op_end': True, 'sample_rate': 500000}

start_run(**start_run_msg)

Emulate an actual run with the LUCIDAC Run queue and FlexIO data aquisition. This will return the ADC measurements on the requested sampling points. There are no constraints for the sampling rate, in contrast to real LUCIDAC.

This function does it all in one rush “in sync” , no need for a dedicated queue. Internally, it just prepares all envelopes and sends them out then alltogether.

Current limitation: The emulator cannot make use of ACL_IN/OUT, i.e. the frontpanel analog inputs and outputs.

Should react on a message such as the following:

example_start_run_message = {
'id': '417ebb51-40b4-4afe-81ce-277bb9d162eb',
'session': None,
'config': {
    'halt_on_external_trigger': False, # will ignore
    'halt_on_overload': True,          # 
    'ic_time': 123456,                 # will ignore
    'op_time': 234567                  # most important, determines simulation time
},
'daq_config': {
    'num_channels': 0,                 # should obey
    'sample_op': True,                 # will ignore
    'sample_op_end': True,             # will ignore
    'sample_rate': 500000              # will ignore
}}

help()

exposed_methods()

Returns a dictionary of exposed methods with string key names and callables as values

handle_request(line, return_always_list=False)

Handles incoming JSONL encoded envelope and respons with a string encoded JSONL envelope.

Parameters

• line – String encoded JSONL input envelope

• return_always_list – Returns always a list of strings

Returns

String encoded JSONL single envelope. If out-of-bound messages are generated, will return a list of such strings.

serve_forking()

Starts TCP server in a seperate process. Furthermore, the TCP Server will fork for each incoming connection, thus allowing multiple clients to connect at the same time. Since this distributed memory model cannot exchange information, each client sees its own version of a LUCIDAC.

Returns a multiprocessing.Process class which can be asked for the process id, waited for, etc. Example:

proc = emu.serve_forking()
print(f"Waiting for server {emu.endpoint()} process to finish, can also do other work")
proc.join()

serve_threading()

Starts TCP server in a seperate thread. Furthermore, the TCP Server will reply to each incoming connection in a seperate thread. Theoretically this could give each client a view to a common hardware memory model (not taking into account locking etc.)

Yields the running thread from a context manager, thus allows to continue work, also with the thread as well as the Emulation object in main thread.

emu = Emulation(bind_addr="0.0.0.0", bind_port=0)
thread = next(emu.serve_threading())
print(f"Waiting for server {emu.endpoint()} thread to finish, can also do other work")
thread.join()

Warning

There is still some bug here and the server started within a thread never responds.

serve_blocking()

Starts TCP server in main thread. This hands over control to the socket server event queue. Only one client can connect at a time. The function will never return except user interaction.

endpoint()

Determines endpoint URL if some server has been started. Endpoints are Strings. If a server has been started, this returns the actual Port assigned if port 0 was requested.