Bridging Data over Sockets

This example creates two separate instances of the datalogd process, and bridges a data connection between them using SocketDataFilters. In this case the running processes are on the same computer, but the connection can be made across a network connection by selecting the correct host addresses.

A socket connection must have one server and at least one client, so one instance is configured to start a server listening on an address (the local loopback address) and port, and the other instance is configured as a client to connect to that port. A

The server configuration is:

recipes/socket_server.config

[datalogd]
connection_graph =
  digraph {
    r [class=RandomWalkDataSource, walkers="[[100, 10]]"];
    s [class=SocketDataFilter, role="server", host="127.0.0.1", port=4567];
    l [class=LoggingDataSink];
    r -> s -> l;
  }

$ datalogd -c recipes/socket_server.config

INFO:main:Initialising DataLogDaemon.
INFO:DataLogDaemon:Loaded config from: recipes/socket_server.config
INFO:pluginlib:Loading plugins from standard library
INFO:DataLogDaemon:Detected source plugins: NullDataSource, PicoTC08DataSource, RandomWalkDataSource, SerialDataSource
INFO:DataLogDaemon:Detected filter plugins: NullDataFilter, AggregatorDataFilter, CoolingPowerDataFilter, CSVDataFilter, FlowSensorCalibrationDataFilter, JoinDataFilter, KeyValDataFilter, PolynomialFunctionDataFilter, SocketDataFilter, TimeStampDataFilter
INFO:DataLogDaemon:Detected sink plugins: NullDataSink, FileDataSink, InfluxDB2DataSink, LoggingDataSink, MatplotlibDataSink, PrintDataSink, PyqtgraphDataSink
INFO:DataLogDaemon:Initialising node r:RandomWalkDataSource(walkers=[[100, 10]])
INFO:DataLogDaemon:Initialising node s:SocketDataFilter(role=server, host=127.0.0.1, port=4567)
INFO:DataLogDaemon:Initialising node l:LoggingDataSink()
INFO:DataLogDaemon:Connecting r:RandomWalkDataSource -> s:SocketDataFilter
INFO:DataLogDaemon:Connecting s:SocketDataFilter -> l:LoggingDataSink
INFO:main:Starting event loop.
INFO:SocketDataFilter:Server started on ('127.0.0.1', 4567)

At the moment, nothing will happen, as no client(s) have connected. Open another terminal window and start a new instance using the client configuration:

recipes/socket_client.config

[datalogd]
connection_graph =
  digraph {
    r [class=RandomWalkDataSource, walkers="[[1.0, 0.1]]"];
    s [class=SocketDataFilter, role="client", host="127.0.0.1", port=4567];
    l [class=LoggingDataSink];
    r -> s -> l;
  }

$ datalogd -c recipes/socket_client.config

INFO:main:Initialising DataLogDaemon.
INFO:DataLogDaemon:Loaded config from: recipes/socket_client.config
INFO:pluginlib:Loading plugins from standard library
INFO:DataLogDaemon:Detected source plugins: NullDataSource, PicoTC08DataSource, RandomWalkDataSource, SerialDataSource
INFO:DataLogDaemon:Detected filter plugins: NullDataFilter, AggregatorDataFilter, CoolingPowerDataFilter, CSVDataFilter, FlowSensorCalibrationDataFilter, JoinDataFilter, KeyValDataFilter, PolynomialFunctionDataFilter, SocketDataFilter, TimeStampDataFilter
INFO:DataLogDaemon:Detected sink plugins: NullDataSink, FileDataSink, InfluxDB2DataSink, LoggingDataSink, MatplotlibDataSink, PrintDataSink, PyqtgraphDataSink
INFO:DataLogDaemon:Initialising node r:RandomWalkDataSource(walkers=[[1.0, 0.1]])
INFO:DataLogDaemon:Initialising node s:SocketDataFilter(role=client, host=127.0.0.1, port=4567)
INFO:DataLogDaemon:Initialising node l:LoggingDataSink()
INFO:DataLogDaemon:Connecting r:RandomWalkDataSource -> s:SocketDataFilter
INFO:DataLogDaemon:Connecting s:SocketDataFilter -> l:LoggingDataSink
INFO:main:Starting event loop.
INFO:SocketDataFilter:Connection established to ('127.0.0.1', 4567).
INFO:LoggingDataSink:Data received:
INFO:LoggingDataSink:  {'type': 'randomwalk', 'id': '0', 'value': 40.0}
INFO:LoggingDataSink:Data received:
INFO:LoggingDataSink:  {'type': 'randomwalk', 'id': '0', 'value': 50.0}
INFO:LoggingDataSink:Data received:
INFO:LoggingDataSink:  {'type': 'randomwalk', 'id': '0', 'value': 50.0}
INFO:LoggingDataSink:Data received:
INFO:LoggingDataSink:  {'type': 'randomwalk', 'id': '0', 'value': 40.0}

Similarly, on the server instance, you should see the data sent by the client being logged to the terminal window.