.. _plugins:

Plugins
=======

This section describes how to create plugins, specify the connections between
them, and then run the complete data logging pipeline.

Plugins are python classes which extend one of the base plugin types. The class
can be defined in any python source code file, and multiple plugin classes may
be defined in a single file. The directory containing plugin source code can be
specified in a configuration file, or in command line parameters.

Data Sources
------------

Data source classes must extend :class:`~datalogd.DataSource`. In addition,
their class name must have the suffix "DataSource".

The following code is a simple example of a functional data source plugin. It
sends the string "Hello, World!" to any connected sinks once every 10 seconds:

.. literalinclude:: ../../plugin_demos/helloworld_datasource.py
  :caption: ``plugin_demos/helloworld_datasource.py``

Note the use of the :mod:`asyncio` event loop to schedule method calls, as
opposed to a separate loop/sleep thread or similar.

Data sources will have their :meth:`~datalogd.DataSource.close` method called when the application
is shutting down so that any resources (devices, files) used can be released.

Data Sinks
----------

Data sinks accept data using their :meth:`~datalogd.DataSink.receive` method,
and do something with it, such as committing it to a database. They must extend
:class:`~datalogd.DataSink`, and must use the suffix "DataSink".

A simple example data sink which prints received data in uppercase to the
console is:

.. literalinclude:: ../../plugin_demos/shout_datasink.py
  :caption: ``plugin_demos/shout_datasink.py``

Data sinks will have their :meth:`~datalogd.DataSink.close` method called when the application
is shutting down so that any resources (devices, files) used can be released.

Data Filters
------------

Data filters are in fact both :class:`~datalogd.DataSource`\ s and
:class:`~datalogd.DataSink`\ s, and thus share the functionality of both. They
must extend :class:`~datalogd.DataFilter`, and must use the suffix "DataFilter".

The :class:`~datalogd.NullDataFilter` is the most simple example of a filter,
which accepts data and passes it unchanged to any connected sinks. A slightly
more functional filter which performs some processing on the data is:

.. literalinclude:: ../../plugin_demos/helloworld_datafilter.py
  :caption: ``plugin_demos/helloworld_datafilter.py``

Connecting
----------

To connect the above source, filter and sink to a complete "Hello, World!"
data logger, the connection graph could be specified as:

.. code-block:: none

  digraph {
      source [class=HelloWorldDataSource];
      filter [class=HelloWorldDataFilter];
      sink   [class=ShoutDataSink];
      source -> filter -> sink;
  }

Running
-------

See the :ref:`commandline` section for details on specifying configuration from
the command line.

.. container:: toggle

  .. container:: header

    .. code-block:: none

      $ datalogd --plugindir plugin_demos --graph 'digraph{a[class=HelloWorldDataSource];b[class=HelloWorldDataFilter];c[class=ShoutDataSink];a->b->c;}'

  .. code-block:: none

    INFO:main:Initialising DataLogDaemon.
    INFO:DataLogDaemon:Loaded config from: /etc/xdg/datalogd/datalogd.conf
    INFO:pluginlib:Loading plugins from standard library
    INFO:pluginlib:Adding plugin_demos as a plugin search path
    INFO:DataLogDaemon:Detected source plugins: NullDataSource, LibSensorsDataSource, RandomWalkDataSource, SerialDataSource, HelloWorldDataSource
    INFO:DataLogDaemon:Detected filter plugins: NullDataFilter, AggregatorDataFilter, CSVDataFilter, KeyValDataFilter, TimeStampDataFilter, HelloWorldDataFilter
    INFO:DataLogDaemon:Detected sink plugins: NullDataSink, FileDataSink, InfluxDBDataSink, LoggingDataSink, MatplotlibDataSink, PrintDataSink, ShoutDataSink
    INFO:DataLogDaemon:Initialising node a:HelloWorldDataSource()
    INFO:DataLogDaemon:Initialising node b:HelloWorldDataFilter()
    INFO:DataLogDaemon:Initialising node c:ShoutDataSink()
    INFO:DataLogDaemon:Connecting a:HelloWorldDataSource -> b:HelloWorldDataFilter
    INFO:DataLogDaemon:Connecting b:HelloWorldDataFilter -> c:ShoutDataSink
    INFO:main:Starting event loop.
    GREETINGS, WORLD!
    GREETINGS, WORLD!
    GREETINGS, WORLD!
    ^CINFO:main:Stopping event loop.