Random Data to CSV File
=======================

This generates random data, adds a timestamp, formats as CSV, and writes to a
file. The default settings for the
:class:`~datalogd.plugins.file_datasink.FileDataSink` is to flush out to the file
every 10 seconds.

..  literalinclude:: ../../../recipes/randomwalk_timestamp_csv_file.config
  :language:  none
  :caption: ``recipes/randomwalk_timestamp_csv_file.config``

.. container:: toggle

  .. container:: header

    .. code-block:: none

      $ datalogd -c recipes/randomwalk_timestamp_csv_file.config

  .. code-block:: none

    INFO:main:Initialising DataLogDaemon.
    INFO:DataLogDaemon:Loaded config from: /etc/xdg/datalogd/datalogd.conf, recipes/randomwalk_timestamp_csv_file.config
    INFO:pluginlib:Loading plugins from standard library
    INFO:DataLogDaemon:Detected source plugins: NullDataSource, LibSensorsDataSource, RandomWalkDataSource, SerialDataSource
    INFO:DataLogDaemon:Detected filter plugins: NullDataFilter, AggregatorDataFilter, CSVDataFilter, KeyValDataFilter, TimeStampDataFilter
    INFO:DataLogDaemon:Detected sink plugins: NullDataSink, FileDataSink, InfluxDBDataSink, LoggingDataSink, MatplotlibDataSink, PrintDataSink
    INFO:DataLogDaemon:Initialising node a:RandomWalkDataSource()
    INFO:DataLogDaemon:Initialising node b:TimeStampDataFilter()
    INFO:DataLogDaemon:Initialising node d:CSVDataFilter(labels=None)
    INFO:DataLogDaemon:Initialising node s:FileDataSink(filename=randomwalk.csv)
    INFO:DataLogDaemon:Connecting a:RandomWalkDataSource -> b:TimeStampDataFilter
    INFO:DataLogDaemon:Connecting b:TimeStampDataFilter -> d:CSVDataFilter
    INFO:DataLogDaemon:Connecting d:CSVDataFilter -> s:FileDataSink
    INFO:main:Starting event loop.


.. container:: toggle

  .. container:: header

    .. code-block:: bash

      $ cat randomwalk.csv

  .. code-block:: none

    2020-04-17 22:02:51.887071+09:30,-1.0,2020-04-17 22:02:51.887071+09:30,-2.0
    2020-04-17 22:02:52.888208+09:30,0.0,2020-04-17 22:02:52.888208+09:30,0.0
    2020-04-17 22:02:53.889423+09:30,-1.0,2020-04-17 22:02:53.889423+09:30,-2.0
    2020-04-17 22:02:54.889818+09:30,-1.0,2020-04-17 22:02:54.889818+09:30,-4.0
    2020-04-17 22:02:55.891065+09:30,-2.0,2020-04-17 22:02:55.891065+09:30,-6.0
    2020-04-17 22:02:56.892261+09:30,-3.0,2020-04-17 22:02:56.892261+09:30,-8.0
    2020-04-17 22:02:57.893478+09:30,-4.0,2020-04-17 22:02:57.893478+09:30,-10.0
    2020-04-17 22:02:58.894673+09:30,-3.0,2020-04-17 22:02:58.894673+09:30,-12.0
    2020-04-17 22:02:59.895897+09:30,-2.0,2020-04-17 22:02:59.895897+09:30,-14.0
    ...

There are no labels for the column headers, and the file will grow infinitely
large with time. It might be better to aggregate the data and update the file
with only the latest data. Note the use of ``mode="w"`` and ``flush_interval=None``, which causes
the file to be opened, written, and closed on each reciept of data. In this way,
each block of aggregated data will overwrite the old one.

..  literalinclude:: ../../../recipes/randomwalk_timestamp_aggregator_csv_file.config
  :language:  none
  :caption: ``recipes/randomwalk_timestamp_aggregator_csv_file.config``

.. container:: toggle

  .. container:: header

    .. code-block:: none

      $ datalogd -c recipes/randomwalk_timestamp_aggregator_csv_file.config

  .. code-block:: none

    INFO:main:Initialising DataLogDaemon.
    INFO:DataLogDaemon:Loaded config from: /etc/xdg/datalogd/datalogd.conf, recipes/randomwalk_timestamp_aggregator_csv_file.config
    INFO:pluginlib:Loading plugins from standard library
    INFO:DataLogDaemon:Detected source plugins: NullDataSource, LibSensorsDataSource, RandomWalkDataSource, SerialDataSource
    INFO:DataLogDaemon:Detected filter plugins: NullDataFilter, AggregatorDataFilter, CSVDataFilter, KeyValDataFilter, TimeStampDataFilter
    INFO:DataLogDaemon:Detected sink plugins: NullDataSink, FileDataSink, InfluxDBDataSink, LoggingDataSink, MatplotlibDataSink, PrintDataSink
    INFO:DataLogDaemon:Initialising node a:RandomWalkDataSource()
    INFO:DataLogDaemon:Initialising node b:TimeStampDataFilter()
    INFO:DataLogDaemon:Initialising node c:AggregatorDataFilter(buffer_size=3600, send_every=30)
    INFO:DataLogDaemon:Initialising node d:CSVDataFilter()
    INFO:DataLogDaemon:Initialising node s:FileDataSink(filename=randomwalk.csv)
    INFO:DataLogDaemon:Connecting a:RandomWalkDataSource -> b:TimeStampDataFilter
    INFO:DataLogDaemon:Connecting b:TimeStampDataFilter -> c:AggregatorDataFilter
    INFO:DataLogDaemon:Connecting c:AggregatorDataFilter -> d:CSVDataFilter
    INFO:DataLogDaemon:Connecting d:CSVDataFilter -> s:FileDataSink
    INFO:main:Starting event loop.

.. container:: toggle

  .. container:: header

    .. code-block:: bash

      $ cat randomwalk.csv

  .. code-block:: none

    timestamp,analog_randomwalk0,timestamp,analog_randomwalk1
    2020-04-17 22:27:36.264577+09:30,1.0,2020-04-17 22:27:36.264577+09:30,0.0
    2020-04-17 22:27:37.265669+09:30,1.0,2020-04-17 22:27:37.265669+09:30,2.0
    2020-04-17 22:27:38.266249+09:30,2.0,2020-04-17 22:27:38.266249+09:30,0.0
    2020-04-17 22:27:39.267433+09:30,3.0,2020-04-17 22:27:39.267433+09:30,-2.0
    2020-04-17 22:27:40.268600+09:30,2.0,2020-04-17 22:27:40.268600+09:30,0.0
    2020-04-17 22:27:41.269554+09:30,2.0,2020-04-17 22:27:41.269554+09:30,0.0
    2020-04-17 22:27:42.270715+09:30,3.0,2020-04-17 22:27:42.270715+09:30,-2.0
    2020-04-17 22:27:43.271873+09:30,4.0,2020-04-17 22:27:43.271873+09:30,0.0
    2020-04-17 22:27:44.272887+09:30,4.0,2020-04-17 22:27:44.272887+09:30,0.0
    2020-04-17 22:27:45.274042+09:30,3.0,2020-04-17 22:27:45.274042+09:30,-2.0
    2020-04-17 22:27:46.275201+09:30,2.0,2020-04-17 22:27:46.275201+09:30,0.0
    2020-04-17 22:27:47.276220+09:30,1.0,2020-04-17 22:27:47.276220+09:30,0.0
    2020-04-17 22:27:48.277396+09:30,1.0,2020-04-17 22:27:48.277396+09:30,0.0
    2020-04-17 22:27:49.278583+09:30,0.0,2020-04-17 22:27:49.278583+09:30,-2.0
    2020-04-17 22:27:50.279763+09:30,1.0,2020-04-17 22:27:50.279763+09:30,-2.0
    2020-04-17 22:27:51.280956+09:30,0.0,2020-04-17 22:27:51.280956+09:30,-2.0
    2020-04-17 22:27:52.282120+09:30,0.0,2020-04-17 22:27:52.282120+09:30,-2.0
    2020-04-17 22:27:53.283198+09:30,0.0,2020-04-17 22:27:53.283198+09:30,-2.0
    2020-04-17 22:27:54.284388+09:30,1.0,2020-04-17 22:27:54.284388+09:30,-2.0
    2020-04-17 22:27:55.285719+09:30,0.0,2020-04-17 22:27:55.285719+09:30,-4.0
    2020-04-17 22:27:56.286249+09:30,0.0,2020-04-17 22:27:56.286249+09:30,-4.0
    2020-04-17 22:27:57.287438+09:30,-1.0,2020-04-17 22:27:57.287438+09:30,-6.0
    2020-04-17 22:27:58.288604+09:30,-2.0,2020-04-17 22:27:58.288604+09:30,-8.0
    2020-04-17 22:27:59.289765+09:30,-2.0,2020-04-17 22:27:59.289765+09:30,-10.0
    2020-04-17 22:28:00.290927+09:30,-1.0,2020-04-17 22:28:00.290927+09:30,-10.0
    2020-04-17 22:28:01.292095+09:30,0.0,2020-04-17 22:28:01.292095+09:30,-12.0
    2020-04-17 22:28:02.292911+09:30,0.0,2020-04-17 22:28:02.292911+09:30,-10.0
    2020-04-17 22:28:03.294073+09:30,0.0,2020-04-17 22:28:03.294073+09:30,-12.0
    2020-04-17 22:28:04.295262+09:30,-1.0,2020-04-17 22:28:04.295262+09:30,-12.0
    2020-04-17 22:28:05.296247+09:30,-2.0,2020-04-17 22:28:05.296247+09:30,-10.0
    ...