Using Multiple Data Sources, Filters, and Sinks
===============================================

This recipe demonstrates a more complicated scenario where multiple data sources are used. The data
is manipulated using filters, and then sent to three different destinations.

A PicoTech TC-08 thermocouple data logger is used to monitor several temperatures. Additionally, an
Arduino interfaces with `commonly available <https://www.adafruit.com/product/381>`_ `DS18B20
<https://datasheets.maximintegrated.com/en/ds/DS18B20.pdf>`__ temperature sensors and a fluid flow
sensor (such as the `YF-S401, YF-S402B <https://www.google.com/search?q=yf-s401+yf-s402b>`__ etc).
Data filters are used to calibrate/correct the sensor values, and finally to compute power
dissipation into the cooling water.

Data is timestamped, converted to CSV, and saved to files. It is also displayed on the console and
sent to a remote InfluxDB instance.

Hardware
--------

The PicoTech TC-08 is connected via USB. The appropriate drivers and interface library
(``libusbtc08.so`` on Linux, ``libusbtc08.dll`` on Windows) must be installed. Six type-K
thermocouples are connected to ports 1, 2, 4, 5, 7, 8.

For the Arduino, two temperature sensors are required, one each for the inlet and outlet water
temperatures. The flow sensor should be connected in line, preferably on the cold side. The
calculations need to convert the pulse counts from the flow sensor to a fluid flow rate. Combined
with the temperature difference of the water, the power dissipated into the liquid can be
determined.

.. container:: toggle

    .. container:: header

        Get the `Arduino code from GitLab
        <https://gitlab.com/ptapping/datalogd/-/tree/master/arduino>`__ or copy
        and paste below into a new Arduino sketch. Upload it to your Arduino.

    .. literalinclude:: ../../../arduino/datalog/datalog.ino
        :language: c++
        :caption: ``datalog.ino``

Ensure the appropriate sensors are selected in the Arduino code using the define statements, for
example:

    .. code-block:: c

        #define USE_DIGITAL_PINS false
        #define USE_ANALOG_PINS false
        #define USE_DS18B20_TEMPERATURE true
        #define USE_BH1750_LUX false
        #define USE_COUNTER true

Connect the thermometer's VCC to +5 V, GND to ground, and DATA to pin 12 (or another, to match that
specified in the code for the 1-Wire bus). You may need a pullup resistor between the VCC and DATA
pins if your thermometer modules don't include one. Connect the flow meter's VCC to +5 V, GND to
ground, and sense wire to pin 7 (or another interrupt-enabled pin, to match that specified in the
code for the pulse counter). Plug the Arduino into your computer USB cable.

Recipe
------

..  literalinclude:: ../../../recipes/serial_tc08_csv_file_influxdb.config
  :language:  none
  :caption: ``recipes/serial_tc08_csv_file_influxdb.config``

.. image:: images/serial_tc08_csv_file_influxdb.svg
  :align: center

The data input and processing steps are:

  * Thermocouple readings are taken from the TC-08 using the
    :class:`~datalogd.plugins.picotc08_datasource.PicoTC08DataSource` data source. The parameters
    specify the sampling interval (in seconds) and the types of probes attached. Each probe is
    specified with the channel it is attached to, a label, thermocouple type, and desired units.
  * The serial data is read in from the Arduino using the
    :class:`~datalogd.plugins.serial_datasource.SerialDataSource` data source. The data contains the
    temperatures from the thermometers and the pulse counter's number of pulses per second. The IDs
    of the temperatures correspond to the serial numbers of the thermometer devices.
  * Timestamps are added to both sources of data using two instances of
    :class:`~datalogd.plugins.timestamp_datafilter.TimeStampDataFilter`.
  * Next, the pulse counter's pulses-per-second is converted to a flow rate in litres-per-minute
    using the
    :class:`~datalogd.plugins.flowsensorcalibration_datafilter.FlowSensorCalibrationDataFilter`. The
    parameters are used in a calibration curve and have been experimentally determined for a YF-S401
    flow sensor.
  * A simple offset is applied to one of the thermometers to correct for a slight difference in
    readings between the two thermometers. This uses a
    :class:`~datalogd.plugins.polynomialfunction_datafilter.PolynomialFunctionDataFilter` to
    subtract 0.44 ℃ from the reading.

The data flow is then split to several separate destinations:

  * The data is formatted into a row of comma-separated values and written to a file. This uses a
    :class:`~datalogd.plugins.csv_datafilter.CSVDataFilter` feeding in to a
    :class:`~datalogd.plugins.file_datasink.FileDataSink`. There is a separate file for each of the
    two data sources.
  * The raw data is displayed on the console using a
    :class:`~datalogd.plugins.logging_datasink.LoggingDataSink`.
  * The raw data is sent to a InfluxDB database instance. Example database details have been used
    here, and will need to be changed to match your specific database.