Simplest way to load IoT timeseries data from local disk or S3

Hi all,

We’re looking to create a Grafana dashboard for the following use case:

  • We have a set of IoT devices in the field that upload parameters in a custom binary format to S3
  • We have a Python script running on a backend, which will convert the raw data to custom formats
  • We wish to display times series data of e.g. 3 parameters in Grafana in the simplest possible way

We believe Grafana is ideal for this, but we’re a bit in doubt on the data source. Our original thinking was to have the Python script output the processed data into e.g. sqllite or CSV. The Python script periodically checks for new data files on the server and then updates the output file with the new data.

We understand that we can probably set up a server to handle this - but this involves a number of extra steps that may create some complexity. We’re looking for the simplest way to achieve the above so that other end users can easily deploy a comparable solution.

Any thoughts are welcome.

If the only reasonable way is to set up a database, what would be the “simplest” solution that could work for our setup (i.e. where a Python script processes data periodically and would need to append this to e.g. a database)?