dataclerk

A fast, reliable, small data remote logging tool with a very small footprint. dataclerk is similar to a log file, but you can send it messages over the web.

If Apache Kafka is too bulky for you, consider dataclerk.

It's intended to be a zero-fuss solution that will easily get out of your way when you outgrow it.

Example Session

To start, we create a server that's listening locally:

 $ dataclerk localhost:4499 iot-logs.sqlite

Its primary user interface is curl (or perhaps httpie). Assuming a dataclerk server is live at clerk.example.com:4499, we're able to register channels and begin to record entries:

  $ curl -X PUT localhost:4499/v1/channel/mesh
  $ curl localhost:4499/+/mesh -d unit=borg-h1a42 -d status=ok

On the backend, a new table has been created within Sqlite database at iot-logs.sqlite called "mesh". The second line has created a new row within that within that table:

• id

1

• created_at

2019-04-09 08:37:42

• uuid

641c8210-5aa2-11e9-8b61-674d210d2b16

• data

{"unit":"borg-h1a42","status":"ok"}

Adding data to dataclerk should be very fast. You should expect a response within 10ms, although the system may be constrained by how long things take to be stored onto physical storage media. You can see from the logs generated from this session that that we used around 3-4ms to make changes:

 $ dataclerk localhost:4499 iot-logs.sqlite
 [2019-04-09T09:04:48Z INFO  dataclerk] Hello!
 [2019-04-09T09:04:48Z INFO  actix_server::builder] Starting 12 workers
 [2019-04-09T09:04:48Z INFO  actix_server::builder] Starting server on 127.0.0.1:4499
 [2019-04-09T09:05:14Z INFO  dataclerk] registering channel "mesh"
 [2019-04-09T09:05:14Z INFO  actix_web::middleware::logger]  "PUT /v1/channel/mesh HTTP/1.1" 201 0 "-" "curl/7.61.0" 0.033640
 [2019-04-09T09:05:46Z DEBUG dataclerk] recv: channel:"mesh", data: {"status": "ok", "unit": "borg-h1a42"}
 [2019-04-09T09:05:46Z INFO  actix_web::middleware::logger]  "POST /+/mesh HTTP/1.1" 204 0 "-" "curl/7.61.0" 0.04191
 ^C[2019-04-09T09:07:53Z INFO  actix_server::builder] SIGINT received, exiting
 [2019-04-09T09:07:53Z INFO  dataclerk] Goodbye