Scratch Data is a wrapper that lets you stream data into and out of your analytics database. It takes arbitrary JSON as input and lets you perform analytical queries.
Clone the repo:
$ git clone [email protected]:scratchdata/scratchdata.git $ cd scratchdata
Start the service:
$ go run . With no configuration, this will automatically set up a local DuckDB database ready for reading and writing.
Create a config.yaml file with all of your settings and run:
$ go run . config.yaml$ curl -X POST "http://localhost:8080/api/data/insert/events?api_key=local" \ --data '{"user": "alice", "event": "click"}'
The "events" table and columns are automatically created.
curl -G "http://localhost:8080/api/data/query" \ --data-urlencode "api_key=local" \ --data-urlencode "query=select * from events"
You can share data as CSV or JSON by creating "share links".
$ curl -X POST "http://localhost:8080/api/data/query/share?api_key=local" \ --data '{"query": "select * from events", "duration": 120}'
This will produce a query ID that expires in 120 seconds. From there, send the following link to users:
http://localhost:8080/share/<query_id>/data.csv
http://localhost:8080/share/<query_id>/data.json
You can set up multiple databases and copy data between them. You can run a SQL query against your source database and Scratch will automatically create a table and insert data into a destination.
$ curl -X POST "http://localhost:8080/api/data/copy?api_key=local" \ --data '{"query": "select * from events", "destination_id": 3, "destination_table": "events"}'
To see the full list of options, look at: https://docs.scratchdata.com