DBOPS - serialization between record notation and database rows.#

Requires python3, sqlite3

Uses a generic data table format for capturing any python record. Contains functionality for converting between database- and record type-data representation.

See usage in main section of dbops.py

Design#

The database has one table holding key-val data and some metadata needed for aggregation (session_id, data_id):

create table if not exists T1 (session_id data_id, key, val)

When inserting data records into the table, using rec_to_table, every record is split upon key-val pairs using the same data_id. So data_id can then be used to assemble the record again from k-v rows.

session_id is used for logging the same kind of data at different points in time. It is currently the same as the epoch timestamp.

Things to do#

Add update functionality providing a parent session_id reference (like in couchdb). In this way lineage can be tracked and data consistency can be managed. This could open up for e.g. branched versioning if desired. This would require extension of the database schema to also include columns parent_session_id and is_head.