The problem with sqlite for this is that as you write more and rows into a table then the writes get progressively slower -- it simply isn't able to do O(1) writes over a long period of time so you'd have to rotate the database files themselves.
I've been solving this problem by writing a non-pretty printed JSON blob to one line in a log file. Structured, but still text and very easy to stream. I build a separate binary index that gives more efficient time based lookups, but it isn't needed to understand and process the log file.
No. It's in a monitoring system we've been working on, but the principle is pretty simple. We may well end up open sourcing quite the rest of it sometime in the future -- a lot of the code already is.
I've been solving this problem by writing a non-pretty printed JSON blob to one line in a log file. Structured, but still text and very easy to stream. I build a separate binary index that gives more efficient time based lookups, but it isn't needed to understand and process the log file.