Hello, I’m trying to figure out what would be the best way to create a nice data pipeline. Right now the system uses timescale db. Everything is fine BUT it is no realtime. The reason is, one process is connected to a websocket, it inserts data in chunks of 5000 events. To insert them they must be enriched from an other table. So begote inserting the chunk the table from timescale is fatched, resampled so that the timestamps matched and then inserted to the final destination . The delay isn’t that big (about one minute). The question is more of more engineering curiosity if there may be a better maybe some design pattern or some fancy big data lake technology or whatever:) looking forward to any hint!