The part about building on top of Postgres… the author does talk about something similar to pub sub, not abusing a table to sort tasks by create_date, correct?
Because I’ve built my own “task queue” on top of “SELECT * FROM tasks WHERE done = 0 ORDER BY create_date LIMIT 1;” and have also seen others do it. Just don’t do it like this. You’ll get spikes of tasks and then your table will be filled faster than your worker can finish the tasks. Just use something else.
Yeah so we didn’t do this, we used Redis and never really had any performance issues at that layer. You’re right that it’s A Bit More Complicated Than This to use Postgres, but it’s still tractable within a few days for someone who knows Postgres (more than just basic SQL). Maybe a little longer if you need high throughput.
Because I’ve built my own “task queue” on top of “SELECT * FROM tasks WHERE done = 0 ORDER BY create_date LIMIT 1;” and have also seen others do it. Just don’t do it like this. You’ll get spikes of tasks and then your table will be filled faster than your worker can finish the tasks. Just use something else.