> One thing that needs to be emphasized with “durable execution” engines is they don’t actually get you out of having to handle errors, rollbacks, etc.
I think this is a gross misrepresentation of what durable executions are. DEs were never expected to magically eliminate the need to handle errors. What DEs do is provide an high-level abstraction of the same pattern that is recurrent on all workflow engines, and they provide a simpler way for developers to implement rollback and compensation steps when workflows fail.
If you are designing and implementing a transaction with a DE, you still need to design and implement a transaction. DEs simplify much of the logic, but you still need to design and implement a transaction. There is no silver bullet.
> Even the canonical examples everyone uses - so you’re using a DE engine to restart a sales transaction, but the part of that transaction that failed was “charging the customer” - did it fail before or after the charge went through? (...)
That's immaterial to the discussion on DEs. You, as a software engineer, still need to design and implement a transaction. DEs greatly simplify your job, but you still need to analyze failure modes and perform the necessary compensation steps.
> All of these problems are tractable, but once you’ve solved them - once you’ve built sufficient atomicity into your system to handle the actual failure cases - the benefits of taking on the complexity of a DE system are substantially lower than the marketing pitch.
I completely disagree, but you do you. Some durable execution engines greatly simplify tracking state and implementing activities and rollback logic. Some cloud providers even provide services that allow you to implement long-running workflows with function-as-a-service components that provide out-of-the-box support for manual approvals. If you feel you are better off rolling your own support, good for you. Meanwhile, everyone around you is delivering the same value with much less work.
I think this is a gross misrepresentation of what durable executions are. DEs were never expected to magically eliminate the need to handle errors. What DEs do is provide an high-level abstraction of the same pattern that is recurrent on all workflow engines, and they provide a simpler way for developers to implement rollback and compensation steps when workflows fail.
If you are designing and implementing a transaction with a DE, you still need to design and implement a transaction. DEs simplify much of the logic, but you still need to design and implement a transaction. There is no silver bullet.
> Even the canonical examples everyone uses - so you’re using a DE engine to restart a sales transaction, but the part of that transaction that failed was “charging the customer” - did it fail before or after the charge went through? (...)
That's immaterial to the discussion on DEs. You, as a software engineer, still need to design and implement a transaction. DEs greatly simplify your job, but you still need to analyze failure modes and perform the necessary compensation steps.
> All of these problems are tractable, but once you’ve solved them - once you’ve built sufficient atomicity into your system to handle the actual failure cases - the benefits of taking on the complexity of a DE system are substantially lower than the marketing pitch.
I completely disagree, but you do you. Some durable execution engines greatly simplify tracking state and implementing activities and rollback logic. Some cloud providers even provide services that allow you to implement long-running workflows with function-as-a-service components that provide out-of-the-box support for manual approvals. If you feel you are better off rolling your own support, good for you. Meanwhile, everyone around you is delivering the same value with much less work.