You probably need to define "fails" for me. I've never had a VPS straight up "fail" before, as in a hardware level, can't access the data, dead. You may temporarily lose access to the data, but I've always been able to recover said data.
It’s going to have much lower write throughput, since SQLite is single-writer and on top of that you need to do Raft consensus. TiDB and CockroachDB can handle concurrent writes easily. Cockroach runs raft per “range” of 128mb of the key space, I’m not as familiar with TiDB. Vitess is an orchestration layer over MySQL, and MySQL handles concurrent writes easily.
That's correct, there is a write-performance hit for the reasons you say. All Raft systems will take the same hit, and SQLite is intrinsically single-writer -- nothing about rqlite changes that[2]. That said, there are approaches to increasing write-performance substantially. See [1] for much more information.
Write-performance is not the only thing to consider though (assuming one has sufficient performance in that dimension). Ease of deployment and operation are also important, and that's an area in which rqlite excels[3] (at least I think so, but I'm biased).
Oh, I also presented some performance numbers in a presentation to a CMU a couple of years back. A little out-of-date, but gives a order-of-magnitude sense. https://youtu.be/JLlIAWjvHxM?t=2690
Relevant to the original inquiry — I really admire that you bring up the etcd and consul comparison right up front in the readme. For my own comprehension at least, it makes obvious the type of workloads for which you're optimizing and I appreciate that context as a past user of both of those stacks.
Depending on what you’re using these tools for. If you want a locking manager and some meta data storage to help your distributed system maintain state, etcd is better for the job than rqlite for that. It’s a better zookeeper. With etcd you can hold a lock and defer unlocking if the connection is disrupted. Rqlite is not a good option for this.
Agreed, in the sense that while rqlite has a lot in common with etcd (and Consul too -- Consul and rqlite share the same Raft implementation[1]) rqlite's primary use case is not about making it easy to build other distributed systems on top of it.
Every time I've looked at rqlite, it just falls short features-wise in what I would want to do with it. A single raft group does not scale horizontally, so to me rqlite is a toy rather than a tool worth using (because someone might mistake the toy as production grade software).
That's clearly a mistaken attitude because both Consul and etcd also use a single "Raft group" and they are production-grade software.
Ruling out a piece of software simply because it doesn't "scale horizontally" (and only writes don't scale horizontally in practice) is a naive attitude.
The qualifier here is for /my/ use cases. However I couldn't recommend rqlite over better options at the level of scale that it can fill.
One of the problems is if you're working with developers, the log replication contents is the queries, instead of the sqlite WAL like in dqlite. I know this is a work around to integrate mattn/sqlite3, but it's untenable in enterprise applications where developers are going to just think "oh, I can do sqlite stuff!". This is a footgun that someone will inevitably trigger at some point if rqlite is in their infrastructure for anything substantial. In enterprise, it's plainly untenable.
Another issue is if I want to architect a system around rqlite, it wont be "consistent" with rqlite alone. The client must operate the transaction and get feedback from the system, which you can not do with an HTTP API the way you've implemented it. There was a post today where you can observe that with the jetcd library against etcd. Furthermore to this point, you can't even design a consistent system around rqlite alone because you can't use it as a locking service. If I want locks, I end up deploying etcd, consul, or zookeeper anyways.
If I had to choose a distributed database with schema support right now for a small scale operation, it would probably be yugabyte or cockroachdb. They're simply better at doing what rqlite is trying to do.
At the end of the day, the type of people needing to do data replication also need to distribute their data. They need a more robust design and better safety guarantees than rqlite can offer today. This is literally the reason one of my own projects has been in the prototyping stage for nearly 10 years now. If building a reliable database was as easy as integrating sqlite with a raft library, I would have shipped nearly 10 years ago. Unfortunately, I'm still testing non-conventional implementations to guarantee safety before I go sharing something that people are going to put their valuable data into.
To simply say I'm "ruling out a piece of software because it doesn't scale horizontally" is incorrect. The software lacks designs and features required for the audience you probably want to use it.
Hopefully you find my thoughts helpful in understanding where I'm coming from with the context I've shared.
>One of the problems is if you're working with developers, the log replication contents is the queries, instead of the sqlite WAL like in dqlite.
I think you mean rqlite does "statement-based replication"? Yes, that is correct, it has its drawbacks, and is clearly called out in the docs[1].
>Another issue is if I want to architect a system around rqlite, it wont be "consistent" with rqlite alone. The client must operate the transaction and get feedback from the system, which you can not do with an HTTP API the way you've implemented it.
I don't understand this statement. rqlite docs are quite clear about the types of transactions it supports. It doesn't support traditional transactions because of the nature of the HTTP API (though that could be addressed).
>Furthermore to this point, you can't even design a consistent system around rqlite alone because you can't use it as a locking service. If I want locks, I end up deploying etcd, consul, or zookeeper anyways.
rqlite is not about allowing developers build consistent systems on top of it. That's not its use case. It's highly-available, fault-tolerant store, the aims for ease-of-use and ease-of-operation -- and aims to do what it does do very well.
>If I had to choose a distributed database with schema support right now for a small scale operation, it would probably be yugabyte or cockroachdb. They're simply better at doing what rqlite is trying to do.
Of course, you should always pick the database that meets your needs.
>If building a reliable database was as easy as integrating sqlite with a raft library, I would have shipped nearly 10 years ago.
Who said it was easy? It's taken almost 10 years of programming to get to the level of maturity it's at today.
>They need a more robust design and better safety guarantees than rqlite can offer today.
That is an assertion without any evidence. What are the safety issues with rqlite within the context of its design goals and scope? I would very much like to know so I can address them. Quality is very important to me.
This seems like a lack of knowledge issue. The problems with rqlite are inherit in it's design as I've already articulated. You can literally start reading jepsen analyses right now and understand it if you don't already: https://jepsen.io/analyses
"Evidence Dump Fallacy." This fallacy occurs when a person claims that a certain proposition is true but, instead of providing clear and specific evidence to support the claim, directs the questioner to a large amount of information, asserting that the evidence is contained within.
rqlite -- to the best of my knowledge and as a result of extensive testing -- offers strict linearizability due to its use of the Raft protocol. Each write request to rqlite is atomic because it's encapsulated in a single Raft log entry -- this is distinct from the other form of transactions offered by rqlite[1], but that second form of transaction functionality has zero effect on the guarantees offered by Raft and rqlite (they are completely different things, operating at different levels in the design). If you know otherwise I'd very much like to know precisely why and how.
I won't be following up further. I've shared all I have to share on this topic. On a personal level, I'm actually disappointed in how you take to critical feedback about your product and don't seem to be interested in understanding the problem domain you're developing for.
no malice intended, but I will take my cloud provider's managed database service.. for which we get support, SOC2 compliance, years of proven stability, no major approval needed by my organization, and budget already approved - rather than jump on the latest 'kubernetes is eating the world' fad.
As the founder and CEO of a two-year-old startup, seeking certainty in the direction amidst uncertainty (determining which products can bring customers and profits) should become my instinct.
See some development frameworks for local-first apps support both SQLite and PostgreSQL. The advantage of using PostgreSQL is that when you add a cloud option to your local-first app, the migration becomes much easier.
I bet that all message queues and log databases will support S3, as these types of data generally have a large volume and aren't as economically valuable (don't get me wrong, what I mean is that these databases won't be frequently read and processed).
Can't agree more! S3 will be the modern data storage primitive. Also, the move towards shared storage and separating compute from storage is a key trend in cloud-native architecture, enhancing scalability and cost-efficiency.
That's a giant stretch. Kraft is a surname. I assumed it was an April fools joke that Kraft the cheese company was making their own cloud (which would be hilarious).
I wonder whether we could get Kraft to sponsor us :) . Unikraft used to be called Unicraft but was changed to a "k" for name clash reasons. Agree that names are hard.
I caught on to that immediately. My statement was meant to mean that it would so so unlikely to indicate the Raft protocol, that it’s more likely to have been a reference to these other things. It is not a reference to those things, but it even less of a reference to Raft.
The parent is downvoted because the original comment contained some stretch logic talking about possible confusion with Raft consensus protocol due to naming similarities, the current comment is after the ninja edit.
