> it is even more expensive/slow to have producers try to continuously anticipate the needs of all possible current and future consumers
I agree, but I don't think Taxi encourages that. (I sure hope it doesn't).
All that's really happening is defining a system-agnostic set of terms, embedding those in producer schemas, and then letting consumers using those same terms to ask for data.
Those terms can be used to automate integration, and in many (but not all) cases, alleviate the need for low-value plumbing code.
Semantic Metadata is a growing trend in the data ecosystem (cube.dev is doing a nice job here too). The Agile folks have had Ubiquitous Language as a first class concept since day dot.
> an approach will work only in a company below a certain size/complexity/age
I think that GraphQL federation is an example of mid-to-large companies trying to solve the federation problem, suggesting they both feel the pain, and are looking for solutions.
The goals of Taxi feel at least somewhat like 'creating a domain driven design style Ubiquitous Language to describe data and relationships' and the deliberate not-forcing-a-single-global-thing seems like it'll at least make it harder for enterprise dysfunction to bend it into having the failure modes the grandparent describes.
What happens when a consumer does not understand the producer’s domain and misuses the data? As we load metadata with semantics of their own, we just kick the can down the road. Metadata is itself semantics about data. In many regulated environments, as producer you are responsible for downstream aggregations etc. I think the idea works great in prototyping, but similar to data mesh, ownership is poorly defined in the presence of aggregate domains and it’s more tuned to operational point to point interfaces.
> What happens when a consumer does not understand the producer’s domain and misuses the data?
That's a problem with or without Taxi/Orbital.
Today, it's generally left to consumers to pick the fields that look right, and hope they get it right. That's a risky approach. And, by leaving it to consumers, you run the risk on every new integration.
I think by asking producers to annotate their attributes with a strongly defined semantic contract, you reduce the risk of consumers getting it wrong.
Producers understand their data much better than consumers, so are better informed on how to map their attributes to a set of semantic contracts that consumers can leverage.
I agree, but I don't think Taxi encourages that. (I sure hope it doesn't).
All that's really happening is defining a system-agnostic set of terms, embedding those in producer schemas, and then letting consumers using those same terms to ask for data.
Those terms can be used to automate integration, and in many (but not all) cases, alleviate the need for low-value plumbing code.
Semantic Metadata is a growing trend in the data ecosystem (cube.dev is doing a nice job here too). The Agile folks have had Ubiquitous Language as a first class concept since day dot.
> an approach will work only in a company below a certain size/complexity/age I think that GraphQL federation is an example of mid-to-large companies trying to solve the federation problem, suggesting they both feel the pain, and are looking for solutions.