Talk · TDWI München 2026
Open Standards for Data Products
Dr. Simon Harrer (CEO & Co-Founder, Entropy Data) · June 23, 2026
A solo talk at TDWI München 2026, in three movements. Simon opens by teasing the future: he kicks off a coding agent to build a data product on its own, then lets it run in the background. He spends the middle of the talk on the stack of open standards that makes that possible (ODCS for data contracts, ODPS for data products, OSI for semantics) and the open-source tooling around them. By the end the agent is done, and the future turns out to be already here. The closing ask: help make history by getting these standards over the line.
Live at TDWI München 2026. The annotation below is an edited summary of the slides.
The Speaker
Simon Harrer describes himself as a software engineer at heart who moved into data about five years ago. He co-authored "Java by Comparison" (now part of several large language models' training data, as he notes with a grin), and he translated Zhamak Dehghani's "Data Mesh" into German. The German print edition is in full color, which no English original is.
Today he is co-founder and CEO of Entropy Data, a small startup building a data product marketplace and metadata layer, with customers worldwide, six people, and a lot of agents doing the work. He sits on the Technical Steering Committee of Bitol at the Linux Foundation, contributes to the Open Semantic Interchange (the standardization initiative Snowflake started), and builds open-source tooling around all of it.
He also maintains data-landscape.com, an open overview of the open standards in the data world. Worth a look, he says: it is all open and free.
The Plan
Three things. First, peek into the future for a bit of motivation: what could it look like? Then learn about standards, the open kind: why they exist and what they can do.
And last, make history together. Keep your phones ready and log into GitHub, Simon tells the room, because we are going to need it before the end.
A Glimpse Into the Future (Just a Teaser)
The framing question: what if we already were using open standards for data products, and really lived them? To answer it, Simon runs a live demo (with a backup recording ready, just in case). In one directory sit two files: a standard-conformant data contract describing the data he wants to offer, and an ODPS data product file describing the box in the architecture diagram, with its purpose, domain, output ports, and links into the semantic layer.
Then he fires the coding agent: claude --dangerously-skip-permissions, "make this data product a reality." He only wrote down what he wants. The agent figures out the rest, and the build takes about 15 minutes.
So he lets it run in the background, with the tokens working hard for us, and turns to the real subject of the talk. Hold this image. The payoff comes at the end.
First, What Is a Data Product?
The crowd at a data conference knows this, so Simon moves fast. A data product is product thinking applied to data: you optimize for your consumers, take ownership, have a team that stands behind it, and build interfaces you are accountable for. It is the second of the four Data Mesh principles.
The free, open-source Data Product Canvas is where you design one on the drawing board, in a workshop with people, starting from the use case. Simon calls the finished canvas a "birth certificate" for the product.
As a software architect at heart, he also sees a data product as an architectural unit: it takes things in through input ports, does a lot inside, and exposes results through output ports. In the opening demo he specified only the output port and the box, then let the coding agent loose on it.
And What Is a Standard?
The obligatory xkcd makes the case. We need standards to agree on something and cut complexity: the one correct date format is the ISO standard, and everything else is wrong. Same with counting down before a start, "3, 2, 1, go," so nobody is unsure whether the action begins on "one" or after it.
And the obligatory risk: there are 14 standards, a big player wants in, and now there are 15. It happens everywhere, the data world included. Useful when it cuts complexity, painful when it just adds another contender.
What Makes a Good Standard
Simon lays out four dimensions, and he wants the left side of each:
- De facto vs. de jure (usage): if everyone simply uses it, it is a standard in practice, even if no body declares it so.
- Initiative vs. vendor (owner): better when an initiative stands behind it, so the power does not sit with one company that can fully control it.
- Many vs. one (control): is the power distributed across contributors, or held by a single party?
- Open vs. paywall (cost): can anyone read and adopt it, or do you pay a body just to read the spec?
The standards worth betting on lean left on all four: used in the wild, community-owned, broadly governed, and open.
A Map of the Open Standards
Standards live everywhere in a data product: describing it, reading other products, processing, storage, scheduling, monitoring, observability. To see the field at once, Simon maintains data-landscape.com, an opinionated, interactive map of the metadata and data standards he rates adopt, situational, assess, or caution.
For this talk he focuses on three he is personally close to and can speak to honestly: ODCS, ODPS, and OSI. The others he knows only from the outside.
What Is a Data Contract?
A data contract exists to build trust. A producer offers data, a consumer wants to use it, and the contract in between carries the producer's promises and guarantees about that dataset (which can be several tables). It covers structure, semantics, ownership, quality, and terms of use.
The need for data contracts grew up at many companies in parallel a few years ago. Everyone built their own format in YAML, JSON, Excel, Word, or Confluence, but the underlying problem was always the same: capture the guarantees a provider gives for an offer of data. (More in What is a Data Contract?)
The Great Format Merge
Here is the history. PayPal open-sourced its data contract format as version 2.2 and donated it to the Linux Foundation. Simon's own company had a competing format too, as did many others. He joined the standardization effort, and the group reworked PayPal's version (which was tuned for one data platform, because not everyone is PayPal and runs only BigQuery) so it would fit any enterprise.
With version 3.1 they merged the two lineages and retired their own, because one strong standard beats two competing ones. ODCS 3.2 is around the corner. The lesson: rather than make it 15 standards, the camps combined into one, governed at Bitol.
Inside the Open Data Contract Standard
ODCS v3 carries schema information, data quality, pricing (in the standard, though Simon has never seen anyone use it, a fun academic discussion), ownership and team, security, service level agreements, infrastructure, and more. You can read it all on datacontract.com.
The interesting part is the schema: it stores things like data classification and PII flags, example values, and inline data quality checks. One check might enforce a set of valid values; another can be raw SQL that runs as a standard-conformant quality check. You can also record who owns the product and how to contact them, the terms of use (what you may and may not do with the data, and links to a privacy policy or license for purchased data), and SLAs like retention and freshness.
And it says where the data lives. In the example that is Postgres on Supabase, and as of ODCS 3.2 there is a Hana location too. (Background: Open Data Contract Standard.)
Now What? Automate All the Things
Why was the standard worth it? Because you do not have to invent the format, and you can attach logic to it. These are the best metadata you can have about a dataset, Simon argues, and it does not get better, especially once you also bind it to an ontology. Once a contract is machine-readable, it drives the rest:
- Code generation: Java, Pydantic, dbt models, SQL DDL
- Testing: check the data on Hana or any other system against the contract, at every point in the lifecycle, plus breaking-change detection and monitoring
- Metadata distribution: push into metastores, catalogs, and marketplaces
- Infrastructure provisioning: output ports, input ports, anonymization, access control
- Collaboration and governance: naming conventions, schema evolution, usage agreements, approval workflows
Humans can use it, deterministic systems can use it, and so can agents, which is exactly what the one building in the background is doing right now.
Open Standards Get Open Tooling
An open standard lets you build open tooling that works hand in hand with it, so adopting the standard gets you the automation almost for free. The open-source Data Contract CLI (around 900 GitHub stars) reads the YAML, connects to all the usual data providers, runs the checks, and hands back a test report you can take anywhere. It also imports and exports, so you can get to a contract fast and keep working from it.
Standard and tooling are tightly coupled. The star-history chart shows both climbing together toward 1,000 stars: the CLI led for a long stretch, helped by a rewrite from Go to Python (the data world lives in Python), while the standard shows a hockey-stick at the end. A standard without tooling is worthless at first, since you have to build everything yourself; give people the automation for free and the standard takes hold.
An Editor, and the Obligatory Excel
Because not everyone is fluent in YAML, there is also an open-source Data Contract Editor with a form view, a YAML view with a live preview, and a diagram view that works like a data modeling tool.
And the obligatory Excel. Simon hoped to avoid it, but reality kept showing companies building their own spreadsheets to capture ODCS contracts, so the team built one with automatic conversion from Excel to YAML. It turned out to be very popular. Bet on the standard and you get the editor, the Excel template, and the CLI automation as a package. If you want open standards to win, pitch in on the tools, because that is what drives adoption, and adoption is what pushes vendors to support them too.
The Open Data Product Standard
Data contracts do not talk about data products; a contract is really the interface, the dataset you share. A data product, in Simon's view, can expose several contracts on several platforms at once, and there is a standard that fits it well: ODPS.
A data product points to a contract for each output port (what it shares) and for each input port (what it relies on). In the example, one Orders product has four output ports: orders in version 1 and version 2, plus non-PII variants, because data without personal information is far easier to shop for, while the sensitive ports sit behind a manual approval process that can take weeks. One input port references a Kafka topic that a pipeline turns into the offer.
Because ports point at contracts, you get product-level lineage: an order service feeds the Orders product, which feeds further consumers. (See What is a Data Product?)
What's Coming in Bitol
A quick show of hands: about 10% of the room already uses ODCS. ODCS 3.2 lands soon, ODPS 1.1 is coming, and the working group is drafting further standards around observability, data domains, access agreements, and orchestration.
It takes time, Simon notes. The committee does not meet often, and everyone does this alongside the work that actually pays the bills.
A Warning: Two Things Called ODPS
Mind the confusion: there are two standards called ODPS, both at the Linux Foundation. One is part of the Bitol family, works well with ODCS through input ports, and can reference several contracts. The other is separate, has no concept of input ports, and maps to only one ODCS.
They embody genuinely different ideas of what a data product is. The Bitol one is weaker on pricing and internationalization; the other is strong on pricing plans and i18n. One is called the Standard, the other the Specification. It is, Simon admits, really sorry.
Open Semantic Interchange
Data products land in a marketplace, which is where agents go to ask what exists, so that layer needs standards too, or you are back to proprietary metadata formats. The third standard is semantics: what do the data mean? Open Semantic Interchange (OSI), heavily pushed by Snowflake, started with the classic semantic model: datasets, relationships, and, importantly, metrics. You already have your data in Snowflake, Databricks, Oracle, or SAP, and you lay a semantic layer over it.
What stands out is how AI-first it is. The format carries instructions, synonyms, and examples, so that filling them in makes agents work much better: give your metadata instructions and the agents do more with it. Otherwise it looks a lot like a contract: you describe datasets, relationships, and now metrics, for example total_revenue as the sum of order amounts, or full_name as first name plus space plus last name.
And, as everywhere, there is open-source tooling, including an editor for modeling it. (More: Semantics.)
Standards Learn From Each Other
ODCS and OSI are actually fairly similar. The comparison shows where each is ahead: OSI leads on metrics, dynamic fields, and AI context, while ODCS covers terms of use and quality and SLAs.
The gaps are closing through the committees. AI context is coming to ODCS 3.2, along with dynamic fields and metrics like the full_name example. The standards are growing together and learning from one another, which is a nice thing to watch.
Ontologies, and a Rename to Apache Ossie
OSI has many working groups, and a particularly active one on ontologies, which Simon likes a lot. It is all very early (version 0.2.0.dev), but heavily backed by Snowflake. You can model an ontology of concepts and relationships, with internationalization, and link a contract column to a concept.
A wink to the computer-science crowd: calling your initiative "OSI" when there is already a famous networking model by that name is a bold move. They seem to agree, because just yesterday they contributed the initiative to the Apache Foundation and renamed it Apache Ossie. A lot of vendors stand behind it, Entropy Data included, so the format is coming.
Live during the talk: the Open Data Contract Standard repository crosses 1,000 stars.
Let's Make History
Here is the part the room can do together. Just before the talk, the Open Data Contract Standard sat at 978 GitHub stars. At 1,000 it reaches the next level and can graduate within the Linux Foundation. So this is your chance: 22 stars, surely we can manage that.
And the room did. Phones came out, the counter ticked up live, 998, then over the line.
We actually did it. In under five minutes, the Bitol community pushed the Open Data Contract Standard from 978 past 1,000 GitHub stars, live in the room. That clears the bar for ODCS to graduate within the Linux Foundation. Thank you to everyone who scanned the QR code and starred. Together we can move mountains.
If these standards are useful to you, give the ODCS repo a star and help keep it growing.
Thank You
That is the case: data products built on a stack of open standards, ODCS for contracts, ODPS for products, OSI for semantics, let agents and humans build and consume data the same way. The future is not coming; it is a git clone away.
Try the contract-based data product marketplace yourself at demo.entropy-data.com, reach Simon at simon.harrer@entropy-data.com or on LinkedIn, and give datacontract-cli a star on GitHub if it has been useful.
Q&A
Selected questions from the audience after the talk.
Q: Are Data Mesh and data products finally on the verge of a breakthrough in the AI era, now that metadata quality is the real lever?
That is the thesis. Data Mesh often stalled on politics, the swing from centralization back to decentralization, and in a BI context it frequently failed to fly. What changed is that metadata quality used to interest no one; governance could swing all the sticks and carrots it liked, and no one cared. Now you tell an agent "go," and with bad metadata it confidently produces garbage, while with good metadata, driven by standards, it produces something genuinely useful. That creates a real feedback loop to care about metadata quality. Standards are no silver bullet and no fire hose that fixes everything, but they nudge you in the right direction and help raise mediocre agents to good ones. Even something as small as the instructions, synonyms, and examples in OSI noticeably improves agent output.
Q: Isn't metadata a competitive product for the big platforms, and a new form of lock-in?
Yes, and that is exactly why you should bet on open standards. The success of agents is decided by metadata, so every big vendor wants your metadata living with them. A huge metadata lock-in is forming right now, and it is even stronger if your metadata is in a proprietary format. As a customer you need some position of power to negotiate prices; if they can say "but all your metadata is already here," you are locked in. So control your metadata, keep it in-house, perhaps even private. Metadata itself is not expensive and not large; the expensive part is compute and inference.
Q: ODPS feels light on what happens between input and output ports. Will the logic be standardized, or do I use custom properties?
Something will happen there, but it is not the main focus right now; the focus is on ODCS. There is already a bill-of-materials-style part, the idea from manufacturing of what a product consists of, and you can model some of it that way today, though I have not seen it used much. The bigger point is that with ODCS and ODPS you have a kind of spec for your coding agent, and the skills make it build compliant products. Encoding "anonymize this column" in the contract, and how anonymization is done in the skills, the agent puts together. How much you really need to describe is an open question: maybe less than we think. Describe the goal and the quality you want, not every step.
Q: How do I avoid expensive metadata maintenance when the business knowledge sits with domain experts who will never write YAML?
One answer is a domain ontology, captured with the business in workshops, the same way you fill out a data product canvas. The technical and physical details are for the engineers, but you can still bring the business along in the discussion. There is no magic cure. What I do observe is that with AI, bureaucracy hurts less: AI automates the bureaucratic parts, so you get the upside without it grinding on you as a human. The old "data contracts are too much effort" objection has shrunk dramatically. One small anecdote: you can draft the input contract with AI, feeding it your guidelines for a good contract plus, say, the transcript of a meeting you had an hour earlier. It still matters that the artifact exists as the source of truth.
Q: Is this already used in practice, where the agent builds most of it and you review and refine?
Yes. We have US customers using it impressively, contract-first: they use AI to draft the contract, it gets reviewed, and then another agent builds the pipeline. The decisive artifact is the contract. That is where people agree, where the review loop sits, where the human steps in. Think of the contract as a chest of drawers: you put your quality requirements in one drawer, your anonymization or pseudonymization strategy in another, your protection class in another. Humans fill the drawers, the agent does the building, and there is still room for people in the process, which is rather nice.