Clemens Vasters
Bringing Clarity to Event Streams: Enabling Analytics and AI Through Rich Metadata
#1about 3 minutes
The problem of ambiguous data without clear context
Examples with LLMs demonstrate how data without metadata, like temperature units, leads to inconsistent and unreliable interpretations.
#2about 5 minutes
Using schemas to provide necessary data constraints
Providing a schema with explicit constraints and descriptions, such as units, enables LLMs to interpret data correctly and generate consistent results.
#3about 1 minute
Adopting a schema-first design for data structures
Data schemas should be created first and be independent of code assets, allowing code generation for polyglot applications and improving LLM performance.
#4about 3 minutes
Standardizing event structure using CNCF CloudEvents
Using a standard like CloudEvents provides a consistent envelope for events, preventing the chaotic and varied structures that LLMs might otherwise generate.
#5about 5 minutes
Automating client code generation with detailed metadata
By providing a comprehensive metadata document that includes endpoint and protocol details, LLMs can automatically generate complete and correct client code for publishers.
#6about 2 minutes
Moving beyond tribal knowledge in event pipelines
Current event pipelines often rely on informal communication, but a metadata-driven approach creates a formal contract that improves maintainability and compatibility.
#7about 6 minutes
A practical metadata-driven pipeline with Microsoft Fabric
A real-world example processing USGS water data demonstrates how a complete metadata definition using X Registry enables automated tooling and contract enforcement.
#8about 2 minutes
Introducing JSON Structure as a better schema language
JSON Structure is proposed as a stricter alternative to JSON Schema, designed as a data definition language with a better type system and namespace support.
#9about 4 minutes
Key standards for modern event-driven architecture
A summary of three key open standards—CloudEvents, X Registry, and JSON Structure—that provide the tools for building robust, metadata-driven systems.
Related jobs
Jobs that call for the skills explored in this talk.
Matching moments
01:06 MIN
Creating a standard for micro-frontend metadata
Micro-Frontends Discovery
04:18 MIN
Why modern applications adopt event streaming
Event Messaging and Streaming with Apache Pulsar
03:06 MIN
Building an open source data mesh and metadata mesh
A Data Mesh needs Open Metadata
03:41 MIN
Decoupling microservices with event streams
From event streaming to event sourcing 101
04:18 MIN
Using streaming data to power real-time agent applications
Unlocking Value from Data: The Key to Smarter Business Decisions-
05:40 MIN
Evolving from classic microservices to event-driven design
Kafka Streams Microservices
04:49 MIN
Introducing Egeria as an open metadata standard
A Data Mesh needs Open Metadata
06:00 MIN
Q&A on data ownership, monetization, and competing tools
A Data Mesh needs Open Metadata
Featured Partners
Related Videos
20:29Introducing JSON Structure - A Better Schema
Clemens Vasters
38:36A Data Mesh needs Open Metadata
Ferd Scheepers
45:36What is a Message Queue and when and why would I use it?
Clemens Vasters
19:44Blueprints for Success: Steering a Global Data & AI Architecture
Dominik Schneider
45:48Kafka Streams Microservices
Denis Washington & Olli Salonen
43:27CQRS and Event Sourcing without the pixie dust
Allard Buijze
32:03Implementing an Event Sourcing strategy on Azure
Eldert Grootenboer & Olena Borzenko-Turianska
27:33From event streaming to event sourcing 101
Gerard Klijs
Related Articles
View all articles
From learning to earning
Jobs that call for the skills explored in this talk.
AUTO1 Group SE
Berlin, Germany
Intermediate
Senior
ELK
Terraform
Elasticsearch
PRODYNA SE
Hamburg, Germany
Intermediate
Senior
Terraform
PRODYNA SE
Düsseldorf, Germany
Intermediate
Senior
Terraform
PRODYNA SE
Stuttgart, Germany
Intermediate
Senior
Terraform
evoila Frankfurt GmbH
Mainz, Germany
Intermediate
Senior
Kubernetes