XDM Kafka

Apache Kafka powers real-time, event-driven architectures, but using Kafka data for testing is often hard to control: messages are distributed across brokers, evolve over time, and frequently contain sensitive information. The XDM Kafka Connector integrates Kafka topics into XDM and enables the controlled provisioning of test data into other Kafka topics and across topic and broker boundaries, so teams can build reliable, repeatable test scenarios without disrupting production streams.

From Streaming Complexity to Self-Service Test Data

While engineering teams manage partitions, offsets, retention and consumer groups, testers and developers need a simple way to request the right data at the right time. XDM fully abstracts Kafka specifics and automates the process end-to-end: end users can request Kafka-based test data via standard XDM workflows and the Data Shop, with no Kafka expertise required. Data can be selected and filtered by timestamp, offset and partition, enabling reproducible extracts as well as incremental propagation for ongoing test cycles.

From Streaming Complexity to Self-Service Test Data

Fast, Direct Kafka-to-Kafka Copy with JSON-Aware Processing

XDM uses the fastest possible path to copy data out of Kafka and provision it into target topics. JSON payload formats are supported natively and can be modified during the copy process, ranging from changing individual fields to restructuring the JSON document. This allows teams to adapt event data to downstream test requirements (e.g., different contract versions, synthetic attributes, or normalized structures) while keeping the process automated and governed.

Fast, Direct Kafka-to-Kafka Copy with JSON-Aware Processing

Reusable Modifications Through Payload Abstraction

To keep test data preparation consistent, XDM introduces an abstraction of message payloads so modification methods become reusable building blocks. The same modification logic can be applied across different Kafka messages and topics, even when payloads differ in structure. This creates a standardized, repeatable approach to transforming streaming test data, reducing manual effort, improving consistency, and accelerating delivery across teams and environments.

More about masking

Reusable Modifications Through Payload Abstraction

Learn more

Test Data Management

XDM is a complete test data management platform that enhances agile teamwork, automates test data workflows, and supports multiple databases with secure masking.

Learn more

Mask sensitive data

Secure sensitive data with XDM’s Masking Tool. Protect privacy and ensure compliance with advanced data masking for all environments.

Learn more

White paper

Data Masking with XDM

Solution: XDM

Download

Optimize your test data management with XDM

Get a free demo