Load management for test data
In many organizations, the biggest peak load in test data management is not caused by individual ad-hoc requests, but by regular, large-scale provisioning across multiple system landscapes for example, the quarterly creation of a Golden Copy.
In this phase, it is often necessary to orchestrate a very large number of copies (sometimes >100 copy jobs), reproducibly, with reliable run times, and without permanently sizing the platform for peak hardware.
At the same time, there is of course the variable day-to-day operation: developers and testers request the required test case data via self-service in a portal.
XDM provides easy access through the web UI and the DataShop, so ordering, traceability, and administration can take place in the browser.
In practice, however, the central bottleneck often remains: How does the TDM platform survive recurring mass executions (Golden Copy) without congestion and without operational risk?
Especially during Golden Copy waves and comparable mass provisioning, classic TDM setups typically run into these problems:
- Infrastructure bottleneck instead of parallelization:
When many copy jobs are supposed to run at the same time, the platform quickly hits fixed capacities.
This leads to queues, long cycle times, and unreliable planning. - Overprovisioning or deadline pressure:
Either you size the platform high permanently (expensive/inefficient), or you force execution into very tight maintenance windows because there is not enough buffer during the day. - Too little transparency when bottlenecks occur:
During mass execution, it is crucial to know whether CPU, memory, disk space, or specific services are the limiting factor—otherwise scaling becomes guesswork. - Additional variable load from daily business:
In addition to Golden Copy, ad-hoc requests come in via self-service.
Without control, this can add further pressure on resources during peak times.
XDM addresses these peaks specifically through *cloud-ready operation* and *scaling via Kubernetes*—supplemented by control mechanisms from the DataShop for day-to-day operations.
Kubernetes scaling as the core: Throughput for mass execution instead of a hardware bottleneck
XDM is delivered via Docker containers and HELM charts and can be run in Kubernetes/OpenShift.
The decisive advantage for Golden Copy waves: the Docker containers can be scaled in the cloud via Kubernetes to the required throughput.
On such a platform, the resources for executing test data provisioning can be scaled across any number of containers.
For QA leads and platform owners, this means in practice:
The platform can “grow” for the quarterly peak, instead of forcing all copy jobs to run sequentially through a rigidly sized installation.
Parallelism becomes a predictable tool (more throughput for Golden Copy), rather than a risk (congestion/timeboxing).
Standardized installation & operations (important for recurring Golden Copy cycles)
XDM provides a HELM chart for installation in Kubernetes or OpenShift.
This ensures a standardized installation and update path: product delivery (Docker images) is clearly separated from infrastructure adjustments, and updates include an automatic migration of the configuration database.
Especially for recurring mass runs, this matters because operational stability and repeatability are often more important than “one-off” performance tuning actions.
Transparency & bottleneck analysis during mass execution (Grafana/system dashboard)
For Golden Copy peaks, not only scaling is important, but also visibility.
XDM includes a preconfigured Grafana that provides information on the usage of test data provisioning and task execution.
In addition, the system dashboard shows memory consumption, disk space, and CPU utilization for running services, and supports the identification of resource bottlenecks and optimization opportunities.
This enables data-based answers to questions such as:
- “Are we CPU-limited right now or I/O-limited?”
- “Do we need more parallelism, or are individual tasks the bottlenecks?”
- “Which services should we scale first for the Golden Copy run?”
Day-to-day operations as an additional load: Self-service via DataShop
For variable day-to-day demand, the DataShop is the central component: tailored input forms simplify provisioning for developers and testers; processes defined once move the data to the desired test environment in the background.
Behind the request forms are typically workflows that execute the steps for copying and making the test data usable.
Permissions can be controlled so it is clearly regulated which teams are allowed to request which use cases.
To ensure Golden Copy phases are not destabilized by additional self-service load, XDM also offers the ability to define an *execution window* per process: requests are collected and, for example, processed during the night shift.
Practical example: Quarterly Golden Copy across multiple system landscapes
Initial situation:
Every three months, a consistent reference state (Golden Copy) must be built across multiple landscapes.
There are >100 copy jobs, which must reliably complete within a limited time window.
With XDM on Kubernetes:
- XDM runs containerized and can be scaled via Kubernetes to the required throughput.
- Mass execution becomes operationally manageable because resources for test data provisioning can be scaled across many containers.
- Bottlenecks become visible via Grafana/system dashboard, so scaling and optimization measures can be taken deliberately rather than “just in case.”
- In addition, variable day-to-day operations can be de-risked via time windows so Golden Copy runs do not collide with ad-hoc requests.
Interested?
If you regularly have to orchestrate Golden Copy cycles or similar mass provisioning, XDM on Kubernetes can be a very pragmatic way to technically cushion peak loads while still enabling self-service in day-to-day operations.
