Skip to content
Webpartner Kommunikationsdienste GmbH

DataHub

One set of data, any number of systems

Your data lives where it is created: products in the PIM, orders in the shop, customers in the CRM, subscriptions in the ERP. DataHub sits in between — it reads in, validates, maps, and from that supplies every partner system in the format it expects. Fully automatically, night after night.

SOURCES CSV / Excel JSON / XML REST API ERP / CRM / host ONIX 01 Ingestor read in 02 Transformer refine 03 Distributor distribute Shop Marketplace Email marketing Reporting Partners SYSTEMS LOGGING · VALIDATION · FULL AND DELTA RUNS · SCHEDULING
The path of the data: read in, refine, distribute — every step logged.

01 — The three steps

Read in, refine, distribute

01

Ingestor

read in
CSV, Excel, JSON, XML, REST interfaces, databases, host exports, SFTP pickups, ONIX: DataHub takes what comes — including the file with the idiosyncratic delimiter and the encoding from 2003. Every source gets a reusable definition, and everything arriving runs against a schema. A faulty delivery shows up at import time, long before it would surface in the shop.
02

Transformer

refine
Field mapping, value lists, conversions, text clean-up, lookup tables: this is where raw data becomes something a shop can work with. Controlled vocabularies collapse three spellings of the same term into one value. Your business rules — gross prices per tax rate, delivery status, bundle resolution, subscription terms — are described once and reused everywhere.
03

Distributor

distribute
Every recipient gets exactly its format: CSV with its own column names for the marketplace, ONIX for the book wholesaler, an API call into the shop, a segment for email marketing, an extract for reporting. Full run or delta, dropped by SFTP or called directly — and logged, so you know who received which record when.

02 — In operation

Built for everyday use

Everything is a definition

Sources, transformations and targets are versionable configurations. A new marketplace with its own column scheme costs one definition and is live the same day.

Workflows instead of manual work

Individual tasks can be chained into workflows that start on a schedule or on an event: the full reconciliation at night, deltas by the minute during the day, and in between a run triggered by an incoming webhook.

Traceability

Every run is logged: which file came in, which rule changed which value, what went out to whom. If a marketplace reports a wrong price, the path of that figure can be traced back.

Errors make themselves heard

Validation on import, notification on failure, runs that can be repeated at will. Every failed run reports itself — and can simply be started again once the cause is fixed.

In short

DataHub is a flexible data exchange system: it reads in any format — CSV, JSON, XML, ONIX, REST interfaces — brings the data into the required structure and hands it out again in whatever format is asked for.

Every step is a task of its own. Tasks can be chained into workflows that run on a schedule or are triggered by an event. A new source, a new target format, a new sequence: one more definition — documented, versioned and readable by everyone on the team.

Laravel · MySQL · file-based definitions · scheduled workflows · on-premise or hosted