§04The Systems Lab

The Systems Lab.

Problems we see often, taken apart and put back together. Run a data pipeline end to end and break it on purpose, then explore four architectures from problem to approach to result.

Demonstrations with simulated data. Not deployed client systems.

An engineer walking down a data-centre aisle between rows of server racks.

§04.1From scattered data to one dashboard

Stage 1 of 5 · Demonstration · illustrative data

Three systems, three versions of the truth

Sales, finance and the field team each keep their own records. Ask for this month's revenue and you get two different numbers, and nobody can say which is right.

01Sources

Sales appOrders · CRM

SO-1001 · Amani Traders · Nairobi · 1,200

SO-1002 · Kivu Fresh Ltd · Kigali · 860

SO-1003 · AMANI TRADERS LTD · NBO · 450Name variant

SO-1004 · Lagos Home Supply · Lagos · 2,100

SO-1002 · Kivu Fresh Ltd · Kigali · 860Sent twice

Finance systemInvoices · ERP

INV-77 · Amani Traders · USD · 1,200

INV-78 · Kivu Fresh · USD · 860

INV-79 · Amani Traders Ltd · USD · 450Name variant

INV-80 · Lagos Home Supply · NGN · 3,150,000NGN

Field spreadsheetDeliveries · shared sheet

SO-1001 · promised 10-04 · delivered 10-04

SO-1002 · promised 10-04 · delivered 10-05

SO-1003 · promised 10-05 · delivered 4/10Date format

SO-1004 · promised 10-06 · delivered 10-06

02Pipeline

Landing zone · change data capturewaiting

Records captured0

sales.orders waiting
finance.invoices waiting
field.deliveries waiting

Quality ruleswaiting

Merge duplicatesOne customer, one record: name variants are matched, and an order sent twice is counted once.
One currencyInvoices in local currency are converted to USD at the agreed rate before anything is added up.
Standard dates and regionsRegion codes become names and every date is read in one format, so nothing is silently dropped.

5 issues waiting

Model · one definitionwaiting

–

Customers

–

Orders

–

Deliveries

revenue = Σ order amount in USD, each order once

03Monthly report (by hand)

Revenue, October

Sales saysUSD 5,470Finance says3,152,510 (mixed)

The numbers disagree, and nobody can say which is right.

Assembled from three exports and a spreadsheet, two weeks after month end.

Illustrative data · not a real organization

§04.2Four architectures

Problem, approach, result.

Scenario 01 · Demonstration

What if every report didn't start with an export?

Data in silos, reports by hand

Sales, finance and field teams each keep their own data. Every report starts with exports and a spreadsheet, numbers disagree between departments, and by the time a problem shows up it is weeks old.

Manual exports
Conflicting numbers
Weeks of delay

From siloed systems to a data platform

Simulated

Event streamSimulated · illustrative only

Waiting for the result stage…

Have a scenario of your own?

Tell us about the system you have and the one you need. We will sketch the route between them with you.

Start a project Or explore the Systems Lab