A portfolio build of the pattern I keep coming back to: medallion layering, applied end to end with dbt Core against a Databricks SQL Warehouse.
The shape of it
Four layers, each with one job.
Sources declare the landing tables and carry the first round of tests, so bad data is caught at the door rather than three models downstream.
Bronze is a 1:1 copy of the raw tables, tagged contain_pii. No cleaning, no reshaping — the point is a layer that always matches what arrived.
Silver cleans, types and conforms. This is where the real modelling decisions live.
Gold holds the business aggregates: gold_daily_sales and gold_avg_ratings.
models/
├── sources/ # landing declarations + source tests
├── bronze/ # raw 1:1, tagged contain_pii
├── silver/ # cleaned, typed, conformed
└── gold/ # business aggregates
snapshots/ # SCD2 history
tests/ # singular SQL tests
macros/ # custom schema naming
Testing as part of the pipeline
The generic tests cover the usual ground — uniqueness, not-null, relationships, accepted values. The one worth calling out is a singular test asserting that quantity * unit_price is never zero or negative, because that’s the kind of error that passes every schema check and still produces nonsense revenue.
Keeping history
Product attributes change. A snapshot tracks them as SCD Type 2, so an order from six months ago can still be read against the product as it was then, rather than as it is now.