Skip to content

Commit e25daf1

Browse files
committed
docs: update buckets recreation procedure
1 parent 50eb125 commit e25daf1

1 file changed

Lines changed: 10 additions & 4 deletions

File tree

services/libs/tinybird/bucketing-architecture.md

Lines changed: 10 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -252,11 +252,17 @@ WHERE cityHash64(segmentId) % 10 = {bucket_number}
252252

253253
### Bootstrap Procedure
254254

255+
The snapshot pipes use **append COPY MODE**. This means running a snapshot pipe will add rows to the existing datasource rather than replacing it. To recreate the buckets from scratch, follow these steps for each bucket `#` (0–9):
256+
255257
1. **Prepare**: Ensure all bucket datasources exist
256-
2. **Execute Snapshots**: Run all 10 snapshot pipes manually
257-
3. **Verify**: Check each bucket has ~10% of total records
258-
4. **Enable Enrichment**: Allow scheduled copy pipes to run
259-
5. **Monitor**: Watch for snapshotId updates in cleaned buckets
258+
2. **Pause** the `activityRelations_bucket_clean_enrich_copy_pipe_#` pipe
259+
3. **Truncate** `activityRelations_bucket_MV_ds_#`
260+
4. **Run** the `activityRelations_bucket_MV_snapshot_#` pipe
261+
5. **Confirm** that `activityRelations_bucket_MV_ds_#` was populated and has 10% of total records
262+
6. **Resume** the `activityRelations_bucket_clean_enrich_copy_pipe_#` pipe
263+
7. **Monitor**: Watch for snapshotId updates in cleaned buckets
264+
265+
> **Why pause the enrichment pipe first?** The enrichment copy pipe reads from `activityRelations_bucket_MV_ds_#`. Truncating while it runs could cause it to produce empty or partial results. Pausing ensures a clean window for the snapshot to populate the datasource.
260266
261267
## Query Patterns
262268

0 commit comments

Comments
 (0)