Backfill.
One backfill of a source over a window, read in blocks with a ledger row (a Job) each: what it reads, by which way, at which code version, from a parse already made by the connector's own implementation where one is held, and how far it has got
Figure 1Backfillspecified
- kind
- work
- scope
- platform
- key
- the source, the way and the window
- store
- postgres
- family
- sources
Fields#
| field | type | required | note |
|---|---|---|---|
source | Source | yes | |
way | AccessMethod | yes | |
window | Window | yes | |
plane | Plane | yes | |
blocks | count | yes | the blocks of the window, one a month for a monthly archive |
blocks_done | count | yes | |
code_version | text | yes | the refine’s code version: the parser’s commit, or the release |
held_parse | SourceProcedure | no | Names the source procedure that records the folder that the connector’s own parser wrote before the platform ran. The engine reads that folder in place (18 §2.25). Absent when the platform parses the block itself. |
Routes that use it#
No route takes or returns Backfill directly. The record holds it, and the objects that point at it reach it.