SE population QA cycle 2 downloader: freeze/catalogue SCB, household-prior, geography, anchor sources
done synth-downloader
Task metadata
| id | t_1bbf9f63 |
|---|---|
| title | SE population QA cycle 2 downloader: freeze/catalogue SCB, household-prior, geography, anchor sources |
| assignee | synth-downloader |
| status | done |
| tenant | synthestat |
| priority | 110 |
| workspace_kind | dir |
| workspace_path | /home/synthestat |
| created_by | synth-manager |
| created_at | 2026-05-19 20:00:15 CEST |
| started_at | 2026-05-19 20:18:41 CEST |
| completed_at | 2026-05-19 20:55:33 CEST |
Latest summary
Froze/catalogued the SE population QA cycle-2 source bundle with a 33-record manifest, CSV index, downloader handoff, and latest snapshot. Complete model-ready assets include the reused full SCB P0 mirror plus newly downloaded/checksummed SCB DeSO/RegSO geodata; absent/licensed/proxy/hidden-overlay gaps are explicitly represented as blocked records rather than silent degradations.
Body
Country: SE (Sweden) Project root / allowed write root: /home/synthestat Parent manager task: t_0c611b3b Depends on: - t_ff30afb9 (SE marginal/source research) - t_f9075c12 (SE distribution/prior research) Mission: Freeze/catalogue the concrete Sweden source artifacts selected by the two researcher tasks for the next SE population QA modeler pass. Do not invent sources beyond researcher recommendations unless needed to resolve exact downloader parameters; if a recommended source is unavailable/licensed/blocked, log that explicitly. Required inputs to read after parents complete: - Parent handoffs from t_ff30afb9 and t_f9075c12 via kanban_show. - /home/synthestat/workspace/manager_handoffs/SE_other_synthesis_ingest.md - /home/synthestat/workspace/manager_handoffs/modeller/2026-05-19_1803_missing_requirements.md - docs/specs/research_knowledge_base.md Download/freeze priorities: P0: 1. SCB target artifacts for DeSO/municipality/county/national population, household, education, labour, tenure/building-type, income, OD commuters, and passenger cars as identified by researchers. 2. Household-composition prior bundle artifacts or reconstructed source tables/manifests. 3. DeSO/RegSO/municipality/county geography assets and concordance metadata. 4. Residential building/dwelling/home anchor source artifacts or explicit blocked/licensed/proxy/scaffold manifests. P1: 5. School/workplace/second-home source artifacts selected by researchers. 6. Hidden-population overlay source artifacts selected by researchers. Deliverable: Write/freeze machine-readable artifacts under existing Synthestat source/catalogue/output conventions (prefer output/catalogue and docs/intelligence/catalogue/raw/extracted patterns if already used in the repo; do not create a new incompatible convention). Write a downloader handoff under /home/synthestat/workspace/manager_handoffs/downloader/ named with timestamp and SE_population_source_freeze. Every frozen artifact or blocked source must include: source_id, URL/API/table ID/citation, retrieval timestamp, checksum where local file exists, geography level, reference period, classification/variables, license/access caveat, quality flag, and candidate use in the review bundle. Definition of done: - A modeler can read the handoff and know exactly which local files/manifests to consume. - Missing/blocked/licensed sources are explicit; no silent degradation. - HARD/FIRM candidate sources are separated from SOFT/GUIDE/INFORMATIONAL priors.
Parents
[ "t_f9075c12", "t_ff30afb9" ]
Children
[ "t_29a0c9c4" ]
Runs
| ID | Profile | Status | Outcome | Started | Ended | Summary/error |
|---|---|---|---|---|---|---|
| 111 | synth-downloader | done | completed | 2026-05-19 20:18:41 CEST | 2026-05-19 20:55:33 CEST | Froze/catalogued the SE population QA cycle-2 source bundle with a 33-record manifest, CSV index, downloader handoff, and latest snapshot. Complete model-ready assets include the reused full SCB P0 mirror plus newly downloaded/checksummed SCB DeSO/RegSO geodata; absent/licensed/proxy/hidden-overlay gaps are explicitly represented as blocked records rather than silent degradations. |
Events
| Time | Kind | Payload |
|---|---|---|
| 2026-05-19 20:00:15 CEST | created | {
"assignee": "synth-downloader",
"status": "todo",
"parents": [
"t_ff30afb9",
"t_f9075c12"
],
"tenant": "synthestat",
"skills": null
} |
| 2026-05-19 20:17:43 CEST | promoted | null |
| 2026-05-19 20:18:41 CEST | claimed | {
"lock": "vmi3188806:1590352",
"expires": 1779215621,
"run_id": 111
} |
| 2026-05-19 20:18:41 CEST | spawned | {
"pid": 1673286
} |
| 2026-05-19 20:25:06 CEST | heartbeat | {
"note": "Running timestamped SE source-freeze script for SCB PxWeb/geodata/blocked-source manifests under /home/synthestat/data/ingestion/se/population_source_freeze."
} |
| 2026-05-19 20:36:48 CEST | heartbeat | {
"note": "Source inputs read; reusing prior SCB bulk mirror where complete, now freezing missing HushallDesoTyp via chunked SCB v1 queries and preparing explicit blocked-source manifests/handoff."
} |
| 2026-05-19 20:52:06 CEST | claim_extended | {
"reason": "pid_alive",
"worker_pid": 1673286,
"claim_lock": "vmi3188806:1590352",
"claim_expires_was": 1779216708,
"claim_expires_now": 1779217626,
"last_heartbeat_at": 1779215808
} |
| 2026-05-19 20:55:33 CEST | completed | {
"result_len": 0,
"summary": "Froze/catalogued the SE population QA cycle-2 source bundle with a 33-record manifest, CSV index, downloader handoff, and latest snapshot. Complete model-ready assets include the reused full SCB P0 mirror plus newly downloaded/checksummed SCB DeSO/RegSO geodata; absent/licensed/proxy/hidden-overlay gaps are explicitly represented as blocked records rather than silent degradations."
} |
Comments
No comments yet.