LT downloader: freeze Eurostat distribution evidence pulls and OSP native table discovery
done synth-downloader
Task metadata
| id | t_fbb4483c |
|---|---|
| title | LT downloader: freeze Eurostat distribution evidence pulls and OSP native table discovery |
| assignee | synth-downloader |
| status | done |
| tenant | synthestat |
| priority | 90 |
| workspace_kind | dir |
| workspace_path | /home/synthestat |
| created_by | synth-distributions-researcher |
| created_at | 2026-05-19 18:52:29 CEST |
| started_at | 2026-05-19 18:52:52 CEST |
| completed_at | 2026-05-19 19:00:06 CEST |
Latest summary
Froze the approved Lithuania Eurostat distribution evidence queue: all 26 datasets returned HTTP 200 and are stored with raw JSON-stat payloads, headers, URLs, checksums, retrieval timestamps, dimensions/geographies/periods, and flag/status metadata under /home/synthestat/data/ingestion/lt/official_sources/2026-05-19_1656_lt_eurostat_distributions. Wrote downloader handoffs and sidecars under /home/synthestat/workspace/manager_handoffs/downloader/; OSP native table discovery remains blocked by Cloudflare/JS challenge plus browser timeout, so no municipality/seniunija table IDs are validated.
Body
Parent t_fd1d7d8f found actionable Lithuania distribution evidence. Freeze deterministic downloads/metadata for these Eurostat datasets with all available LT NUTS geographies, code lists, Eurostat flags/status, retrieval timestamps, and source metadata: household/family `cens_21hhcs_r3`, `cens_21hhct_r3`, `cens_21fhcs_r3`; co-residence `cens_21resh_r2`; occupation/origin/workplace `cens_21empo_r2`, `cens_21loc_r2`, `cens_21cobo_r2`, `cens_21ctzo_r2`, `cens_21arco_r2`, `cens_21reso_r2`; LFS priors `lfsa_egised`, `lfsa_eisn2`, `lfsa_egaisedm`; commuting `lfso_19plwk28`, `lfso_19plwk29`, `lfso_19plwk30`, `lfst_r_lfe2ecomm`; income/earnings `ilc_di04`, `ilc_lvph04`, `earn_ses22_47`, `earn_ses22_48`, `nama_10r_2hhinc`; fertility `demo_fordagec`, `demo_frate`, `demo_r_frate2`, `demo_r_find3`. Also attempt OSP native table-ID discovery via `https://osp.stat.gov.lt/en_GB/rdb-rest`, Census 2021 pages, or documented API/browser-capable fetch to determine whether municipality/seniunija detail exists. Inputs: `/home/synthestat/workspace/manager_handoffs/distributions/2026-05-19_1651_findings.md`, `/home/synthestat/workspace/manager_handoffs/distributions/2026-05-19_1651_extraction_specs.md`, `/home/synthestat/workspace/manager_handoffs/distributions/latest.md`. Do not promote to config/output catalogue without validation; produce downloader handoff with local paths, checksums/request metadata, licences/provenance, geography/reference-period semantics, and any failed/blocked OSP discovery.
Parents
[ "t_fd1d7d8f" ]
Children
[ "t_cd2a944e" ]
Runs
| ID | Profile | Status | Outcome | Started | Ended | Summary/error |
|---|---|---|---|---|---|---|
| 49 | synth-downloader | done | completed | 2026-05-19 18:52:52 CEST | 2026-05-19 19:00:06 CEST | Froze the approved Lithuania Eurostat distribution evidence queue: all 26 datasets returned HTTP 200 and are stored with raw JSON-stat payloads, headers, URLs, checksums, retrieval timestamps, dimensions/geographies/periods, and flag/status metadata under /home/synthestat/data/ingestion/lt/official_sources/2026-05-19_1656_lt_eurostat_distributions. Wrote downloader handoffs and sidecars under /home/synthestat/workspace/manager_handoffs/downloader/; OSP native table discovery remains blocked by Cloudflare/JS challenge plus browser timeout, so no municipality/seniunija table IDs are validated. |
Events
| Time | Kind | Payload |
|---|---|---|
| 2026-05-19 18:52:29 CEST | created | {
"assignee": "synth-downloader",
"status": "todo",
"parents": [
"t_fd1d7d8f"
],
"tenant": "synthestat",
"skills": null
} |
| 2026-05-19 18:52:43 CEST | promoted | null |
| 2026-05-19 18:52:52 CEST | claimed | {
"lock": "vmi3188806:1590352",
"expires": 1779210472,
"run_id": 49
} |
| 2026-05-19 18:52:52 CEST | spawned | {
"pid": 1632663
} |
| 2026-05-19 19:00:06 CEST | completed | {
"result_len": 0,
"summary": "Froze the approved Lithuania Eurostat distribution evidence queue: all 26 datasets returned HTTP 200 and are stored with raw JSON-stat payloads, headers, URLs, checksums, retrieval timestamps, dimensions/geographies/periods, and flag/status metadata under /home/synthestat/data/ingestion/lt/official_sources/2026-05-19_1656_lt_eurostat_distributions. Wrote downloader handoffs and sidecars under /hom",
"artifacts": [
"/home/synthestat/workspace/manager_handoffs/downloader/2026-05-19_1656_download_log.md",
"/home/synthestat/workspace/manager_handoffs/downloader/2026-05-19_1656_catalogue_updates.md",
"/home/synthestat/workspace/manager_handoffs/downloader/latest.md",
"/home/synthestat/workspace/manager_handoffs/downloader/2026-05-19_1656_lt_eurostat_distribution_freeze_catalogue.json",
"/home/synthestat/workspace/manager_handoffs/downloader/2026-05-19_1656_lt_eurostat_distribution_freeze_catalogue.csv"
]
} |
Comments
No comments yet.