3 Commits

Author	SHA1	Message	Date
John Doe	a35bcd6647	M4: ingestion management (incremental, GS1 supplement, dedup/conflict, quality, scheduler) ... - OFF incremental fetch via search API + persistent watermark (ingest_state, migration 0004) - GS1 barcode supplement adapter (offline mapping + GS1-style API) filling only gaps with field-level provenance - Non-GTIN dedup with canonical selection + merge_log; field-level conflict resolution (source trust > recency) - Quality scoring (0.4 completeness + 0.3 source trust + 0.2 multi-source + 0.1 freshness) wired into load/merge - Jobs: update_off, dedup, schedule; docs/ingestion-management.md - 19 new tests (pure + DB-integration), ruff clean Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-06-08 09:27:42 +00:00
lixu	766a573989	feat(M2): Open Food Facts 采集导入 ETL ... - adapters/openfoodfacts.py: OFF API 适配器(限速+User-Agent) + JSONL/.gz dump 读取 - etl/transform.py: 纯函数转换(GTIN 校验/净含量解析+归一/营养 per_100g 双能量/过敏原添加剂清洗/关键词分类映射) - etl/load.py: psycopg upsert(product/food_detail/product_image) + product_source 字段级溯源 - jobs/seed_off.py: CLI(--barcodes API / --dump 文件 / --limit) - 测试: 13 个离线 transform 单测(fixture) + 可跳过的 DB 集成测试 - 依赖: 增加 psycopg[binary] - 实跑: 从 OFF API 拉 5 个真实条码入本地 postgres 验证通过 - docs/etl-openfoodfacts.md: 流程/运行/字段映射 Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-06-08 07:01:04 +00:00
lixu	786b7d3721	feat(M0): 工程地基 - Go API + Python 采集骨架 + CI + 数据契约 ... - api/: Go(chi) 只读 API 骨架, /healthz + 版本化路由(占位), Dockerfile, 单测 - ingestion/: Python 采集/ETL 包骨架, units 单位归一化(纯函数+测试), adapter 协议 - docker-compose.yml: postgres + redis + minio + api - .github/workflows/ci.yml: Go build/vet/test + Python ruff/pytest - docs/data-contract.md(两端共享契约) + docs/disclaimer.md(不提供购买声明) - migrations/ 占位(M1 起填充) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>	2026-06-08 06:18:31 +00:00