Compare commits

...

86 Commits

Author SHA1 Message Date
lixu 53ade9eeea Merge pull request 'feat(public): 首页搜索居中、移动端自适应,移除下方三卡片' (#37) from devin/1782372352-home-mobile-center into main
CI / Go (api) (push) Successful in 56s
CI / Python (ingestion) (push) Successful in 23s
CI / Migrations (postgres) (push) Successful in 27s
2026-06-25 15:28:54 +08:00
lixu 313bc84cb0 feat(public): 首页搜索居中、移动端自适应,移除下方三卡片
CI / Go (api) (pull_request) Successful in 52s
CI / Python (ingestion) (pull_request) Successful in 25s
CI / Migrations (postgres) (pull_request) Successful in 29s
2026-06-25 07:25:52 +00:00
lixu 8c8df2b8a2 Merge pull request 'fix(admin): 回流接口校验/规范化 GTIN' (#36) from devin/1782352663-backflow-gtin-validate into main
CI / Go (api) (push) Successful in 1m7s
CI / Python (ingestion) (push) Successful in 24s
CI / Migrations (postgres) (push) Successful in 25s
2026-06-25 10:00:29 +08:00
busenalbantoglu51424 f04891d0b9 fix(admin): 回流接口校验/规范化 GTIN,非法条码标记 invalid 而非中断批次
CI / Go (api) (pull_request) Successful in 56s
CI / Python (ingestion) (pull_request) Successful in 28s
CI / Migrations (postgres) (pull_request) Successful in 29s
2026-06-25 01:57:43 +00:00
lixu 3a5575f51b Merge pull request 'feat(admin): 新增档案回流接口 POST /api/public/backflow' (#35) from devin/1782350840-backflow-api into main
CI / Go (api) (push) Successful in 53s
CI / Python (ingestion) (push) Successful in 28s
CI / Migrations (postgres) (push) Successful in 47s
2026-06-25 09:52:43 +08:00
busenalbantoglu51424 a61d1c558c docs: 补充档案回流接口调用说明(docs/api.md + 前台 API 文档页)
CI / Go (api) (pull_request) Successful in 1m2s
CI / Python (ingestion) (pull_request) Successful in 27s
CI / Migrations (postgres) (pull_request) Successful in 24s
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-25 01:41:39 +00:00
busenalbantoglu51424 e7e5ce22ab feat(admin): 新增档案回流接口 POST /api/public/backflow
CI / Go (api) (pull_request) Successful in 55s
CI / Python (ingestion) (pull_request) Successful in 16s
CI / Migrations (postgres) (pull_request) Successful in 25s
进销存软件可用公开 API Key(og_live_) 批量回流未收录商品,进入现有审核
队列,审核通过后收录。按 GTIN 去重(已收录跳过 exists,已有待审跳过
duplicate),来源标记 source=backflow,在后台队列与公众投稿区分。

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-25 01:27:31 +00:00
lixu 856b81125c Merge PR #34: public MSRP display
CI / Go (api) (push) Successful in 54s
CI / Python (ingestion) (push) Successful in 16s
CI / Migrations (postgres) (push) Successful in 26s
feat: embed MSRP in public product detail (hide zero amounts)
2026-06-25 08:59:21 +08:00
busenalbantoglu51424 5fe5774251 feat(public): 商品详情页展示厂商建议零售价(MSRP)
CI / Go (api) (pull_request) Successful in 1m59s
CI / Python (ingestion) (pull_request) Successful in 1m13s
CI / Migrations (postgres) (pull_request) Successful in 27s
后端 ProductByID/ProductByGTIN 在商品详情 JSON 中内嵌 msrp 字段(按 effective_date 倒序,过滤 amount=0 的无效快照);公开前台详情页新增「建议零售价」展示。

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-25 00:42:18 +00:00
lixu 0da57a2fb3 Merge pull request 'feat(admin): 新增 /api/import/bypos 导入接口(修复采集器导入失败)' (#33) from devin/1782307670-bypos-import-endpoint into main
CI / Go (api) (push) Successful in 53s
CI / Python (ingestion) (push) Successful in 16s
CI / Migrations (postgres) (push) Successful in 28s
2026-06-24 21:33:02 +08:00
rosemariejebbjtxbfp 85c58fbd52 feat(admin): add /api/import/bypos endpoint for bypos-collector import
CI / Go (api) (pull_request) Successful in 58s
CI / Python (ingestion) (pull_request) Successful in 16s
CI / Migrations (postgres) (pull_request) Successful in 27s
Brings the bypos JSONL import backend (handler + store) into main so the
bypos-collector tool's import feature works end-to-end. Records are upserted
on GTIN with manufacturer/MSRP/barcode/source provenance, quality recomputed
per product. Only status=hit records with a name are imported.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-24 13:30:27 +00:00
lixu 3aec0b84fe Merge pull request 'feat(web+api): 贡献表单按 archive_kind 动态生成字段' (#32) from devin/1782295354-contribution-form-dynamic into main
CI / Go (api) (push) Successful in 55s
CI / Python (ingestion) (push) Successful in 32s
CI / Migrations (postgres) (push) Successful in 26s
2026-06-24 18:07:05 +08:00
rosemariejebbjtxbfp bd1d3bde7c feat(web+api): 贡献表单按 archive_kind 动态生成字段
CI / Go (api) (pull_request) Successful in 1m3s
CI / Python (ingestion) (pull_request) Successful in 23s
CI / Migrations (postgres) (pull_request) Successful in 26s
- categories 接口返回 archive_kind
- 新增公开只读接口 /api/v1/kind-fields?kind=X 返回字段模板
- 投稿 payload 支持通用 attributes,审核通过写入 product.attributes (JSONB 合并)
- food_detail 仅在含食品数据时才 upsert (药品/3C/通用不再产生空行)
- 前端 Contribute 按所选品类 archive_kind 动态渲染字段
  (食品营养 / 药品 18 字段 / 3C / 通用),除商品名外均选填

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-24 10:02:46 +00:00
lixu 998471f6df Merge pull request 'feat: 新增药品(drug)商品档案格式' (#31) from devin/1782286467-drug-archive-kind into main
CI / Go (api) (push) Successful in 47s
CI / Python (ingestion) (push) Successful in 18s
CI / Migrations (postgres) (push) Successful in 27s
2026-06-24 15:39:07 +08:00
rosemariejebbjtxbfp 1f9b1ecd30 feat(migrations): add drug (药品) archive kind template + category subtree
CI / Go (api) (pull_request) Successful in 55s
CI / Python (ingestion) (pull_request) Successful in 23s
CI / Migrations (postgres) (pull_request) Successful in 27s
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-24 07:36:43 +00:00
lixu 1d71617f71 Merge pull request 'chore(web): 添加微信域名校验文件' (#30) from devin/1782285002-wechat-domain-verify into main
CI / Go (api) (push) Successful in 51s
CI / Python (ingestion) (push) Successful in 30s
CI / Migrations (postgres) (push) Successful in 26s
2026-06-24 15:11:58 +08:00
rosemariejebbjtxbfp d18fc2470b chore(web): add WeChat domain verification file
CI / Go (api) (pull_request) Successful in 54s
CI / Python (ingestion) (pull_request) Successful in 26s
CI / Migrations (postgres) (pull_request) Successful in 27s
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-24 07:10:07 +00:00
lixu f29696607a Merge pull request 'feat(api): Redis read cache for product details and search (stage 0)' (#29) from devin/1782283345-redis-read-cache into main
CI / Go (api) (push) Successful in 50s
CI / Python (ingestion) (push) Successful in 37s
CI / Migrations (postgres) (push) Successful in 27s
2026-06-24 14:54:59 +08:00
rosemariejebbjtxbfp a75318b811 feat(api): Redis read cache for product details and search
CI / Go (api) (pull_request) Successful in 54s
CI / Python (ingestion) (pull_request) Successful in 19s
CI / Migrations (postgres) (pull_request) Successful in 27s
Stage-0 caching from docs/scalability.md. The Go API now caches hot
product-detail and search-result reads in Redis with a fail-open,
epoch-versioned scheme; Python ingestion bumps the epoch after a write
run to invalidate the cache globally in O(1).

- api/internal/cache: fail-open Cache (GetJSON/SetJSON) namespaced by an
  epoch counter (og:cache:epoch); disabled when Redis is unconfigured.
- store: ProductByID/ProductByGTIN (24h TTL) and SearchProducts (1h TTL)
  read-through the cache via WithCache.
- ingestion: bump_cache_epoch() called after update_off/seed_off/
  import_bypos/dedup commits when rows changed; best-effort, never fails
  a run. Adds redis dependency.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-24 06:52:56 +00:00
lixu 20b9eb0fc9 Merge pull request 'ci: 修复 main 上 Python CI 历史失败(在 python3.12+node 容器内运行)' (#28) from devin/1782280936-fix-python-ci into main
CI / Go (api) (push) Successful in 54s
CI / Python (ingestion) (push) Successful in 23s
CI / Migrations (postgres) (push) Successful in 27s
2026-06-24 14:32:51 +08:00
rosemariejebbjtxbfp 17bc0ed680 ci: remove GitHub-hosted actions, manual checkout + Go from CN mirror
CI / Go (api) (pull_request) Successful in 51s
CI / Python (ingestion) (pull_request) Successful in 21s
CI / Migrations (postgres) (pull_request) Successful in 26s
Self-hosted Gitea runner has flaky/blocked access to github.com, causing
actions/checkout and actions/setup-go to time out intermittently across all
jobs. Replace them with:
- manual git checkout against $GITHUB_SERVER_URL (the Gitea host, reachable
  from job containers)
- Go installed from mirrors.aliyun.com

Python job stays on the python3.12-nodejs20 container (fixes the original
PEP 660 editable-install failure). No more github.com network dependency.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-24 06:30:03 +00:00
rosemariejebbjtxbfp 5020fdcc19 ci: re-trigger with public gitea actions url
CI / Go (api) (pull_request) Failing after 11s
CI / Python (ingestion) (pull_request) Failing after 31s
CI / Migrations (postgres) (pull_request) Failing after 11s
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-24 06:24:52 +00:00
rosemariejebbjtxbfp 885f6c2b01 ci: re-trigger with self-hosted actions
CI / Go (api) (pull_request) Failing after 12s
CI / Python (ingestion) (pull_request) Failing after 1m31s
CI / Migrations (postgres) (pull_request) Failing after 32s
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-24 06:21:01 +00:00
rosemariejebbjtxbfp 2aff6287d2 ci: re-trigger after configuring action proxy
CI / Go (api) (pull_request) Failing after 13s
CI / Python (ingestion) (pull_request) Failing after 31s
CI / Migrations (postgres) (pull_request) Failing after 1m32s
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-24 06:10:22 +00:00
rosemariejebbjtxbfp 3364a50c68 ci: run Python job in python3.12+node container
CI / Go (api) (pull_request) Failing after 1m32s
CI / Python (ingestion) (pull_request) Failing after 31s
CI / Migrations (postgres) (pull_request) Failing after 19m16s
setup-python@v5 can't fetch CPython 3.12 on the self-hosted Gitea runner
(it queries the Gitea API for actions/python-versions, which 404s), so the
job silently fell back to the image's system Python 3.10 + pip 22.0.2. That
old pip lacks PEP 660 editable support, so 'pip install -e' failed with
'build backend is missing the build_editable hook', and 3.10 is below the
project's requires-python>=3.11 (code uses datetime.UTC).

Run the job inside nikolaik/python-nodejs:python3.12-nodejs20 which bundles
Python 3.12, Node 20 (for actions/checkout) and modern pip, removing the
GitHub download dependency entirely.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-24 06:02:28 +00:00
lixu 936828abae Merge pull request 'perf(search): 浏览路径部分复合索引 (0012)' (#27) from devin/1782279211-search-indexes into main
CI / Go (api) (push) Successful in 49s
CI / Python (ingestion) (push) Failing after 15s
CI / Migrations (postgres) (push) Successful in 31s
2026-06-24 13:35:50 +08:00
rosemariejebbjtxbfp ab7a964934 perf(search): 为浏览路径补部分复合索引 (0012)
CI / Go (api) (pull_request) Successful in 58s
CI / Python (ingestion) (pull_request) Failing after 21s
CI / Migrations (postgres) (pull_request) Successful in 32s
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-24 05:33:31 +00:00
lixu 649fcc711c Merge pull request 'docs: 可扩展性设计与路线图 (scalability roadmap)' (#26) from devin/1782278890-scalability-doc into main
CI / Go (api) (push) Successful in 52s
CI / Python (ingestion) (push) Failing after 16s
CI / Migrations (postgres) (push) Successful in 26s
2026-06-24 13:31:21 +08:00
rosemariejebbjtxbfp 3b62f61288 docs: 新增可扩展性设计与路线图 (scalability roadmap)
CI / Go (api) (pull_request) Successful in 59s
CI / Python (ingestion) (pull_request) Failing after 20s
CI / Migrations (postgres) (pull_request) Successful in 34s
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-24 05:29:08 +00:00
lixu ba36e3ff4c Merge pull request '新增联系我们页面' (#25) from devin/1782277231-contact-us into main
CI / Go (api) (push) Failing after 32s
CI / Python (ingestion) (push) Failing after 31s
CI / Migrations (postgres) (push) Successful in 35s
2026-06-24 13:02:07 +08:00
rosemariejebbjtxbfp d837dd38bf feat(public-frontend): 新增联系我们页面
CI / Go (api) (pull_request) Successful in 56s
CI / Python (ingestion) (pull_request) Failing after 1m31s
CI / Migrations (postgres) (pull_request) Failing after 32s
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-24 05:00:31 +00:00
lixu 53ce705572 Merge pull request '移除首页 API 调用说明入口' (#23) from devin/1782276245-home-remove-api-cta into main
CI / Go (api) (push) Failing after 16m4s
CI / Python (ingestion) (push) Failing after 17s
CI / Migrations (postgres) (push) Successful in 29s
2026-06-24 12:46:03 +08:00
rosemariejebbjtxbfp 836ee73d73 feat(public-frontend): 移除首页 API 调用说明入口
CI / Go (api) (pull_request) Failing after 1m31s
CI / Python (ingestion) (pull_request) Failing after 16s
CI / Migrations (postgres) (pull_request) Successful in 25s
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-24 04:44:05 +00:00
lixu dbbad274b6 Merge pull request 'feat(public-frontend): 公开前端 UI 视觉升级' (#22) from devin/1782270985-public-ui-redesign into main
CI / Go (api) (push) Successful in 55s
CI / Python (ingestion) (push) Failing after 14s
CI / Migrations (postgres) (push) Successful in 23s
2026-06-24 12:36:17 +08:00
rosemariejebbjtxbfp d58f46bc80 feat(public-frontend): 公开前端 UI 视觉升级
CI / Go (api) (pull_request) Failing after 32s
CI / Python (ingestion) (pull_request) Failing after 31s
CI / Migrations (postgres) (pull_request) Failing after 32s
引入统一品牌主题(色阶/字体/阴影/动效),重做首页 hero、卡片、导航与页脚,统一各页面的卡片/输入框/按钮样式。

- tailwind: 新增 brand 色阶、Inter 字体、card/glow 阴影与 fade-up 动效
- index.html: 引入 Inter 字体与 theme-color/description meta
- index.css: 双径向渐变背景 + @layer 组件类(.card/.input/.btn-primary 等)
- App/Home/ProductView/Contribute/ApiDocs/Account: 套用新设计系统

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-24 03:16:41 +00:00
lixu 8f9a03a929 Merge pull request 'feat(ingestion): import bypos-collector JSONL into goods' (#20) from devin/1782268418-bypos-importer into main
CI / Python (ingestion) (push) Successful in 11s
CI / Migrations (postgres) (push) Successful in 24s
CI / Go (api) (push) Successful in 49s
2026-06-24 10:35:27 +08:00
novaalphastrikeomegaz663 f04da0a135 feat(ingestion): import bypos-collector JSONL into goods
CI / Python (ingestion) (pull_request) Successful in 16s
CI / Migrations (postgres) (pull_request) Successful in 24s
CI / Go (api) (pull_request) Successful in 51s
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-24 02:33:45 +00:00
lixu a2ef7319e9 Merge pull request 'chore(tools): add bypos-collector (条码批量采集器源码备份)' (#19) from devin/1782267629-add-bypos-collector into main
CI / Python (ingestion) (push) Successful in 11s
CI / Migrations (postgres) (push) Successful in 22s
CI / Go (api) (push) Successful in 48s
2026-06-24 10:27:15 +08:00
novaalphastrikeomegaz663 e746b9cd31 chore(tools): add bypos-collector (条码批量采集器源码备份)
CI / Python (ingestion) (pull_request) Successful in 12s
CI / Migrations (postgres) (pull_request) Successful in 23s
CI / Go (api) (pull_request) Successful in 48s
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-24 02:20:34 +00:00
lixu ddda61252b Merge pull request '后台商品档案:点击表头排序' (#18) from devin/1782028836-product-sort into main
CI / Go (api) (push) Successful in 9s
CI / Python (ingestion) (push) Successful in 10s
CI / Migrations (postgres) (push) Successful in 15s
2026-06-21 16:01:56 +08:00
sulaimaannaasif6866 241fd38a56 feat(admin): sortable product list column headers
CI / Go (api) (pull_request) Successful in 11s
CI / Python (ingestion) (pull_request) Successful in 9s
CI / Migrations (postgres) (pull_request) Successful in 14s
Click a column header (名称/品牌/条码/品类/状态/质量分) to sort asc, click
again for desc, and a third time to clear back to the default
most-recently-updated order. Sort key/direction are whitelisted server-side.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-21 08:00:36 +00:00
lixu 3178f8a85a Merge pull request 'API 配额分级:免费累计 1000 次 + 公开注册自助领取更高配额密钥' (#17) from devin/1782026795-api-quota-registration into main
CI / Go (api) (push) Successful in 10s
CI / Python (ingestion) (push) Successful in 9s
CI / Migrations (postgres) (push) Successful in 17s
2026-06-21 15:28:59 +08:00
sulaimaannaasif6866 7434e5195e feat(api): tiered cumulative quota + self-service registration
CI / Go (api) (pull_request) Successful in 15s
CI / Python (ingestion) (pull_request) Successful in 10s
CI / Migrations (postgres) (pull_request) Successful in 16s
Anonymous callers get a free cumulative quota (1000 calls per IP); once
exhausted they get 403 quota_exhausted and must register. Public users can
self-register (email+password) to obtain a higher-quota API key, view usage,
and regenerate the key. Quota counters live in Redis; the public API stays
read-only except for the registration writes.

- migration 0011: app_user table + api_key.quota_total + 'registered' tier
- ratelimit: IncrTotal/TotalUsed/CopyTotal lifetime counters
- middleware: enforce cumulative quota + X-Quota-* headers
- store: RegisterUser/Authenticate/RegenerateKey (bcrypt)
- handlers: POST /api/v1/register, /account, /account/regenerate
- admin: quota_total column + registered tier
- public: 'API 密钥' account page + API docs quota section

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-21 07:26:41 +00:00
lixu 7f66aad779 Merge pull request 'feat: 可扩展档案模式框架 + 内置 3C(电子) 模式' (#16) from devin/1782022411-archive-kind-3c into main
CI / Go (api) (push) Successful in 12s
CI / Python (ingestion) (push) Successful in 10s
CI / Migrations (postgres) (push) Successful in 14s
2026-06-21 14:15:21 +08:00
sulaimaannaasif6866 916bdd9c7c feat: 可扩展档案模式框架 + 内置 3C(电子) 模式
CI / Go (api) (pull_request) Successful in 13s
CI / Python (ingestion) (pull_request) Successful in 9s
CI / Migrations (postgres) (pull_request) Successful in 14s
- 新增 category.archive_kind 与 kind_field 字段模板表(迁移 0010)
- 种子 electronics 字段模板 + 3C 品类树(手机/笔记本/平板等)
- 后端按档案模式动态计算完整度/合格:食品沿用 food_detail,
  其它模式走 product.attributes + kind_field
- 新增 GET /api/kind-fields?kind= 接口
- 后台编辑页按品类模式动态渲染规格参数表单
- 公开详情页/接口输出带标签的规格表

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-21 06:13:42 +00:00
lixu 98eb01f4ea Merge pull request 'feat(admin): 商品编辑页 上一个/下一个 导航' (#15) from devin/1782021020-detail-prevnext into main
CI / Go (api) (push) Successful in 9s
CI / Python (ingestion) (push) Successful in 11s
CI / Migrations (postgres) (push) Successful in 14s
2026-06-21 13:51:19 +08:00
sulaimaannaasif6866 51304d782a feat(admin): 商品编辑页增加上一个/下一个导航
CI / Go (api) (pull_request) Successful in 9s
CI / Python (ingestion) (pull_request) Successful in 10s
CI / Migrations (postgres) (pull_request) Successful in 14s
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-21 05:50:20 +00:00
lixu 6e81a7eb36 Merge pull request 'feat(admin): 商品列表分页增强(每页数量+跳页)' (#14) from devin/1782012403-list-pagination into main
CI / Go (api) (push) Successful in 8s
CI / Python (ingestion) (push) Successful in 9s
CI / Migrations (postgres) (push) Successful in 14s
2026-06-21 11:27:41 +08:00
sulaimaannaasif6866 8bee570cb2 feat(admin): 商品列表底部增加每页数量与跳页控制
CI / Go (api) (pull_request) Successful in 9s
CI / Python (ingestion) (pull_request) Successful in 9s
CI / Migrations (postgres) (pull_request) Successful in 14s
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-21 03:26:43 +00:00
lixu ed06d269c6 Merge pull request 'style(public): 页脚内容居中' (#13) from devin/1782012124-footer-center into main
CI / Go (api) (push) Successful in 9s
CI / Python (ingestion) (push) Successful in 9s
CI / Migrations (postgres) (push) Successful in 18s
2026-06-21 11:23:02 +08:00
sulaimaannaasif6866 0214253b48 style(public): 页脚内容居中
CI / Go (api) (pull_request) Successful in 8s
CI / Python (ingestion) (pull_request) Successful in 9s
CI / Migrations (postgres) (pull_request) Successful in 14s
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-21 03:22:04 +00:00
lixu afced01ba6 Merge pull request 'style(public): 更新页脚免责文案' (#12) from devin/1782011674-footer-text into main
CI / Go (api) (push) Successful in 9s
CI / Python (ingestion) (push) Successful in 10s
CI / Migrations (postgres) (push) Successful in 14s
2026-06-21 11:15:29 +08:00
sulaimaannaasif6866 8812e5f5e3 style(public): 更新页脚免责文案
CI / Go (api) (pull_request) Successful in 9s
CI / Python (ingestion) (pull_request) Successful in 10s
CI / Migrations (postgres) (pull_request) Successful in 13s
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-21 03:14:34 +00:00
lixu 01055ded02 Merge pull request 'style(admin): 后台各页容器统一居中' (#11) from devin/1782011126-admin-center into main
CI / Go (api) (push) Successful in 8s
CI / Python (ingestion) (push) Successful in 9s
CI / Migrations (postgres) (push) Successful in 15s
2026-06-21 11:06:31 +08:00
sulaimaannaasif6866 01593dcbdc style(admin): 后台各页容器统一居中 (mx-auto)
CI / Go (api) (pull_request) Successful in 8s
CI / Python (ingestion) (pull_request) Successful in 13s
CI / Migrations (postgres) (pull_request) Successful in 14s
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-21 03:05:26 +00:00
lixu afdd26cb59 Merge pull request 'feat(public): 首页移除品牌/产地筛选,合格档案数移至页脚' (#10) from devin/1782010793-home-tweaks into main
CI / Go (api) (push) Successful in 8s
CI / Python (ingestion) (push) Successful in 10s
CI / Migrations (postgres) (push) Successful in 14s
2026-06-21 11:01:03 +08:00
sulaimaannaasif6866 4b4758a60f feat(public): 首页移除品牌/产地筛选,合格档案数移至页脚
CI / Go (api) (pull_request) Successful in 14s
CI / Python (ingestion) (pull_request) Successful in 10s
CI / Migrations (postgres) (pull_request) Successful in 14s
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-21 02:59:53 +00:00
lixu f9360c15e3 Merge pull request 'feat(admin): 后台完善 — 分类管理 + 品牌管理 + 新建商品' (#9) from devin/1782002626-admin-category-management into main
CI / Go (api) (push) Successful in 10s
CI / Python (ingestion) (push) Successful in 9s
CI / Migrations (postgres) (push) Successful in 15s
feat(admin): 后台完善 — 分类/品牌管理、新建商品、数据概览、操作日志、批量操作 + 首页 (#9)
2026-06-21 10:00:38 +08:00
sulaimaannaasif6866 f382c27200 feat: 数据概览/操作日志/批量操作 + 首页合格档案数
CI / Go (api) (pull_request) Successful in 13s
CI / Python (ingestion) (pull_request) Successful in 9s
CI / Migrations (postgres) (pull_request) Successful in 14s
后台新增「数据概览」(商品/合格/按状态/品牌/分类/待审核) 与「操作日志」(全局审计分页);商品列表支持多选批量改状态/分类。公开首页标题改为「天工」并展示合格档案数;新增公开接口 /api/v1/stats 与后台 /api/stats、/api/audit、/api/products/bulk。合格口径=quality_score≥0.6 且在用。

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-21 01:45:28 +00:00
sulaimaannaasif6866 a36700076e feat(admin): 品牌管理 + 新建商品
CI / Go (api) (pull_request) Successful in 37s
CI / Python (ingestion) (pull_request) Successful in 10s
CI / Migrations (postgres) (pull_request) Successful in 16s
品牌管理:列出品牌及引用商品数,支持改名、合并重复品牌(把源品牌的商品并入目标后删除源)、删除未被引用的品牌。新建商品:商品列表新增「新建商品」入口,填写名称/条码/品牌/品类后创建并进入详情页继续补全。

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-21 01:31:07 +00:00
sulaimaannaasif6866 4a693c8fe9 feat(admin): 分类管理(分类树增删改移 + 后台页面)
CI / Go (api) (pull_request) Successful in 1m15s
CI / Python (ingestion) (pull_request) Successful in 10s
CI / Migrations (postgres) (pull_request) Successful in 16s
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-21 00:43:49 +00:00
lixu 50124af833 Merge pull request 'feat(search+docs): 搜索升级(trgm 模糊 + 品牌/产地过滤 + 排序)+ 开发者文档 (M5b)' (#8) from devin/1781948300-m5b-search-docs into main
CI / Python (ingestion) (push) Successful in 10s
CI / Migrations (postgres) (push) Successful in 22s
CI / Go (api) (push) Successful in 44s
2026-06-20 17:40:07 +08:00
novaalphastrikeomegaz663 69a0149bbe feat(search+docs): trigram fuzzy search, brand/country filters, developer docs
CI / Python (ingestion) (pull_request) Successful in 12s
CI / Migrations (postgres) (pull_request) Successful in 22s
CI / Go (api) (pull_request) Successful in 47s
Search:
- migration 0009: trigram GIN index on brand.name + btree on country_of_origin
- SearchProducts: typo-tolerant word_similarity matching (>=0.42) on top of
  ILIKE substring + barcode; new brand/country filters; rank by
  similarity * (0.5 + quality_score). Response gains country_of_origin,
  quality_score and per-result relevance score.
- public search UI: brand/country filter inputs; show country in results

Docs:
- serve embedded OpenAPI 3 spec at GET /api/v1/openapi.json (not rate limited)
- ApiDocs page: auth + rate-limit section, updated search params/response
- docs/api.md developer guide

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-20 09:38:27 +00:00
lixu cf255e2380 Merge pull request 'feat(api): API Key + Redis 限流 + 用量统计 (M5a)' (#7) from devin/1781943842-m5a-api-key-ratelimit into main
CI / Python (ingestion) (push) Successful in 11s
CI / Migrations (postgres) (push) Successful in 23s
CI / Go (api) (push) Successful in 48s
2026-06-20 16:27:13 +08:00
novaalphastrikeomegaz663 2820823b36 feat(api): API keys + Redis rate limiting + usage stats
CI / Python (ingestion) (pull_request) Successful in 12s
CI / Migrations (postgres) (pull_request) Successful in 24s
CI / Go (api) (pull_request) Successful in 53s
Add an optional API-key layer to the public read-only API. Keys grant
higher per-minute rate limits and attribute usage; anonymous callers are
still allowed at a lower IP-based budget.

- migration 0008_api_key: api_key table (sha256 hash only, plaintext shown once)
- apikey pkg: key generation + hashing
- ratelimit pkg: Redis fixed-window limiter + per-key usage counters; fails open
- public API middleware: X-API-Key / Bearer auth, X-RateLimit-* headers, 429+Retry-After
- admin: issue/list/revoke keys + usage view (API + UI tab)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-20 08:24:07 +00:00
lixu 7d7a8f1baf Merge pull request 'feat(ingestion): harden OFF ingestion for bulk seeding' (#6) from devin/1781938549-ingestion-resilience into main
CI / Python (ingestion) (push) Successful in 11s
CI / Migrations (postgres) (push) Successful in 22s
CI / Go (api) (push) Successful in 36s
2026-06-20 15:54:07 +08:00
lixu e34e007f28 Merge pull request 'feat(barcode): 多条码管理(迁移 + GTIN 校验 + 后台/公开 API + 后台 UI)' (#5) from devin/1781934649-multi-barcode into main
CI / Python (ingestion) (push) Successful in 11s
CI / Migrations (postgres) (push) Successful in 22s
CI / Go (api) (push) Successful in 37s
2026-06-20 15:53:43 +08:00
novaalphastrikeomegaz663 5b9338175e style(api): gofmt gtin_test.go
CI / Python (ingestion) (pull_request) Successful in 8s
CI / Migrations (postgres) (pull_request) Successful in 15s
CI / Go (api) (pull_request) Successful in 31s
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-20 07:33:10 +00:00
novaalphastrikeomegaz663 c090bcdc9b style(ingest): ruff format country-seed tests
CI / Python (ingestion) (pull_request) Successful in 9s
CI / Migrations (postgres) (pull_request) Successful in 13s
CI / Go (api) (pull_request) Successful in 29s
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-20 07:32:30 +00:00
novaalphastrikeomegaz663 2255243081 feat(ingest): country-focused seeding (collect domestic CN products)
CI / Python (ingestion) (pull_request) Failing after 6s
CI / Migrations (postgres) (pull_request) Successful in 16s
CI / Go (api) (pull_request) Failing after 11m35s
Add a market-focused seeding path so the catalogue can be built from
domestic products rather than the English-heavy global default:

- adapter.fetch_by_country(country): OFF search filtered by
  countries_tags_en, sorted by unique_scans_n (most-scanned first),
  de-duplicated across pages since OFF popularity ordering is unstable.
- is_cn_gs1(code): True for GS1-China company prefixes (690-699),
  i.e. genuinely domestic items vs. imports merely sold in China.
- seed_off --country <slug> [--domestic-only] [--page-size/--max-pages]:
  e.g. 'seed_off --country china --domestic-only' loads only 69x
  barcodes.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-20 07:14:45 +00:00
novaalphastrikeomegaz663 98b5f1575d feat(barcode): admin barcode CRUD endpoints + UI; show extra barcodes publicly
CI / Python (ingestion) (pull_request) Successful in 10s
CI / Go (api) (pull_request) Failing after 15s
CI / Migrations (postgres) (pull_request) Failing after 11m35s
Wire the multi-barcode store layer to HTTP and the operator console:

- adminhandler: add POST /products/{id}/barcodes, DELETE
  /products/{id}/barcodes/{barcodeID}, and POST .../primary. A barcode
  owned by another product returns 409 with the conflicting product
  (gtin/product_id/product_name); an invalid GTIN returns 400.
- admin-frontend: BarcodesCard on the product detail page lists all
  barcodes (primary starred), adds with type/pack-level/region, sets
  primary, and deletes; audit labels for the new actions.
- public-frontend: product detail surfaces non-primary barcodes so a
  case/region code resolves and is visible to consumers.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-20 07:06:07 +00:00
novaalphastrikeomegaz663 044c870df7 feat(ingestion): harden OFF ingestion for bulk seeding
CI / Go (api) (pull_request) Failing after 22s
CI / Python (ingestion) (pull_request) Successful in 14s
CI / Migrations (postgres) (pull_request) Failing after 18s
- Add retry/backoff (429 + 5xx, Retry-After aware) to the OFF adapter so
  transient API errors no longer abort a run.
- Clamp bounded text fields (serving_size, net_content_unit,
  country_of_origin) to their column widths in transform; long OFF values
  previously raised StringDataRightTruncation and rolled back the batch.
- Load each record inside a savepoint (load_record_safe) so one malformed
  source record is skipped instead of aborting the whole import; jobs now
  report an errored count.
- Tests for retry behaviour, serving_size clamping, and per-record isolation.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-20 06:55:56 +00:00
oyaegeli98668 1d5f775d33 feat(barcode): 多条码管理(后端+迁移+GTIN校验)WIP
CI / Go (api) (pull_request) Failing after 18s
CI / Python (ingestion) (pull_request) Successful in 7s
CI / Migrations (postgres) (pull_request) Failing after 22s
- 迁移 0007: 新增 product_barcode 表(一品多码),回填旧 product.gtin 为主码,
  全局唯一索引保证「一码一品」,每品至多一个主码
- internal/gtin: GS1 GTIN-8/12/13/14 校验(校验位 + 拒收店内码/变量重量码/优惠券码)
- 公开只读 API: 任一条码命中商品、详情返回 barcodes、搜索匹配条码
- adminstore: 商品详情含 barcodes;新增 AddBarcode/DeleteBarcode/SetPrimaryBarcode,
  一码命中其他商品返回 ConflictError 供后台去重

待办(按用户要求暂停): 后台 handler 路由、投稿/审核多条码、前后端 UI

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-20 05:54:18 +00:00
lixu 88d5766e8e Merge pull request 'feat(admin): 运营后台(登录/查看/编辑补全)+ 写入API + 审计留痕' (#4) from devin/1781923221-admin-console into main
CI / Go (api) (push) Failing after 19s
CI / Python (ingestion) (push) Successful in 8s
CI / Migrations (postgres) (push) Failing after 16s
2026-06-20 13:42:09 +08:00
oyaegeli98668 5bfb2d3a5f chore: 站名改为「天工商品档案公共仓」(公开站 + 后台标题/页头/页脚/API 文案)
CI / Go (api) (pull_request) Failing after 15s
CI / Python (ingestion) (pull_request) Successful in 7s
CI / Migrations (postgres) (pull_request) Failing after 19s
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-20 05:37:23 +00:00
oyaegeli98668 8909ddcb69 chore(public): 页脚添加 ICP 备案号 鲁ICP备2025185218号-6
CI / Go (api) (pull_request) Failing after 20s
CI / Python (ingestion) (pull_request) Successful in 8s
CI / Migrations (postgres) (pull_request) Failing after 16s
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-20 05:33:58 +00:00
oyaegeli98668 2d95052bd9 feat(public): 首页新增 API 调用说明页(只读端点/参数/示例/免责声明)
CI / Python (ingestion) (pull_request) Successful in 6s
CI / Go (api) (pull_request) Failing after 17s
CI / Migrations (postgres) (pull_request) Failing after 20s
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-20 05:26:53 +00:00
oyaegeli98668 c9a4404052 feat: 公开首页(搜索+商品详情) + 好心人投稿 + 后台审核收纳
CI / Go (api) (pull_request) Failing after 20s
CI / Python (ingestion) (pull_request) Successful in 8s
CI / Migrations (postgres) (pull_request) Failing after 17s
- 公开前端 SPA(根路径 /):首页大搜索框、检索结果、只读商品详情、贡献档案表单
- 公开写入端点 POST /api/public/submissions(无需登录,基础频率限流),投稿进入 submission 待审核队列,不直接写 product
- 迁移 0006:submission 投稿表 + community 来源(trust=0.50)
- 后台审核队列:列表(待审核/已通过/已驳回) → 查看投稿 → 通过(创建/补全商品 + 记 source=community + 字段级溯源 + 审计 + 重算质量分) / 驳回(记原因)
- 公开只读 api 服务内嵌公开 SPA;Dockerfile.prod 增加 node 构建阶段 + 内嵌 dist

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-20 05:11:07 +00:00
oyaegeli98668 bad771e172 build(admin): npm 走 npmmirror 镜像以适配部署环境网络
CI / Go (api) (pull_request) Failing after 19s
CI / Python (ingestion) (pull_request) Successful in 7s
CI / Migrations (postgres) (pull_request) Failing after 21s
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-20 02:54:32 +00:00
oyaegeli98668 d90a539e6b feat(admin): 运营后台(登录/查看/审核编辑/补全)+ 写入API + 审计留痕
CI / Go (api) (pull_request) Failing after 18s
CI / Python (ingestion) (pull_request) Successful in 7s
CI / Migrations (postgres) (pull_request) Failing after 18s
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-20 02:53:30 +00:00
lixu c272b2c5d7 Merge pull request 'chore(deploy): 生产 docker compose 部署配置' (#3) from devin/1781922421-prod-deploy into main
CI / Go (api) (push) Failing after 20s
CI / Python (ingestion) (push) Successful in 6s
CI / Migrations (postgres) (push) Failing after 19s
2026-06-20 10:32:22 +08:00
oyaegeli98668 6777461268 chore(deploy): 生产 docker compose + scratch Dockerfile + 部署文档
CI / Go (api) (pull_request) Failing after 41s
CI / Python (ingestion) (pull_request) Successful in 43s
CI / Migrations (postgres) (pull_request) Failing after 17s
- docker-compose.prod.yml:仅 api 绑 127.0.0.1:8120,DB/redis/minio 不暴露,restart=unless-stopped,凭据走 .env
- api/Dockerfile.prod:runtime 改用 scratch + 拷贝 ca-certs(gcr.io/distroless 不可达环境),Go 模块走 goproxy.cn
- .env.example / docs/deploy.md

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-20 02:27:09 +00:00
lixu e3e5c7979c ci: 修复 Gitea Actions 数据库连接(用服务名 postgres) (#2)
CI / Go (api) (push) Successful in 6s
CI / Python (ingestion) (push) Successful in 6s
CI / Migrations (postgres) (push) Successful in 16s
2026-06-19 17:47:41 +08:00
rosemariejebbjtxbfp 19b7c43f37 ci: connect to postgres via service hostname for Gitea Actions
CI / Go (api) (pull_request) Successful in 30s
CI / Python (ingestion) (pull_request) Successful in 7s
CI / Migrations (postgres) (pull_request) Successful in 16s
Gitea Actions runs jobs inside a container, so a service container is reachable by its service name (postgres), not localhost; localhost:5432 yields connection refused. Also drop the unnecessary 5432:5432 host port mapping that caused 'port already allocated' collisions.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-19 08:54:18 +00:00
lixu 57537c880d M4: ingestion management (#1)
CI / Go (api) (push) Failing after 14s
CI / Migrations (postgres) (push) Failing after 33s
CI / Python (ingestion) (push) Failing after 13m10s
Incremental OFF + GS1 supplement + dedup/conflict + quality scoring + scheduler
2026-06-08 17:40:26 +08:00
John Doe a35bcd6647 M4: ingestion management (incremental, GS1 supplement, dedup/conflict, quality, scheduler)
CI / Go (api) (pull_request) Has been cancelled
CI / Python (ingestion) (pull_request) Has been cancelled
CI / Migrations (postgres) (pull_request) Has been cancelled
- OFF incremental fetch via search API + persistent watermark (ingest_state, migration 0004)
- GS1 barcode supplement adapter (offline mapping + GS1-style API) filling only gaps with field-level provenance
- Non-GTIN dedup with canonical selection + merge_log; field-level conflict resolution (source trust > recency)
- Quality scoring (0.4 completeness + 0.3 source trust + 0.2 multi-source + 0.1 freshness) wired into load/merge
- Jobs: update_off, dedup, schedule; docs/ingestion-management.md
- 19 new tests (pure + DB-integration), ruff clean

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-08 09:27:42 +00:00
167 changed files with 22632 additions and 104 deletions
+14
View File
@@ -0,0 +1,14 @@
.git
**/node_modules
admin-frontend/dist
public-frontend/dist
api/server
*.test
*.out
__pycache__
.venv
.pytest_cache
.ruff_cache
.env
.env.*
!.env.example
+12
View File
@@ -0,0 +1,12 @@
# Copy to .env and fill in real values before running docker-compose.prod.yml.
# Used by docker-compose.prod.yml for production deployment.
POSTGRES_USER=opengoods
POSTGRES_PASSWORD=change-me
POSTGRES_DB=opengoods
MINIO_ROOT_USER=opengoods
MINIO_ROOT_PASSWORD=change-me
# Admin console (served at /ping). Set a strong password and a random JWT secret.
GOODS_ADMIN_USER=admin
GOODS_ADMIN_PASSWORD=change-me
GOODS_ADMIN_JWT_SECRET=change-me-to-a-long-random-string
+47 -19
View File
@@ -5,6 +5,10 @@ on:
branches: [main] branches: [main]
pull_request: pull_request:
env:
GO_VERSION: "1.23.12"
GOPROXY: "https://goproxy.cn,direct"
jobs: jobs:
go: go:
name: Go (api) name: Go (api)
@@ -19,19 +23,28 @@ jobs:
POSTGRES_USER: opengoods POSTGRES_USER: opengoods
POSTGRES_PASSWORD: opengoods POSTGRES_PASSWORD: opengoods
POSTGRES_DB: opengoods POSTGRES_DB: opengoods
ports:
- "5432:5432"
options: >- options: >-
--health-cmd "pg_isready -U opengoods" --health-cmd "pg_isready -U opengoods"
--health-interval 5s --health-timeout 5s --health-retries 10 --health-interval 5s --health-timeout 5s --health-retries 10
env: env:
OPENGOODS_DATABASE_URL: postgres://opengoods:opengoods@localhost:5432/opengoods?sslmode=disable OPENGOODS_DATABASE_URL: postgres://opengoods:opengoods@postgres:5432/opengoods?sslmode=disable
steps: steps:
- uses: actions/checkout@v4 - name: Checkout
- uses: actions/setup-go@v5 working-directory: ${{ github.workspace }}
with: run: |
go-version: "1.23" git config --global --add safe.directory '*'
cache-dependency-path: api/go.sum git init -q .
git remote add origin "${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}.git"
git -c protocol.version=2 fetch -q --no-tags --depth 1 origin "${GITHUB_REF}"
git checkout -q --force FETCH_HEAD
- name: Setup Go
working-directory: ${{ github.workspace }}
run: |
curl -fsSL -o /tmp/go.tgz "https://mirrors.aliyun.com/golang/go${GO_VERSION}.linux-amd64.tar.gz"
rm -rf /usr/local/go
tar -C /usr/local -xzf /tmp/go.tgz
echo "/usr/local/go/bin" >> "$GITHUB_PATH"
echo "$HOME/go/bin" >> "$GITHUB_PATH"
- name: Apply migrations - name: Apply migrations
working-directory: . working-directory: .
run: | run: |
@@ -46,14 +59,20 @@ jobs:
python: python:
name: Python (ingestion) name: Python (ingestion)
runs-on: ubuntu-latest runs-on: ubuntu-latest
container:
image: nikolaik/python-nodejs:python3.12-nodejs20
defaults: defaults:
run: run:
working-directory: ingestion working-directory: ingestion
steps: steps:
- uses: actions/checkout@v4 - name: Checkout
- uses: actions/setup-python@v5 working-directory: ${{ github.workspace }}
with: run: |
python-version: "3.12" git config --global --add safe.directory '*'
git init -q .
git remote add origin "${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}.git"
git -c protocol.version=2 fetch -q --no-tags --depth 1 origin "${GITHUB_REF}"
git checkout -q --force FETCH_HEAD
- name: Install - name: Install
run: pip install -e ".[dev]" run: pip install -e ".[dev]"
- name: Ruff lint - name: Ruff lint
@@ -73,18 +92,27 @@ jobs:
POSTGRES_USER: opengoods POSTGRES_USER: opengoods
POSTGRES_PASSWORD: opengoods POSTGRES_PASSWORD: opengoods
POSTGRES_DB: opengoods POSTGRES_DB: opengoods
ports:
- "5432:5432"
options: >- options: >-
--health-cmd "pg_isready -U opengoods" --health-cmd "pg_isready -U opengoods"
--health-interval 5s --health-timeout 5s --health-retries 10 --health-interval 5s --health-timeout 5s --health-retries 10
env: env:
DBURL: postgres://opengoods:opengoods@localhost:5432/opengoods?sslmode=disable DBURL: postgres://opengoods:opengoods@postgres:5432/opengoods?sslmode=disable
steps: steps:
- uses: actions/checkout@v4 - name: Checkout
- uses: actions/setup-go@v5 working-directory: ${{ github.workspace }}
with: run: |
go-version: "1.23" git config --global --add safe.directory '*'
git init -q .
git remote add origin "${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}.git"
git -c protocol.version=2 fetch -q --no-tags --depth 1 origin "${GITHUB_REF}"
git checkout -q --force FETCH_HEAD
- name: Setup Go
run: |
curl -fsSL -o /tmp/go.tgz "https://mirrors.aliyun.com/golang/go${GO_VERSION}.linux-amd64.tar.gz"
rm -rf /usr/local/go
tar -C /usr/local -xzf /tmp/go.tgz
echo "/usr/local/go/bin" >> "$GITHUB_PATH"
echo "$HOME/go/bin" >> "$GITHUB_PATH"
- name: Install golang-migrate - name: Install golang-migrate
run: go install -tags 'postgres' github.com/golang-migrate/migrate/v4/cmd/migrate@v4.18.1 run: go install -tags 'postgres' github.com/golang-migrate/migrate/v4/cmd/migrate@v4.18.1
- name: Migrate up - name: Migrate up
+9
View File
@@ -18,6 +18,15 @@ dist/
.env.* .env.*
!.env.example !.env.example
# Node / admin frontend
node_modules/
# Keep the embedded SPA placeholders (real builds are injected during Docker build)
!api/internal/adminweb/dist/
!api/internal/adminweb/dist/index.html
!api/internal/publicweb/dist/
!api/internal/publicweb/dist/index.html
# OS / editors # OS / editors
.DS_Store .DS_Store
*.swp *.swp
+12
View File
@@ -0,0 +1,12 @@
<!doctype html>
<html lang="zh">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>天工商品档案公共仓 · 管理后台</title>
</head>
<body>
<div id="root"></div>
<script type="module" src="/src/main.tsx"></script>
</body>
</html>
+2690
View File
File diff suppressed because it is too large Load Diff
+26
View File
@@ -0,0 +1,26 @@
{
"name": "opengoods-admin-frontend",
"private": true,
"version": "1.0.0",
"type": "module",
"scripts": {
"dev": "vite",
"build": "tsc && vite build",
"preview": "vite preview"
},
"dependencies": {
"react": "^18.2.0",
"react-dom": "^18.2.0",
"lucide-react": "^0.344.0"
},
"devDependencies": {
"@types/react": "^18.2.55",
"@types/react-dom": "^18.2.19",
"@vitejs/plugin-react": "^4.2.1",
"autoprefixer": "^10.4.17",
"postcss": "^8.4.35",
"tailwindcss": "^3.4.1",
"typescript": "^5.3.3",
"vite": "^5.1.0"
}
}
+6
View File
@@ -0,0 +1,6 @@
export default {
plugins: {
tailwindcss: {},
autoprefixer: {},
},
};
+211
View File
@@ -0,0 +1,211 @@
import { useEffect, useState } from "react";
import { api, clearToken, getToken } from "./api";
import Login from "./components/Login";
import ProductList from "./components/ProductList";
import ProductDetail from "./components/ProductDetail";
import SubmissionsPage from "./components/SubmissionsPage";
import ApiKeysPage from "./components/ApiKeysPage";
import CategoriesPage from "./components/CategoriesPage";
import BrandsPage from "./components/BrandsPage";
import StatsPage from "./components/StatsPage";
import AuditLogPage from "./components/AuditLogPage";
import { BarChart3, FolderTree, Inbox, KeyRound, LogOut, Package, ScrollText, Tag } from "lucide-react";
type Tab =
| "overview"
| "products"
| "submissions"
| "categories"
| "brands"
| "audit"
| "keys";
type View = { name: "list" } | { name: "detail"; id: string };
export default function App() {
const [authed, setAuthed] = useState(false);
const [checking, setChecking] = useState(true);
const [username, setUsername] = useState("");
const [tab, setTab] = useState<Tab>("products");
const [pending, setPending] = useState<number | null>(null);
const [view, setView] = useState<View>({ name: "list" });
const [navIds, setNavIds] = useState<string[]>([]);
useEffect(() => {
if (!authed) return;
api
.listSubmissions("pending", 1, 1)
.then((r) => setPending(r.pending))
.catch(() => undefined);
}, [authed]);
useEffect(() => {
if (!getToken()) {
setChecking(false);
return;
}
api
.me()
.then((r) => {
setUsername(r.username);
setAuthed(true);
})
.catch(() => clearToken())
.finally(() => setChecking(false));
}, []);
function onLoggedIn(name: string) {
setUsername(name);
setAuthed(true);
setView({ name: "list" });
}
function logout() {
clearToken();
setAuthed(false);
setUsername("");
}
if (checking) {
return (
<div className="flex h-full items-center justify-center text-gray-500">
</div>
);
}
if (!authed) return <Login onLoggedIn={onLoggedIn} />;
return (
<div className="flex h-full flex-col">
<header className="flex items-center justify-between bg-white px-6 py-3 shadow-sm">
<div className="flex items-center gap-6">
<div className="flex items-center gap-2 text-lg font-semibold text-gray-800">
<Package className="h-5 w-5 text-emerald-600" />
·
</div>
<nav className="flex items-center gap-1 text-sm">
<button
onClick={() => setTab("overview")}
className={`px-3 py-1.5 rounded-md flex items-center gap-1.5 ${
tab === "overview"
? "bg-emerald-50 text-emerald-700"
: "text-gray-600 hover:bg-gray-100"
}`}
>
<BarChart3 className="h-4 w-4" />
</button>
<button
onClick={() => {
setTab("products");
setView({ name: "list" });
}}
className={`px-3 py-1.5 rounded-md flex items-center gap-1.5 ${
tab === "products"
? "bg-emerald-50 text-emerald-700"
: "text-gray-600 hover:bg-gray-100"
}`}
>
<Package className="h-4 w-4" />
</button>
<button
onClick={() => setTab("submissions")}
className={`px-3 py-1.5 rounded-md flex items-center gap-1.5 ${
tab === "submissions"
? "bg-emerald-50 text-emerald-700"
: "text-gray-600 hover:bg-gray-100"
}`}
>
<Inbox className="h-4 w-4" /> 稿
{pending != null && pending > 0 && (
<span className="ml-1 text-xs bg-amber-500 text-white rounded-full px-1.5">
{pending}
</span>
)}
</button>
<button
onClick={() => {
setTab("categories");
setView({ name: "list" });
}}
className={`px-3 py-1.5 rounded-md flex items-center gap-1.5 ${
tab === "categories"
? "bg-emerald-50 text-emerald-700"
: "text-gray-600 hover:bg-gray-100"
}`}
>
<FolderTree className="h-4 w-4" />
</button>
<button
onClick={() => setTab("brands")}
className={`px-3 py-1.5 rounded-md flex items-center gap-1.5 ${
tab === "brands"
? "bg-emerald-50 text-emerald-700"
: "text-gray-600 hover:bg-gray-100"
}`}
>
<Tag className="h-4 w-4" />
</button>
<button
onClick={() => setTab("audit")}
className={`px-3 py-1.5 rounded-md flex items-center gap-1.5 ${
tab === "audit"
? "bg-emerald-50 text-emerald-700"
: "text-gray-600 hover:bg-gray-100"
}`}
>
<ScrollText className="h-4 w-4" />
</button>
<button
onClick={() => setTab("keys")}
className={`px-3 py-1.5 rounded-md flex items-center gap-1.5 ${
tab === "keys"
? "bg-emerald-50 text-emerald-700"
: "text-gray-600 hover:bg-gray-100"
}`}
>
<KeyRound className="h-4 w-4" /> API
</button>
</nav>
</div>
<div className="flex items-center gap-4 text-sm text-gray-600">
<span>{username}</span>
<button
onClick={logout}
className="flex items-center gap-1 rounded px-2 py-1 text-gray-500 hover:bg-gray-100 hover:text-gray-800"
>
<LogOut className="h-4 w-4" /> 退
</button>
</div>
</header>
<main className="flex-1 overflow-auto p-6">
{tab === "overview" ? (
<StatsPage />
) : tab === "audit" ? (
<AuditLogPage />
) : tab === "keys" ? (
<ApiKeysPage />
) : tab === "categories" ? (
<CategoriesPage />
) : tab === "brands" ? (
<BrandsPage />
) : tab === "submissions" ? (
<SubmissionsPage onPending={setPending} />
) : view.name === "list" ? (
<ProductList
onOpen={(id, ids) => {
setNavIds(ids);
setView({ name: "detail", id });
}}
/>
) : (
<ProductDetail
id={view.id}
ids={navIds}
onNavigate={(id) => setView({ name: "detail", id })}
onBack={() => setView({ name: "list" })}
/>
)}
</main>
</div>
);
}
+206
View File
@@ -0,0 +1,206 @@
// API base derives from Vite's BASE_URL (/ping/) so it matches the nginx prefix.
const API_BASE = `${import.meta.env.BASE_URL}api`;
const TOKEN_KEY = "opengoods_admin_token";
export function getToken(): string | null {
return localStorage.getItem(TOKEN_KEY);
}
export function setToken(token: string) {
localStorage.setItem(TOKEN_KEY, token);
}
export function clearToken() {
localStorage.removeItem(TOKEN_KEY);
}
export class ApiError extends Error {
status: number;
constructor(status: number, message: string) {
super(message);
this.status = status;
}
}
async function request<T>(path: string, options: RequestInit = {}): Promise<T> {
const headers: Record<string, string> = {
"Content-Type": "application/json",
...(options.headers as Record<string, string>),
};
const token = getToken();
if (token) headers.Authorization = `Bearer ${token}`;
const res = await fetch(`${API_BASE}${path}`, { ...options, headers });
if (res.status === 401) {
clearToken();
throw new ApiError(401, "登录已过期,请重新登录");
}
const text = await res.text();
const data = text ? JSON.parse(text) : null;
if (!res.ok) {
const msg = data?.error?.message || `请求失败 (${res.status})`;
throw new ApiError(res.status, msg);
}
return data as T;
}
export const api = {
login: (username: string, password: string) =>
request<{ token: string; username: string }>("/login", {
method: "POST",
body: JSON.stringify({ username, password }),
}),
me: () => request<{ username: string }>("/me"),
listProducts: (
q: string,
page: number,
size: number,
sort?: string,
order?: string,
) =>
request<{
items: import("./types").ProductRow[];
page: number;
size: number;
total: number;
completeness_fields: string[];
}>(
`/products?q=${encodeURIComponent(q)}&page=${page}&size=${size}` +
(sort ? `&sort=${sort}&order=${order || "asc"}` : ""),
),
getProduct: (id: string) =>
request<import("./types").ProductDetail>(`/products/${id}`),
createProduct: (body: unknown) =>
request<import("./types").ProductDetail>("/products", {
method: "POST",
body: JSON.stringify(body),
}),
updateProduct: (id: string, body: unknown) =>
request<import("./types").ProductDetail>(`/products/${id}`, {
method: "PUT",
body: JSON.stringify(body),
}),
listAudit: (id: string) =>
request<{ items: import("./types").AuditEntry[] }>(`/products/${id}/audit`),
addImage: (id: string, url: string, kind: string) =>
request<import("./types").ProductImage>(`/products/${id}/images`, {
method: "POST",
body: JSON.stringify({ url, kind }),
}),
deleteImage: (id: string, imageId: string) =>
request<{ status: string }>(`/products/${id}/images/${imageId}`, {
method: "DELETE",
}),
addMsrp: (id: string, body: unknown) =>
request<import("./types").MSRP>(`/products/${id}/msrp`, {
method: "POST",
body: JSON.stringify(body),
}),
deleteMsrp: (id: string, msrpId: string) =>
request<{ status: string }>(`/products/${id}/msrp/${msrpId}`, {
method: "DELETE",
}),
addBarcode: (id: string, body: unknown) =>
request<import("./types").Barcode>(`/products/${id}/barcodes`, {
method: "POST",
body: JSON.stringify(body),
}),
deleteBarcode: (id: string, barcodeId: string) =>
request<{ status: string }>(`/products/${id}/barcodes/${barcodeId}`, {
method: "DELETE",
}),
setPrimaryBarcode: (id: string, barcodeId: string) =>
request<import("./types").Barcode>(
`/products/${id}/barcodes/${barcodeId}/primary`,
{ method: "POST" },
),
listBrands: () =>
request<{ items: import("./types").Brand[] }>("/brands"),
createBrand: (name: string) =>
request<import("./types").Brand>("/brands", {
method: "POST",
body: JSON.stringify({ name }),
}),
updateBrand: (id: string, name: string) =>
request<import("./types").Brand>(`/brands/${id}`, {
method: "PUT",
body: JSON.stringify({ name }),
}),
mergeBrands: (id: string, targetId: string) =>
request<import("./types").Brand>(`/brands/${id}/merge`, {
method: "POST",
body: JSON.stringify({ target_id: targetId }),
}),
deleteBrand: (id: string) =>
request<{ status: string }>(`/brands/${id}`, { method: "DELETE" }),
listKindFields: (kind: string) =>
request<{ items: import("./types").KindField[]; kind: string }>(
`/kind-fields?kind=${encodeURIComponent(kind)}`,
),
listCategories: () =>
request<{ items: import("./types").Category[] }>("/categories"),
createCategory: (body: import("./types").CategoryInput) =>
request<import("./types").Category>("/categories", {
method: "POST",
body: JSON.stringify(body),
}),
updateCategory: (id: string, body: import("./types").CategoryInput) =>
request<import("./types").Category>(`/categories/${id}`, {
method: "PUT",
body: JSON.stringify(body),
}),
deleteCategory: (id: string) =>
request<{ status: string }>(`/categories/${id}`, { method: "DELETE" }),
listSubmissions: (status: string, page: number, size: number) =>
request<{
items: import("./types").SubmissionRow[];
page: number;
size: number;
total: number;
pending: number;
}>(`/submissions?status=${encodeURIComponent(status)}&page=${page}&size=${size}`),
getSubmission: (id: string) =>
request<import("./types").SubmissionDetail>(`/submissions/${id}`),
approveSubmission: (id: string) =>
request<import("./types").ProductDetail>(`/submissions/${id}/approve`, {
method: "POST",
}),
rejectSubmission: (id: string, note: string) =>
request<{ status: string }>(`/submissions/${id}/reject`, {
method: "POST",
body: JSON.stringify({ note }),
}),
listApiKeys: () =>
request<{ items: import("./types").ApiKey[] }>("/keys"),
createApiKey: (body: {
name: string;
owner_email?: string;
tier?: string;
rate_limit_per_min?: number;
quota_total?: number;
}) =>
request<{ key: string; item: import("./types").ApiKey; warning: string }>(
"/keys",
{ method: "POST", body: JSON.stringify(body) },
),
revokeApiKey: (id: string) =>
request<{ status: string }>(`/keys/${id}`, { method: "DELETE" }),
stats: () => request<import("./types").AdminStats>("/stats"),
auditLog: (page: number, size: number) =>
request<{
items: import("./types").AuditLogRow[];
page: number;
size: number;
total: number;
}>(`/audit?page=${page}&size=${size}`),
bulkProducts: (body: {
ids: string[];
action: "status" | "category";
status?: string;
category_id?: string | null;
}) =>
request<{ status: string; affected: number }>("/products/bulk", {
method: "POST",
body: JSON.stringify(body),
}),
};
@@ -0,0 +1,304 @@
import { useEffect, useState } from "react";
import { api, ApiError } from "../api";
import type { ApiKey } from "../types";
import { Copy, KeyRound, Plus, Trash2 } from "lucide-react";
const TIERS = [
{ key: "free", label: "免费 (free)", rate: 120, quota: 1000 },
{ key: "registered", label: "注册用户 (registered)", rate: 300, quota: 100000 },
{ key: "partner", label: "合作方 (partner)", rate: 600, quota: 0 },
{ key: "internal", label: "内部 (internal)", rate: 6000, quota: 0 },
];
function tierLabel(tier: string): string {
return TIERS.find((t) => t.key === tier)?.label ?? tier;
}
export default function ApiKeysPage() {
const [rows, setRows] = useState<ApiKey[]>([]);
const [error, setError] = useState("");
const [creating, setCreating] = useState(false);
const [newKey, setNewKey] = useState<string | null>(null);
async function load() {
setError("");
try {
const res = await api.listApiKeys();
setRows(res.items);
} catch (e) {
setError(e instanceof ApiError ? e.message : "加载失败");
}
}
useEffect(() => {
load();
}, []);
async function revoke(id: string, name: string) {
if (!confirm(`确认吊销密钥「${name}」?使用该密钥的请求将立即被拒绝。`)) return;
try {
await api.revokeApiKey(id);
await load();
} catch (e) {
setError(e instanceof ApiError ? e.message : "操作失败");
}
}
return (
<div className="mx-auto max-w-4xl">
<div className="flex items-center justify-between mb-4">
<div>
<h2 className="text-lg font-semibold text-gray-800 flex items-center gap-2">
<KeyRound className="h-5 w-5 text-emerald-600" /> API
</h2>
<p className="text-sm text-gray-500 mt-1">
API
</p>
</div>
<button
onClick={() => setCreating(true)}
className="px-4 py-2 rounded-lg bg-emerald-600 text-white text-sm font-medium hover:bg-emerald-700 flex items-center gap-1.5"
>
<Plus className="h-4 w-4" />
</button>
</div>
{error && (
<div className="mb-3 bg-red-50 text-red-700 text-sm rounded px-4 py-2">{error}</div>
)}
{newKey && (
<div className="mb-4 bg-amber-50 border border-amber-200 rounded-lg p-4">
<div className="text-sm font-medium text-amber-800 mb-1">
</div>
<div className="flex items-center gap-2">
<code className="flex-1 bg-white border rounded px-3 py-2 text-sm break-all">
{newKey}
</code>
<button
onClick={() => navigator.clipboard?.writeText(newKey)}
className="px-3 py-2 rounded border text-sm text-gray-600 hover:bg-gray-50 flex items-center gap-1"
>
<Copy className="h-4 w-4" />
</button>
<button
onClick={() => setNewKey(null)}
className="px-3 py-2 rounded text-sm text-gray-500 hover:bg-gray-100"
>
</button>
</div>
</div>
)}
{creating && (
<CreateKeyForm
onClose={() => setCreating(false)}
onCreated={(plaintext) => {
setCreating(false);
setNewKey(plaintext);
load();
}}
/>
)}
<div className="bg-white border rounded-lg overflow-hidden">
<table className="w-full text-sm">
<thead className="bg-gray-50 text-gray-500 text-left">
<tr>
<th className="px-4 py-2 font-medium"></th>
<th className="px-4 py-2 font-medium"></th>
<th className="px-4 py-2 font-medium"></th>
<th className="px-4 py-2 font-medium">/</th>
<th className="px-4 py-2 font-medium">(/)</th>
<th className="px-4 py-2 font-medium"></th>
<th className="px-4 py-2 font-medium"></th>
<th className="px-4 py-2 font-medium"></th>
</tr>
</thead>
<tbody className="divide-y">
{rows.length === 0 ? (
<tr>
<td colSpan={8} className="px-4 py-8 text-center text-gray-400">
</td>
</tr>
) : (
rows.map((k) => (
<tr key={k.id} className={k.revoked_at ? "opacity-50" : ""}>
<td className="px-4 py-2 text-gray-800">
{k.name}
{k.owner_email && (
<span className="block text-xs text-gray-400">{k.owner_email}</span>
)}
</td>
<td className="px-4 py-2 text-gray-500">
<code>{k.key_prefix}</code>
</td>
<td className="px-4 py-2 text-gray-600">{tierLabel(k.tier)}</td>
<td className="px-4 py-2 text-gray-600">{k.rate_limit_per_min}</td>
<td className="px-4 py-2 text-gray-600">
{k.usage.today} / {k.usage.total}
</td>
<td className="px-4 py-2 text-gray-600">
{k.quota_total > 0 ? (
<span className={k.usage.total >= k.quota_total ? "text-red-600" : ""}>
{k.usage.total.toLocaleString()} / {k.quota_total.toLocaleString()}
</span>
) : (
<span className="text-gray-400"></span>
)}
</td>
<td className="px-4 py-2">
{k.revoked_at ? (
<span className="text-xs rounded px-2 py-0.5 bg-red-50 text-red-700">
</span>
) : (
<span className="text-xs rounded px-2 py-0.5 bg-emerald-50 text-emerald-700">
</span>
)}
</td>
<td className="px-4 py-2 text-right">
{!k.revoked_at && (
<button
onClick={() => revoke(k.id, k.name)}
className="text-gray-400 hover:text-red-600"
title="吊销"
>
<Trash2 className="h-4 w-4" />
</button>
)}
</td>
</tr>
))
)}
</tbody>
</table>
</div>
</div>
);
}
function CreateKeyForm({
onClose,
onCreated,
}: {
onClose: () => void;
onCreated: (plaintext: string) => void;
}) {
const [name, setName] = useState("");
const [ownerEmail, setOwnerEmail] = useState("");
const [tier, setTier] = useState("free");
const [rate, setRate] = useState(120);
const [quota, setQuota] = useState(1000);
const [busy, setBusy] = useState(false);
const [error, setError] = useState("");
function pickTier(t: string) {
setTier(t);
const def = TIERS.find((x) => x.key === t);
if (def) {
setRate(def.rate);
setQuota(def.quota);
}
}
async function submit() {
if (!name.trim()) {
setError("名称不能为空");
return;
}
setBusy(true);
setError("");
try {
const res = await api.createApiKey({
name: name.trim(),
owner_email: ownerEmail.trim() || undefined,
tier,
rate_limit_per_min: rate,
quota_total: quota,
});
onCreated(res.key);
} catch (e) {
setError(e instanceof ApiError ? e.message : "创建失败");
} finally {
setBusy(false);
}
}
return (
<div className="mb-4 bg-white border rounded-lg p-5">
<h3 className="font-medium text-gray-700 mb-3"></h3>
{error && <div className="mb-3 bg-red-50 text-red-700 text-sm rounded px-3 py-2">{error}</div>}
<div className="grid grid-cols-2 gap-4">
<label className="block">
<span className="text-xs text-gray-500"> *</span>
<input
className="w-full border rounded-md px-3 py-2 text-sm mt-1"
value={name}
onChange={(e) => setName(e.target.value)}
placeholder="例如:我的 App / 合作方 X"
/>
</label>
<label className="block">
<span className="text-xs text-gray-500"></span>
<input
className="w-full border rounded-md px-3 py-2 text-sm mt-1"
value={ownerEmail}
onChange={(e) => setOwnerEmail(e.target.value)}
placeholder="owner@example.com"
/>
</label>
<label className="block">
<span className="text-xs text-gray-500"></span>
<select
className="w-full border rounded-md px-3 py-2 text-sm mt-1 bg-white"
value={tier}
onChange={(e) => pickTier(e.target.value)}
>
{TIERS.map((t) => (
<option key={t.key} value={t.key}>
{t.label}
</option>
))}
</select>
</label>
<label className="block">
<span className="text-xs text-gray-500">/</span>
<input
type="number"
min={1}
className="w-full border rounded-md px-3 py-2 text-sm mt-1"
value={rate}
onChange={(e) => setRate(Math.max(1, parseInt(e.target.value || "1", 10)))}
/>
</label>
<label className="block">
<span className="text-xs text-gray-500">0=</span>
<input
type="number"
min={0}
className="w-full border rounded-md px-3 py-2 text-sm mt-1"
value={quota}
onChange={(e) => setQuota(Math.max(0, parseInt(e.target.value || "0", 10)))}
/>
</label>
</div>
<div className="mt-4 flex gap-2">
<button
onClick={submit}
disabled={busy}
className="px-4 py-2 rounded bg-emerald-600 text-white text-sm hover:bg-emerald-700 disabled:opacity-60"
>
</button>
<button onClick={onClose} className="px-4 py-2 rounded border text-sm text-gray-600">
</button>
</div>
</div>
);
}
@@ -0,0 +1,115 @@
import { useEffect, useState } from "react";
import { api, ApiError } from "../api";
import type { AuditLogRow } from "../types";
import { ScrollText } from "lucide-react";
const ACTION_LABEL: Record<string, string> = {
create: "新建",
update: "修改",
delete: "删除",
add_image: "添加图片",
delete_image: "删除图片",
bulk_status: "批量改状态",
bulk_category: "批量改分类",
};
const ENTITY_LABEL: Record<string, string> = {
product: "商品",
category: "分类",
brand: "品牌",
};
export default function AuditLogPage() {
const [rows, setRows] = useState<AuditLogRow[]>([]);
const [total, setTotal] = useState(0);
const [page, setPage] = useState(1);
const size = 30;
const [error, setError] = useState("");
useEffect(() => {
setError("");
api
.auditLog(page, size)
.then((r) => {
setRows(r.items);
setTotal(r.total);
})
.catch((e) => setError(e instanceof ApiError ? e.message : "加载失败"));
}, [page]);
const pages = Math.max(1, Math.ceil(total / size));
return (
<div className="mx-auto max-w-5xl">
<h2 className="mb-4 flex items-center gap-2 text-lg font-semibold text-gray-800">
<ScrollText className="h-5 w-5 text-emerald-600" />
<span className="text-sm font-normal text-gray-400"> {total} </span>
</h2>
{error && (
<div className="mb-3 rounded bg-red-50 px-4 py-2 text-sm text-red-700">{error}</div>
)}
<div className="overflow-hidden rounded-lg border bg-white">
<table className="w-full text-sm">
<thead className="bg-gray-50 text-left text-xs uppercase text-gray-500">
<tr>
<th className="px-4 py-2 font-medium"></th>
<th className="px-4 py-2 font-medium"></th>
<th className="px-4 py-2 font-medium"></th>
<th className="px-4 py-2 font-medium"></th>
<th className="px-4 py-2 font-medium"></th>
</tr>
</thead>
<tbody className="divide-y">
{rows.length === 0 ? (
<tr>
<td colSpan={5} className="px-4 py-8 text-center text-gray-400">
</td>
</tr>
) : (
rows.map((e) => (
<tr key={e.id} className="hover:bg-gray-50">
<td className="whitespace-nowrap px-4 py-2 text-gray-500">
{new Date(e.created_at).toLocaleString()}
</td>
<td className="px-4 py-2 text-gray-700">{e.actor}</td>
<td className="px-4 py-2 text-gray-700">
{ACTION_LABEL[e.action] || e.action}
</td>
<td className="px-4 py-2 text-gray-600">
{ENTITY_LABEL[e.entity] || e.entity}
</td>
<td className="px-4 py-2 text-xs text-gray-400">
{e.fields.length ? e.fields.join(", ") : "—"}
</td>
</tr>
))
)}
</tbody>
</table>
</div>
<div className="mt-4 flex items-center justify-end gap-2 text-sm text-gray-600">
<button
disabled={page <= 1}
onClick={() => setPage((p) => p - 1)}
className="rounded border border-gray-300 px-3 py-1 disabled:opacity-50"
>
</button>
<span>
{page} / {pages}
</span>
<button
disabled={page >= pages}
onClick={() => setPage((p) => p + 1)}
className="rounded border border-gray-300 px-3 py-1 disabled:opacity-50"
>
</button>
</div>
</div>
);
}
@@ -0,0 +1,305 @@
import { useEffect, useMemo, useState } from "react";
import { api, ApiError } from "../api";
import type { Brand } from "../types";
import { Tag, Pencil, Plus, Trash2, GitMerge, Search } from "lucide-react";
type EditState = { id: string; name: string };
type MergeState = { source: Brand; targetId: string };
export default function BrandsPage() {
const [rows, setRows] = useState<Brand[]>([]);
const [error, setError] = useState("");
const [query, setQuery] = useState("");
const [creating, setCreating] = useState(false);
const [newName, setNewName] = useState("");
const [edit, setEdit] = useState<EditState | null>(null);
const [merge, setMerge] = useState<MergeState | null>(null);
const [busy, setBusy] = useState(false);
async function load() {
setError("");
try {
const res = await api.listBrands();
setRows(res.items);
} catch (e) {
setError(e instanceof ApiError ? e.message : "加载失败");
}
}
useEffect(() => {
load();
}, []);
const filtered = useMemo(() => {
const q = query.trim().toLowerCase();
if (!q) return rows;
return rows.filter((b) => b.name.toLowerCase().includes(q));
}, [rows, query]);
function fail(e: unknown, fallback: string) {
setError(e instanceof ApiError ? e.message : fallback);
}
async function create() {
if (!newName.trim()) return;
setBusy(true);
setError("");
try {
await api.createBrand(newName.trim());
setNewName("");
setCreating(false);
await load();
} catch (e) {
fail(e, "新建失败");
} finally {
setBusy(false);
}
}
async function saveEdit() {
if (!edit || !edit.name.trim()) return;
setBusy(true);
setError("");
try {
await api.updateBrand(edit.id, edit.name.trim());
setEdit(null);
await load();
} catch (e) {
fail(e, "保存失败");
} finally {
setBusy(false);
}
}
async function remove(b: Brand) {
if (!confirm(`确认删除品牌「${b.name}」?`)) return;
setError("");
try {
await api.deleteBrand(b.id);
await load();
} catch (e) {
fail(e, "删除失败");
}
}
async function doMerge() {
if (!merge || !merge.targetId) return;
const target = rows.find((b) => b.id === merge.targetId);
if (
!confirm(
`将「${merge.source.name}」的 ${merge.source.product_count} 个商品并入「${target?.name}」,并删除「${merge.source.name}」?`,
)
)
return;
setBusy(true);
setError("");
try {
await api.mergeBrands(merge.source.id, merge.targetId);
setMerge(null);
await load();
} catch (e) {
fail(e, "合并失败");
} finally {
setBusy(false);
}
}
return (
<div className="mx-auto max-w-4xl">
<div className="flex items-center justify-between mb-4">
<div>
<h2 className="text-lg font-semibold text-gray-800 flex items-center gap-2">
<Tag className="h-5 w-5 text-emerald-600" />
</h2>
<p className="text-sm text-gray-500 mt-1">
</p>
</div>
<button
onClick={() => {
setCreating(true);
setNewName("");
}}
className="px-4 py-2 rounded-lg bg-emerald-600 text-white text-sm font-medium hover:bg-emerald-700 flex items-center gap-1.5"
>
<Plus className="h-4 w-4" />
</button>
</div>
{error && (
<div className="mb-3 bg-red-50 text-red-700 text-sm rounded px-4 py-2">{error}</div>
)}
<div className="mb-3 relative w-72">
<Search className="absolute left-2 top-2.5 h-4 w-4 text-gray-400" />
<input
value={query}
onChange={(e) => setQuery(e.target.value)}
placeholder="筛选品牌名"
className="w-full rounded border border-gray-300 py-2 pl-8 pr-3 text-sm focus:border-emerald-500 focus:outline-none"
/>
</div>
{creating && (
<div className="mb-4 bg-white border rounded-lg p-5">
<h3 className="font-medium text-gray-700 mb-3"></h3>
<div className="flex items-end gap-2">
<label className="block flex-1">
<span className="text-xs text-gray-500"> *</span>
<input
autoFocus
className="w-full border rounded-md px-3 py-2 text-sm mt-1"
value={newName}
onChange={(e) => setNewName(e.target.value)}
onKeyDown={(e) => e.key === "Enter" && create()}
placeholder="例如:可口可乐"
/>
</label>
<button
onClick={create}
disabled={busy}
className="px-4 py-2 rounded bg-emerald-600 text-white text-sm hover:bg-emerald-700 disabled:opacity-60"
>
</button>
<button
onClick={() => setCreating(false)}
className="px-4 py-2 rounded border text-sm text-gray-600"
>
</button>
</div>
</div>
)}
{merge && (
<div className="mb-4 bg-white border rounded-lg p-5">
<h3 className="font-medium text-gray-700 mb-3 flex items-center gap-1.5">
<GitMerge className="h-4 w-4 text-emerald-600" />
</h3>
<p className="text-sm text-gray-500 mb-3">
{merge.source.name}{merge.source.product_count}
</p>
<div className="flex items-end gap-2">
<label className="block flex-1">
<span className="text-xs text-gray-500"></span>
<select
className="w-full border rounded-md px-3 py-2 text-sm mt-1 bg-white"
value={merge.targetId}
onChange={(e) => setMerge({ ...merge, targetId: e.target.value })}
>
<option value=""></option>
{rows
.filter((b) => b.id !== merge.source.id)
.map((b) => (
<option key={b.id} value={b.id}>
{b.name}{b.product_count}
</option>
))}
</select>
</label>
<button
onClick={doMerge}
disabled={busy || !merge.targetId}
className="px-4 py-2 rounded bg-emerald-600 text-white text-sm hover:bg-emerald-700 disabled:opacity-60"
>
</button>
<button
onClick={() => setMerge(null)}
className="px-4 py-2 rounded border text-sm text-gray-600"
>
</button>
</div>
</div>
)}
<div className="bg-white border rounded-lg overflow-hidden">
<table className="w-full text-sm">
<thead className="bg-gray-50 text-gray-500 text-left">
<tr>
<th className="px-4 py-2 font-medium"></th>
<th className="px-4 py-2 font-medium"></th>
<th className="px-4 py-2 font-medium"></th>
</tr>
</thead>
<tbody className="divide-y">
{filtered.length === 0 ? (
<tr>
<td colSpan={3} className="px-4 py-8 text-center text-gray-400">
{rows.length === 0 ? "暂无品牌" : "无匹配品牌"}
</td>
</tr>
) : (
filtered.map((b) => (
<tr key={b.id} className="hover:bg-gray-50">
<td className="px-4 py-2 text-gray-800">
{edit && edit.id === b.id ? (
<input
autoFocus
className="border rounded px-2 py-1 text-sm w-64"
value={edit.name}
onChange={(e) => setEdit({ ...edit, name: e.target.value })}
onKeyDown={(e) => {
if (e.key === "Enter") saveEdit();
if (e.key === "Escape") setEdit(null);
}}
/>
) : (
<span className="font-medium">{b.name}</span>
)}
</td>
<td className="px-4 py-2 text-gray-600">{b.product_count}</td>
<td className="px-4 py-2 text-right whitespace-nowrap">
{edit && edit.id === b.id ? (
<>
<button
onClick={saveEdit}
disabled={busy}
className="text-emerald-600 hover:text-emerald-700 text-xs mr-3"
>
</button>
<button
onClick={() => setEdit(null)}
className="text-gray-400 hover:text-gray-600 text-xs"
>
</button>
</>
) : (
<>
<button
onClick={() => setEdit({ id: b.id, name: b.name })}
className="text-gray-400 hover:text-emerald-600 mr-3"
title="重命名"
>
<Pencil className="h-4 w-4" />
</button>
<button
onClick={() => setMerge({ source: b, targetId: "" })}
className="text-gray-400 hover:text-emerald-600 mr-3"
title="合并到其它品牌"
>
<GitMerge className="h-4 w-4" />
</button>
<button
onClick={() => remove(b)}
className="text-gray-400 hover:text-red-600"
title="删除"
>
<Trash2 className="h-4 w-4" />
</button>
</>
)}
</td>
</tr>
))
)}
</tbody>
</table>
</div>
</div>
);
}
@@ -0,0 +1,311 @@
import { useEffect, useMemo, useState } from "react";
import { api, ApiError } from "../api";
import type { Category, CategoryInput } from "../types";
import { FolderTree, Pencil, Plus, Trash2 } from "lucide-react";
type FormState = {
mode: "create" | "edit";
id?: string;
name_zh: string;
name_en: string;
slug: string;
parent_id: string; // "" = top level
gpc_brick_code: string;
};
function emptyForm(parentId = ""): FormState {
return {
mode: "create",
name_zh: "",
name_en: "",
slug: "",
parent_id: parentId,
gpc_brick_code: "",
};
}
export default function CategoriesPage() {
const [rows, setRows] = useState<Category[]>([]);
const [error, setError] = useState("");
const [form, setForm] = useState<FormState | null>(null);
async function load() {
setError("");
try {
const res = await api.listCategories();
setRows(res.items);
} catch (e) {
setError(e instanceof ApiError ? e.message : "加载失败");
}
}
useEffect(() => {
load();
}, []);
async function remove(c: Category) {
if (!confirm(`确认删除分类「${c.name_zh}」(${c.path})`)) return;
setError("");
try {
await api.deleteCategory(c.id);
await load();
} catch (e) {
setError(e instanceof ApiError ? e.message : "删除失败");
}
}
return (
<div className="mx-auto max-w-5xl">
<div className="flex items-center justify-between mb-4">
<div>
<h2 className="text-lg font-semibold text-gray-800 flex items-center gap-2">
<FolderTree className="h-5 w-5 text-emerald-600" />
</h2>
<p className="text-sm text-gray-500 mt-1">
(slug)
</p>
</div>
<button
onClick={() => setForm(emptyForm())}
className="px-4 py-2 rounded-lg bg-emerald-600 text-white text-sm font-medium hover:bg-emerald-700 flex items-center gap-1.5"
>
<Plus className="h-4 w-4" />
</button>
</div>
{error && (
<div className="mb-3 bg-red-50 text-red-700 text-sm rounded px-4 py-2">{error}</div>
)}
{form && (
<CategoryForm
form={form}
categories={rows}
onClose={() => setForm(null)}
onSaved={() => {
setForm(null);
load();
}}
onError={setError}
/>
)}
<div className="bg-white border rounded-lg overflow-hidden">
<table className="w-full text-sm">
<thead className="bg-gray-50 text-gray-500 text-left">
<tr>
<th className="px-4 py-2 font-medium"></th>
<th className="px-4 py-2 font-medium"></th>
<th className="px-4 py-2 font-medium">GPC</th>
<th className="px-4 py-2 font-medium"></th>
<th className="px-4 py-2 font-medium"></th>
</tr>
</thead>
<tbody className="divide-y">
{rows.length === 0 ? (
<tr>
<td colSpan={5} className="px-4 py-8 text-center text-gray-400">
</td>
</tr>
) : (
rows.map((c) => (
<tr key={c.id} className="hover:bg-gray-50">
<td className="px-4 py-2 text-gray-800">
<span style={{ paddingLeft: `${c.level * 18}px` }} className="inline-flex items-center gap-2">
{c.level > 0 && <span className="text-gray-300"></span>}
<span className="font-medium">{c.name_zh}</span>
{c.name_en && <span className="text-xs text-gray-400">{c.name_en}</span>}
</span>
</td>
<td className="px-4 py-2 text-gray-500">
<code className="text-xs">{c.path}</code>
</td>
<td className="px-4 py-2 text-gray-500 text-xs">{c.gpc_brick_code || "—"}</td>
<td className="px-4 py-2 text-gray-600">{c.product_count}</td>
<td className="px-4 py-2 text-right whitespace-nowrap">
<button
onClick={() => setForm(emptyForm(c.id))}
className="text-gray-400 hover:text-emerald-600 mr-3"
title="新增子分类"
>
<Plus className="h-4 w-4" />
</button>
<button
onClick={() =>
setForm({
mode: "edit",
id: c.id,
name_zh: c.name_zh,
name_en: c.name_en ?? "",
slug: c.path,
parent_id: c.parent_id ?? "",
gpc_brick_code: c.gpc_brick_code ?? "",
})
}
className="text-gray-400 hover:text-emerald-600 mr-3"
title="编辑"
>
<Pencil className="h-4 w-4" />
</button>
<button
onClick={() => remove(c)}
className="text-gray-400 hover:text-red-600"
title="删除"
>
<Trash2 className="h-4 w-4" />
</button>
</td>
</tr>
))
)}
</tbody>
</table>
</div>
</div>
);
}
function CategoryForm({
form,
categories,
onClose,
onSaved,
onError,
}: {
form: FormState;
categories: Category[];
onClose: () => void;
onSaved: () => void;
onError: (msg: string) => void;
}) {
const [state, setState] = useState<FormState>(form);
const [busy, setBusy] = useState(false);
// When editing, the node itself and its descendants are not valid parents.
const parentOptions = useMemo(() => {
if (state.mode === "create") return categories;
const self = categories.find((c) => c.id === state.id);
if (!self) return categories;
return categories.filter(
(c) => c.id !== self.id && !c.path.startsWith(self.path + "."),
);
}, [categories, state.mode, state.id]);
function set<K extends keyof FormState>(key: K, value: FormState[K]) {
setState((s) => ({ ...s, [key]: value }));
}
async function submit() {
if (!state.name_zh.trim()) {
onError("分类名称不能为空");
return;
}
setBusy(true);
onError("");
const body: CategoryInput = {
name_zh: state.name_zh.trim(),
name_en: state.name_en.trim() || null,
parent_id: state.parent_id || null,
gpc_brick_code: state.gpc_brick_code.trim() || null,
};
if (state.mode === "create") body.slug = state.slug.trim() || null;
try {
if (state.mode === "create") {
await api.createCategory(body);
} else if (state.id) {
await api.updateCategory(state.id, body);
}
onSaved();
} catch (e) {
onError(e instanceof ApiError ? e.message : "保存失败");
} finally {
setBusy(false);
}
}
return (
<div className="mb-4 bg-white border rounded-lg p-5">
<h3 className="font-medium text-gray-700 mb-3">
{state.mode === "create" ? "新建分类" : "编辑分类"}
</h3>
<div className="grid grid-cols-2 gap-4">
<label className="block">
<span className="text-xs text-gray-500"> *</span>
<input
className="w-full border rounded-md px-3 py-2 text-sm mt-1"
value={state.name_zh}
onChange={(e) => set("name_zh", e.target.value)}
placeholder="例如:饮料"
/>
</label>
<label className="block">
<span className="text-xs text-gray-500"></span>
<input
className="w-full border rounded-md px-3 py-2 text-sm mt-1"
value={state.name_en}
onChange={(e) => set("name_en", e.target.value)}
placeholder="Beverages"
/>
</label>
<label className="block">
<span className="text-xs text-gray-500"></span>
<select
className="w-full border rounded-md px-3 py-2 text-sm mt-1 bg-white"
value={state.parent_id}
onChange={(e) => set("parent_id", e.target.value)}
>
<option value=""></option>
{parentOptions.map((c) => (
<option key={c.id} value={c.id}>
{"\u00A0".repeat(c.level * 2)}
{c.name_zh} ({c.path})
</option>
))}
</select>
</label>
{state.mode === "create" ? (
<label className="block">
<span className="text-xs text-gray-500"> slug</span>
<input
className="w-full border rounded-md px-3 py-2 text-sm mt-1"
value={state.slug}
onChange={(e) => set("slug", e.target.value)}
placeholder="beverages"
/>
</label>
) : (
<label className="block">
<span className="text-xs text-gray-500"></span>
<input
className="w-full border rounded-md px-3 py-2 text-sm mt-1 bg-gray-50 text-gray-400"
value={state.slug}
disabled
/>
</label>
)}
<label className="block">
<span className="text-xs text-gray-500">GPC Brick </span>
<input
className="w-full border rounded-md px-3 py-2 text-sm mt-1"
value={state.gpc_brick_code}
onChange={(e) => set("gpc_brick_code", e.target.value)}
placeholder="10000224"
/>
</label>
</div>
<div className="mt-4 flex gap-2">
<button
onClick={submit}
disabled={busy}
className="px-4 py-2 rounded bg-emerald-600 text-white text-sm hover:bg-emerald-700 disabled:opacity-60"
>
</button>
<button onClick={onClose} className="px-4 py-2 rounded border text-sm text-gray-600">
</button>
</div>
</div>
);
}
+75
View File
@@ -0,0 +1,75 @@
import { useState } from "react";
import { api, setToken } from "../api";
import { Package } from "lucide-react";
export default function Login({
onLoggedIn,
}: {
onLoggedIn: (username: string) => void;
}) {
const [username, setUsername] = useState("");
const [password, setPassword] = useState("");
const [error, setError] = useState("");
const [loading, setLoading] = useState(false);
async function submit(e: React.FormEvent) {
e.preventDefault();
setError("");
setLoading(true);
try {
const r = await api.login(username, password);
setToken(r.token);
onLoggedIn(r.username);
} catch (err) {
setError(err instanceof Error ? err.message : "登录失败");
} finally {
setLoading(false);
}
}
return (
<div className="flex h-full items-center justify-center">
<form
onSubmit={submit}
className="w-80 rounded-xl bg-white p-8 shadow-md"
>
<div className="mb-6 flex flex-col items-center gap-2">
<Package className="h-8 w-8 text-emerald-600" />
<h1 className="text-lg font-semibold text-gray-800">
·
</h1>
</div>
{error && (
<div className="mb-4 rounded bg-red-50 px-3 py-2 text-sm text-red-600">
{error}
</div>
)}
<label className="mb-3 block">
<span className="mb-1 block text-sm text-gray-600"></span>
<input
value={username}
onChange={(e) => setUsername(e.target.value)}
className="w-full rounded border border-gray-300 px-3 py-2 text-sm focus:border-emerald-500 focus:outline-none"
autoFocus
/>
</label>
<label className="mb-5 block">
<span className="mb-1 block text-sm text-gray-600"></span>
<input
type="password"
value={password}
onChange={(e) => setPassword(e.target.value)}
className="w-full rounded border border-gray-300 px-3 py-2 text-sm focus:border-emerald-500 focus:outline-none"
/>
</label>
<button
type="submit"
disabled={loading}
className="w-full rounded bg-emerald-600 py-2 text-sm font-medium text-white hover:bg-emerald-700 disabled:opacity-60"
>
{loading ? "登录中…" : "登录"}
</button>
</form>
</div>
);
}
@@ -0,0 +1,972 @@
import { useEffect, useMemo, useState } from "react";
import { api } from "../api";
import {
AuditEntry,
Brand,
Category,
FIELD_LABELS,
KindField,
ProductDetail as Detail,
} from "../types";
import {
ArrowLeft,
ChevronLeft,
ChevronRight,
Plus,
Save,
Trash2,
AlertCircle,
History,
Star,
} from "lucide-react";
const GTIN_TYPES = ["EAN13", "EAN8", "UPC", "ITF14", "GTIN14"];
const PACK_LEVELS: { value: string; label: string }[] = [
{ value: "each", label: "消费单元" },
{ value: "case", label: "箱" },
{ value: "pallet", label: "托盘" },
];
const NUTRIMENT_KEYS: { key: string; label: string }[] = [
{ key: "energy_kcal", label: "能量 (kcal)" },
{ key: "energy_kj", label: "能量 (kJ)" },
{ key: "fat", label: "脂肪 (g)" },
{ key: "saturated_fat", label: "饱和脂肪 (g)" },
{ key: "carbohydrates", label: "碳水 (g)" },
{ key: "sugars", label: "糖 (g)" },
{ key: "proteins", label: "蛋白质 (g)" },
{ key: "salt", label: "盐 (g)" },
];
const STATUS_OPTIONS = [
{ value: "active", label: "在用" },
{ value: "merged", label: "已合并" },
{ value: "deprecated", label: "已停用" },
];
const ACTION_LABEL: Record<string, string> = {
update: "编辑",
add_image: "新增图片",
delete_image: "删除图片",
add_msrp: "新增建议零售价",
delete_msrp: "删除建议零售价",
add_barcode: "新增条码",
delete_barcode: "删除条码",
set_primary_barcode: "设为主条码",
};
function Card({
title,
children,
}: {
title: string;
children: React.ReactNode;
}) {
return (
<div className="rounded-lg border border-gray-200 bg-white p-5">
<h3 className="mb-4 text-sm font-semibold text-gray-700">{title}</h3>
{children}
</div>
);
}
function Field({
label,
children,
}: {
label: string;
children: React.ReactNode;
}) {
return (
<label className="block">
<span className="mb-1 block text-xs text-gray-500">{label}</span>
{children}
</label>
);
}
const inputCls =
"w-full rounded border border-gray-300 px-3 py-2 text-sm focus:border-emerald-500 focus:outline-none";
export default function ProductDetail({
id,
onBack,
ids = [],
onNavigate,
}: {
id: string;
onBack: () => void;
ids?: string[];
onNavigate?: (id: string) => void;
}) {
const navIndex = ids.indexOf(id);
const prevId = navIndex > 0 ? ids[navIndex - 1] : null;
const nextId =
navIndex >= 0 && navIndex < ids.length - 1 ? ids[navIndex + 1] : null;
const [d, setD] = useState<Detail | null>(null);
const [brands, setBrands] = useState<Brand[]>([]);
const [categories, setCategories] = useState<Category[]>([]);
const [audit, setAudit] = useState<AuditEntry[]>([]);
const [loading, setLoading] = useState(true);
const [saving, setSaving] = useState(false);
const [msg, setMsg] = useState("");
const [error, setError] = useState("");
// editable form state
const [name, setName] = useState("");
const [gtin, setGtin] = useState("");
const [brandName, setBrandName] = useState("");
const [categoryId, setCategoryId] = useState("");
const [netValue, setNetValue] = useState("");
const [netUnit, setNetUnit] = useState("");
const [country, setCountry] = useState("");
const [status, setStatus] = useState("active");
const [ingredients, setIngredients] = useState("");
const [allergens, setAllergens] = useState("");
const [additives, setAdditives] = useState("");
const [nutriments, setNutriments] = useState<Record<string, string>>({});
const [basis, setBasis] = useState("");
const [serving, setServing] = useState("");
const [nutriScore, setNutriScore] = useState("");
const [kindFields, setKindFields] = useState<KindField[]>([]);
const [attrs, setAttrs] = useState<Record<string, string>>({});
function hydrate(detail: Detail) {
setD(detail);
setName(detail.name);
setGtin(detail.gtin || "");
setBrandName(detail.brand || "");
setCategoryId(detail.category_id || "");
setNetValue(detail.net_content_value?.toString() || "");
setNetUnit(detail.net_content_unit || "");
setCountry(detail.country_of_origin || "");
setStatus(detail.status);
setIngredients(detail.ingredients_text || "");
setAllergens(detail.allergens.join(", "));
setAdditives(detail.additives.join(", "));
const nm: Record<string, string> = {};
if (detail.nutriments) {
for (const [k, v] of Object.entries(detail.nutriments)) nm[k] = String(v);
}
setNutriments(nm);
setBasis(detail.nutrition_basis || "");
setServing(detail.serving_size || "");
setNutriScore(detail.nutri_score || "");
const am: Record<string, string> = {};
if (detail.attributes) {
for (const [k, v] of Object.entries(detail.attributes)) {
am[k] = v == null ? "" : Array.isArray(v) ? v.join(", ") : String(v);
}
}
setAttrs(am);
}
function reload() {
setLoading(true);
Promise.all([api.getProduct(id), api.listAudit(id)])
.then(([detail, a]) => {
hydrate(detail);
setAudit(a.items);
})
.catch((e) => setError(e.message))
.finally(() => setLoading(false));
}
useEffect(() => {
reload();
api.listBrands().then((r) => setBrands(r.items)).catch(() => {});
api.listCategories().then((r) => setCategories(r.items)).catch(() => {});
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [id]);
const missing = useMemo(() => d?.missing ?? [], [d]);
const selectedKind = useMemo(() => {
const c = categories.find((x) => x.id === categoryId);
return c?.archive_kind || d?.archive_kind || "generic";
}, [categories, categoryId, d]);
useEffect(() => {
if (selectedKind && selectedKind !== "food") {
api
.listKindFields(selectedKind)
.then((r) => setKindFields(r.items))
.catch(() => setKindFields([]));
} else {
setKindFields([]);
}
}, [selectedKind]);
const specGroups = useMemo(() => {
const groups: { label: string; fields: KindField[] }[] = [];
for (const f of kindFields) {
let g = groups.find((x) => x.label === f.group_label);
if (!g) {
g = { label: f.group_label, fields: [] };
groups.push(g);
}
g.fields.push(f);
}
return groups;
}, [kindFields]);
const attrLabels = useMemo(() => {
const m: Record<string, string> = {};
for (const f of kindFields) m[f.field_key] = f.label_zh;
return m;
}, [kindFields]);
function parseList(s: string): string[] {
return s
.split(",")
.map((x) => x.trim())
.filter(Boolean);
}
async function save() {
setSaving(true);
setMsg("");
setError("");
const nm: Record<string, number> = {};
for (const [k, v] of Object.entries(nutriments)) {
const n = parseFloat(v);
if (!Number.isNaN(n)) nm[k] = n;
}
let attributes: Record<string, unknown> | undefined;
if (selectedKind !== "food") {
attributes = {};
for (const f of kindFields) {
const raw = (attrs[f.field_key] ?? "").trim();
if (raw === "") continue;
if (f.field_type === "number") {
const n = parseFloat(raw);
if (!Number.isNaN(n)) attributes[f.field_key] = n;
} else if (f.field_type === "list") {
attributes[f.field_key] = parseList(raw);
} else {
attributes[f.field_key] = raw;
}
}
}
const body = {
gtin: gtin.trim() || null,
name: name.trim(),
brand_name: brandName.trim() || null,
brand_id: brandName.trim() ? undefined : null,
category_id: categoryId || null,
net_content_value: netValue.trim() ? parseFloat(netValue) : null,
net_content_unit: netUnit.trim() || null,
country_of_origin: country.trim() || null,
status,
ingredients_text: ingredients.trim() || null,
allergens: parseList(allergens),
additives: parseList(additives),
nutriments: nm,
nutrition_basis: basis || null,
serving_size: serving.trim() || null,
nutri_score: nutriScore || null,
...(attributes !== undefined ? { attributes } : {}),
};
try {
const updated = await api.updateProduct(id, body);
hydrate(updated);
const a = await api.listAudit(id);
setAudit(a.items);
setMsg("已保存");
setTimeout(() => setMsg(""), 2500);
} catch (e) {
setError(e instanceof Error ? e.message : "保存失败");
} finally {
setSaving(false);
}
}
if (loading) {
return <div className="text-gray-400"></div>;
}
if (!d) {
return (
<div>
<button onClick={onBack} className="text-emerald-600">
</button>
<p className="mt-4 text-red-600">{error || "未找到商品"}</p>
</div>
);
}
return (
<div className="mx-auto max-w-5xl space-y-5">
<div className="flex items-center justify-between">
<div className="flex items-center gap-2">
<button
onClick={onBack}
className="flex items-center gap-1 text-sm text-gray-600 hover:text-gray-900"
>
<ArrowLeft className="h-4 w-4" />
</button>
{ids.length > 1 && navIndex >= 0 && (
<div className="ml-2 flex items-center gap-1 text-sm">
<button
onClick={() => prevId && onNavigate?.(prevId)}
disabled={!prevId}
className="flex items-center gap-1 rounded border border-gray-300 px-2 py-1 text-gray-600 hover:bg-gray-50 disabled:opacity-40"
>
<ChevronLeft className="h-4 w-4" />
</button>
<span className="text-xs text-gray-400">
{navIndex + 1} / {ids.length}
</span>
<button
onClick={() => nextId && onNavigate?.(nextId)}
disabled={!nextId}
className="flex items-center gap-1 rounded border border-gray-300 px-2 py-1 text-gray-600 hover:bg-gray-50 disabled:opacity-40"
>
<ChevronRight className="h-4 w-4" />
</button>
</div>
)}
</div>
<div className="flex items-center gap-3">
{msg && <span className="text-sm text-emerald-600">{msg}</span>}
{error && <span className="text-sm text-red-600">{error}</span>}
<span className="text-xs text-gray-400">
{Math.round(d.quality_score * 100)}
</span>
<button
onClick={save}
disabled={saving}
className="flex items-center gap-1 rounded bg-emerald-600 px-4 py-2 text-sm text-white hover:bg-emerald-700 disabled:opacity-60"
>
<Save className="h-4 w-4" /> {saving ? "保存中…" : "保存"}
</button>
</div>
</div>
{missing.length > 0 && (
<div className="flex items-center gap-2 rounded-lg border border-amber-200 bg-amber-50 px-4 py-3 text-sm text-amber-700">
<AlertCircle className="h-4 w-4" />
{missing.map((f) => FIELD_LABELS[f] || attrLabels[f] || f).join("、")}
</div>
)}
<Card title="基础信息">
<div className="grid grid-cols-2 gap-4">
<Field label="名称 *">
<input
className={inputCls}
value={name}
onChange={(e) => setName(e.target.value)}
/>
</Field>
<Field label="条码 (GTIN)">
<input
className={inputCls}
value={gtin}
onChange={(e) => setGtin(e.target.value)}
/>
</Field>
<Field label="品牌(不存在将自动创建)">
<input
className={inputCls}
list="brand-list"
value={brandName}
onChange={(e) => setBrandName(e.target.value)}
/>
<datalist id="brand-list">
{brands.map((b) => (
<option key={b.id} value={b.name} />
))}
</datalist>
</Field>
<Field label="品类">
<select
className={inputCls}
value={categoryId}
onChange={(e) => setCategoryId(e.target.value)}
>
<option value=""></option>
{categories.map((c) => (
<option key={c.id} value={c.id}>
{"\u00A0".repeat(c.level * 2)}
{c.name_zh} ({c.path})
</option>
))}
</select>
</Field>
<Field label="净含量">
<input
className={inputCls}
type="number"
step="any"
value={netValue}
onChange={(e) => setNetValue(e.target.value)}
/>
</Field>
<Field label="净含量单位 (g/ml/cl…)">
<input
className={inputCls}
value={netUnit}
onChange={(e) => setNetUnit(e.target.value)}
/>
</Field>
<Field label="产地">
<input
className={inputCls}
value={country}
onChange={(e) => setCountry(e.target.value)}
/>
</Field>
<Field label="状态">
<select
className={inputCls}
value={status}
onChange={(e) => setStatus(e.target.value)}
>
{STATUS_OPTIONS.map((o) => (
<option key={o.value} value={o.value}>
{o.label}
</option>
))}
</select>
</Field>
</div>
</Card>
{selectedKind === "food" && (
<Card title="配料与营养">
<div className="mb-4 grid grid-cols-2 gap-4">
<Field label="配料表">
<textarea
className={inputCls}
rows={3}
value={ingredients}
onChange={(e) => setIngredients(e.target.value)}
/>
</Field>
<div className="grid grid-cols-2 gap-4">
<Field label="过敏原(逗号分隔)">
<input
className={inputCls}
value={allergens}
onChange={(e) => setAllergens(e.target.value)}
/>
</Field>
<Field label="添加剂(逗号分隔)">
<input
className={inputCls}
value={additives}
onChange={(e) => setAdditives(e.target.value)}
/>
</Field>
<Field label="营养基准">
<select
className={inputCls}
value={basis}
onChange={(e) => setBasis(e.target.value)}
>
<option value=""></option>
<option value="per_100g"> 100g</option>
<option value="per_100ml"> 100ml</option>
<option value="per_serving"></option>
</select>
</Field>
<Field label="份量">
<input
className={inputCls}
value={serving}
onChange={(e) => setServing(e.target.value)}
/>
</Field>
<Field label="Nutri-Score (A-E)">
<input
className={inputCls}
maxLength={1}
value={nutriScore}
onChange={(e) =>
setNutriScore(e.target.value.toUpperCase())
}
/>
</Field>
</div>
</div>
<div className="grid grid-cols-4 gap-3">
{NUTRIMENT_KEYS.map((n) => (
<Field key={n.key} label={n.label}>
<input
className={inputCls}
type="number"
step="any"
value={nutriments[n.key] ?? ""}
onChange={(e) =>
setNutriments((prev) => ({ ...prev, [n.key]: e.target.value }))
}
/>
</Field>
))}
</div>
</Card>
)}
{selectedKind !== "food" && kindFields.length > 0 && (
<Card title="规格参数">
{specGroups.map((grp) => (
<div key={grp.label} className="mb-4 last:mb-0">
{grp.label && (
<h4 className="mb-2 text-xs font-medium text-gray-500">
{grp.label}
</h4>
)}
<div className="grid grid-cols-3 gap-3">
{grp.fields.map((f) => (
<Field
key={f.field_key}
label={f.unit ? `${f.label_zh} (${f.unit})` : f.label_zh}
>
{f.field_type === "select" ? (
<select
className={inputCls}
value={attrs[f.field_key] ?? ""}
onChange={(e) =>
setAttrs((prev) => ({
...prev,
[f.field_key]: e.target.value,
}))
}
>
<option value=""></option>
{f.options.map((o) => (
<option key={o} value={o}>
{o}
</option>
))}
</select>
) : f.field_type === "textarea" ? (
<textarea
className={inputCls}
rows={3}
value={attrs[f.field_key] ?? ""}
onChange={(e) =>
setAttrs((prev) => ({
...prev,
[f.field_key]: e.target.value,
}))
}
/>
) : (
<input
className={inputCls}
type={f.field_type === "number" ? "number" : "text"}
step={f.field_type === "number" ? "any" : undefined}
placeholder={f.placeholder ?? undefined}
value={attrs[f.field_key] ?? ""}
onChange={(e) =>
setAttrs((prev) => ({
...prev,
[f.field_key]: e.target.value,
}))
}
/>
)}
</Field>
))}
</div>
</div>
))}
</Card>
)}
<BarcodesCard product={d} onChange={reload} onError={setError} />
<ImagesCard
product={d}
onChange={reload}
onError={setError}
/>
<MsrpCard product={d} onChange={reload} onError={setError} />
<Card title="操作记录">
{audit.length === 0 ? (
<p className="text-sm text-gray-400"></p>
) : (
<ul className="space-y-2 text-sm">
{audit.map((a) => (
<li
key={a.id}
className="flex items-center gap-3 text-gray-600"
>
<History className="h-3.5 w-3.5 text-gray-400" />
<span className="text-gray-400">{a.created_at}</span>
<span className="font-medium text-gray-700">{a.actor}</span>
<span>{ACTION_LABEL[a.action] || a.action}</span>
{a.fields.length > 0 && (
<span className="text-gray-400">
[{a.fields.map((f) => FIELD_LABELS[f] || f).join("、")}]
</span>
)}
</li>
))}
</ul>
)}
</Card>
</div>
);
}
function BarcodesCard({
product,
onChange,
onError,
}: {
product: Detail;
onChange: () => void;
onError: (m: string) => void;
}) {
const [gtin, setGtin] = useState("");
const [gtinType, setGtinType] = useState("EAN13");
const [packLevel, setPackLevel] = useState("each");
const [region, setRegion] = useState("");
const [busy, setBusy] = useState(false);
async function add() {
if (!gtin.trim()) return;
setBusy(true);
try {
await api.addBarcode(product.id, {
gtin: gtin.trim(),
gtin_type: gtinType,
pack_level: packLevel,
region: region.trim() || null,
is_primary: false,
});
setGtin("");
setRegion("");
onChange();
} catch (e) {
onError(e instanceof Error ? e.message : "添加失败");
} finally {
setBusy(false);
}
}
async function remove(barcodeId: string) {
try {
await api.deleteBarcode(product.id, barcodeId);
onChange();
} catch (e) {
onError(e instanceof Error ? e.message : "删除失败");
}
}
async function makePrimary(barcodeId: string) {
try {
await api.setPrimaryBarcode(product.id, barcodeId);
onChange();
} catch (e) {
onError(e instanceof Error ? e.message : "设置失败");
}
}
return (
<Card title="条码(一品多码,主条码镜像到 GTIN)">
<div className="mb-3 space-y-2">
{product.barcodes.length === 0 && (
<span className="text-sm text-gray-400"></span>
)}
{product.barcodes.map((b) => (
<div
key={b.id}
className="flex items-center gap-3 rounded border border-gray-100 bg-gray-50 px-3 py-2 text-sm"
>
<button
onClick={() => !b.is_primary && makePrimary(b.id)}
title={b.is_primary ? "主条码" : "设为主条码"}
disabled={b.is_primary}
className={
b.is_primary
? "text-amber-500"
: "text-gray-300 hover:text-amber-500"
}
>
<Star
className="h-4 w-4"
fill={b.is_primary ? "currentColor" : "none"}
/>
</button>
<span className="font-mono font-medium text-gray-800">
{b.gtin}
</span>
<span className="rounded bg-gray-200 px-1.5 py-0.5 text-[11px] text-gray-600">
{b.gtin_type}
</span>
<span className="text-gray-500">
{PACK_LEVELS.find((p) => p.value === b.pack_level)?.label ||
b.pack_level}
</span>
<span className="flex-1 text-gray-400">{b.region || ""}</span>
<button
onClick={() => remove(b.id)}
className="text-gray-400 hover:text-red-600"
>
<Trash2 className="h-4 w-4" />
</button>
</div>
))}
</div>
<div className="flex flex-wrap items-end gap-2">
<Field label="条码 (GTIN)">
<input
className="w-44 rounded border border-gray-300 px-3 py-2 text-sm"
value={gtin}
onChange={(e) => setGtin(e.target.value)}
placeholder="8/12/13/14 位"
/>
</Field>
<Field label="类型">
<select
className="rounded border border-gray-300 px-2 py-2 text-sm"
value={gtinType}
onChange={(e) => setGtinType(e.target.value)}
>
{GTIN_TYPES.map((t) => (
<option key={t} value={t}>
{t}
</option>
))}
</select>
</Field>
<Field label="包装层级">
<select
className="rounded border border-gray-300 px-2 py-2 text-sm"
value={packLevel}
onChange={(e) => setPackLevel(e.target.value)}
>
{PACK_LEVELS.map((p) => (
<option key={p.value} value={p.value}>
{p.label}
</option>
))}
</select>
</Field>
<Field label="地区(可选)">
<input
className="w-20 rounded border border-gray-300 px-3 py-2 text-sm"
value={region}
onChange={(e) => setRegion(e.target.value.toUpperCase())}
/>
</Field>
<button
onClick={add}
disabled={busy}
className="flex items-center gap-1 rounded bg-gray-700 px-3 py-2 text-sm text-white hover:bg-gray-800 disabled:opacity-60"
>
<Plus className="h-4 w-4" />
</button>
</div>
</Card>
);
}
function ImagesCard({
product,
onChange,
onError,
}: {
product: Detail;
onChange: () => void;
onError: (m: string) => void;
}) {
const [url, setUrl] = useState("");
const [kind, setKind] = useState("front");
async function add() {
if (!url.trim()) return;
try {
await api.addImage(product.id, url.trim(), kind);
setUrl("");
onChange();
} catch (e) {
onError(e instanceof Error ? e.message : "添加失败");
}
}
async function remove(imageId: string) {
try {
await api.deleteImage(product.id, imageId);
onChange();
} catch (e) {
onError(e instanceof Error ? e.message : "删除失败");
}
}
return (
<Card title="图片(仅存 URL">
<div className="mb-3 flex flex-wrap gap-3">
{product.images.length === 0 && (
<span className="text-sm text-gray-400"></span>
)}
{product.images.map((im) => (
<div
key={im.id}
className="relative h-24 w-24 overflow-hidden rounded border border-gray-200"
>
<img
src={im.url}
alt={im.kind}
className="h-full w-full object-cover"
/>
<button
onClick={() => remove(im.id)}
className="absolute right-1 top-1 rounded bg-black/50 p-1 text-white hover:bg-black/70"
>
<Trash2 className="h-3 w-3" />
</button>
<span className="absolute bottom-0 left-0 bg-black/50 px-1 text-[10px] text-white">
{im.kind}
</span>
</div>
))}
</div>
<div className="flex items-center gap-2">
<input
className={inputCls}
placeholder="图片 URL"
value={url}
onChange={(e) => setUrl(e.target.value)}
/>
<select
className="rounded border border-gray-300 px-2 py-2 text-sm"
value={kind}
onChange={(e) => setKind(e.target.value)}
>
<option value="front"></option>
<option value="ingredients"></option>
<option value="nutrition"></option>
<option value="other"></option>
</select>
<button
onClick={add}
className="flex items-center gap-1 whitespace-nowrap rounded bg-gray-700 px-3 py-2 text-sm text-white hover:bg-gray-800"
>
<Plus className="h-4 w-4" />
</button>
</div>
</Card>
);
}
function MsrpCard({
product,
onChange,
onError,
}: {
product: Detail;
onChange: () => void;
onError: (m: string) => void;
}) {
const [amount, setAmount] = useState("");
const [currency, setCurrency] = useState("CNY");
const [region, setRegion] = useState("CN");
const [date, setDate] = useState("");
const [note, setNote] = useState("");
async function add() {
const a = parseFloat(amount);
if (Number.isNaN(a)) return;
try {
await api.addMsrp(product.id, {
amount: a,
currency,
region,
effective_date: date || null,
note: note.trim() || null,
});
setAmount("");
setNote("");
setDate("");
onChange();
} catch (e) {
onError(e instanceof Error ? e.message : "添加失败");
}
}
async function remove(msrpId: string) {
try {
await api.deleteMsrp(product.id, msrpId);
onChange();
} catch (e) {
onError(e instanceof Error ? e.message : "删除失败");
}
}
return (
<Card title="官方建议零售价(MSRP 快照,非售卖)">
<div className="mb-3 space-y-2">
{product.msrp.length === 0 && (
<span className="text-sm text-gray-400"></span>
)}
{product.msrp.map((m) => (
<div
key={m.id}
className="flex items-center gap-3 rounded border border-gray-100 bg-gray-50 px-3 py-2 text-sm"
>
<span className="font-medium text-gray-800">
{m.amount} {m.currency}
</span>
<span className="text-gray-500">{m.region}</span>
<span className="text-gray-400">{m.effective_date || ""}</span>
<span className="flex-1 text-gray-400">{m.note || ""}</span>
<button
onClick={() => remove(m.id)}
className="text-gray-400 hover:text-red-600"
>
<Trash2 className="h-4 w-4" />
</button>
</div>
))}
</div>
<div className="flex flex-wrap items-end gap-2">
<Field label="金额">
<input
className="w-28 rounded border border-gray-300 px-3 py-2 text-sm"
type="number"
step="any"
value={amount}
onChange={(e) => setAmount(e.target.value)}
/>
</Field>
<Field label="币种">
<input
className="w-20 rounded border border-gray-300 px-3 py-2 text-sm"
value={currency}
onChange={(e) => setCurrency(e.target.value.toUpperCase())}
/>
</Field>
<Field label="地区">
<input
className="w-20 rounded border border-gray-300 px-3 py-2 text-sm"
value={region}
onChange={(e) => setRegion(e.target.value.toUpperCase())}
/>
</Field>
<Field label="生效日期">
<input
className="rounded border border-gray-300 px-3 py-2 text-sm"
type="date"
value={date}
onChange={(e) => setDate(e.target.value)}
/>
</Field>
<Field label="备注">
<input
className="w-40 rounded border border-gray-300 px-3 py-2 text-sm"
value={note}
onChange={(e) => setNote(e.target.value)}
/>
</Field>
<button
onClick={add}
className="flex items-center gap-1 rounded bg-gray-700 px-3 py-2 text-sm text-white hover:bg-gray-800"
>
<Plus className="h-4 w-4" />
</button>
</div>
</Card>
);
}
@@ -0,0 +1,574 @@
import { useEffect, useState } from "react";
import { api, ApiError } from "../api";
import { Brand, Category, FIELD_LABELS, ProductRow } from "../types";
import { Search, AlertCircle, Plus, ChevronUp, ChevronDown, ChevronsUpDown } from "lucide-react";
type SortKey =
| "name"
| "brand"
| "gtin"
| "category_path"
| "status"
| "quality_score";
const STATUS_LABEL: Record<string, string> = {
active: "在用",
merged: "已合并",
deprecated: "已停用",
};
function QualityBadge({ score }: { score: number }) {
const pct = Math.round(score * 100);
const color =
score >= 0.8
? "bg-emerald-100 text-emerald-700"
: score >= 0.5
? "bg-amber-100 text-amber-700"
: "bg-red-100 text-red-700";
return (
<span className={`rounded px-2 py-0.5 text-xs font-medium ${color}`}>
{pct}
</span>
);
}
function SortableTh({
label,
sortKey,
sort,
order,
onSort,
}: {
label: string;
sortKey: SortKey;
sort: SortKey | "";
order: "asc" | "desc";
onSort: (key: SortKey) => void;
}) {
const active = sort === sortKey;
return (
<th className="px-4 py-3">
<button
type="button"
onClick={() => onSort(sortKey)}
className={`flex items-center gap-1 uppercase hover:text-gray-700 ${
active ? "text-emerald-600" : ""
}`}
>
{label}
{!active ? (
<ChevronsUpDown className="h-3.5 w-3.5 text-gray-300" />
) : order === "asc" ? (
<ChevronUp className="h-3.5 w-3.5" />
) : (
<ChevronDown className="h-3.5 w-3.5" />
)}
</button>
</th>
);
}
export default function ProductList({
onOpen,
}: {
onOpen: (id: string, ids: string[]) => void;
}) {
const [q, setQ] = useState("");
const [input, setInput] = useState("");
const [page, setPage] = useState(1);
const [size, setSize] = useState(20);
const [sort, setSort] = useState<SortKey | "">("");
const [order, setOrder] = useState<"asc" | "desc">("asc");
const [jump, setJump] = useState("");
const [rows, setRows] = useState<ProductRow[]>([]);
const [total, setTotal] = useState(0);
const [loading, setLoading] = useState(false);
const [error, setError] = useState("");
const [creating, setCreating] = useState(false);
const [selected, setSelected] = useState<Set<string>>(new Set());
const [categories, setCategories] = useState<Category[]>([]);
const [bulkStatus, setBulkStatus] = useState("");
const [bulkCategory, setBulkCategory] = useState("");
const [bulkBusy, setBulkBusy] = useState(false);
function reload() {
setLoading(true);
setError("");
api
.listProducts(q, page, size, sort || undefined, order)
.then((r) => {
setRows(r.items);
setTotal(r.total);
})
.catch((e) => setError(e.message))
.finally(() => setLoading(false));
}
useEffect(() => {
setSelected(new Set());
reload();
}, [q, page, size, sort, order]);
function toggleSort(key: SortKey) {
setPage(1);
if (sort !== key) {
setSort(key);
setOrder("asc");
} else if (order === "asc") {
setOrder("desc");
} else {
setSort("");
setOrder("asc");
}
}
useEffect(() => {
api.listCategories().then((r) => setCategories(r.items)).catch(() => {});
}, []);
function toggle(id: string) {
setSelected((prev) => {
const next = new Set(prev);
if (next.has(id)) next.delete(id);
else next.add(id);
return next;
});
}
function toggleAll() {
setSelected((prev) =>
prev.size === rows.length ? new Set() : new Set(rows.map((r) => r.id)),
);
}
async function applyBulkStatus() {
if (!bulkStatus || selected.size === 0) return;
setBulkBusy(true);
setError("");
try {
await api.bulkProducts({
ids: [...selected],
action: "status",
status: bulkStatus,
});
setSelected(new Set());
setBulkStatus("");
reload();
} catch (e) {
setError(e instanceof ApiError ? e.message : "批量操作失败");
} finally {
setBulkBusy(false);
}
}
async function applyBulkCategory() {
if (selected.size === 0) return;
setBulkBusy(true);
setError("");
try {
await api.bulkProducts({
ids: [...selected],
action: "category",
category_id: bulkCategory || null,
});
setSelected(new Set());
setBulkCategory("");
reload();
} catch (e) {
setError(e instanceof ApiError ? e.message : "批量操作失败");
} finally {
setBulkBusy(false);
}
}
const pages = Math.max(1, Math.ceil(total / size));
return (
<div className="mx-auto max-w-6xl">
<div className="mb-4 flex items-center justify-between">
<h2 className="text-xl font-semibold text-gray-800">
<span className="text-sm font-normal text-gray-400"> {total} </span>
</h2>
<form
onSubmit={(e) => {
e.preventDefault();
setPage(1);
setQ(input.trim());
}}
className="flex items-center gap-2"
>
<div className="relative">
<Search className="absolute left-2 top-2.5 h-4 w-4 text-gray-400" />
<input
value={input}
onChange={(e) => setInput(e.target.value)}
placeholder="按名称 / 条码搜索"
className="w-64 rounded border border-gray-300 py-2 pl-8 pr-3 text-sm focus:border-emerald-500 focus:outline-none"
/>
</div>
<button className="rounded bg-emerald-600 px-3 py-2 text-sm text-white hover:bg-emerald-700">
</button>
<button
type="button"
onClick={() => setCreating(true)}
className="flex items-center gap-1 rounded bg-gray-700 px-3 py-2 text-sm text-white hover:bg-gray-800"
>
<Plus className="h-4 w-4" />
</button>
</form>
</div>
{creating && (
<CreateProductModal
onClose={() => setCreating(false)}
onCreated={(id) => {
setCreating(false);
onOpen(id, [id]);
}}
/>
)}
{error && (
<div className="mb-3 rounded bg-red-50 px-3 py-2 text-sm text-red-600">
{error}
</div>
)}
{selected.size > 0 && (
<div className="mb-3 flex flex-wrap items-center gap-3 rounded-lg border border-emerald-200 bg-emerald-50 px-4 py-3 text-sm">
<span className="font-medium text-emerald-800">
{selected.size}
</span>
<div className="flex items-center gap-1.5">
<span className="text-gray-500"></span>
<select
value={bulkStatus}
onChange={(e) => setBulkStatus(e.target.value)}
className="rounded border border-gray-300 bg-white px-2 py-1"
>
<option value=""></option>
<option value="active"></option>
<option value="deprecated"></option>
<option value="merged"></option>
</select>
<button
onClick={applyBulkStatus}
disabled={bulkBusy || !bulkStatus}
className="rounded bg-emerald-600 px-3 py-1 text-white hover:bg-emerald-700 disabled:opacity-50"
>
</button>
</div>
<div className="flex items-center gap-1.5">
<span className="text-gray-500"></span>
<select
value={bulkCategory}
onChange={(e) => setBulkCategory(e.target.value)}
className="rounded border border-gray-300 bg-white px-2 py-1"
>
<option value=""></option>
{categories.map((c) => (
<option key={c.id} value={c.id}>
{"\u00A0".repeat(c.level * 2)}
{c.name_zh}
</option>
))}
</select>
<button
onClick={applyBulkCategory}
disabled={bulkBusy}
className="rounded bg-emerald-600 px-3 py-1 text-white hover:bg-emerald-700 disabled:opacity-50"
>
</button>
</div>
<button
onClick={() => setSelected(new Set())}
className="text-gray-500 hover:text-gray-700"
>
</button>
</div>
)}
<div className="overflow-hidden rounded-lg border border-gray-200 bg-white">
<table className="w-full text-sm">
<thead className="bg-gray-50 text-left text-xs uppercase text-gray-500">
<tr>
<th className="w-10 px-4 py-3">
<input
type="checkbox"
checked={rows.length > 0 && selected.size === rows.length}
onChange={toggleAll}
aria-label="全选"
/>
</th>
<SortableTh label="名称" sortKey="name" sort={sort} order={order} onSort={toggleSort} />
<SortableTh label="品牌" sortKey="brand" sort={sort} order={order} onSort={toggleSort} />
<SortableTh label="条码" sortKey="gtin" sort={sort} order={order} onSort={toggleSort} />
<SortableTh label="品类" sortKey="category_path" sort={sort} order={order} onSort={toggleSort} />
<SortableTh label="状态" sortKey="status" sort={sort} order={order} onSort={toggleSort} />
<SortableTh label="质量分" sortKey="quality_score" sort={sort} order={order} onSort={toggleSort} />
<th className="px-4 py-3"></th>
</tr>
</thead>
<tbody className="divide-y divide-gray-100">
{loading ? (
<tr>
<td colSpan={8} className="px-4 py-8 text-center text-gray-400">
</td>
</tr>
) : rows.length === 0 ? (
<tr>
<td colSpan={8} className="px-4 py-8 text-center text-gray-400">
</td>
</tr>
) : (
rows.map((r) => (
<tr
key={r.id}
onClick={() => onOpen(r.id, rows.map((x) => x.id))}
className={`cursor-pointer hover:bg-emerald-50/50 ${
selected.has(r.id) ? "bg-emerald-50/60" : ""
}`}
>
<td
className="px-4 py-3"
onClick={(e) => e.stopPropagation()}
>
<input
type="checkbox"
checked={selected.has(r.id)}
onChange={() => toggle(r.id)}
aria-label="选择"
/>
</td>
<td className="px-4 py-3 font-medium text-gray-800">{r.name}</td>
<td className="px-4 py-3 text-gray-600">{r.brand || "—"}</td>
<td className="px-4 py-3 font-mono text-xs text-gray-500">
{r.gtin || "—"}
</td>
<td className="px-4 py-3 text-xs text-gray-500">
{r.category_path || "—"}
</td>
<td className="px-4 py-3 text-gray-600">
{STATUS_LABEL[r.status] || r.status}
</td>
<td className="px-4 py-3">
<QualityBadge score={r.quality_score} />
</td>
<td className="px-4 py-3">
{r.missing.length === 0 ? (
<span className="text-xs text-emerald-600"></span>
) : (
<span className="flex items-center gap-1 text-xs text-amber-600">
<AlertCircle className="h-3.5 w-3.5" />
{r.missing
.map((f) => FIELD_LABELS[f] || f)
.join("、")}
</span>
)}
</td>
</tr>
))
)}
</tbody>
</table>
</div>
<div className="mt-4 flex flex-wrap items-center justify-end gap-2 text-sm text-gray-600">
<div className="mr-auto flex items-center gap-1">
<span></span>
<select
value={size}
onChange={(e) => {
setSize(Number(e.target.value));
setPage(1);
}}
className="rounded border border-gray-300 px-2 py-1"
>
{[20, 50, 100].map((n) => (
<option key={n} value={n}>
{n}
</option>
))}
</select>
<span> · {total} </span>
</div>
<button
disabled={page <= 1}
onClick={() => setPage((p) => p - 1)}
className="rounded border border-gray-300 px-3 py-1 disabled:opacity-50"
>
</button>
<span>
{page} / {pages}
</span>
<button
disabled={page >= pages}
onClick={() => setPage((p) => p + 1)}
className="rounded border border-gray-300 px-3 py-1 disabled:opacity-50"
>
</button>
<form
onSubmit={(e) => {
e.preventDefault();
const n = Number(jump);
if (Number.isFinite(n) && n >= 1) {
setPage(Math.min(Math.max(1, Math.trunc(n)), pages));
setJump("");
}
}}
className="flex items-center gap-1"
>
<span></span>
<input
value={jump}
onChange={(e) => setJump(e.target.value.replace(/[^0-9]/g, ""))}
placeholder={String(page)}
className="w-14 rounded border border-gray-300 px-2 py-1 text-center"
aria-label="跳转页码"
/>
<button
type="submit"
className="rounded border border-gray-300 px-3 py-1 hover:bg-gray-50"
>
</button>
</form>
</div>
</div>
);
}
function CreateProductModal({
onClose,
onCreated,
}: {
onClose: () => void;
onCreated: (id: string) => void;
}) {
const [name, setName] = useState("");
const [gtin, setGtin] = useState("");
const [brand, setBrand] = useState("");
const [categoryId, setCategoryId] = useState("");
const [brands, setBrands] = useState<Brand[]>([]);
const [categories, setCategories] = useState<Category[]>([]);
const [busy, setBusy] = useState(false);
const [error, setError] = useState("");
useEffect(() => {
api.listBrands().then((r) => setBrands(r.items)).catch(() => {});
api.listCategories().then((r) => setCategories(r.items)).catch(() => {});
}, []);
async function submit() {
if (!name.trim()) {
setError("名称不能为空");
return;
}
setBusy(true);
setError("");
try {
const created = await api.createProduct({
name: name.trim(),
gtin: gtin.trim() || null,
brand_name: brand.trim() || null,
category_id: categoryId || null,
status: "active",
});
onCreated(created.id);
} catch (e) {
setError(e instanceof ApiError ? e.message : "新建失败");
} finally {
setBusy(false);
}
}
return (
<div className="fixed inset-0 z-20 flex items-center justify-center bg-black/30">
<div className="w-full max-w-md rounded-lg bg-white p-6 shadow-lg">
<h3 className="mb-4 text-base font-semibold text-gray-800"></h3>
{error && (
<div className="mb-3 rounded bg-red-50 px-3 py-2 text-sm text-red-600">
{error}
</div>
)}
<div className="space-y-3">
<label className="block">
<span className="text-xs text-gray-500"> *</span>
<input
autoFocus
className="mt-1 w-full rounded border border-gray-300 px-3 py-2 text-sm"
value={name}
onChange={(e) => setName(e.target.value)}
placeholder="商品名称"
/>
</label>
<label className="block">
<span className="text-xs text-gray-500"> (GTIN)</span>
<input
className="mt-1 w-full rounded border border-gray-300 px-3 py-2 text-sm"
value={gtin}
onChange={(e) => setGtin(e.target.value)}
placeholder="8/12/13/14 位"
/>
</label>
<label className="block">
<span className="text-xs text-gray-500"></span>
<input
list="create-brand-list"
className="mt-1 w-full rounded border border-gray-300 px-3 py-2 text-sm"
value={brand}
onChange={(e) => setBrand(e.target.value)}
/>
<datalist id="create-brand-list">
{brands.map((b) => (
<option key={b.id} value={b.name} />
))}
</datalist>
</label>
<label className="block">
<span className="text-xs text-gray-500"></span>
<select
className="mt-1 w-full rounded border border-gray-300 bg-white px-3 py-2 text-sm"
value={categoryId}
onChange={(e) => setCategoryId(e.target.value)}
>
<option value=""></option>
{categories.map((c) => (
<option key={c.id} value={c.id}>
{"\u00A0".repeat(c.level * 2)}
{c.name_zh} ({c.path})
</option>
))}
</select>
</label>
</div>
<p className="mt-3 text-xs text-gray-400">
</p>
<div className="mt-4 flex justify-end gap-2">
<button
onClick={onClose}
className="rounded border px-4 py-2 text-sm text-gray-600"
>
</button>
<button
onClick={submit}
disabled={busy}
className="rounded bg-emerald-600 px-4 py-2 text-sm text-white hover:bg-emerald-700 disabled:opacity-60"
>
{busy ? "创建中…" : "创建并编辑"}
</button>
</div>
</div>
</div>
);
}
+125
View File
@@ -0,0 +1,125 @@
import { useEffect, useState } from "react";
import { api, ApiError } from "../api";
import type { AdminStats } from "../types";
import { BarChart3, Package, CheckCircle2, Tag, FolderTree, Inbox } from "lucide-react";
const STATUS_LABEL: Record<string, string> = {
active: "在用",
merged: "已合并",
deprecated: "已停用",
};
function Card({
icon,
label,
value,
hint,
}: {
icon: React.ReactNode;
label: string;
value: string | number;
hint?: string;
}) {
return (
<div className="rounded-lg border bg-white p-5">
<div className="flex items-center gap-2 text-sm text-gray-500">
{icon}
{label}
</div>
<div className="mt-2 text-2xl font-semibold text-gray-800">{value}</div>
{hint && <div className="mt-1 text-xs text-gray-400">{hint}</div>}
</div>
);
}
export default function StatsPage() {
const [stats, setStats] = useState<AdminStats | null>(null);
const [error, setError] = useState("");
useEffect(() => {
api
.stats()
.then(setStats)
.catch((e) => setError(e instanceof ApiError ? e.message : "加载失败"));
}, []);
return (
<div className="mx-auto max-w-5xl">
<h2 className="mb-4 flex items-center gap-2 text-lg font-semibold text-gray-800">
<BarChart3 className="h-5 w-5 text-emerald-600" />
</h2>
{error && (
<div className="mb-3 rounded bg-red-50 px-4 py-2 text-sm text-red-700">{error}</div>
)}
{!stats ? (
<div className="text-sm text-gray-400"></div>
) : (
<>
<div className="grid grid-cols-2 gap-4 md:grid-cols-3">
<Card
icon={<Package className="h-4 w-4" />}
label="商品总数"
value={stats.products.toLocaleString()}
/>
<Card
icon={<CheckCircle2 className="h-4 w-4 text-emerald-600" />}
label="合格档案"
value={stats.qualified.toLocaleString()}
hint={`质量分 ≥ ${stats.min_score} 且在用`}
/>
<Card
icon={<BarChart3 className="h-4 w-4" />}
label="平均质量分"
value={Math.round(stats.avg_quality * 100)}
hint="满分 100"
/>
<Card
icon={<Tag className="h-4 w-4" />}
label="品牌数"
value={stats.brands.toLocaleString()}
/>
<Card
icon={<FolderTree className="h-4 w-4" />}
label="分类数"
value={stats.categories.toLocaleString()}
/>
<Card
icon={<Inbox className="h-4 w-4" />}
label="待审核投稿"
value={stats.pending_submissions.toLocaleString()}
/>
</div>
<div className="mt-6 rounded-lg border bg-white p-5">
<h3 className="mb-3 text-sm font-medium text-gray-700"></h3>
<div className="space-y-2">
{Object.keys(stats.by_status).length === 0 ? (
<div className="text-sm text-gray-400"></div>
) : (
Object.entries(stats.by_status).map(([st, n]) => {
const pct = stats.products > 0 ? (n / stats.products) * 100 : 0;
return (
<div key={st} className="flex items-center gap-3 text-sm">
<span className="w-16 text-gray-600">
{STATUS_LABEL[st] || st}
</span>
<div className="h-3 flex-1 overflow-hidden rounded bg-gray-100">
<div
className="h-full bg-emerald-500"
style={{ width: `${pct}%` }}
/>
</div>
<span className="w-12 text-right text-gray-500">{n}</span>
</div>
);
})
)}
</div>
</div>
</>
)}
</div>
);
}
@@ -0,0 +1,314 @@
import { useEffect, useState } from "react";
import { api, ApiError } from "../api";
import type { SubmissionDetail, SubmissionRow } from "../types";
import { FIELD_LABELS } from "../types";
import { ArrowLeft, Check, X } from "lucide-react";
const STATUS_TABS = [
{ key: "pending", label: "待审核" },
{ key: "approved", label: "已通过" },
{ key: "rejected", label: "已驳回" },
];
const STATUS_BADGE: Record<string, string> = {
pending: "bg-amber-50 text-amber-700",
approved: "bg-emerald-50 text-emerald-700",
rejected: "bg-red-50 text-red-700",
};
const STATUS_TEXT: Record<string, string> = {
pending: "待审核",
approved: "已通过",
rejected: "已驳回",
};
export default function SubmissionsPage({ onPending }: { onPending?: (n: number) => void }) {
const [tab, setTab] = useState("pending");
const [rows, setRows] = useState<SubmissionRow[]>([]);
const [openId, setOpenId] = useState<string | null>(null);
const [error, setError] = useState("");
async function load() {
setError("");
try {
const res = await api.listSubmissions(tab, 1, 50);
setRows(res.items);
onPending?.(res.pending);
} catch (e) {
setError(e instanceof ApiError ? e.message : "加载失败");
}
}
useEffect(() => {
load();
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [tab]);
if (openId) {
return (
<SubmissionView
id={openId}
onBack={() => {
setOpenId(null);
load();
}}
/>
);
}
return (
<div className="mx-auto max-w-5xl">
<div className="flex items-center gap-2 mb-4">
{STATUS_TABS.map((t) => (
<button
key={t.key}
onClick={() => setTab(t.key)}
className={`px-3 py-1.5 rounded-md text-sm ${
tab === t.key
? "bg-emerald-600 text-white"
: "bg-white border text-gray-600 hover:bg-gray-50"
}`}
>
{t.label}
</button>
))}
</div>
{error && (
<div className="mb-3 bg-red-50 text-red-700 text-sm rounded px-4 py-2">{error}</div>
)}
<div className="bg-white border rounded-lg overflow-hidden">
<table className="w-full text-sm">
<thead className="bg-gray-50 text-gray-500 text-left">
<tr>
<th className="px-4 py-2 font-medium"></th>
<th className="px-4 py-2 font-medium"></th>
<th className="px-4 py-2 font-medium">稿</th>
<th className="px-4 py-2 font-medium"></th>
<th className="px-4 py-2 font-medium"></th>
<th className="px-4 py-2 font-medium"></th>
</tr>
</thead>
<tbody className="divide-y">
{rows.length === 0 ? (
<tr>
<td colSpan={6} className="px-4 py-8 text-center text-gray-400">
稿
</td>
</tr>
) : (
rows.map((r) => (
<tr
key={r.id}
onClick={() => setOpenId(r.id)}
className="cursor-pointer hover:bg-gray-50"
>
<td className="px-4 py-2 text-gray-800">{r.name}</td>
<td className="px-4 py-2 text-gray-500">{r.gtin || "—"}</td>
<td className="px-4 py-2 text-gray-500">{r.submitter_name || "匿名"}</td>
<td className="px-4 py-2">
<span className="text-xs text-gray-500">
{r.matched ? "补全已有商品" : "新建商品"}
</span>
</td>
<td className="px-4 py-2 text-gray-500">
{new Date(r.created_at).toLocaleString()}
</td>
<td className="px-4 py-2">
<span className={`text-xs rounded px-2 py-0.5 ${STATUS_BADGE[r.status]}`}>
{STATUS_TEXT[r.status]}
</span>
</td>
</tr>
))
)}
</tbody>
</table>
</div>
</div>
);
}
function Field({ label, value }: { label: string; value: React.ReactNode }) {
if (value === null || value === undefined || value === "") return null;
return (
<div className="flex py-1.5 text-sm">
<div className="w-28 shrink-0 text-gray-400">{label}</div>
<div className="text-gray-800 break-all">{value}</div>
</div>
);
}
function SubmissionView({ id, onBack }: { id: string; onBack: () => void }) {
const [d, setD] = useState<SubmissionDetail | null>(null);
const [error, setError] = useState("");
const [busy, setBusy] = useState(false);
const [rejecting, setRejecting] = useState(false);
const [reason, setReason] = useState("");
useEffect(() => {
api.getSubmission(id).then(setD).catch((e) => setError(e.message));
}, [id]);
async function approve() {
if (!confirm("确认通过该投稿?将写入正式商品库并记录来源 community。")) return;
setBusy(true);
setError("");
try {
await api.approveSubmission(id);
onBack();
} catch (e) {
setError(e instanceof ApiError ? e.message : "操作失败");
} finally {
setBusy(false);
}
}
async function reject() {
setBusy(true);
setError("");
try {
await api.rejectSubmission(id, reason.trim());
onBack();
} catch (e) {
setError(e instanceof ApiError ? e.message : "操作失败");
} finally {
setBusy(false);
}
}
if (error && !d) {
return (
<div>
<button onClick={onBack} className="text-sm text-gray-500 flex items-center gap-1 mb-4">
<ArrowLeft className="h-4 w-4" />
</button>
<div className="bg-red-50 text-red-700 text-sm rounded px-4 py-3">{error}</div>
</div>
);
}
if (!d) return <div className="text-gray-400"></div>;
const p = d.payload;
const nutri = Object.entries(p.nutriments || {});
return (
<div className="mx-auto max-w-3xl">
<button onClick={onBack} className="text-sm text-gray-500 flex items-center gap-1 mb-4">
<ArrowLeft className="h-4 w-4" /> 稿
</button>
<div className="flex items-center justify-between">
<h2 className="text-lg font-semibold text-gray-800">{p.name}</h2>
<span className={`text-xs rounded px-2 py-0.5 ${STATUS_BADGE[d.status]}`}>
{STATUS_TEXT[d.status]}
</span>
</div>
{error && <div className="mt-3 bg-red-50 text-red-700 text-sm rounded px-4 py-2">{error}</div>}
{d.target_product_id && (
<div className="mt-3 text-sm bg-blue-50 text-blue-700 rounded px-4 py-2">
<b></b>
</div>
)}
<div className="bg-white border rounded-lg p-5 mt-4">
<h3 className="font-medium text-gray-700 mb-2">稿</h3>
<Field label="商品名称" value={p.name} />
<Field label="条码" value={p.gtin} />
<Field label="品牌" value={p.brand_name} />
<Field
label="净含量"
value={p.net_content_value != null ? `${p.net_content_value} ${p.net_content_unit || ""}` : null}
/>
<Field label="产地" value={p.country_of_origin} />
<Field label="配料" value={p.ingredients_text} />
{nutri.length > 0 && (
<Field
label="营养成分"
value={
<span>
{p.nutrition_basis ? `(${p.nutrition_basis}) ` : ""}
{nutri.map(([k, v]) => `${FIELD_LABELS[k] || k}:${v}`).join("")}
</span>
}
/>
)}
{p.images && p.images.length > 0 && (
<Field
label="图片"
value={
<div className="flex flex-wrap gap-2">
{p.images.map((im, i) => (
<a key={i} href={im.url} target="_blank" rel="noreferrer">
<img src={im.url} alt="" className="h-20 w-20 object-cover rounded border" />
</a>
))}
</div>
}
/>
)}
</div>
<div className="bg-white border rounded-lg p-5 mt-4">
<h3 className="font-medium text-gray-700 mb-2">稿 / </h3>
<Field label="称呼" value={p.submitter_name || "匿名"} />
<Field label="联系方式" value={p.submitter_contact} />
<Field label="备注" value={p.note} />
<Field label="提交时间" value={new Date(d.created_at).toLocaleString()} />
{d.reviewed_by && <Field label="审核人" value={d.reviewed_by} />}
{d.review_note && <Field label="驳回原因" value={d.review_note} />}
{d.result_product_id && <Field label="收录商品ID" value={d.result_product_id} />}
</div>
{d.status === "pending" && (
<div className="mt-5">
{rejecting ? (
<div className="bg-white border rounded-lg p-4">
<label className="block text-xs text-gray-500 mb-1"></label>
<input
className="w-full border rounded-md px-3 py-2 text-sm"
value={reason}
onChange={(e) => setReason(e.target.value)}
placeholder="例如:资料无法核实 / 重复投稿"
/>
<div className="mt-3 flex gap-2">
<button
onClick={reject}
disabled={busy}
className="px-4 py-2 rounded bg-red-600 text-white text-sm hover:bg-red-700 disabled:opacity-60"
>
</button>
<button
onClick={() => setRejecting(false)}
className="px-4 py-2 rounded border text-sm text-gray-600"
>
</button>
</div>
</div>
) : (
<div className="flex gap-3">
<button
onClick={approve}
disabled={busy}
className="px-5 py-2.5 rounded-lg bg-emerald-600 text-white font-medium hover:bg-emerald-700 disabled:opacity-60 flex items-center gap-1.5"
>
<Check className="h-4 w-4" />
</button>
<button
onClick={() => setRejecting(true)}
disabled={busy}
className="px-5 py-2.5 rounded-lg border text-gray-700 font-medium hover:bg-gray-50 flex items-center gap-1.5"
>
<X className="h-4 w-4" />
</button>
</div>
)}
</div>
)}
</div>
);
}
+16
View File
@@ -0,0 +1,16 @@
@tailwind base;
@tailwind components;
@tailwind utilities;
html,
body,
#root {
height: 100%;
}
body {
margin: 0;
background: #f3f4f6;
font-family: system-ui, -apple-system, "Segoe UI", Roboto, "Helvetica Neue",
Arial, "PingFang SC", "Microsoft YaHei", sans-serif;
}
+10
View File
@@ -0,0 +1,10 @@
import React from "react";
import ReactDOM from "react-dom/client";
import App from "./App";
import "./index.css";
ReactDOM.createRoot(document.getElementById("root")!).render(
<React.StrictMode>
<App />
</React.StrictMode>,
);
+219
View File
@@ -0,0 +1,219 @@
export interface ProductRow {
id: string;
gtin: string | null;
name: string;
brand: string | null;
category_path: string | null;
status: string;
quality_score: number;
missing: string[];
updated_at: string;
}
export interface Barcode {
id: string;
gtin: string;
gtin_type: string;
pack_level: string;
region: string | null;
is_primary: boolean;
}
export interface ProductImage {
id: string;
url: string;
kind: string;
license: string | null;
}
export interface MSRP {
id: string;
amount: number;
currency: string;
region: string;
effective_date: string | null;
source_url: string | null;
note: string | null;
}
export interface ProductDetail {
id: string;
gtin: string | null;
name: string;
brand_id: string | null;
brand: string | null;
category_id: string | null;
category_path: string | null;
archive_kind: string;
attributes: Record<string, unknown>;
net_content_value: number | null;
net_content_unit: string | null;
country_of_origin: string | null;
status: string;
quality_score: number;
ingredients_text: string | null;
allergens: string[];
additives: string[];
nutriments: Record<string, number> | null;
nutrition_basis: string | null;
serving_size: string | null;
nutri_score: string | null;
barcodes: Barcode[];
images: ProductImage[];
msrp: MSRP[];
missing: string[];
updated_at: string;
}
export interface Brand {
id: string;
name: string;
product_count: number;
}
export interface Category {
id: string;
name_zh: string;
name_en: string | null;
path: string;
level: number;
parent_id: string | null;
gpc_brick_code: string | null;
archive_kind: string;
product_count: number;
}
export interface KindField {
kind: string;
field_key: string;
group_label: string;
label_zh: string;
field_type: string;
unit: string | null;
options: string[];
placeholder: string | null;
sort_order: number;
qualified: boolean;
}
export interface CategoryInput {
name_zh: string;
name_en?: string | null;
slug?: string | null;
parent_id?: string | null;
gpc_brick_code?: string | null;
}
export interface AuditEntry {
id: string;
actor: string;
action: string;
fields: string[];
created_at: string;
}
export interface AuditLogRow {
id: string;
actor: string;
action: string;
entity: string;
entity_id: string | null;
fields: string[];
created_at: string;
}
export interface AdminStats {
products: number;
qualified: number;
min_score: number;
by_status: Record<string, number>;
brands: number;
categories: number;
pending_submissions: number;
avg_quality: number;
}
export interface SubmissionRow {
id: string;
gtin: string | null;
name: string;
status: string;
submitter_name: string | null;
matched: boolean;
created_at: string;
reviewed_at: string | null;
}
export interface SubmissionImage {
url: string;
kind: string;
}
export interface SubmissionPayload {
gtin: string | null;
name: string;
brand_name: string | null;
category_id: string | null;
net_content_value: number | null;
net_content_unit: string | null;
country_of_origin: string | null;
ingredients_text: string | null;
nutriments: Record<string, number> | null;
nutrition_basis: string | null;
serving_size: string | null;
nutri_score: string | null;
images: SubmissionImage[] | null;
submitter_name: string | null;
submitter_contact: string | null;
note: string | null;
}
export interface SubmissionDetail {
id: string;
status: string;
gtin: string | null;
name: string;
submitter_name: string | null;
submitter_contact: string | null;
note: string | null;
review_note: string | null;
reviewed_by: string | null;
reviewed_at: string | null;
created_at: string;
target_product_id: string | null;
result_product_id: string | null;
payload: SubmissionPayload;
existing_product?: ProductDetail;
}
export interface ApiKeyUsage {
total: number;
today: number;
last_used_at?: number | null;
}
export interface ApiKey {
id: string;
name: string;
key_prefix: string;
owner_email: string | null;
tier: string;
rate_limit_per_min: number;
quota_total: number;
revoked_at: string | null;
created_by: string | null;
created_at: string;
usage: ApiKeyUsage;
}
export const FIELD_LABELS: Record<string, string> = {
name: "名称",
gtin: "条码",
brand: "品牌",
category: "品类",
net_content: "净含量",
country_of_origin: "产地",
nutriments: "营养成分",
ingredients: "配料",
image: "图片",
};
+1
View File
@@ -0,0 +1 @@
/// <reference types="vite/client" />
+6
View File
@@ -0,0 +1,6 @@
/** @type {import('tailwindcss').Config} */
export default {
content: ["./index.html", "./src/**/*.{ts,tsx}"],
theme: { extend: {} },
plugins: [],
};
+21
View File
@@ -0,0 +1,21 @@
{
"compilerOptions": {
"target": "ES2020",
"useDefineForClassFields": true,
"lib": ["ES2020", "DOM", "DOM.Iterable"],
"module": "ESNext",
"skipLibCheck": true,
"moduleResolution": "bundler",
"allowImportingTsExtensions": true,
"resolveJsonModule": true,
"isolatedModules": true,
"noEmit": true,
"jsx": "react-jsx",
"strict": true,
"noUnusedLocals": true,
"noUnusedParameters": true,
"noFallthroughCasesInSwitch": true
},
"include": ["src"],
"references": [{ "path": "./tsconfig.node.json" }]
}
+11
View File
@@ -0,0 +1,11 @@
{
"compilerOptions": {
"composite": true,
"skipLibCheck": true,
"module": "ESNext",
"moduleResolution": "bundler",
"allowSyntheticDefaultImports": true,
"strict": true
},
"include": ["vite.config.ts"]
}
+9
View File
@@ -0,0 +1,9 @@
import { defineConfig } from "vite";
import react from "@vitejs/plugin-react";
// Served under /ping by the admin Go binary; base must match the nginx prefix.
export default defineConfig({
base: "/ping/",
plugins: [react()],
build: { outDir: "dist", emptyOutDir: true },
});
+29
View File
@@ -0,0 +1,29 @@
# Admin console image: builds the SPA (node), embeds it into the Go admin
# binary, and ships a static scratch runtime. Build context is the repo root.
# Stage 1: build the admin SPA.
FROM node:22-alpine AS web
WORKDIR /web
ENV npm_config_registry=https://registry.npmmirror.com
COPY admin-frontend/package.json admin-frontend/package-lock.json* ./
RUN npm ci || npm install
COPY admin-frontend/ ./
RUN npm run build
# Stage 2: build the Go admin binary with the SPA embedded.
FROM golang:1.23-alpine AS build
ENV GOPROXY=https://goproxy.cn,direct
WORKDIR /src
COPY api/go.mod api/go.sum ./
RUN go mod download
COPY api/ ./
RUN rm -rf internal/adminweb/dist && mkdir -p internal/adminweb/dist
COPY --from=web /web/dist/ internal/adminweb/dist/
RUN CGO_ENABLED=0 go build -o /out/admin ./cmd/admin
# Stage 3: minimal runtime.
FROM scratch
COPY --from=build /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/
COPY --from=build /out/admin /admin
EXPOSE 8080
ENTRYPOINT ["/admin"]
+31
View File
@@ -0,0 +1,31 @@
# Public API image: builds the public SPA (homepage + search + contribute),
# embeds it into the Go read-only server binary, and ships a static scratch
# runtime (used where gcr.io/distroless is not reachable). Build context is the
# repo root.
# Stage 1: build the public SPA.
FROM node:22-alpine AS web
WORKDIR /web
ENV npm_config_registry=https://registry.npmmirror.com
COPY public-frontend/package.json public-frontend/package-lock.json* ./
RUN npm ci || npm install
COPY public-frontend/ ./
RUN npm run build
# Stage 2: build the Go server binary with the SPA embedded.
FROM golang:1.23-alpine AS build
ENV GOPROXY=https://goproxy.cn,direct
WORKDIR /src
COPY api/go.mod api/go.sum ./
RUN go mod download
COPY api/ ./
RUN rm -rf internal/publicweb/dist && mkdir -p internal/publicweb/dist
COPY --from=web /web/dist/ internal/publicweb/dist/
RUN CGO_ENABLED=0 go build -o /out/server ./cmd/server
# Stage 3: minimal runtime.
FROM scratch
COPY --from=build /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/
COPY --from=build /out/server /server
EXPOSE 8080
ENTRYPOINT ["/server"]
+86
View File
@@ -0,0 +1,86 @@
// Command admin starts the OpenGoods admin console (authenticated write API +
// embedded SPA), served under a base path (default /ping).
package main
import (
"context"
"crypto/rand"
"log"
"net/http"
"os"
"time"
"github.com/jackc/pgx/v5/pgxpool"
"golang.org/x/crypto/bcrypt"
"github.com/baicai2026-baicai/goods/api/internal/adminhandler"
"github.com/baicai2026-baicai/goods/api/internal/adminstore"
"github.com/baicai2026-baicai/goods/api/internal/adminweb"
"github.com/baicai2026-baicai/goods/api/internal/auth"
"github.com/baicai2026-baicai/goods/api/internal/ratelimit"
)
func getenv(key, fallback string) string {
if v, ok := os.LookupEnv(key); ok && v != "" {
return v
}
return fallback
}
func main() {
addr := getenv("GOODS_ADMIN_ADDR", ":8080")
dbURL := getenv("OPENGOODS_DATABASE_URL", "postgres://opengoods:opengoods@localhost:5432/opengoods?sslmode=disable")
basePath := getenv("GOODS_ADMIN_BASE_PATH", "/ping")
username := getenv("GOODS_ADMIN_USER", "admin")
// Password: prefer a bcrypt hash; otherwise hash a plaintext password.
var passwordHash []byte
if h := os.Getenv("GOODS_ADMIN_PASSWORD_HASH"); h != "" {
passwordHash = []byte(h)
} else if p := os.Getenv("GOODS_ADMIN_PASSWORD"); p != "" {
hashed, err := bcrypt.GenerateFromPassword([]byte(p), bcrypt.DefaultCost)
if err != nil {
log.Fatalf("failed to hash admin password: %v", err)
}
passwordHash = hashed
} else {
log.Fatal("set GOODS_ADMIN_PASSWORD or GOODS_ADMIN_PASSWORD_HASH")
}
secret := []byte(os.Getenv("GOODS_ADMIN_JWT_SECRET"))
if len(secret) == 0 {
secret = make([]byte, 32)
if _, err := rand.Read(secret); err != nil {
log.Fatalf("failed to generate jwt secret: %v", err)
}
log.Print("warning: GOODS_ADMIN_JWT_SECRET not set; using a random secret (tokens invalidate on restart)")
}
ctx := context.Background()
pool, err := pgxpool.New(ctx, dbURL)
if err != nil {
log.Fatalf("failed to create db pool: %v", err)
}
defer pool.Close()
pingCtx, cancel := context.WithTimeout(ctx, 5*time.Second)
defer cancel()
if err := pool.Ping(pingCtx); err != nil {
log.Printf("warning: database not reachable at startup: %v", err)
}
authn := auth.New(username, passwordHash, secret, 12*time.Hour)
usage := ratelimit.New(getenv("OPENGOODS_REDIS_URL", "redis://localhost:6379/0"))
h := adminhandler.New(adminstore.New(pool), authn, basePath, adminweb.Dist()).
WithUsage(usage)
srv := &http.Server{
Addr: addr,
Handler: h.Router(),
ReadHeaderTimeout: 10 * time.Second,
}
log.Printf("OpenGoods admin console listening on %s (base path %s)", addr, basePath)
if err := srv.ListenAndServe(); err != nil && err != http.ErrServerClosed {
log.Fatalf("server error: %v", err)
}
}
+15 -1
View File
@@ -9,8 +9,11 @@ import (
"github.com/jackc/pgx/v5/pgxpool" "github.com/jackc/pgx/v5/pgxpool"
"github.com/baicai2026-baicai/goods/api/internal/cache"
"github.com/baicai2026-baicai/goods/api/internal/config" "github.com/baicai2026-baicai/goods/api/internal/config"
"github.com/baicai2026-baicai/goods/api/internal/handler" "github.com/baicai2026-baicai/goods/api/internal/handler"
"github.com/baicai2026-baicai/goods/api/internal/publicweb"
"github.com/baicai2026-baicai/goods/api/internal/ratelimit"
"github.com/baicai2026-baicai/goods/api/internal/store" "github.com/baicai2026-baicai/goods/api/internal/store"
) )
@@ -30,7 +33,18 @@ func main() {
log.Printf("warning: database not reachable at startup: %v", err) log.Printf("warning: database not reachable at startup: %v", err)
} }
h := handler.New(store.New(pool)) limiter := ratelimit.New(cfg.RedisURL)
if !limiter.Enabled() {
log.Print("warning: Redis not configured; public API rate limiting disabled")
}
readCache := cache.New(cfg.RedisURL)
if !readCache.Enabled() {
log.Print("warning: Redis not configured; public API read cache disabled")
}
h := handler.New(store.New(pool).WithCache(readCache), publicweb.Dist()).
WithRateLimit(limiter, cfg.AnonRateLimitPerMin).
WithQuotas(cfg.AnonTotalQuota, cfg.RegisteredRateLimitPerMin, cfg.RegisteredQuotaTotal)
srv := &http.Server{ srv := &http.Server{
Addr: cfg.Addr, Addr: cfg.Addr,
+5 -1
View File
@@ -5,13 +5,17 @@ go 1.23.4
require ( require (
github.com/go-chi/chi/v5 v5.1.0 github.com/go-chi/chi/v5 v5.1.0
github.com/jackc/pgx/v5 v5.7.2 github.com/jackc/pgx/v5 v5.7.2
github.com/redis/go-redis/v9 v9.18.0
golang.org/x/crypto v0.31.0
) )
require ( require (
github.com/cespare/xxhash/v2 v2.3.0 // indirect
github.com/dgryski/go-rendezvous v0.0.0-20200823014737-9f7001d12a5f // indirect
github.com/jackc/pgpassfile v1.0.0 // indirect github.com/jackc/pgpassfile v1.0.0 // indirect
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 // indirect github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 // indirect
github.com/jackc/puddle/v2 v2.2.2 // indirect github.com/jackc/puddle/v2 v2.2.2 // indirect
golang.org/x/crypto v0.31.0 // indirect go.uber.org/atomic v1.11.0 // indirect
golang.org/x/sync v0.10.0 // indirect golang.org/x/sync v0.10.0 // indirect
golang.org/x/text v0.21.0 // indirect golang.org/x/text v0.21.0 // indirect
) )
+16
View File
@@ -1,6 +1,14 @@
github.com/bsm/ginkgo/v2 v2.12.0 h1:Ny8MWAHyOepLGlLKYmXG4IEkioBysk6GpaRTLC8zwWs=
github.com/bsm/ginkgo/v2 v2.12.0/go.mod h1:SwYbGRRDovPVboqFv0tPTcG1sN61LM1Z4ARdbAV9g4c=
github.com/bsm/gomega v1.27.10 h1:yeMWxP2pV2fG3FgAODIY8EiRE3dy0aeFYt4l7wh6yKA=
github.com/bsm/gomega v1.27.10/go.mod h1:JyEr/xRbxbtgWNi8tIEVPUYZ5Dzef52k01W3YH0H+O0=
github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs=
github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/dgryski/go-rendezvous v0.0.0-20200823014737-9f7001d12a5f h1:lO4WD4F/rVNCu3HqELle0jiPLLBs70cWOduZpkS1E78=
github.com/dgryski/go-rendezvous v0.0.0-20200823014737-9f7001d12a5f/go.mod h1:cuUVRXasLTGF7a8hSLbxyZXjz+1KgoB3wDUb6vlszIc=
github.com/go-chi/chi/v5 v5.1.0 h1:acVI1TYaD+hhedDJ3r54HyA6sExp3HfXq7QWEEY/xMw= github.com/go-chi/chi/v5 v5.1.0 h1:acVI1TYaD+hhedDJ3r54HyA6sExp3HfXq7QWEEY/xMw=
github.com/go-chi/chi/v5 v5.1.0/go.mod h1:DslCQbL2OYiznFReuXYUmQ2hGd1aDpCnlMNITLSKoi8= github.com/go-chi/chi/v5 v5.1.0/go.mod h1:DslCQbL2OYiznFReuXYUmQ2hGd1aDpCnlMNITLSKoi8=
github.com/jackc/pgpassfile v1.0.0 h1:/6Hmqy13Ss2zCq62VdNG8tM1wchn8zjSGOBJ6icpsIM= github.com/jackc/pgpassfile v1.0.0 h1:/6Hmqy13Ss2zCq62VdNG8tM1wchn8zjSGOBJ6icpsIM=
@@ -11,13 +19,21 @@ github.com/jackc/pgx/v5 v5.7.2 h1:mLoDLV6sonKlvjIEsV56SkWNCnuNv531l94GaIzO+XI=
github.com/jackc/pgx/v5 v5.7.2/go.mod h1:ncY89UGWxg82EykZUwSpUKEfccBGGYq1xjrOpsbsfGQ= github.com/jackc/pgx/v5 v5.7.2/go.mod h1:ncY89UGWxg82EykZUwSpUKEfccBGGYq1xjrOpsbsfGQ=
github.com/jackc/puddle/v2 v2.2.2 h1:PR8nw+E/1w0GLuRFSmiioY6UooMp6KJv0/61nB7icHo= github.com/jackc/puddle/v2 v2.2.2 h1:PR8nw+E/1w0GLuRFSmiioY6UooMp6KJv0/61nB7icHo=
github.com/jackc/puddle/v2 v2.2.2/go.mod h1:vriiEXHvEE654aYKXXjOvZM39qJ0q+azkZFrfEOc3H4= github.com/jackc/puddle/v2 v2.2.2/go.mod h1:vriiEXHvEE654aYKXXjOvZM39qJ0q+azkZFrfEOc3H4=
github.com/klauspost/cpuid/v2 v2.0.9 h1:lgaqFMSdTdQYdZ04uHyN2d/eKdOMyi2YLSvlQIBFYa4=
github.com/klauspost/cpuid/v2 v2.0.9/go.mod h1:FInQzS24/EEf25PyTYn52gqo7WaD8xa0213Md/qVLRg=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/redis/go-redis/v9 v9.18.0 h1:pMkxYPkEbMPwRdenAzUNyFNrDgHx9U+DrBabWNfSRQs=
github.com/redis/go-redis/v9 v9.18.0/go.mod h1:k3ufPphLU5YXwNTUcCRXGxUoF1fqxnhFQmscfkCoDA0=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI= github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg= github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.8.1 h1:w7B6lhMri9wdJUVmEZPGGhZzrYTPvgJArz7wNPgYKsk= github.com/stretchr/testify v1.8.1 h1:w7B6lhMri9wdJUVmEZPGGhZzrYTPvgJArz7wNPgYKsk=
github.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4= github.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=
github.com/zeebo/xxh3 v1.0.2 h1:xZmwmqxHZA8AI603jOQ0tMqmBr9lPeFwGg6d+xy9DC0=
github.com/zeebo/xxh3 v1.0.2/go.mod h1:5NWz9Sef7zIDm2JHfFlcQvNekmcEl9ekUZQQKCYaDcA=
go.uber.org/atomic v1.11.0 h1:ZvwS0R+56ePWxUNi+Atn9dWONBPp/AUETXlHW0DxSjE=
go.uber.org/atomic v1.11.0/go.mod h1:LUxbIzbOniOlMKjJjyPfpl4v+PKK2cNJn91OQbhoJI0=
golang.org/x/crypto v0.31.0 h1:ihbySMvVjLAeSH1IbfcRTkD/iNscyz8rGzjF/E5hV6U= golang.org/x/crypto v0.31.0 h1:ihbySMvVjLAeSH1IbfcRTkD/iNscyz8rGzjF/E5hV6U=
golang.org/x/crypto v0.31.0/go.mod h1:kDsLvtWBEx7MV9tJOj9bnXsPbxwJQ6csT/x4KIN4Ssk= golang.org/x/crypto v0.31.0/go.mod h1:kDsLvtWBEx7MV9tJOj9bnXsPbxwJQ6csT/x4KIN4Ssk=
golang.org/x/sync v0.10.0 h1:3NQrjDixjgGwUOCaF8w2+VYHv0Ve/vGYSbdkTa98gmQ= golang.org/x/sync v0.10.0 h1:3NQrjDixjgGwUOCaF8w2+VYHv0Ve/vGYSbdkTa98gmQ=
+70
View File
@@ -0,0 +1,70 @@
package adminhandler
import (
"encoding/json"
"net/http"
"strings"
"github.com/go-chi/chi/v5"
"github.com/baicai2026-baicai/goods/api/internal/adminstore"
"github.com/baicai2026-baicai/goods/api/internal/auth"
"github.com/baicai2026-baicai/goods/api/internal/ratelimit"
)
// apiKeyView is an issued key plus its usage counters.
type apiKeyView struct {
adminstore.APIKeyRow
Usage ratelimit.UsageStat `json:"usage"`
}
// ListAPIKeys returns all issued keys with usage stats merged in.
func (h *Handler) ListAPIKeys(w http.ResponseWriter, r *http.Request) {
keys, err := h.store.ListAPIKeys(r.Context())
if h.handleErr(w, err) {
return
}
views := make([]apiKeyView, 0, len(keys))
for _, k := range keys {
v := apiKeyView{APIKeyRow: k}
if h.usage != nil {
v.Usage = h.usage.Usage(r.Context(), k.ID)
}
views = append(views, v)
}
writeJSON(w, http.StatusOK, map[string]any{"items": views})
}
// CreateAPIKey issues a new key and returns its plaintext exactly once.
func (h *Handler) CreateAPIKey(w http.ResponseWriter, r *http.Request) {
var in adminstore.APIKeyInput
if err := json.NewDecoder(r.Body).Decode(&in); err != nil {
writeError(w, http.StatusBadRequest, "bad_request", "invalid body")
return
}
if strings.TrimSpace(in.Name) == "" {
writeError(w, http.StatusBadRequest, "bad_request", "名称不能为空")
return
}
if in.Tier != "" && in.Tier != "free" && in.Tier != "registered" && in.Tier != "partner" && in.Tier != "internal" {
writeError(w, http.StatusBadRequest, "bad_request", "tier 取值无效")
return
}
plaintext, row, err := h.store.CreateAPIKey(r.Context(), in, auth.UserFrom(r.Context()))
if h.handleErr(w, err) {
return
}
writeJSON(w, http.StatusCreated, map[string]any{
"key": plaintext,
"item": row,
"warning": "请立即复制保存此密钥,它只显示这一次,无法再次查看。",
})
}
// RevokeAPIKey disables a key. Subsequent requests with it are rejected.
func (h *Handler) RevokeAPIKey(w http.ResponseWriter, r *http.Request) {
if err := h.store.RevokeAPIKey(r.Context(), chi.URLParam(r, "id")); h.handleErr(w, err) {
return
}
writeJSON(w, http.StatusOK, map[string]string{"status": "revoked"})
}
+812
View File
@@ -0,0 +1,812 @@
// Package adminhandler wires up the authenticated admin console: a JSON write
// API mounted under a base path (default /ping) plus the embedded SPA.
package adminhandler
import (
"encoding/json"
"errors"
"io/fs"
"net/http"
"strings"
"time"
"github.com/go-chi/chi/v5"
"github.com/go-chi/chi/v5/middleware"
"github.com/baicai2026-baicai/goods/api/internal/adminstore"
"github.com/baicai2026-baicai/goods/api/internal/apikey"
"github.com/baicai2026-baicai/goods/api/internal/auth"
"github.com/baicai2026-baicai/goods/api/internal/gtin"
"github.com/baicai2026-baicai/goods/api/internal/ratelimit"
)
// Handler holds the admin dependencies.
type Handler struct {
store *adminstore.Store
authn *auth.Authenticator
basePath string
spa fs.FS
submitLimit *rateLimiter
usage *ratelimit.Limiter
}
// New constructs an admin Handler. basePath is e.g. "/ping" (no trailing slash).
func New(store *adminstore.Store, authn *auth.Authenticator, basePath string, spa fs.FS) *Handler {
basePath = "/" + strings.Trim(basePath, "/")
return &Handler{
store: store,
authn: authn,
basePath: basePath,
spa: spa,
submitLimit: newRateLimiter(5, 10*time.Minute),
}
}
// WithUsage attaches a Redis-backed limiter used to read per-key usage counters
// for the API-key management view. Optional; without it usage shows as zero.
func (h *Handler) WithUsage(l *ratelimit.Limiter) *Handler {
h.usage = l
return h
}
// Router builds the HTTP handler.
func (h *Handler) Router() http.Handler {
r := chi.NewRouter()
r.Use(middleware.RequestID)
r.Use(middleware.RealIP)
r.Use(middleware.Recoverer)
r.Route(h.basePath, func(r chi.Router) {
r.Get("/healthz", func(w http.ResponseWriter, _ *http.Request) {
writeJSON(w, http.StatusOK, map[string]string{"status": "ok"})
})
r.Post("/api/login", h.Login)
r.Group(func(r chi.Router) {
r.Use(h.authn.Middleware)
r.Get("/api/me", h.Me)
r.Get("/api/stats", h.Stats)
r.Get("/api/audit", h.ListAllAudit)
r.Get("/api/products", h.ListProducts)
r.Post("/api/products", h.CreateProduct)
r.Post("/api/products/bulk", h.BulkProducts)
r.Get("/api/products/{id}", h.GetProduct)
r.Put("/api/products/{id}", h.UpdateProduct)
r.Get("/api/products/{id}/audit", h.ListAudit)
r.Post("/api/products/{id}/images", h.AddImage)
r.Delete("/api/products/{id}/images/{imageID}", h.DeleteImage)
r.Post("/api/products/{id}/msrp", h.AddMSRP)
r.Delete("/api/products/{id}/msrp/{msrpID}", h.DeleteMSRP)
r.Post("/api/products/{id}/barcodes", h.AddBarcode)
r.Delete("/api/products/{id}/barcodes/{barcodeID}", h.DeleteBarcode)
r.Post("/api/products/{id}/barcodes/{barcodeID}/primary", h.SetPrimaryBarcode)
r.Get("/api/brands", h.ListBrands)
r.Post("/api/brands", h.CreateBrand)
r.Put("/api/brands/{id}", h.UpdateBrand)
r.Post("/api/brands/{id}/merge", h.MergeBrands)
r.Delete("/api/brands/{id}", h.DeleteBrand)
r.Get("/api/kind-fields", h.ListKindFields)
r.Get("/api/categories", h.ListCategories)
r.Post("/api/categories", h.CreateCategory)
r.Put("/api/categories/{id}", h.UpdateCategory)
r.Delete("/api/categories/{id}", h.DeleteCategory)
r.Post("/api/import/bypos", h.ImportBypos)
r.Get("/api/submissions", h.ListSubmissions)
r.Get("/api/submissions/{id}", h.GetSubmission)
r.Post("/api/submissions/{id}/approve", h.ApproveSubmission)
r.Post("/api/submissions/{id}/reject", h.RejectSubmission)
r.Get("/api/keys", h.ListAPIKeys)
r.Post("/api/keys", h.CreateAPIKey)
r.Delete("/api/keys/{id}", h.RevokeAPIKey)
})
r.Handle("/*", http.HandlerFunc(h.serveSPA))
})
// Public, unauthenticated contribution endpoint (proxied at /api/public/*).
// Submissions enter a moderation queue and never touch products until an
// admin approves them.
r.Post("/api/public/submissions", h.CreateSubmission)
// Archive backflow: inventory-management software pushes products missing
// from the archive. Authenticated with a public API key; records enter the
// same moderation queue and are archived only after admin approval.
r.Post("/api/public/backflow", h.Backflow)
return r
}
func (h *Handler) serveSPA(w http.ResponseWriter, r *http.Request) {
rel := strings.TrimPrefix(r.URL.Path, h.basePath)
rel = strings.TrimPrefix(rel, "/")
if rel == "" {
rel = "index.html"
}
if f, err := h.spa.Open(rel); err == nil {
f.Close()
http.StripPrefix(h.basePath+"/", http.FileServer(http.FS(h.spa))).ServeHTTP(w, r)
return
}
// SPA fallback: serve index.html for client-side routes.
index, err := h.spa.Open("index.html")
if err != nil {
http.NotFound(w, r)
return
}
defer index.Close()
data, _ := fs.ReadFile(h.spa, "index.html")
w.Header().Set("Content-Type", "text/html; charset=utf-8")
_, _ = w.Write(data)
}
// ---------- auth ----------
// Login authenticates and returns a bearer token.
func (h *Handler) Login(w http.ResponseWriter, r *http.Request) {
var body struct {
Username string `json:"username"`
Password string `json:"password"`
}
if err := json.NewDecoder(r.Body).Decode(&body); err != nil {
writeError(w, http.StatusBadRequest, "bad_request", "invalid body")
return
}
token, err := h.authn.Login(body.Username, body.Password)
if err != nil {
writeError(w, http.StatusUnauthorized, "unauthorized", "用户名或密码错误")
return
}
writeJSON(w, http.StatusOK, map[string]string{"token": token, "username": body.Username})
}
// Me returns the current authenticated user.
func (h *Handler) Me(w http.ResponseWriter, r *http.Request) {
writeJSON(w, http.StatusOK, map[string]string{"username": auth.UserFrom(r.Context())})
}
// ---------- products ----------
// ListProducts returns a paginated product list.
func (h *Handler) ListProducts(w http.ResponseWriter, r *http.Request) {
q := r.URL.Query().Get("q")
sort := r.URL.Query().Get("sort")
order := r.URL.Query().Get("order")
page, size := pageParams(r)
items, total, err := h.store.ListProducts(r.Context(), q, sort, order, size, (page-1)*size)
if h.handleErr(w, err) {
return
}
writeJSON(w, http.StatusOK, map[string]any{
"items": items, "page": page, "size": size, "total": total,
"completeness_fields": adminstore.CompletenessFields,
})
}
// GetProduct returns full editable detail.
func (h *Handler) GetProduct(w http.ResponseWriter, r *http.Request) {
d, err := h.store.GetProduct(r.Context(), chi.URLParam(r, "id"))
if h.handleErr(w, err) {
return
}
writeJSON(w, http.StatusOK, d)
}
// CreateProduct adds a new product with core fields; the rest is filled in via
// the detail editor.
func (h *Handler) CreateProduct(w http.ResponseWriter, r *http.Request) {
var in adminstore.ProductInput
if err := json.NewDecoder(r.Body).Decode(&in); err != nil {
writeError(w, http.StatusBadRequest, "bad_request", "invalid body")
return
}
if strings.TrimSpace(in.Name) == "" {
writeError(w, http.StatusBadRequest, "bad_request", "名称不能为空")
return
}
d, err := h.store.CreateProduct(r.Context(), auth.UserFrom(r.Context()), in)
if errors.Is(err, adminstore.ErrDuplicateGTIN) {
writeError(w, http.StatusConflict, "duplicate_gtin", "该条码(GTIN)已被其它商品使用")
return
}
if h.handleErr(w, err) {
return
}
writeJSON(w, http.StatusCreated, d)
}
// Stats returns the dashboard overview counters.
func (h *Handler) Stats(w http.ResponseWriter, r *http.Request) {
st, err := h.store.Stats(r.Context())
if h.handleErr(w, err) {
return
}
writeJSON(w, http.StatusOK, st)
}
// ListAllAudit returns a page of the global operations audit log.
func (h *Handler) ListAllAudit(w http.ResponseWriter, r *http.Request) {
page, size := pageParams(r)
items, total, err := h.store.ListAllAudit(r.Context(), size, (page-1)*size)
if h.handleErr(w, err) {
return
}
writeJSON(w, http.StatusOK, map[string]any{
"items": items, "page": page, "size": size, "total": total,
})
}
type bulkInput struct {
IDs []string `json:"ids"`
Action string `json:"action"`
Status string `json:"status"`
CategoryID *string `json:"category_id"`
}
// BulkProducts applies a status or category change to many products at once.
func (h *Handler) BulkProducts(w http.ResponseWriter, r *http.Request) {
var in bulkInput
if err := json.NewDecoder(r.Body).Decode(&in); err != nil {
writeError(w, http.StatusBadRequest, "bad_request", "invalid body")
return
}
if len(in.IDs) == 0 {
writeError(w, http.StatusBadRequest, "bad_request", "未选择任何商品")
return
}
actor := auth.UserFrom(r.Context())
var (
affected int
err error
)
switch in.Action {
case "status":
affected, err = h.store.BulkSetStatus(r.Context(), actor, in.IDs, in.Status)
case "category":
affected, err = h.store.BulkSetCategory(r.Context(), actor, in.IDs, in.CategoryID)
default:
writeError(w, http.StatusBadRequest, "bad_request", "未知的批量操作")
return
}
switch {
case errors.Is(err, adminstore.ErrInvalidStatus):
writeError(w, http.StatusBadRequest, "invalid_status", "无效的状态值")
return
case errors.Is(err, adminstore.ErrInvalidParent):
writeError(w, http.StatusBadRequest, "invalid_parent", "目标分类无效")
return
case errors.Is(err, adminstore.ErrNoTargets):
writeError(w, http.StatusBadRequest, "bad_request", "未选择任何商品")
return
}
if h.handleErr(w, err) {
return
}
writeJSON(w, http.StatusOK, map[string]any{"status": "ok", "affected": affected})
}
// UpdateProduct applies an edit.
func (h *Handler) UpdateProduct(w http.ResponseWriter, r *http.Request) {
var in adminstore.ProductInput
if err := json.NewDecoder(r.Body).Decode(&in); err != nil {
writeError(w, http.StatusBadRequest, "bad_request", "invalid body")
return
}
if strings.TrimSpace(in.Name) == "" {
writeError(w, http.StatusBadRequest, "bad_request", "名称不能为空")
return
}
d, err := h.store.UpdateProduct(r.Context(), chi.URLParam(r, "id"), auth.UserFrom(r.Context()), in)
if h.handleErr(w, err) {
return
}
writeJSON(w, http.StatusOK, d)
}
// ListAudit returns audit history for a product.
func (h *Handler) ListAudit(w http.ResponseWriter, r *http.Request) {
items, err := h.store.ListAudit(r.Context(), chi.URLParam(r, "id"), 100)
if h.handleErr(w, err) {
return
}
writeJSON(w, http.StatusOK, map[string]any{"items": items})
}
// AddImage adds an image URL.
func (h *Handler) AddImage(w http.ResponseWriter, r *http.Request) {
var body struct {
URL string `json:"url"`
Kind string `json:"kind"`
}
if err := json.NewDecoder(r.Body).Decode(&body); err != nil || strings.TrimSpace(body.URL) == "" {
writeError(w, http.StatusBadRequest, "bad_request", "图片 URL 不能为空")
return
}
im, err := h.store.AddImage(r.Context(), chi.URLParam(r, "id"), auth.UserFrom(r.Context()), body.URL, body.Kind)
if h.handleErr(w, err) {
return
}
writeJSON(w, http.StatusCreated, im)
}
// DeleteImage removes an image.
func (h *Handler) DeleteImage(w http.ResponseWriter, r *http.Request) {
err := h.store.DeleteImage(r.Context(), chi.URLParam(r, "id"), chi.URLParam(r, "imageID"), auth.UserFrom(r.Context()))
if h.handleErr(w, err) {
return
}
writeJSON(w, http.StatusOK, map[string]string{"status": "deleted"})
}
// AddMSRP adds a suggested-retail-price snapshot.
func (h *Handler) AddMSRP(w http.ResponseWriter, r *http.Request) {
var in adminstore.MSRPInput
if err := json.NewDecoder(r.Body).Decode(&in); err != nil {
writeError(w, http.StatusBadRequest, "bad_request", "invalid body")
return
}
m, err := h.store.AddMSRP(r.Context(), chi.URLParam(r, "id"), auth.UserFrom(r.Context()), in)
if h.handleErr(w, err) {
return
}
writeJSON(w, http.StatusCreated, m)
}
// DeleteMSRP removes an MSRP snapshot.
func (h *Handler) DeleteMSRP(w http.ResponseWriter, r *http.Request) {
err := h.store.DeleteMSRP(r.Context(), chi.URLParam(r, "id"), chi.URLParam(r, "msrpID"), auth.UserFrom(r.Context()))
if h.handleErr(w, err) {
return
}
writeJSON(w, http.StatusOK, map[string]string{"status": "deleted"})
}
// ---------- barcodes ----------
// AddBarcode validates and attaches a barcode to a product. A code already
// owned by another product yields 409 with the conflicting product so the
// operator can de-duplicate; an invalid GTIN yields 400.
func (h *Handler) AddBarcode(w http.ResponseWriter, r *http.Request) {
var in adminstore.BarcodeInput
if err := json.NewDecoder(r.Body).Decode(&in); err != nil {
writeError(w, http.StatusBadRequest, "bad_request", "invalid body")
return
}
b, err := h.store.AddBarcode(r.Context(), chi.URLParam(r, "id"), auth.UserFrom(r.Context()), in)
if h.handleBarcodeErr(w, err) {
return
}
writeJSON(w, http.StatusCreated, b)
}
// DeleteBarcode removes a barcode; a primary one is replaced automatically.
func (h *Handler) DeleteBarcode(w http.ResponseWriter, r *http.Request) {
err := h.store.DeleteBarcode(r.Context(), chi.URLParam(r, "id"), chi.URLParam(r, "barcodeID"), auth.UserFrom(r.Context()))
if h.handleErr(w, err) {
return
}
writeJSON(w, http.StatusOK, map[string]string{"status": "deleted"})
}
// SetPrimaryBarcode marks one barcode primary and mirrors it to product.gtin.
func (h *Handler) SetPrimaryBarcode(w http.ResponseWriter, r *http.Request) {
b, err := h.store.SetPrimaryBarcode(r.Context(), chi.URLParam(r, "id"), chi.URLParam(r, "barcodeID"), auth.UserFrom(r.Context()))
if h.handleErr(w, err) {
return
}
writeJSON(w, http.StatusOK, b)
}
// handleBarcodeErr maps barcode-specific errors (GTIN validation, ownership
// conflict) to client-facing statuses, falling back to handleErr otherwise.
func (h *Handler) handleBarcodeErr(w http.ResponseWriter, err error) bool {
if err == nil {
return false
}
var conflict *adminstore.ConflictError
if errors.As(err, &conflict) {
writeJSON(w, http.StatusConflict, map[string]any{
"error": map[string]string{"code": "barcode_conflict", "message": err.Error()},
"conflict": map[string]string{
"gtin": conflict.GTIN,
"product_id": conflict.ProductID,
"product_name": conflict.ProductName,
},
})
return true
}
if errors.Is(err, gtin.ErrEmpty) || errors.Is(err, gtin.ErrFormat) ||
errors.Is(err, gtin.ErrCheck) || errors.Is(err, gtin.ErrRestricted) {
writeError(w, http.StatusBadRequest, "invalid_gtin", err.Error())
return true
}
return h.handleErr(w, err)
}
// ListBrands returns brand options.
func (h *Handler) ListBrands(w http.ResponseWriter, r *http.Request) {
items, err := h.store.ListBrands(r.Context())
if h.handleErr(w, err) {
return
}
writeJSON(w, http.StatusOK, map[string]any{"items": items})
}
// ListKindFields returns the editable spec field template for an archive kind.
func (h *Handler) ListKindFields(w http.ResponseWriter, r *http.Request) {
kind := strings.TrimSpace(r.URL.Query().Get("kind"))
if kind == "" {
writeError(w, http.StatusBadRequest, "bad_request", "缺少 kind 参数")
return
}
items, err := h.store.ListKindFields(r.Context(), kind)
if h.handleErr(w, err) {
return
}
writeJSON(w, http.StatusOK, map[string]any{"items": items, "kind": kind})
}
// ListCategories returns category options.
func (h *Handler) ListCategories(w http.ResponseWriter, r *http.Request) {
items, err := h.store.ListCategories(r.Context())
if h.handleErr(w, err) {
return
}
writeJSON(w, http.StatusOK, map[string]any{"items": items})
}
// CreateCategory adds a category node.
func (h *Handler) CreateCategory(w http.ResponseWriter, r *http.Request) {
var in adminstore.CategoryInput
if err := json.NewDecoder(r.Body).Decode(&in); err != nil {
writeError(w, http.StatusBadRequest, "bad_request", "invalid body")
return
}
if strings.TrimSpace(in.NameZH) == "" {
writeError(w, http.StatusBadRequest, "bad_request", "分类名称不能为空")
return
}
c, err := h.store.CreateCategory(r.Context(), auth.UserFrom(r.Context()), in)
if h.handleCategoryErr(w, err) {
return
}
writeJSON(w, http.StatusCreated, c)
}
// UpdateCategory renames and/or moves a category node.
func (h *Handler) UpdateCategory(w http.ResponseWriter, r *http.Request) {
var in adminstore.CategoryInput
if err := json.NewDecoder(r.Body).Decode(&in); err != nil {
writeError(w, http.StatusBadRequest, "bad_request", "invalid body")
return
}
if strings.TrimSpace(in.NameZH) == "" {
writeError(w, http.StatusBadRequest, "bad_request", "分类名称不能为空")
return
}
c, err := h.store.UpdateCategory(r.Context(), chi.URLParam(r, "id"), auth.UserFrom(r.Context()), in)
if h.handleCategoryErr(w, err) {
return
}
writeJSON(w, http.StatusOK, c)
}
// DeleteCategory removes a leaf category that no product uses.
func (h *Handler) DeleteCategory(w http.ResponseWriter, r *http.Request) {
err := h.store.DeleteCategory(r.Context(), chi.URLParam(r, "id"), auth.UserFrom(r.Context()))
if h.handleCategoryErr(w, err) {
return
}
writeJSON(w, http.StatusOK, map[string]string{"status": "deleted"})
}
// handleCategoryErr maps category-specific errors to client statuses, falling
// back to handleErr otherwise.
func (h *Handler) handleCategoryErr(w http.ResponseWriter, err error) bool {
if err == nil {
return false
}
switch {
case errors.Is(err, adminstore.ErrDuplicatePath):
writeError(w, http.StatusConflict, "duplicate_path", "该分类路径已存在,请换一个英文标识(slug)")
return true
case errors.Is(err, adminstore.ErrCategoryHasChildren):
writeError(w, http.StatusConflict, "has_children", "该分类存在子分类,请先删除或移动其子分类")
return true
case errors.Is(err, adminstore.ErrCategoryInUse):
writeError(w, http.StatusConflict, "in_use", "仍有商品归属于该分类,请先改归其它分类")
return true
case errors.Is(err, adminstore.ErrInvalidParent):
writeError(w, http.StatusBadRequest, "invalid_parent", "上级分类无效(不存在或不能移动到自身/子级下)")
return true
}
return h.handleErr(w, err)
}
type brandInput struct {
Name string `json:"name"`
}
type brandMergeInput struct {
TargetID string `json:"target_id"`
}
// CreateBrand adds a brand.
func (h *Handler) CreateBrand(w http.ResponseWriter, r *http.Request) {
var in brandInput
if err := json.NewDecoder(r.Body).Decode(&in); err != nil {
writeError(w, http.StatusBadRequest, "bad_request", "invalid body")
return
}
if strings.TrimSpace(in.Name) == "" {
writeError(w, http.StatusBadRequest, "bad_request", "品牌名称不能为空")
return
}
b, err := h.store.CreateBrand(r.Context(), auth.UserFrom(r.Context()), in.Name)
if h.handleBrandErr(w, err) {
return
}
writeJSON(w, http.StatusCreated, b)
}
// UpdateBrand renames a brand.
func (h *Handler) UpdateBrand(w http.ResponseWriter, r *http.Request) {
var in brandInput
if err := json.NewDecoder(r.Body).Decode(&in); err != nil {
writeError(w, http.StatusBadRequest, "bad_request", "invalid body")
return
}
if strings.TrimSpace(in.Name) == "" {
writeError(w, http.StatusBadRequest, "bad_request", "品牌名称不能为空")
return
}
b, err := h.store.UpdateBrand(r.Context(), chi.URLParam(r, "id"), auth.UserFrom(r.Context()), in.Name)
if h.handleBrandErr(w, err) {
return
}
writeJSON(w, http.StatusOK, b)
}
// MergeBrands folds one brand's products into another, then deletes the source.
func (h *Handler) MergeBrands(w http.ResponseWriter, r *http.Request) {
var in brandMergeInput
if err := json.NewDecoder(r.Body).Decode(&in); err != nil {
writeError(w, http.StatusBadRequest, "bad_request", "invalid body")
return
}
if strings.TrimSpace(in.TargetID) == "" {
writeError(w, http.StatusBadRequest, "bad_request", "请选择合并目标品牌")
return
}
b, err := h.store.MergeBrands(r.Context(), chi.URLParam(r, "id"), in.TargetID, auth.UserFrom(r.Context()))
if h.handleBrandErr(w, err) {
return
}
writeJSON(w, http.StatusOK, b)
}
// DeleteBrand removes a brand no product references.
func (h *Handler) DeleteBrand(w http.ResponseWriter, r *http.Request) {
err := h.store.DeleteBrand(r.Context(), chi.URLParam(r, "id"), auth.UserFrom(r.Context()))
if h.handleBrandErr(w, err) {
return
}
writeJSON(w, http.StatusOK, map[string]string{"status": "deleted"})
}
// handleBrandErr maps brand-specific errors to client statuses.
func (h *Handler) handleBrandErr(w http.ResponseWriter, err error) bool {
if err == nil {
return false
}
switch {
case errors.Is(err, adminstore.ErrDuplicateBrand):
writeError(w, http.StatusConflict, "duplicate_brand", "该品牌名称已存在")
return true
case errors.Is(err, adminstore.ErrBrandInUse):
writeError(w, http.StatusConflict, "in_use", "仍有商品使用该品牌,请先改用其它品牌或合并")
return true
case errors.Is(err, adminstore.ErrInvalidMerge):
writeError(w, http.StatusBadRequest, "invalid_merge", "合并目标无效(不存在或与源品牌相同)")
return true
}
return h.handleErr(w, err)
}
// ---------- submissions ----------
// CreateSubmission accepts an anonymous public contribution into the queue.
func (h *Handler) CreateSubmission(w http.ResponseWriter, r *http.Request) {
if !h.submitLimit.allow(realIP(r)) {
writeError(w, http.StatusTooManyRequests, "rate_limited", "提交过于频繁,请稍后再试")
return
}
var in adminstore.SubmissionInput
if err := json.NewDecoder(http.MaxBytesReader(w, r.Body, 1<<20)).Decode(&in); err != nil {
writeError(w, http.StatusBadRequest, "bad_request", "invalid body")
return
}
if strings.TrimSpace(in.Name) == "" {
writeError(w, http.StatusBadRequest, "bad_request", "商品名称不能为空")
return
}
id, err := h.store.CreateSubmission(r.Context(), in, realIP(r))
if err != nil {
writeError(w, http.StatusInternalServerError, "internal_error", err.Error())
return
}
writeJSON(w, http.StatusCreated, map[string]string{"id": id, "status": "pending"})
}
// Backflow accepts a batch of products pushed by inventory-management software.
// It authenticates with a public API key (X-API-Key or Bearer), enqueues each
// item for admin review (deduplicating by GTIN), and returns a per-item summary.
func (h *Handler) Backflow(w http.ResponseWriter, r *http.Request) {
raw := presentedAPIKey(r)
if raw == "" {
writeError(w, http.StatusUnauthorized, "missing_api_key", "缺少 API key(请在 X-API-Key 或 Authorization: Bearer 中提供)")
return
}
if !apikey.IsWellFormed(raw) {
writeError(w, http.StatusUnauthorized, "invalid_api_key", "API key 格式无效")
return
}
if _, err := h.store.APIKeyByHash(r.Context(), apikey.Hash(raw)); err != nil {
if errors.Is(err, adminstore.ErrNotFound) {
writeError(w, http.StatusUnauthorized, "invalid_api_key", "API key 无效或已吊销")
return
}
writeError(w, http.StatusInternalServerError, "internal_error", err.Error())
return
}
var items []adminstore.SubmissionInput
if err := json.NewDecoder(http.MaxBytesReader(w, r.Body, 16<<20)).Decode(&items); err != nil {
writeError(w, http.StatusBadRequest, "bad_request", "请求体应为商品数组 (JSON array)")
return
}
if len(items) == 0 {
writeError(w, http.StatusBadRequest, "bad_request", "回流列表为空")
return
}
if len(items) > 1000 {
writeError(w, http.StatusBadRequest, "too_many", "单次回流最多 1000 条")
return
}
sum, err := h.store.CreateBackflowSubmissions(r.Context(), items, realIP(r))
if err != nil {
writeError(w, http.StatusInternalServerError, "internal_error", err.Error())
return
}
writeJSON(w, http.StatusOK, sum)
}
// presentedAPIKey extracts a public API key from X-API-Key or a Bearer token.
func presentedAPIKey(r *http.Request) string {
if v := strings.TrimSpace(r.Header.Get("X-API-Key")); v != "" {
return v
}
if v := r.Header.Get("Authorization"); strings.HasPrefix(v, "Bearer ") {
return strings.TrimSpace(strings.TrimPrefix(v, "Bearer "))
}
return ""
}
// ListSubmissions returns the moderation queue (admin).
func (h *Handler) ListSubmissions(w http.ResponseWriter, r *http.Request) {
status := r.URL.Query().Get("status")
page, size := pageParams(r)
items, total, err := h.store.ListSubmissions(r.Context(), status, size, (page-1)*size)
if h.handleErr(w, err) {
return
}
pending, _ := h.store.PendingSubmissionCount(r.Context())
writeJSON(w, http.StatusOK, map[string]any{
"items": items, "page": page, "size": size, "total": total, "pending": pending,
})
}
// GetSubmission returns full submission detail (admin).
func (h *Handler) GetSubmission(w http.ResponseWriter, r *http.Request) {
d, err := h.store.GetSubmission(r.Context(), chi.URLParam(r, "id"))
if h.handleErr(w, err) {
return
}
writeJSON(w, http.StatusOK, d)
}
// ApproveSubmission applies a contribution to the product store (admin).
func (h *Handler) ApproveSubmission(w http.ResponseWriter, r *http.Request) {
d, err := h.store.ApproveSubmission(r.Context(), chi.URLParam(r, "id"), auth.UserFrom(r.Context()))
if h.handleErr(w, err) {
return
}
writeJSON(w, http.StatusOK, d)
}
// RejectSubmission rejects a contribution with a reviewer note (admin).
func (h *Handler) RejectSubmission(w http.ResponseWriter, r *http.Request) {
var body struct {
Note string `json:"note"`
}
_ = json.NewDecoder(r.Body).Decode(&body)
err := h.store.RejectSubmission(r.Context(), chi.URLParam(r, "id"), auth.UserFrom(r.Context()), body.Note)
if h.handleErr(w, err) {
return
}
writeJSON(w, http.StatusOK, map[string]string{"status": "rejected"})
}
// ---------- helpers ----------
func realIP(r *http.Request) string {
if ip := r.Header.Get("X-Forwarded-For"); ip != "" {
if i := strings.IndexByte(ip, ','); i >= 0 {
return strings.TrimSpace(ip[:i])
}
return strings.TrimSpace(ip)
}
if ip := r.Header.Get("X-Real-IP"); ip != "" {
return ip
}
return r.RemoteAddr
}
func (h *Handler) handleErr(w http.ResponseWriter, err error) bool {
if err == nil {
return false
}
if errors.Is(err, adminstore.ErrNotFound) {
writeError(w, http.StatusNotFound, "not_found", "资源不存在")
return true
}
if errors.Is(err, adminstore.ErrConflict) {
writeError(w, http.StatusConflict, "conflict", "该投稿已被处理")
return true
}
writeError(w, http.StatusInternalServerError, "internal_error", err.Error())
return true
}
func pageParams(r *http.Request) (page, size int) {
page = atoiDefault(r.URL.Query().Get("page"), 1)
if page < 1 {
page = 1
}
size = atoiDefault(r.URL.Query().Get("size"), 20)
if size < 1 {
size = 20
}
if size > 100 {
size = 100
}
return page, size
}
func atoiDefault(s string, fallback int) int {
if s == "" {
return fallback
}
n := 0
for _, c := range s {
if c < '0' || c > '9' {
return fallback
}
n = n*10 + int(c-'0')
}
return n
}
func writeJSON(w http.ResponseWriter, status int, body any) {
w.Header().Set("Content-Type", "application/json; charset=utf-8")
w.WriteHeader(status)
_ = json.NewEncoder(w).Encode(body)
}
func writeError(w http.ResponseWriter, status int, code, message string) {
writeJSON(w, status, map[string]any{"error": map[string]string{"code": code, "message": message}})
}
+30
View File
@@ -0,0 +1,30 @@
package adminhandler
import (
"encoding/json"
"net/http"
"github.com/baicai2026-baicai/goods/api/internal/adminstore"
)
func (h *Handler) ImportBypos(w http.ResponseWriter, r *http.Request) {
var records []adminstore.ByposRecord
dec := json.NewDecoder(http.MaxBytesReader(w, r.Body, 50<<20))
for dec.More() {
var rec adminstore.ByposRecord
if err := dec.Decode(&rec); err != nil {
continue
}
records = append(records, rec)
}
if len(records) == 0 {
writeError(w, http.StatusBadRequest, "empty", "\u6ca1\u6709\u53ef\u5bfc\u5165\u7684\u8bb0\u5f55")
return
}
result, err := h.store.ImportByposRecords(r.Context(), records)
if err != nil {
writeError(w, http.StatusInternalServerError, "import_error", err.Error())
return
}
writeJSON(w, http.StatusOK, result)
}
+41
View File
@@ -0,0 +1,41 @@
package adminhandler
import (
"sync"
"time"
)
// rateLimiter is a simple fixed-window per-key limiter used to throttle
// anonymous public submissions (basic anti-spam; captcha can be added later).
type rateLimiter struct {
mu sync.Mutex
hits map[string][]time.Time
limit int
window time.Duration
}
func newRateLimiter(limit int, window time.Duration) *rateLimiter {
return &rateLimiter{hits: map[string][]time.Time{}, limit: limit, window: window}
}
// allow reports whether the key may proceed, recording the hit if so.
func (r *rateLimiter) allow(key string) bool {
now := time.Now()
cutoff := now.Add(-r.window)
r.mu.Lock()
defer r.mu.Unlock()
kept := r.hits[key][:0]
for _, t := range r.hits[key] {
if t.After(cutoff) {
kept = append(kept, t)
}
}
if len(kept) >= r.limit {
r.hits[key] = kept
return false
}
r.hits[key] = append(kept, now)
return true
}
+349
View File
@@ -0,0 +1,349 @@
// Package adminstore is the read/write data-access layer for the admin console.
// Unlike the public store (read-only), it performs INSERT/UPDATE/DELETE and
// records field-level provenance (source = "manual") plus an audit_log entry
// for every write, then recomputes product.quality_score.
package adminstore
import (
"context"
"encoding/json"
"errors"
"strconv"
"strings"
"time"
"github.com/jackc/pgx/v5"
"github.com/jackc/pgx/v5/pgxpool"
)
// ErrNotFound is returned when a requested row does not exist.
var ErrNotFound = errors.New("not found")
// Store wraps a pgx pool for admin operations.
type Store struct {
pool *pgxpool.Pool
}
// New constructs an admin Store.
func New(pool *pgxpool.Pool) *Store { return &Store{pool: pool} }
// Ping verifies DB connectivity.
func (s *Store) Ping(ctx context.Context) error { return s.pool.Ping(ctx) }
// CompletenessFields mirrors ingestion/opengoods/etl/quality.py COMPLETENESS_FIELDS.
var CompletenessFields = []string{
"name", "gtin", "brand", "category", "net_content",
"country_of_origin", "nutriments", "ingredients", "image",
}
// ---------- list ----------
// ProductRow is a list-view row for the admin product table.
type ProductRow struct {
ID string `json:"id"`
GTIN *string `json:"gtin"`
Name string `json:"name"`
Brand *string `json:"brand"`
CategoryPath *string `json:"category_path"`
Status string `json:"status"`
QualityScore float64 `json:"quality_score"`
Missing []string `json:"missing"`
UpdatedAt string `json:"updated_at"`
}
// productSortColumns whitelists the sortable list columns, mapping the API sort
// key to a SQL expression. NULLs sort last regardless of direction.
var productSortColumns = map[string]string{
"name": "p.name",
"brand": "b.name",
"gtin": "p.gtin",
"category_path": "c.path",
"status": "p.status",
"quality_score": "p.quality_score",
"updated_at": "p.updated_at",
}
// productOrderBy returns a safe ORDER BY clause for the given sort key/direction,
// falling back to the default (most recently updated first) for unknown keys.
func productOrderBy(sort, order string) string {
col, ok := productSortColumns[sort]
if !ok {
return "p.updated_at DESC"
}
dir := "ASC"
if strings.EqualFold(order, "desc") {
dir = "DESC"
}
return col + " " + dir + " NULLS LAST, p.updated_at DESC"
}
// ListProducts returns a paginated, optionally name/gtin-filtered list.
func (s *Store) ListProducts(ctx context.Context, q, sort, order string, limit, offset int) ([]ProductRow, int, error) {
args := []any{}
where := "WHERE 1=1"
if q != "" {
args = append(args, q)
where += " AND (p.name ILIKE '%' || $1 || '%' OR p.gtin ILIKE '%' || $1 || '%')"
}
var total int
if err := s.pool.QueryRow(ctx, "SELECT count(*) FROM product p "+where, args...).Scan(&total); err != nil {
return nil, 0, err
}
qualified, err := s.kindQualifiedKeys(ctx, s.pool)
if err != nil {
return nil, 0, err
}
args = append(args, limit, offset)
sql := `
SELECT p.id, p.gtin, p.name, b.name, c.path::text, p.status, p.quality_score,
p.updated_at, COALESCE(c.archive_kind, 'generic'), p.attributes,
(p.brand_id IS NOT NULL) AS has_brand,
(p.category_id IS NOT NULL) AS has_cat,
(p.net_content_canonical IS NOT NULL) AS has_net,
(p.country_of_origin IS NOT NULL AND p.country_of_origin <> '') AS has_country,
(f.nutriments IS NOT NULL AND f.nutriments::text <> '{}') AS has_nutri,
(f.ingredients_text IS NOT NULL AND f.ingredients_text <> '') AS has_ing,
EXISTS (SELECT 1 FROM product_image pi WHERE pi.product_id = p.id) AS has_img
FROM product p
LEFT JOIN brand b ON b.id = p.brand_id
LEFT JOIN category c ON c.id = p.category_id
LEFT JOIN food_detail f ON f.product_id = p.id ` + where +
" ORDER BY " + productOrderBy(sort, order) +
" LIMIT $" + strconv.Itoa(len(args)-1) + " OFFSET $" + strconv.Itoa(len(args))
rows, err := s.pool.Query(ctx, sql, args...)
if err != nil {
return nil, 0, err
}
defer rows.Close()
out := []ProductRow{}
for rows.Next() {
var r ProductRow
var hasBrand, hasCat, hasNet, hasCountry, hasNutri, hasIng, hasImg bool
var kind string
var attributes []byte
var updated time.Time
if err := rows.Scan(&r.ID, &r.GTIN, &r.Name, &r.Brand, &r.CategoryPath, &r.Status,
&r.QualityScore, &updated, &kind, &attributes,
&hasBrand, &hasCat, &hasNet, &hasCountry,
&hasNutri, &hasIng, &hasImg); err != nil {
return nil, 0, err
}
r.UpdatedAt = updated.Format(time.RFC3339)
attrs := map[string]any{}
if len(attributes) > 0 {
_ = json.Unmarshal(attributes, &attrs)
}
qkeys := qualified[kind]
present := map[string]bool{
"name": r.Name != "",
"gtin": r.GTIN != nil && *r.GTIN != "",
"brand": hasBrand,
"category": hasCat,
"net_content": hasNet,
"country_of_origin": hasCountry,
"nutriments": hasNutri,
"ingredients": hasIng,
"image": hasImg,
}
for _, k := range qkeys {
present[k] = attrPresent(attrs, k)
}
r.Missing = []string{}
for _, f := range completenessKeys(kind, qkeys) {
if !present[f] {
r.Missing = append(r.Missing, f)
}
}
out = append(out, r)
}
return out, total, rows.Err()
}
// ---------- detail ----------
// ProductImage is one image row.
type ProductImage struct {
ID string `json:"id"`
URL string `json:"url"`
Kind string `json:"kind"`
License *string `json:"license"`
}
// MSRP is one suggested-retail-price snapshot.
type MSRP struct {
ID string `json:"id"`
Amount float64 `json:"amount"`
Currency string `json:"currency"`
Region string `json:"region"`
EffectiveDate *string `json:"effective_date"`
SourceURL *string `json:"source_url"`
Note *string `json:"note"`
}
// ProductDetail is the full editable view of a product.
type ProductDetail struct {
ID string `json:"id"`
GTIN *string `json:"gtin"`
Name string `json:"name"`
BrandID *string `json:"brand_id"`
Brand *string `json:"brand"`
CategoryID *string `json:"category_id"`
CategoryPath *string `json:"category_path"`
ArchiveKind string `json:"archive_kind"`
Attributes map[string]any `json:"attributes"`
NetContentValue *float64 `json:"net_content_value"`
NetContentUnit *string `json:"net_content_unit"`
CountryOfOrigin *string `json:"country_of_origin"`
Status string `json:"status"`
QualityScore float64 `json:"quality_score"`
IngredientsText *string `json:"ingredients_text"`
Allergens []string `json:"allergens"`
Additives []string `json:"additives"`
Nutriments map[string]any `json:"nutriments"`
NutritionBasis *string `json:"nutrition_basis"`
ServingSize *string `json:"serving_size"`
NutriScore *string `json:"nutri_score"`
Barcodes []Barcode `json:"barcodes"`
Images []ProductImage `json:"images"`
MSRP []MSRP `json:"msrp"`
Missing []string `json:"missing"`
UpdatedAt string `json:"updated_at"`
}
// GetProduct returns the full editable detail for one product.
func (s *Store) GetProduct(ctx context.Context, id string) (*ProductDetail, error) {
var d ProductDetail
var nutriments []byte
var attributes []byte
var updated time.Time
err := s.pool.QueryRow(ctx, `
SELECT p.id, p.gtin, p.name, p.brand_id, b.name, p.category_id, c.path::text,
COALESCE(c.archive_kind, 'generic'), p.attributes,
p.net_content_value, p.net_content_unit, p.country_of_origin, p.status,
p.quality_score, p.updated_at,
f.ingredients_text, f.allergens, f.additives, f.nutriments,
f.nutrition_basis, f.serving_size, f.nutri_score
FROM product p
LEFT JOIN brand b ON b.id = p.brand_id
LEFT JOIN category c ON c.id = p.category_id
LEFT JOIN food_detail f ON f.product_id = p.id
WHERE p.id = $1`, id).Scan(
&d.ID, &d.GTIN, &d.Name, &d.BrandID, &d.Brand, &d.CategoryID, &d.CategoryPath,
&d.ArchiveKind, &attributes,
&d.NetContentValue, &d.NetContentUnit, &d.CountryOfOrigin, &d.Status,
&d.QualityScore, &updated,
&d.IngredientsText, &d.Allergens, &d.Additives, &nutriments,
&d.NutritionBasis, &d.ServingSize, &d.NutriScore,
)
if errors.Is(err, pgx.ErrNoRows) {
return nil, ErrNotFound
}
if err != nil {
return nil, err
}
d.UpdatedAt = updated.Format(time.RFC3339)
d.Attributes = map[string]any{}
if len(attributes) > 0 {
_ = json.Unmarshal(attributes, &d.Attributes)
}
if len(nutriments) > 0 {
_ = json.Unmarshal(nutriments, &d.Nutriments)
}
if d.Allergens == nil {
d.Allergens = []string{}
}
if d.Additives == nil {
d.Additives = []string{}
}
bcs, err := s.listBarcodes(ctx, id)
if err != nil {
return nil, err
}
d.Barcodes = bcs
imgs, err := s.listImages(ctx, id)
if err != nil {
return nil, err
}
d.Images = imgs
msrps, err := s.listMSRP(ctx, id)
if err != nil {
return nil, err
}
d.MSRP = msrps
qualified, err := s.kindQualifiedKeys(ctx, s.pool)
if err != nil {
return nil, err
}
d.Missing = missingFromDetail(&d, qualified[d.ArchiveKind])
return &d, nil
}
func missingFromDetail(d *ProductDetail, qualifiedAttrKeys []string) []string {
present := map[string]bool{
"name": d.Name != "",
"gtin": d.GTIN != nil && *d.GTIN != "",
"brand": d.BrandID != nil,
"category": d.CategoryID != nil,
"net_content": d.NetContentValue != nil,
"country_of_origin": d.CountryOfOrigin != nil && *d.CountryOfOrigin != "",
"nutriments": len(d.Nutriments) > 0,
"ingredients": d.IngredientsText != nil && *d.IngredientsText != "",
"image": len(d.Images) > 0,
}
for _, k := range qualifiedAttrKeys {
present[k] = attrPresent(d.Attributes, k)
}
missing := []string{}
for _, f := range completenessKeys(d.ArchiveKind, qualifiedAttrKeys) {
if !present[f] {
missing = append(missing, f)
}
}
return missing
}
func (s *Store) listImages(ctx context.Context, productID string) ([]ProductImage, error) {
rows, err := s.pool.Query(ctx,
"SELECT id, url, kind, license FROM product_image WHERE product_id = $1 ORDER BY id", productID)
if err != nil {
return nil, err
}
defer rows.Close()
out := []ProductImage{}
for rows.Next() {
var im ProductImage
if err := rows.Scan(&im.ID, &im.URL, &im.Kind, &im.License); err != nil {
return nil, err
}
out = append(out, im)
}
return out, rows.Err()
}
func (s *Store) listMSRP(ctx context.Context, productID string) ([]MSRP, error) {
rows, err := s.pool.Query(ctx, `
SELECT id, amount, currency, region, effective_date::text, source_url, note
FROM product_msrp WHERE product_id = $1 ORDER BY effective_date DESC NULLS LAST`, productID)
if err != nil {
return nil, err
}
defer rows.Close()
out := []MSRP{}
for rows.Next() {
var m MSRP
if err := rows.Scan(&m.ID, &m.Amount, &m.Currency, &m.Region, &m.EffectiveDate, &m.SourceURL, &m.Note); err != nil {
return nil, err
}
out = append(out, m)
}
return out, rows.Err()
}
+136
View File
@@ -0,0 +1,136 @@
package adminstore
import (
"context"
"errors"
"strings"
"time"
"github.com/jackc/pgx/v5/pgconn"
"github.com/baicai2026-baicai/goods/api/internal/apikey"
)
// APIKeyRow is an admin-facing view of an issued API key (never the secret).
type APIKeyRow struct {
ID string `json:"id"`
Name string `json:"name"`
KeyPrefix string `json:"key_prefix"`
OwnerEmail *string `json:"owner_email"`
Tier string `json:"tier"`
RateLimitPerMin int `json:"rate_limit_per_min"`
QuotaTotal int64 `json:"quota_total"`
RevokedAt *string `json:"revoked_at"`
CreatedBy *string `json:"created_by"`
CreatedAt string `json:"created_at"`
}
// APIKeyInput holds the fields accepted when issuing a key.
type APIKeyInput struct {
Name string `json:"name"`
OwnerEmail string `json:"owner_email"`
Tier string `json:"tier"`
RateLimitPerMin int `json:"rate_limit_per_min"`
QuotaTotal int64 `json:"quota_total"`
}
// CreateAPIKey issues a new key, returning the one-time plaintext alongside the
// stored row. Only the SHA-256 hash and a short display prefix are persisted.
func (s *Store) CreateAPIKey(ctx context.Context, in APIKeyInput, createdBy string) (plaintext string, row APIKeyRow, err error) {
tier := in.Tier
if tier == "" {
tier = "free"
}
rate := in.RateLimitPerMin
if rate <= 0 {
rate = 120
}
quota := in.QuotaTotal
if quota < 0 {
quota = 0
}
var owner *string
if e := strings.TrimSpace(in.OwnerEmail); e != "" {
owner = &e
}
key, hash, prefix, err := apikey.Generate()
if err != nil {
return "", row, err
}
var revoked, created *time.Time
var createdByOut *string
err = s.pool.QueryRow(ctx, `
INSERT INTO api_key (name, key_prefix, key_hash, owner_email, tier, rate_limit_per_min, quota_total, created_by)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8)
RETURNING id, name, key_prefix, owner_email, tier, rate_limit_per_min, quota_total, revoked_at, created_by, created_at`,
strings.TrimSpace(in.Name), prefix, hash, owner, tier, rate, quota, createdBy,
).Scan(&row.ID, &row.Name, &row.KeyPrefix, &row.OwnerEmail, &row.Tier,
&row.RateLimitPerMin, &row.QuotaTotal, &revoked, &createdByOut, &created)
if err != nil {
return "", row, err
}
row.CreatedBy = createdByOut
if created != nil {
row.CreatedAt = created.Format(time.RFC3339)
}
return key, row, nil
}
// ListAPIKeys returns all keys (active first, newest first).
func (s *Store) ListAPIKeys(ctx context.Context) ([]APIKeyRow, error) {
rows, err := s.pool.Query(ctx, `
SELECT id, name, key_prefix, owner_email, tier, rate_limit_per_min, quota_total, revoked_at, created_by, created_at
FROM api_key
ORDER BY (revoked_at IS NULL) DESC, created_at DESC`)
if err != nil {
return nil, err
}
defer rows.Close()
out := []APIKeyRow{}
for rows.Next() {
var r APIKeyRow
var revoked, created *time.Time
if err := rows.Scan(&r.ID, &r.Name, &r.KeyPrefix, &r.OwnerEmail, &r.Tier,
&r.RateLimitPerMin, &r.QuotaTotal, &revoked, &r.CreatedBy, &created); err != nil {
return nil, err
}
if revoked != nil {
v := revoked.Format(time.RFC3339)
r.RevokedAt = &v
}
if created != nil {
r.CreatedAt = created.Format(time.RFC3339)
}
out = append(out, r)
}
return out, rows.Err()
}
// RevokeAPIKey marks a key revoked. Revoking an already-revoked or missing key
// returns ErrNotFound.
func (s *Store) RevokeAPIKey(ctx context.Context, id string) error {
tag, err := s.pool.Exec(ctx,
"UPDATE api_key SET revoked_at = now() WHERE id = $1 AND revoked_at IS NULL", id)
if err != nil {
if isInvalidUUID(err) {
return ErrNotFound
}
return err
}
if tag.RowsAffected() == 0 {
return ErrNotFound
}
return nil
}
// isInvalidUUID reports whether err is a Postgres invalid-UUID-text error,
// which happens when a non-UUID id is supplied.
func isInvalidUUID(err error) bool {
var pgErr *pgconn.PgError
if errors.As(err, &pgErr) {
return pgErr.Code == "22P02"
}
return false
}
+158
View File
@@ -0,0 +1,158 @@
package adminstore
import (
"context"
"encoding/json"
"errors"
"strings"
"github.com/baicai2026-baicai/goods/api/internal/gtin"
"github.com/jackc/pgx/v5"
)
// backflowSource is the value stored in submission.payload->>'source' for
// records pushed by inventory-management software via the backflow API.
const backflowSource = "backflow"
// APIKeyAuth is the minimal key metadata needed to authenticate a backflow
// caller. Only active (non-revoked) keys resolve.
type APIKeyAuth struct {
ID string
Name string
}
// APIKeyByHash returns the active key matching a SHA-256 hash, or ErrNotFound
// when no such active key exists. Used to authenticate machine callers (e.g.
// the backflow endpoint) with the same public API keys issued to API users.
func (s *Store) APIKeyByHash(ctx context.Context, hash string) (*APIKeyAuth, error) {
var k APIKeyAuth
err := s.pool.QueryRow(ctx,
"SELECT id, name FROM api_key WHERE key_hash = $1 AND revoked_at IS NULL", hash,
).Scan(&k.ID, &k.Name)
if errors.Is(err, pgx.ErrNoRows) {
return nil, ErrNotFound
}
if err != nil {
return nil, err
}
return &k, nil
}
// BackflowResult reports the outcome of one item in a backflow batch.
type BackflowResult struct {
GTIN *string `json:"gtin"`
Name string `json:"name"`
Status string `json:"status"` // queued | exists | duplicate | invalid
ID string `json:"id,omitempty"` // submission id when queued
Reason string `json:"reason,omitempty"`
}
// BackflowSummary aggregates a backflow batch outcome.
type BackflowSummary struct {
Total int `json:"total"`
Queued int `json:"queued"`
Exists int `json:"exists"`
Duplicate int `json:"duplicate"`
Invalid int `json:"invalid"`
Results []BackflowResult `json:"results"`
}
// CreateBackflowSubmissions enqueues products pushed by inventory software for
// admin review. Each item is deduplicated by GTIN: items whose barcode already
// matches an archived product are skipped ("exists"), and items that duplicate
// a pending submission are skipped ("duplicate"). Accepted items are tagged
// with source="backflow" and enter the same moderation queue as public
// contributions; approval creates the product exactly as ApproveSubmission does.
//
// Items are processed independently: a bad item never rolls back accepted ones.
func (s *Store) CreateBackflowSubmissions(ctx context.Context, items []SubmissionInput, remoteIP string) (BackflowSummary, error) {
sum := BackflowSummary{Total: len(items), Results: make([]BackflowResult, 0, len(items))}
for _, in := range items {
in.Name = strings.TrimSpace(in.Name)
res := BackflowResult{Name: in.Name}
if in.GTIN != nil {
g := strings.TrimSpace(*in.GTIN)
if g == "" {
in.GTIN = nil
} else {
// Validate/normalize so a malformed barcode is reported as an
// invalid item instead of aborting the batch (the gtin column
// is varchar(14) and only GS1 codes are archived).
norm, err := gtin.Normalize(g)
if err != nil {
res.GTIN = &g
res.Status = "invalid"
res.Reason = err.Error()
sum.Invalid++
sum.Results = append(sum.Results, res)
continue
}
in.GTIN = &norm
res.GTIN = &norm
}
}
if in.Name == "" {
res.Status = "invalid"
res.Reason = "商品名称不能为空"
sum.Invalid++
sum.Results = append(sum.Results, res)
continue
}
if in.GTIN != nil {
// Already archived: backflow only carries products we don't have yet.
var pid string
err := s.pool.QueryRow(ctx, "SELECT id FROM product WHERE gtin = $1", *in.GTIN).Scan(&pid)
if err == nil {
res.Status = "exists"
res.Reason = "该条码商品已收录"
sum.Exists++
sum.Results = append(sum.Results, res)
continue
} else if !errors.Is(err, pgx.ErrNoRows) {
return sum, err
}
// Collapse repeated auto-pushes of the same barcode in the queue.
var sid string
err = s.pool.QueryRow(ctx,
"SELECT id FROM submission WHERE gtin = $1 AND status = 'pending' LIMIT 1", *in.GTIN).Scan(&sid)
if err == nil {
res.Status = "duplicate"
res.Reason = "已有待审核的同条码回流记录"
res.ID = sid
sum.Duplicate++
sum.Results = append(sum.Results, res)
continue
} else if !errors.Is(err, pgx.ErrNoRows) {
return sum, err
}
}
src := backflowSource
in.Source = &src
payload, err := json.Marshal(in)
if err != nil {
return sum, err
}
var id string
err = s.pool.QueryRow(ctx, `
INSERT INTO submission (gtin, name, payload, submitter_name, submitter_contact, note, remote_ip)
VALUES ($1,$2,$3,$4,$5,$6,$7) RETURNING id`,
in.GTIN, in.Name, payload, in.SubmitterName, in.SubmitterContact, in.Note, remoteIP).Scan(&id)
if err != nil {
return sum, err
}
res.Status = "queued"
res.ID = id
sum.Queued++
sum.Results = append(sum.Results, res)
}
return sum, nil
}
+270
View File
@@ -0,0 +1,270 @@
package adminstore
import (
"context"
"errors"
"github.com/jackc/pgx/v5"
"github.com/baicai2026-baicai/goods/api/internal/gtin"
)
// Barcode is one GS1 trade item number attached to a product.
type Barcode struct {
ID string `json:"id"`
GTIN string `json:"gtin"`
GTINType string `json:"gtin_type"`
PackLevel string `json:"pack_level"`
Region *string `json:"region"`
IsPrimary bool `json:"is_primary"`
}
// BarcodeInput is the payload for attaching a barcode to a product.
type BarcodeInput struct {
GTIN string `json:"gtin"`
GTINType string `json:"gtin_type"`
PackLevel string `json:"pack_level"`
Region *string `json:"region"`
IsPrimary bool `json:"is_primary"`
}
// ConflictError signals that a barcode is already attached to another product,
// so the operator must de-duplicate instead of creating a clash.
type ConflictError struct {
GTIN string
ProductID string
ProductName string
}
func (e *ConflictError) Error() string { return "条码已被其他商品占用:" + e.GTIN }
func validPackLevel(p string) string {
switch p {
case "each", "case", "pallet":
return p
default:
return "each"
}
}
func validGTINType(t, normalized string) string {
switch t {
case "EAN8", "UPC", "EAN13", "ITF14", "GTIN14":
return t
default:
return gtin.InferType(normalized)
}
}
func (s *Store) listBarcodes(ctx context.Context, productID string) ([]Barcode, error) {
rows, err := s.pool.Query(ctx,
`SELECT id, gtin, gtin_type, pack_level, region, is_primary
FROM product_barcode WHERE product_id = $1
ORDER BY is_primary DESC, gtin`, productID)
if err != nil {
return nil, err
}
defer rows.Close()
out := []Barcode{}
for rows.Next() {
var b Barcode
if err := rows.Scan(&b.ID, &b.GTIN, &b.GTINType, &b.PackLevel, &b.Region, &b.IsPrimary); err != nil {
return nil, err
}
out = append(out, b)
}
return out, rows.Err()
}
// barcodeOwner returns the product currently owning a barcode, if any.
func barcodeOwner(ctx context.Context, q pgx.Tx, code string) (productID, productName string, found bool, err error) {
err = q.QueryRow(ctx,
`SELECT pb.product_id, p.name FROM product_barcode pb
JOIN product p ON p.id = pb.product_id WHERE pb.gtin = $1`, code).
Scan(&productID, &productName)
if errors.Is(err, pgx.ErrNoRows) {
return "", "", false, nil
}
if err != nil {
return "", "", false, err
}
return productID, productName, true, nil
}
// AddBarcode validates and attaches a barcode to a product, recording audit.
// A barcode already owned by another product yields a *ConflictError.
func (s *Store) AddBarcode(ctx context.Context, productID, actor string, in BarcodeInput) (*Barcode, error) {
code, err := gtin.Normalize(in.GTIN)
if err != nil {
return nil, err
}
tx, err := s.pool.Begin(ctx)
if err != nil {
return nil, err
}
defer tx.Rollback(ctx)
// Product must exist.
var exists bool
if err := tx.QueryRow(ctx, "SELECT EXISTS(SELECT 1 FROM product WHERE id=$1)", productID).Scan(&exists); err != nil {
return nil, err
}
if !exists {
return nil, ErrNotFound
}
// Globally unique: a barcode owned by any product (this one included)
// is a conflict the operator must resolve by de-duplicating.
if owner, name, found, err := barcodeOwner(ctx, tx, code); err != nil {
return nil, err
} else if found {
return nil, &ConflictError{GTIN: code, ProductID: owner, ProductName: name}
}
// Make this the primary barcode when requested or when none exists yet.
makePrimary := in.IsPrimary
if !makePrimary {
var hasPrimary bool
if err := tx.QueryRow(ctx,
"SELECT EXISTS(SELECT 1 FROM product_barcode WHERE product_id=$1 AND is_primary)", productID).
Scan(&hasPrimary); err != nil {
return nil, err
}
makePrimary = !hasPrimary
}
if makePrimary {
if _, err := tx.Exec(ctx,
"UPDATE product_barcode SET is_primary=false WHERE product_id=$1 AND is_primary", productID); err != nil {
return nil, err
}
}
srcID, _ := s.manualSourceID(ctx, tx)
var srcArg any
if srcID != "" {
srcArg = srcID
}
var b Barcode
err = tx.QueryRow(ctx, `
INSERT INTO product_barcode (product_id, gtin, gtin_type, pack_level, region, is_primary, source_id)
VALUES ($1,$2,$3,$4,$5,$6,$7)
RETURNING id, gtin, gtin_type, pack_level, region, is_primary`,
productID, code, validGTINType(in.GTINType, code), validPackLevel(in.PackLevel),
in.Region, makePrimary, srcArg).
Scan(&b.ID, &b.GTIN, &b.GTINType, &b.PackLevel, &b.Region, &b.IsPrimary)
if err != nil {
return nil, err
}
if makePrimary {
if _, err := tx.Exec(ctx, "UPDATE product SET gtin=$2 WHERE id=$1", productID, code); err != nil {
return nil, err
}
}
if _, err := s.recomputeQualityTx(ctx, tx, productID); err != nil {
return nil, err
}
if err := tx.Commit(ctx); err != nil {
return nil, err
}
_ = s.writeAudit(ctx, actor, "add_barcode", "product", &productID, []string{"gtin"}, nil, b)
return &b, nil
}
// DeleteBarcode removes a barcode; if it was primary, another is promoted.
func (s *Store) DeleteBarcode(ctx context.Context, productID, barcodeID, actor string) error {
tx, err := s.pool.Begin(ctx)
if err != nil {
return err
}
defer tx.Rollback(ctx)
var code string
var wasPrimary bool
err = tx.QueryRow(ctx,
"DELETE FROM product_barcode WHERE id=$1 AND product_id=$2 RETURNING gtin, is_primary",
barcodeID, productID).Scan(&code, &wasPrimary)
if errors.Is(err, pgx.ErrNoRows) {
return ErrNotFound
}
if err != nil {
return err
}
if wasPrimary {
var newID, newGTIN string
e := tx.QueryRow(ctx,
"SELECT id, gtin FROM product_barcode WHERE product_id=$1 ORDER BY gtin LIMIT 1", productID).
Scan(&newID, &newGTIN)
if e == nil {
if _, err := tx.Exec(ctx, "UPDATE product_barcode SET is_primary=true WHERE id=$1", newID); err != nil {
return err
}
if _, err := tx.Exec(ctx, "UPDATE product SET gtin=$2 WHERE id=$1", productID, newGTIN); err != nil {
return err
}
} else if errors.Is(e, pgx.ErrNoRows) {
if _, err := tx.Exec(ctx, "UPDATE product SET gtin=NULL WHERE id=$1", productID); err != nil {
return err
}
} else {
return e
}
}
if _, err := s.recomputeQualityTx(ctx, tx, productID); err != nil {
return err
}
if err := tx.Commit(ctx); err != nil {
return err
}
_ = s.writeAudit(ctx, actor, "delete_barcode", "product", &productID, []string{"gtin"},
map[string]string{"gtin": code}, nil)
return nil
}
// SetPrimaryBarcode marks one barcode primary and mirrors it to product.gtin.
func (s *Store) SetPrimaryBarcode(ctx context.Context, productID, barcodeID, actor string) (*Barcode, error) {
tx, err := s.pool.Begin(ctx)
if err != nil {
return nil, err
}
defer tx.Rollback(ctx)
var code string
err = tx.QueryRow(ctx, "SELECT gtin FROM product_barcode WHERE id=$1 AND product_id=$2", barcodeID, productID).Scan(&code)
if errors.Is(err, pgx.ErrNoRows) {
return nil, ErrNotFound
}
if err != nil {
return nil, err
}
if _, err := tx.Exec(ctx, "UPDATE product_barcode SET is_primary=false WHERE product_id=$1 AND is_primary", productID); err != nil {
return nil, err
}
if _, err := tx.Exec(ctx, "UPDATE product_barcode SET is_primary=true WHERE id=$1", barcodeID); err != nil {
return nil, err
}
if _, err := tx.Exec(ctx, "UPDATE product SET gtin=$2 WHERE id=$1", productID, code); err != nil {
return nil, err
}
if err := tx.Commit(ctx); err != nil {
return nil, err
}
_ = s.writeAudit(ctx, actor, "set_primary_barcode", "product", &productID, []string{"gtin"}, nil,
map[string]string{"gtin": code})
bcs, err := s.listBarcodes(ctx, productID)
if err != nil {
return nil, err
}
for i := range bcs {
if bcs[i].ID == barcodeID {
return &bcs[i], nil
}
}
return nil, ErrNotFound
}
+154
View File
@@ -0,0 +1,154 @@
package adminstore
import (
"context"
"errors"
"fmt"
"strings"
"github.com/jackc/pgx/v5"
)
// Brand-management errors, mapped to client statuses by the handler.
var (
// ErrDuplicateBrand is returned when a brand name already exists.
ErrDuplicateBrand = errors.New("duplicate brand name")
// ErrBrandInUse blocks deleting a brand still referenced by products.
ErrBrandInUse = errors.New("brand in use")
// ErrInvalidMerge is returned when a merge target is missing or equal to
// the source.
ErrInvalidMerge = errors.New("invalid merge target")
)
// CreateBrand inserts a new brand. Names are unique by normalized form.
func (s *Store) CreateBrand(ctx context.Context, actor, name string) (*Brand, error) {
name = strings.TrimSpace(name)
if name == "" {
return nil, errors.New("name required")
}
var b Brand
err := s.pool.QueryRow(ctx,
`INSERT INTO brand (name, normalized_name) VALUES ($1, $2) RETURNING id, name, 0`,
name, normBrand(name)).Scan(&b.ID, &b.Name, &b.ProductCount)
if isUniqueViolation(err) {
return nil, ErrDuplicateBrand
}
if err != nil {
return nil, err
}
_ = s.writeAudit(ctx, actor, "create", "brand", &b.ID, []string{"name"}, nil, b)
return &b, nil
}
// UpdateBrand renames a brand, keeping the normalized name in sync.
func (s *Store) UpdateBrand(ctx context.Context, id, actor, name string) (*Brand, error) {
name = strings.TrimSpace(name)
if name == "" {
return nil, errors.New("name required")
}
ct, err := s.pool.Exec(ctx,
"UPDATE brand SET name = $1, normalized_name = $2 WHERE id = $3",
name, normBrand(name), id)
if isUniqueViolation(err) {
return nil, ErrDuplicateBrand
}
if err != nil {
return nil, err
}
if ct.RowsAffected() == 0 {
return nil, ErrNotFound
}
out, err := s.getBrand(ctx, id)
if err != nil {
return nil, err
}
_ = s.writeAudit(ctx, actor, "update", "brand", &id, []string{"name"}, nil, out)
return out, nil
}
// MergeBrands reassigns every product of src to dst, then deletes src. Useful
// for collapsing duplicate brands (e.g. "可口可乐" and "Coca-Cola").
func (s *Store) MergeBrands(ctx context.Context, srcID, dstID, actor string) (*Brand, error) {
if srcID == dstID {
return nil, ErrInvalidMerge
}
tx, err := s.pool.Begin(ctx)
if err != nil {
return nil, err
}
defer tx.Rollback(ctx)
var dstName string
err = tx.QueryRow(ctx, "SELECT name FROM brand WHERE id = $1", dstID).Scan(&dstName)
if errors.Is(err, pgx.ErrNoRows) {
return nil, ErrInvalidMerge
}
if err != nil {
return nil, err
}
var srcName string
err = tx.QueryRow(ctx, "SELECT name FROM brand WHERE id = $1", srcID).Scan(&srcName)
if errors.Is(err, pgx.ErrNoRows) {
return nil, ErrNotFound
}
if err != nil {
return nil, err
}
if _, err := tx.Exec(ctx, "UPDATE product SET brand_id = $1 WHERE brand_id = $2", dstID, srcID); err != nil {
return nil, err
}
if _, err := tx.Exec(ctx, "DELETE FROM brand WHERE id = $1", srcID); err != nil {
return nil, err
}
if err := tx.Commit(ctx); err != nil {
return nil, err
}
out, err := s.getBrand(ctx, dstID)
if err != nil {
return nil, err
}
_ = s.writeAudit(ctx, actor, "merge", "brand", &srcID, []string{"name"},
map[string]string{"name": srcName},
map[string]string{"merged_into": dstName, "merged_into_id": dstID})
return out, nil
}
// DeleteBrand removes a brand not referenced by any product.
func (s *Store) DeleteBrand(ctx context.Context, id, actor string) error {
before, err := s.getBrand(ctx, id)
if err != nil {
return err
}
if before.ProductCount > 0 {
return fmt.Errorf("%w: %d products", ErrBrandInUse, before.ProductCount)
}
ct, err := s.pool.Exec(ctx, "DELETE FROM brand WHERE id = $1", id)
if err != nil {
if isForeignKeyViolation(err) {
return ErrBrandInUse
}
return err
}
if ct.RowsAffected() == 0 {
return ErrNotFound
}
_ = s.writeAudit(ctx, actor, "delete", "brand", &id, []string{"name"}, before, nil)
return nil
}
func (s *Store) getBrand(ctx context.Context, id string) (*Brand, error) {
var b Brand
err := s.pool.QueryRow(ctx, `
SELECT b.id, b.name,
(SELECT count(*) FROM product p WHERE p.brand_id = b.id)
FROM brand b WHERE b.id = $1`, id).Scan(&b.ID, &b.Name, &b.ProductCount)
if errors.Is(err, pgx.ErrNoRows) {
return nil, ErrNotFound
}
if err != nil {
return nil, err
}
return &b, nil
}
+78
View File
@@ -0,0 +1,78 @@
package adminstore
import (
"context"
"errors"
"testing"
)
func TestBrandLifecycle(t *testing.T) {
s := newTestStore(t)
ctx := context.Background()
a, err := s.CreateBrand(ctx, "tester", "品牌A "+randomHex(4))
if err != nil {
t.Fatalf("create A: %v", err)
}
t.Cleanup(func() { _, _ = s.pool.Exec(ctx, "DELETE FROM brand WHERE id = $1", a.ID) })
b, err := s.CreateBrand(ctx, "tester", "品牌B "+randomHex(4))
if err != nil {
t.Fatalf("create B: %v", err)
}
t.Cleanup(func() { _, _ = s.pool.Exec(ctx, "DELETE FROM brand WHERE id = $1", b.ID) })
// Duplicate (normalized) name must be rejected.
if _, err := s.CreateBrand(ctx, "tester", " "+a.Name+" "); !errors.Is(err, ErrDuplicateBrand) {
t.Fatalf("expected ErrDuplicateBrand, got %v", err)
}
// Rename.
renamed, err := s.UpdateBrand(ctx, a.ID, "tester", "品牌A改名")
if err != nil {
t.Fatalf("rename: %v", err)
}
if renamed.Name != "品牌A改名" {
t.Fatalf("rename not applied: %q", renamed.Name)
}
// Attach a product to brand A so deletion is blocked and merge moves it.
var prodID string
err = s.pool.QueryRow(ctx,
"INSERT INTO product (name, brand_id, status) VALUES ($1,$2,'active') RETURNING id",
"测试商品 "+randomHex(4), a.ID).Scan(&prodID)
if err != nil {
t.Fatalf("insert product: %v", err)
}
t.Cleanup(func() { _, _ = s.pool.Exec(ctx, "DELETE FROM product WHERE id = $1", prodID) })
// Deleting an in-use brand must fail.
if err := s.DeleteBrand(ctx, a.ID, "tester"); !errors.Is(err, ErrBrandInUse) {
t.Fatalf("expected ErrBrandInUse, got %v", err)
}
// Merge A into B: product reassigned, A deleted.
merged, err := s.MergeBrands(ctx, a.ID, b.ID, "tester")
if err != nil {
t.Fatalf("merge: %v", err)
}
if merged.ID != b.ID || merged.ProductCount < 1 {
t.Fatalf("merge result wrong: %+v", merged)
}
if _, err := s.getBrand(ctx, a.ID); !errors.Is(err, ErrNotFound) {
t.Fatalf("source brand should be gone, got %v", err)
}
// Self-merge is invalid.
if _, err := s.MergeBrands(ctx, b.ID, b.ID, "tester"); !errors.Is(err, ErrInvalidMerge) {
t.Fatalf("expected ErrInvalidMerge, got %v", err)
}
// After moving the product away from B, B can be deleted.
if _, err := s.pool.Exec(ctx, "DELETE FROM product WHERE id = $1", prodID); err != nil {
t.Fatalf("cleanup product: %v", err)
}
if err := s.DeleteBrand(ctx, b.ID, "tester"); err != nil {
t.Fatalf("delete unused brand: %v", err)
}
}
+80
View File
@@ -0,0 +1,80 @@
package adminstore
import (
"context"
"errors"
)
// Bulk-operation errors.
var (
// ErrNoTargets is returned when a bulk request selects no products.
ErrNoTargets = errors.New("no products selected")
// ErrInvalidStatus is returned for an unknown product status value.
ErrInvalidStatus = errors.New("invalid status")
)
var validStatus = map[string]bool{"active": true, "merged": true, "deprecated": true}
// BulkSetStatus updates the status of every selected product in one statement.
func (s *Store) BulkSetStatus(ctx context.Context, actor string, ids []string, status string) (int, error) {
if len(ids) == 0 {
return 0, ErrNoTargets
}
if !validStatus[status] {
return 0, ErrInvalidStatus
}
ct, err := s.pool.Exec(ctx,
"UPDATE product SET status = $1 WHERE id = ANY($2)", status, ids)
if err != nil {
return 0, err
}
n := int(ct.RowsAffected())
_ = s.writeAudit(ctx, actor, "bulk_status", "product", nil,
[]string{"status"}, map[string]any{"ids": ids}, map[string]any{"status": status})
return n, nil
}
// BulkSetCategory reassigns the category of every selected product, syncing the
// GPC brick code and recomputing quality for each one.
func (s *Store) BulkSetCategory(ctx context.Context, actor string, ids []string, categoryID *string) (int, error) {
if len(ids) == 0 {
return 0, ErrNoTargets
}
tx, err := s.pool.Begin(ctx)
if err != nil {
return 0, err
}
defer tx.Rollback(ctx)
var gpc *string
if categoryID != nil && *categoryID != "" {
if err := tx.QueryRow(ctx, "SELECT gpc_brick_code FROM category WHERE id = $1", *categoryID).Scan(&gpc); err != nil {
return 0, ErrInvalidParent
}
} else {
categoryID = nil
}
ct, err := tx.Exec(ctx,
"UPDATE product SET category_id = $1, gpc_brick_code = $2 WHERE id = ANY($3)",
categoryID, gpc, ids)
if err != nil {
return 0, err
}
for _, id := range ids {
if _, err := s.recomputeQualityTx(ctx, tx, id); err != nil {
return 0, err
}
}
if err := tx.Commit(ctx); err != nil {
return 0, err
}
n := int(ct.RowsAffected())
cat := ""
if categoryID != nil {
cat = *categoryID
}
_ = s.writeAudit(ctx, actor, "bulk_category", "product", nil,
[]string{"category"}, map[string]any{"ids": ids}, map[string]any{"category_id": cat})
return n, nil
}
+298
View File
@@ -0,0 +1,298 @@
package adminstore
import (
"context"
"crypto/rand"
"encoding/hex"
"errors"
"fmt"
"regexp"
"strings"
"github.com/jackc/pgx/v5"
"github.com/jackc/pgx/v5/pgconn"
)
// randomHex returns n random lowercase hex characters for fallback ltree slugs.
func randomHex(n int) string {
b := make([]byte, (n+1)/2)
if _, err := rand.Read(b); err != nil {
return "x"
}
return hex.EncodeToString(b)[:n]
}
// Category-management errors, mapped to client statuses by the handler.
var (
// ErrDuplicatePath is returned when a category path already exists.
ErrDuplicatePath = errors.New("duplicate category path")
// ErrCategoryHasChildren blocks deleting a node that still has children.
ErrCategoryHasChildren = errors.New("category has children")
// ErrCategoryInUse blocks deleting a node still referenced by products.
ErrCategoryInUse = errors.New("category in use")
// ErrInvalidParent is returned for a missing parent or an illegal move
// (onto itself or one of its own descendants).
ErrInvalidParent = errors.New("invalid parent category")
)
// CategoryInput is the payload accepted when creating or editing a category.
// Slug is the ltree label (ASCII); when empty it is derived from NameEN, then
// from a random suffix, since ltree labels cannot contain CJK or spaces.
type CategoryInput struct {
NameZH string `json:"name_zh"`
NameEN *string `json:"name_en"`
Slug *string `json:"slug"`
ParentID *string `json:"parent_id"`
GPCBrickCode *string `json:"gpc_brick_code"`
}
var slugInvalid = regexp.MustCompile(`[^a-z0-9_]+`)
// slugify converts a string into a valid ltree label ([a-z0-9_]).
func slugify(s string) string {
s = strings.ToLower(strings.TrimSpace(s))
s = slugInvalid.ReplaceAllString(s, "_")
s = strings.Trim(s, "_")
for strings.Contains(s, "__") {
s = strings.ReplaceAll(s, "__", "_")
}
return s
}
// resolveSlug picks an ltree label from the explicit slug, then NameEN, then a
// random fallback so a Chinese-only category still gets a valid path label.
func resolveSlug(in CategoryInput) string {
if in.Slug != nil {
if s := slugify(*in.Slug); s != "" {
return s
}
}
if in.NameEN != nil {
if s := slugify(*in.NameEN); s != "" {
return s
}
}
return "cat_" + randomHex(6)
}
func trimPtr(p *string) *string {
if p == nil {
return nil
}
t := strings.TrimSpace(*p)
if t == "" {
return nil
}
return &t
}
// CreateCategory inserts a new category node. With no parent it becomes a root
// (level 0); otherwise its path is parentPath.slug and level is parentLevel+1.
func (s *Store) CreateCategory(ctx context.Context, actor string, in CategoryInput) (*Category, error) {
name := strings.TrimSpace(in.NameZH)
if name == "" {
return nil, errors.New("name_zh required")
}
parentPath := ""
parentLevel := -1
kind := DefaultKind
var parentID *string
if pid := trimPtr(in.ParentID); pid != nil {
var path string
var level int
err := s.pool.QueryRow(ctx, "SELECT path::text, level, archive_kind FROM category WHERE id = $1", *pid).Scan(&path, &level, &kind)
if errors.Is(err, pgx.ErrNoRows) {
return nil, ErrInvalidParent
}
if err != nil {
return nil, err
}
parentPath, parentLevel, parentID = path, level, pid
}
slug := resolveSlug(in)
path := slug
if parentPath != "" {
path = parentPath + "." + slug
}
level := parentLevel + 1
var c Category
err := s.pool.QueryRow(ctx, `
INSERT INTO category (name_zh, name_en, parent_id, path, gpc_brick_code, level, archive_kind)
VALUES ($1, $2, $3, $4::ltree, $5, $6, $7)
RETURNING id, name_zh, name_en, path::text, level, parent_id::text, gpc_brick_code, archive_kind, 0`,
name, trimPtr(in.NameEN), parentID, path, trimPtr(in.GPCBrickCode), level, kind).
Scan(&c.ID, &c.NameZH, &c.NameEN, &c.Path, &c.Level, &c.ParentID, &c.GPCBrickCode, &c.ArchiveKind, &c.ProductCount)
if isUniqueViolation(err) {
return nil, ErrDuplicatePath
}
if err != nil {
return nil, err
}
_ = s.writeAudit(ctx, actor, "create", "category", &c.ID, []string{"name_zh", "path"}, nil, c)
return &c, nil
}
// UpdateCategory renames a node and/or moves it under a new parent. Moving
// rewrites the path of the node and every descendant via ltree, keeping level
// in sync. Moving a node onto itself or a descendant is rejected.
func (s *Store) UpdateCategory(ctx context.Context, id, actor string, in CategoryInput) (*Category, error) {
name := strings.TrimSpace(in.NameZH)
if name == "" {
return nil, errors.New("name_zh required")
}
tx, err := s.pool.Begin(ctx)
if err != nil {
return nil, err
}
defer tx.Rollback(ctx)
var oldPath string
var oldLevel int
var oldParent *string
err = tx.QueryRow(ctx, "SELECT path::text, level, parent_id::text FROM category WHERE id = $1", id).
Scan(&oldPath, &oldLevel, &oldParent)
if errors.Is(err, pgx.ErrNoRows) {
return nil, ErrNotFound
}
if err != nil {
return nil, err
}
if _, err := tx.Exec(ctx,
"UPDATE category SET name_zh = $1, name_en = $2, gpc_brick_code = $3 WHERE id = $4",
name, trimPtr(in.NameEN), trimPtr(in.GPCBrickCode), id); err != nil {
return nil, err
}
newParent := trimPtr(in.ParentID)
if !strEq(newParent, oldParent) {
if err := s.moveCategoryTx(ctx, tx, id, oldPath, newParent); err != nil {
return nil, err
}
}
if err := tx.Commit(ctx); err != nil {
return nil, err
}
out, err := s.getCategory(ctx, id)
if err != nil {
return nil, err
}
_ = s.writeAudit(ctx, actor, "update", "category", &id, []string{"name_zh", "name_en", "gpc_brick_code", "parent_id"}, nil, out)
return out, nil
}
// moveCategoryTx re-parents a subtree. The node's slug (last path label) is
// preserved; only its ancestor prefix and level change.
func (s *Store) moveCategoryTx(ctx context.Context, tx pgx.Tx, id, oldPath string, newParent *string) error {
slug := oldPath
if i := strings.LastIndex(oldPath, "."); i >= 0 {
slug = oldPath[i+1:]
}
newBase := slug
if newParent != nil {
var parentPath string
err := tx.QueryRow(ctx, "SELECT path::text FROM category WHERE id = $1", *newParent).Scan(&parentPath)
if errors.Is(err, pgx.ErrNoRows) {
return ErrInvalidParent
}
if err != nil {
return err
}
// Disallow moving a node under itself or one of its descendants.
if parentPath == oldPath || strings.HasPrefix(parentPath, oldPath+".") {
return ErrInvalidParent
}
newBase = parentPath + "." + slug
}
// Rewrite the node and all descendants in one statement; level tracks depth.
_, err := tx.Exec(ctx, `
UPDATE category
SET path = ($1::ltree || subpath(path, nlevel($2::ltree) - 1)),
level = nlevel($1::ltree) + (nlevel(path) - nlevel($2::ltree)) - 1
WHERE path = $2::ltree OR path <@ $2::ltree`, newBase, oldPath)
if isUniqueViolation(err) {
return ErrDuplicatePath
}
if err != nil {
return err
}
_, err = tx.Exec(ctx, "UPDATE category SET parent_id = $1 WHERE id = $2", newParent, id)
return err
}
// DeleteCategory removes a leaf node not referenced by any product. Nodes with
// children or in-use nodes are rejected with a specific error.
func (s *Store) DeleteCategory(ctx context.Context, id, actor string) error {
before, err := s.getCategory(ctx, id)
if err != nil {
return err
}
var children int
if err := s.pool.QueryRow(ctx, "SELECT count(*) FROM category WHERE parent_id = $1", id).Scan(&children); err != nil {
return err
}
if children > 0 {
return ErrCategoryHasChildren
}
var products int
if err := s.pool.QueryRow(ctx, "SELECT count(*) FROM product WHERE category_id = $1", id).Scan(&products); err != nil {
return err
}
if products > 0 {
return fmt.Errorf("%w: %d products", ErrCategoryInUse, products)
}
ct, err := s.pool.Exec(ctx, "DELETE FROM category WHERE id = $1", id)
if err != nil {
// A concurrent product assignment can still trip the FK.
if isForeignKeyViolation(err) {
return ErrCategoryInUse
}
return err
}
if ct.RowsAffected() == 0 {
return ErrNotFound
}
_ = s.writeAudit(ctx, actor, "delete", "category", &id, []string{"path"}, before, nil)
return nil
}
func (s *Store) getCategory(ctx context.Context, id string) (*Category, error) {
var c Category
err := s.pool.QueryRow(ctx, `
SELECT c.id, c.name_zh, c.name_en, c.path::text, c.level, c.parent_id::text,
c.gpc_brick_code, c.archive_kind,
(SELECT count(*) FROM product p WHERE p.category_id = c.id)
FROM category c WHERE c.id = $1`, id).
Scan(&c.ID, &c.NameZH, &c.NameEN, &c.Path, &c.Level, &c.ParentID, &c.GPCBrickCode, &c.ArchiveKind, &c.ProductCount)
if errors.Is(err, pgx.ErrNoRows) {
return nil, ErrNotFound
}
if err != nil {
return nil, err
}
return &c, nil
}
func isUniqueViolation(err error) bool {
var pgErr *pgconn.PgError
return errors.As(err, &pgErr) && pgErr.Code == "23505"
}
func isForeignKeyViolation(err error) bool {
var pgErr *pgconn.PgError
return errors.As(err, &pgErr) && pgErr.Code == "23503"
}
+130
View File
@@ -0,0 +1,130 @@
package adminstore
import (
"context"
"errors"
"os"
"testing"
"time"
"github.com/jackc/pgx/v5/pgxpool"
)
// newTestStore connects to the test database, skipping when it is unreachable
// or migrations have not been applied.
func newTestStore(t *testing.T) *Store {
t.Helper()
dsn := os.Getenv("OPENGOODS_DATABASE_URL")
if dsn == "" {
dsn = "postgres://opengoods:opengoods@localhost:5432/opengoods?sslmode=disable"
}
ctx, cancel := context.WithTimeout(context.Background(), 3*time.Second)
defer cancel()
pool, err := pgxpool.New(ctx, dsn)
if err != nil {
t.Skipf("no database: %v", err)
}
if err := pool.Ping(ctx); err != nil {
pool.Close()
t.Skipf("database not reachable: %v", err)
}
var hasTable bool
if err := pool.QueryRow(ctx, "SELECT to_regclass('public.category') IS NOT NULL").Scan(&hasTable); err != nil || !hasTable {
pool.Close()
t.Skip("migrations not applied (category missing)")
}
t.Cleanup(pool.Close)
return New(pool)
}
func ptr(s string) *string { return &s }
func TestCategoryLifecycle(t *testing.T) {
s := newTestStore(t)
ctx := context.Background()
root, err := s.CreateCategory(ctx, "tester", CategoryInput{
NameZH: "测试根", Slug: ptr("test_root_" + randomHex(6)),
})
if err != nil {
t.Fatalf("create root: %v", err)
}
t.Cleanup(func() { _, _ = s.pool.Exec(ctx, "DELETE FROM category WHERE path <@ $1::ltree", root.Path) })
if root.Level != 0 || root.ParentID != nil {
t.Fatalf("root level/parent wrong: level=%d parent=%v", root.Level, root.ParentID)
}
child, err := s.CreateCategory(ctx, "tester", CategoryInput{
NameZH: "测试子", NameEN: ptr("Test Child"), ParentID: &root.ID,
})
if err != nil {
t.Fatalf("create child: %v", err)
}
if child.Level != 1 || child.ParentID == nil || *child.ParentID != root.ID {
t.Fatalf("child hierarchy wrong: %+v", child)
}
// Deleting a node with children must fail.
if err := s.DeleteCategory(ctx, root.ID, "tester"); !errors.Is(err, ErrCategoryHasChildren) {
t.Fatalf("expected ErrCategoryHasChildren, got %v", err)
}
// Rename child.
renamed, err := s.UpdateCategory(ctx, child.ID, "tester", CategoryInput{NameZH: "测试子-改名"})
if err != nil {
t.Fatalf("rename: %v", err)
}
if renamed.NameZH != "测试子-改名" {
t.Fatalf("rename not applied: %q", renamed.NameZH)
}
// Move child to a second root, descendants' path/level should follow.
root2, err := s.CreateCategory(ctx, "tester", CategoryInput{
NameZH: "测试根2", Slug: ptr("test_root2_" + randomHex(6)),
})
if err != nil {
t.Fatalf("create root2: %v", err)
}
t.Cleanup(func() { _, _ = s.pool.Exec(ctx, "DELETE FROM category WHERE path <@ $1::ltree", root2.Path) })
moved, err := s.UpdateCategory(ctx, child.ID, "tester", CategoryInput{NameZH: "测试子-改名", ParentID: &root2.ID})
if err != nil {
t.Fatalf("move: %v", err)
}
if moved.ParentID == nil || *moved.ParentID != root2.ID {
t.Fatalf("move parent wrong: %+v", moved)
}
if moved.Level != 1 {
t.Fatalf("moved level wrong: %d", moved.Level)
}
// Moving a node under itself must be rejected.
if _, err := s.UpdateCategory(ctx, root2.ID, "tester", CategoryInput{NameZH: "测试根2", ParentID: &child.ID}); !errors.Is(err, ErrInvalidParent) {
t.Fatalf("expected ErrInvalidParent for self-move, got %v", err)
}
// Duplicate path on create must be rejected.
if _, err := s.CreateCategory(ctx, "tester", CategoryInput{NameZH: "dup", Slug: ptr(root.Path)}); !errors.Is(err, ErrDuplicatePath) {
t.Fatalf("expected ErrDuplicatePath, got %v", err)
}
// Now the leaf can be deleted.
if err := s.DeleteCategory(ctx, child.ID, "tester"); err != nil {
t.Fatalf("delete leaf: %v", err)
}
}
func TestSlugify(t *testing.T) {
cases := map[string]string{
"Cooking Oil": "cooking_oil",
" Hello--Wld": "hello_wld",
"食品": "",
"a__b": "a_b",
}
for in, want := range cases {
if got := slugify(in); got != want {
t.Errorf("slugify(%q) = %q, want %q", in, got, want)
}
}
}
+217
View File
@@ -0,0 +1,217 @@
package adminstore
import (
"context"
"encoding/json"
"strings"
"github.com/jackc/pgx/v5"
)
type ByposRecord struct {
Barcode string `json:"barcode"`
Name string `json:"name"`
Spec string `json:"spec"`
Unit string `json:"unit"`
Area string `json:"area"`
Manufacturer string `json:"manufacturer"`
License string `json:"license"`
InPrice string `json:"in_price"`
SellPrice string `json:"sell_price"`
Status string `json:"status"`
RetMsg string `json:"retmsg"`
FetchedAt string `json:"fetched_at"`
Source string `json:"source"`
}
type ImportByposResult struct {
Loaded int `json:"loaded"`
Skipped int `json:"skipped"`
Errored int `json:"errored"`
}
const byposSourceName = "bypos\u4e2d\u5fc3\u5e93"
const byposSourceURL = "https://zc.bypos.net"
func (s *Store) ensureByposSource(ctx context.Context, tx pgx.Tx) (string, error) {
var id string
err := tx.QueryRow(ctx, `
INSERT INTO source (name, homepage, license, trust_weight)
VALUES ($1, $2, 'proprietary', 0.6)
ON CONFLICT (name) DO UPDATE SET homepage = EXCLUDED.homepage
RETURNING id`, byposSourceName, byposSourceURL).Scan(&id)
return id, err
}
func (s *Store) ensureManufacturer(ctx context.Context, tx pgx.Tx, name string, country *string) (string, error) {
norm := strings.Join(strings.Fields(strings.ToLower(name)), " ")
var id string
err := tx.QueryRow(ctx, `
INSERT INTO manufacturer (name, normalized_name, country)
VALUES ($1, $2, $3)
ON CONFLICT (normalized_name) DO UPDATE SET name = manufacturer.name
RETURNING id`, name, norm, country).Scan(&id)
return id, err
}
func (s *Store) importByposRecord(ctx context.Context, tx pgx.Tx, rec ByposRecord, sourceID string) error {
if rec.Status != "hit" || strings.TrimSpace(rec.Name) == "" {
return nil
}
barcode := strings.TrimSpace(rec.Barcode)
name := strings.TrimSpace(rec.Name)
var country *string
if strings.HasPrefix(barcode, "69") {
c := "\u4e2d\u56fd"
country = &c
}
var manufacturerID *string
if mfr := strings.TrimSpace(rec.Manufacturer); mfr != "" {
mid, err := s.ensureManufacturer(ctx, tx, mfr, country)
if err != nil {
return err
}
manufacturerID = &mid
}
attrs := map[string]interface{}{}
if v := strings.TrimSpace(rec.Spec); v != "" {
attrs["spec"] = v
}
if v := strings.TrimSpace(rec.Unit); v != "" {
attrs["pack_unit"] = v
}
if v := strings.TrimSpace(rec.Area); v != "" {
attrs["origin_area"] = v
}
if v := strings.TrimSpace(rec.License); v != "" {
attrs["production_license"] = v
}
attrsJSON, _ := json.Marshal(attrs)
var productID string
if barcode != "" {
err := tx.QueryRow(ctx, `
INSERT INTO product (gtin, name, manufacturer_id, country_of_origin, attributes, status)
VALUES ($1, $2, $3, $4, $5, 'active')
ON CONFLICT (gtin) WHERE gtin IS NOT NULL DO UPDATE SET
name = EXCLUDED.name,
manufacturer_id = COALESCE(EXCLUDED.manufacturer_id, product.manufacturer_id),
country_of_origin = COALESCE(EXCLUDED.country_of_origin, product.country_of_origin),
attributes = product.attributes || EXCLUDED.attributes
RETURNING id`, barcode, name, manufacturerID, country, attrsJSON).Scan(&productID)
if err != nil {
return err
}
} else {
err := tx.QueryRow(ctx, `
INSERT INTO product (name, manufacturer_id, country_of_origin, attributes, status)
VALUES ($1, $2, $3, $4, 'active')
RETURNING id`, name, manufacturerID, country, attrsJSON).Scan(&productID)
if err != nil {
return err
}
}
_, _ = tx.Exec(ctx, "DELETE FROM product_msrp WHERE product_id = $1 AND source_id = $2", productID, sourceID)
if sp := strings.TrimSpace(rec.SellPrice); sp != "" && sp != "0" {
_, _ = tx.Exec(ctx, `
INSERT INTO product_msrp (product_id, amount, currency, region, source_id, source_url)
VALUES ($1, $2::numeric, 'CNY', 'CN', $3, $4)`, productID, sp, sourceID, byposSourceURL)
}
_, _ = tx.Exec(ctx, "DELETE FROM product_source WHERE product_id = $1 AND source_id = $2", productID, sourceID)
fields := []string{"name", "country_of_origin"}
if manufacturerID != nil {
fields = append(fields, "manufacturer")
}
if len(attrs) > 0 {
fields = append(fields, "attributes")
}
if barcode != "" {
fields = append(fields, "gtin")
}
rawJSON, _ := json.Marshal(rec)
fetchedAt := strings.TrimSpace(rec.FetchedAt)
if fetchedAt == "" {
fetchedAt = ""
}
if fetchedAt != "" {
_, err := tx.Exec(ctx, `
INSERT INTO product_source (product_id, source_id, url, fields, fetched_at, raw)
VALUES ($1, $2, $3, $4, $5::timestamptz, $6)`,
productID, sourceID, byposSourceURL, fields, fetchedAt, rawJSON)
if err != nil {
return err
}
} else {
_, err := tx.Exec(ctx, `
INSERT INTO product_source (product_id, source_id, url, fields, fetched_at, raw)
VALUES ($1, $2, $3, $4, now(), $5)`,
productID, sourceID, byposSourceURL, fields, rawJSON)
if err != nil {
return err
}
}
if barcode != "" {
gtinType := "EAN13"
switch len(barcode) {
case 8:
gtinType = "EAN8"
case 12:
gtinType = "UPC"
case 14:
gtinType = "GTIN14"
}
_, _ = tx.Exec(ctx, `
INSERT INTO product_barcode (product_id, gtin, gtin_type, pack_level, is_primary, source_id)
VALUES ($1, $2, $3, 'each', true, $4)
ON CONFLICT (gtin) DO NOTHING`, productID, barcode, gtinType, sourceID)
}
_, err := s.recomputeQualityTx(ctx, tx, productID)
return err
}
func (s *Store) ImportByposRecords(ctx context.Context, records []ByposRecord) (*ImportByposResult, error) {
result := &ImportByposResult{}
tx, err := s.pool.Begin(ctx)
if err != nil {
return nil, err
}
defer tx.Rollback(ctx)
sourceID, err := s.ensureByposSource(ctx, tx)
if err != nil {
return nil, err
}
for _, rec := range records {
if rec.Status != "hit" || strings.TrimSpace(rec.Name) == "" {
result.Skipped++
continue
}
sp, spErr := tx.Begin(ctx)
if spErr != nil {
result.Errored++
continue
}
if err := s.importByposRecord(ctx, sp, rec, sourceID); err != nil {
sp.Rollback(ctx)
result.Errored++
} else {
sp.Commit(ctx)
result.Loaded++
}
}
if err := tx.Commit(ctx); err != nil {
return nil, err
}
return result, nil
}
+101
View File
@@ -0,0 +1,101 @@
package adminstore
import "context"
// DefaultKind is used for products whose category has no archive kind (or no
// category at all).
const DefaultKind = "generic"
// FoodKind keeps the mature, dedicated food_detail path; every other kind is
// driven generically by kind_field + product.attributes.
const FoodKind = "food"
// genericBaseFields are the core completeness fields for any non-food kind.
// Food keeps its own richer CompletenessFields list.
var genericBaseFields = []string{"name", "gtin", "brand", "category", "image"}
// KindField describes one editable spec field for an archive kind. It drives
// both the dynamic admin form and the kind-aware completeness computation.
type KindField struct {
Kind string `json:"kind"`
FieldKey string `json:"field_key"`
GroupLabel string `json:"group_label"`
LabelZH string `json:"label_zh"`
FieldType string `json:"field_type"`
Unit *string `json:"unit"`
Options []string `json:"options"`
Placeholder *string `json:"placeholder"`
SortOrder int `json:"sort_order"`
Qualified bool `json:"qualified"`
}
// ListKindFields returns the ordered field template for one archive kind.
func (s *Store) ListKindFields(ctx context.Context, kind string) ([]KindField, error) {
rows, err := s.pool.Query(ctx, `
SELECT kind, field_key, group_label, label_zh, field_type, unit, options,
placeholder, sort_order, qualified
FROM kind_field WHERE kind = $1 ORDER BY sort_order, field_key`, kind)
if err != nil {
return nil, err
}
defer rows.Close()
out := []KindField{}
for rows.Next() {
var f KindField
if err := rows.Scan(&f.Kind, &f.FieldKey, &f.GroupLabel, &f.LabelZH,
&f.FieldType, &f.Unit, &f.Options, &f.Placeholder, &f.SortOrder,
&f.Qualified); err != nil {
return nil, err
}
out = append(out, f)
}
return out, rows.Err()
}
// kindQualifiedKeys returns, per kind, the attribute keys that count toward the
// completeness/qualified score. Loaded in one query so list views stay cheap.
func (s *Store) kindQualifiedKeys(ctx context.Context, q queryer) (map[string][]string, error) {
rows, err := s.pool.Query(ctx,
"SELECT kind, field_key FROM kind_field WHERE qualified ORDER BY sort_order, field_key")
if err != nil {
return nil, err
}
defer rows.Close()
m := map[string][]string{}
for rows.Next() {
var kind, key string
if err := rows.Scan(&kind, &key); err != nil {
return nil, err
}
m[kind] = append(m[kind], key)
}
return m, rows.Err()
}
// completenessKeys returns the ordered list of field keys that define a full
// archive for the given kind.
func completenessKeys(kind string, qualifiedAttrKeys []string) []string {
if kind == FoodKind {
return CompletenessFields
}
keys := make([]string, 0, len(genericBaseFields)+len(qualifiedAttrKeys))
keys = append(keys, genericBaseFields...)
keys = append(keys, qualifiedAttrKeys...)
return keys
}
// attrPresent reports whether an attribute value is meaningfully filled in.
func attrPresent(attrs map[string]any, key string) bool {
v, ok := attrs[key]
if !ok || v == nil {
return false
}
switch t := v.(type) {
case string:
return t != ""
case []any:
return len(t) > 0
default:
return true
}
}
+119
View File
@@ -0,0 +1,119 @@
package adminstore
import (
"context"
"testing"
)
// contains reports whether s holds v.
func contains(s []string, v string) bool {
for _, x := range s {
if x == v {
return true
}
}
return false
}
// electronicsCategoryID returns the seeded electronics.phone category id,
// skipping the test when the archive-kind migration is not applied.
func electronicsCategoryID(t *testing.T, s *Store) string {
t.Helper()
ctx := context.Background()
var hasTable bool
if err := s.pool.QueryRow(ctx, "SELECT to_regclass('public.kind_field') IS NOT NULL").Scan(&hasTable); err != nil || !hasTable {
t.Skip("archive-kind migration not applied (kind_field missing)")
}
var id string
err := s.pool.QueryRow(ctx, "SELECT id FROM category WHERE path = 'electronics.phone'::ltree").Scan(&id)
if err != nil {
t.Skipf("electronics.phone category not seeded: %v", err)
}
return id
}
func TestListKindFieldsElectronics(t *testing.T) {
s := newTestStore(t)
electronicsCategoryID(t, s) // ensures migration applied
fields, err := s.ListKindFields(context.Background(), "electronics")
if err != nil {
t.Fatalf("list kind fields: %v", err)
}
if len(fields) == 0 {
t.Fatal("expected seeded electronics fields, got none")
}
var sawQualified bool
for _, f := range fields {
if f.FieldKey == "model_number" && f.Qualified {
sawQualified = true
}
}
if !sawQualified {
t.Fatal("expected model_number to be a qualified field")
}
}
// TestElectronicsArchiveKind verifies a non-food product uses the electronics
// completeness rules: food fields (nutriments/ingredients) are not required,
// and qualified spec fields drive both "missing" and the quality score.
func TestElectronicsArchiveKind(t *testing.T) {
s := newTestStore(t)
ctx := context.Background()
catID := electronicsCategoryID(t, s)
created, err := s.CreateProduct(ctx, "tester", ProductInput{
Name: "测试手机 " + randomHex(6),
CategoryID: &catID,
})
if err != nil {
t.Fatalf("create product: %v", err)
}
t.Cleanup(func() { _, _ = s.pool.Exec(ctx, "DELETE FROM product WHERE id = $1", created.ID) })
if created.ArchiveKind != "electronics" {
t.Fatalf("archive_kind = %q, want electronics", created.ArchiveKind)
}
// Food-only completeness fields must not be required for electronics.
if contains(created.Missing, "nutriments") || contains(created.Missing, "ingredients") {
t.Fatalf("electronics product should not require food fields: missing=%v", created.Missing)
}
// Qualified spec fields should appear as missing while empty.
for _, k := range []string{"model_number", "ccc_cert", "screen_size"} {
if !contains(created.Missing, k) {
t.Fatalf("expected %q in missing, got %v", k, created.Missing)
}
}
scoreBefore := created.QualityScore
gtin := "69" + randomHex(11)
brand := "TestPhoneCo"
updated, err := s.UpdateProduct(ctx, created.ID, "tester", ProductInput{
Name: created.Name,
GTIN: &gtin,
BrandName: &brand,
CategoryID: &catID,
Status: "active",
Attributes: map[string]any{
"model_number": "X-100",
"ccc_cert": "2024010101234567",
"screen_size": "6.1",
"color": "黑色",
},
})
if err != nil {
t.Fatalf("update product: %v", err)
}
if got := updated.Attributes["model_number"]; got != "X-100" {
t.Fatalf("attributes not persisted: %v", updated.Attributes)
}
for _, k := range []string{"model_number", "ccc_cert", "screen_size"} {
if contains(updated.Missing, k) {
t.Fatalf("%q should be filled, still missing: %v", k, updated.Missing)
}
}
if updated.QualityScore <= scoreBefore {
t.Fatalf("quality should rise after filling fields: before=%v after=%v", scoreBefore, updated.QualityScore)
}
}
+159
View File
@@ -0,0 +1,159 @@
package adminstore
import (
"context"
"encoding/json"
"math"
"time"
"github.com/jackc/pgx/v5"
)
// Quality weights mirror ingestion/opengoods/etl/quality.py.
const (
wCompleteness = 0.4
wSourceTrust = 0.3
wAgreement = 0.2
wFreshness = 0.1
)
// queryer is satisfied by both *pgxpool.Pool and pgx.Tx.
type queryer interface {
QueryRow(ctx context.Context, sql string, args ...any) pgx.Row
}
func agreementFromSources(n int) float64 {
switch {
case n <= 1:
return 0.5
case n == 2:
return 0.8
default:
return 1.0
}
}
func freshnessFromAge(ageDays *float64) float64 {
if ageDays == nil {
return 0.5
}
d := *ageDays
switch {
case d <= 30:
return 1.0
case d <= 180:
return 0.8
case d <= 365:
return 0.6
case d <= 730:
return 0.4
default:
return 0.2
}
}
func (s *Store) computeQuality(ctx context.Context, q queryer, productID string) (float64, error) {
var name, country *string
var gtin *string
var brandID, categoryID *string
var netCanonical *float64
var ingredients *string
var hasNutri, hasImage bool
var kind string
var attributes []byte
err := q.QueryRow(ctx, `
SELECT p.name, p.gtin, p.brand_id, p.category_id, p.net_content_canonical,
p.country_of_origin, COALESCE(c.archive_kind, 'generic'), p.attributes,
f.ingredients_text,
(f.nutriments IS NOT NULL AND f.nutriments::text <> '{}'),
EXISTS (SELECT 1 FROM product_image pi WHERE pi.product_id = p.id)
FROM product p
LEFT JOIN category c ON c.id = p.category_id
LEFT JOIN food_detail f ON f.product_id = p.id
WHERE p.id = $1`, productID).Scan(
&name, &gtin, &brandID, &categoryID, &netCanonical, &country,
&kind, &attributes, &ingredients, &hasNutri, &hasImage)
if err != nil {
return 0, err
}
attrs := map[string]any{}
if len(attributes) > 0 {
_ = json.Unmarshal(attributes, &attrs)
}
qualified, err := s.kindQualifiedKeys(ctx, s.pool)
if err != nil {
return 0, err
}
qkeys := qualified[kind]
known := map[string]bool{
"name": name != nil && *name != "",
"gtin": gtin != nil && *gtin != "",
"brand": brandID != nil,
"category": categoryID != nil,
"net_content": netCanonical != nil,
"country_of_origin": country != nil && *country != "",
"nutriments": hasNutri,
"ingredients": ingredients != nil && *ingredients != "",
"image": hasImage,
}
for _, k := range qkeys {
known[k] = attrPresent(attrs, k)
}
keys := completenessKeys(kind, qkeys)
present := 0
for _, k := range keys {
if known[k] {
present++
}
}
completeness := float64(present) / float64(len(keys))
var sourceCount int
var sourceTrust *float64
var lastFetched *time.Time
err = q.QueryRow(ctx, `
SELECT count(DISTINCT ps.source_id), COALESCE(max(s.trust_weight),0), max(ps.fetched_at)
FROM product_source ps LEFT JOIN source s ON s.id = ps.source_id
WHERE ps.product_id = $1`, productID).Scan(&sourceCount, &sourceTrust, &lastFetched)
if err != nil {
return 0, err
}
trust := 0.0
if sourceTrust != nil {
trust = *sourceTrust
}
var ageDays *float64
if lastFetched != nil {
d := time.Since(*lastFetched).Hours() / 24.0
if d < 0 {
d = 0
}
ageDays = &d
}
raw := wCompleteness*completeness + wSourceTrust*trust +
wAgreement*agreementFromSources(sourceCount) + wFreshness*freshnessFromAge(ageDays)
raw = math.Max(0, math.Min(1, raw))
return math.Round(raw*1000) / 1000, nil
}
func (s *Store) recomputeQualityTx(ctx context.Context, tx pgx.Tx, productID string) (float64, error) {
v, err := s.computeQuality(ctx, tx, productID)
if err != nil {
return 0, err
}
_, err = tx.Exec(ctx, "UPDATE product SET quality_score=$1 WHERE id=$2", v, productID)
return v, err
}
func (s *Store) recomputeQuality(ctx context.Context, productID string) (float64, error) {
v, err := s.computeQuality(ctx, s.pool, productID)
if err != nil {
return 0, err
}
_, err = s.pool.Exec(ctx, "UPDATE product SET quality_score=$1 WHERE id=$2", v, productID)
return v, err
}
+30
View File
@@ -0,0 +1,30 @@
package adminstore
import "testing"
func TestAgreementFromSources(t *testing.T) {
cases := map[int]float64{0: 0.5, 1: 0.5, 2: 0.8, 3: 1.0, 9: 1.0}
for n, want := range cases {
if got := agreementFromSources(n); got != want {
t.Errorf("agreementFromSources(%d) = %v, want %v", n, got, want)
}
}
}
func TestFreshnessFromAge(t *testing.T) {
mk := func(d float64) *float64 { return &d }
if got := freshnessFromAge(nil); got != 0.5 {
t.Errorf("nil age = %v, want 0.5", got)
}
cases := []struct {
days float64
want float64
}{
{10, 1.0}, {30, 1.0}, {100, 0.8}, {300, 0.6}, {500, 0.4}, {1000, 0.2},
}
for _, c := range cases {
if got := freshnessFromAge(mk(c.days)); got != c.want {
t.Errorf("freshnessFromAge(%v) = %v, want %v", c.days, got, c.want)
}
}
}
+61
View File
@@ -0,0 +1,61 @@
package adminstore
import "context"
// QualifiedMinScore is the quality_score threshold at or above which a product
// counts as "qualified" (合格) in admin and public stats.
const QualifiedMinScore = 0.6
// AdminStats summarizes the catalog for the admin overview dashboard.
type AdminStats struct {
Products int `json:"products"`
Qualified int `json:"qualified"`
MinScore float64 `json:"min_score"`
ByStatus map[string]int `json:"by_status"`
Brands int `json:"brands"`
Categories int `json:"categories"`
Pending int `json:"pending_submissions"`
AvgQuality float64 `json:"avg_quality"`
}
// Stats gathers the dashboard counters in a handful of aggregate queries.
func (s *Store) Stats(ctx context.Context) (*AdminStats, error) {
out := &AdminStats{MinScore: QualifiedMinScore, ByStatus: map[string]int{}}
if err := s.pool.QueryRow(ctx, `
SELECT count(*),
count(*) FILTER (WHERE quality_score >= $1 AND status = 'active'),
COALESCE(avg(quality_score), 0)
FROM product`, QualifiedMinScore).Scan(&out.Products, &out.Qualified, &out.AvgQuality); err != nil {
return nil, err
}
rows, err := s.pool.Query(ctx, "SELECT status, count(*) FROM product GROUP BY status")
if err != nil {
return nil, err
}
defer rows.Close()
for rows.Next() {
var st string
var n int
if err := rows.Scan(&st, &n); err != nil {
return nil, err
}
out.ByStatus[st] = n
}
if err := rows.Err(); err != nil {
return nil, err
}
if err := s.pool.QueryRow(ctx, "SELECT count(*) FROM brand").Scan(&out.Brands); err != nil {
return nil, err
}
if err := s.pool.QueryRow(ctx, "SELECT count(*) FROM category").Scan(&out.Categories); err != nil {
return nil, err
}
if err := s.pool.QueryRow(ctx,
"SELECT count(*) FROM submission WHERE status = 'pending'").Scan(&out.Pending); err != nil {
return nil, err
}
return out, nil
}
@@ -0,0 +1,71 @@
package adminstore
import (
"context"
"testing"
)
func TestStatsAndBulk(t *testing.T) {
s := newTestStore(t)
ctx := context.Background()
base, err := s.Stats(ctx)
if err != nil {
t.Fatalf("stats: %v", err)
}
var p1, p2 string
if err := s.pool.QueryRow(ctx,
"INSERT INTO product (name, status) VALUES ($1,'active') RETURNING id",
"批量测试1 "+randomHex(4)).Scan(&p1); err != nil {
t.Fatalf("insert p1: %v", err)
}
if err := s.pool.QueryRow(ctx,
"INSERT INTO product (name, status) VALUES ($1,'active') RETURNING id",
"批量测试2 "+randomHex(4)).Scan(&p2); err != nil {
t.Fatalf("insert p2: %v", err)
}
t.Cleanup(func() { _, _ = s.pool.Exec(ctx, "DELETE FROM product WHERE id = ANY($1)", []string{p1, p2}) })
after, err := s.Stats(ctx)
if err != nil {
t.Fatalf("stats after: %v", err)
}
if after.Products != base.Products+2 {
t.Fatalf("product count: got %d want %d", after.Products, base.Products+2)
}
// Bulk set status to deprecated.
n, err := s.BulkSetStatus(ctx, "tester", []string{p1, p2}, "deprecated")
if err != nil || n != 2 {
t.Fatalf("bulk status: n=%d err=%v", n, err)
}
var deprecated int
if err := s.pool.QueryRow(ctx,
"SELECT count(*) FROM product WHERE id = ANY($1) AND status='deprecated'",
[]string{p1, p2}).Scan(&deprecated); err != nil {
t.Fatalf("verify: %v", err)
}
if deprecated != 2 {
t.Fatalf("expected 2 deprecated, got %d", deprecated)
}
// Invalid status rejected.
if _, err := s.BulkSetStatus(ctx, "tester", []string{p1}, "nope"); err != ErrInvalidStatus {
t.Fatalf("expected ErrInvalidStatus, got %v", err)
}
// Empty selection rejected.
if _, err := s.BulkSetStatus(ctx, "tester", nil, "active"); err != ErrNoTargets {
t.Fatalf("expected ErrNoTargets, got %v", err)
}
// Global audit log should contain the bulk_status entry.
rows, total, err := s.ListAllAudit(ctx, 10, 0)
if err != nil {
t.Fatalf("audit: %v", err)
}
if total == 0 || len(rows) == 0 {
t.Fatalf("expected audit rows, got total=%d", total)
}
}
+450
View File
@@ -0,0 +1,450 @@
package adminstore
import (
"context"
"encoding/json"
"errors"
"strconv"
"strings"
"time"
"github.com/jackc/pgx/v5"
)
// ErrConflict is returned when a submission has already been reviewed.
var ErrConflict = errors.New("conflict")
// SubmissionImage is one proposed image URL inside a contribution.
type SubmissionImage struct {
URL string `json:"url"`
Kind string `json:"kind"`
}
// SubmissionInput is the public contribution payload (no login required).
type SubmissionInput struct {
GTIN *string `json:"gtin"`
Name string `json:"name"`
BrandName *string `json:"brand_name"`
CategoryID *string `json:"category_id"`
NetContentValue *float64 `json:"net_content_value"`
NetContentUnit *string `json:"net_content_unit"`
CountryOfOrigin *string `json:"country_of_origin"`
IngredientsText *string `json:"ingredients_text"`
Nutriments map[string]any `json:"nutriments"`
Attributes map[string]any `json:"attributes"`
NutritionBasis *string `json:"nutrition_basis"`
ServingSize *string `json:"serving_size"`
NutriScore *string `json:"nutri_score"`
Images []SubmissionImage `json:"images"`
MSRP []MSRPInput `json:"msrp"`
SubmitterName *string `json:"submitter_name"`
SubmitterContact *string `json:"submitter_contact"`
Note *string `json:"note"`
// Source tags the origin of the submission, stored inside the payload so no
// schema change is needed. Empty means the default public contribution
// ("community"); "backflow" marks records pushed by inventory software.
Source *string `json:"source,omitempty"`
}
// SubmissionRow is a queue-list row for the admin review table.
type SubmissionRow struct {
ID string `json:"id"`
GTIN *string `json:"gtin"`
Name string `json:"name"`
Status string `json:"status"`
SubmitterName *string `json:"submitter_name"`
Source *string `json:"source"`
Matched bool `json:"matched"`
CreatedAt string `json:"created_at"`
ReviewedAt *string `json:"reviewed_at"`
}
// SubmissionDetail is the full review view of one contribution.
type SubmissionDetail struct {
ID string `json:"id"`
Status string `json:"status"`
GTIN *string `json:"gtin"`
Name string `json:"name"`
SubmitterName *string `json:"submitter_name"`
SubmitterContact *string `json:"submitter_contact"`
Note *string `json:"note"`
ReviewNote *string `json:"review_note"`
ReviewedBy *string `json:"reviewed_by"`
ReviewedAt *string `json:"reviewed_at"`
CreatedAt string `json:"created_at"`
TargetProductID *string `json:"target_product_id"`
ResultProductID *string `json:"result_product_id"`
Payload SubmissionInput `json:"payload"`
ExistingProduct *ProductDetail `json:"existing_product,omitempty"`
}
// CreateSubmission validates and stores a public contribution as pending.
func (s *Store) CreateSubmission(ctx context.Context, in SubmissionInput, remoteIP string) (string, error) {
in.Name = strings.TrimSpace(in.Name)
if in.Name == "" {
return "", errors.New("商品名称不能为空")
}
if in.GTIN != nil {
g := strings.TrimSpace(*in.GTIN)
if g == "" {
in.GTIN = nil
} else {
in.GTIN = &g
}
}
// Link to an existing product when the barcode already exists (supplement).
var target *string
if in.GTIN != nil {
var pid string
err := s.pool.QueryRow(ctx, "SELECT id FROM product WHERE gtin = $1", *in.GTIN).Scan(&pid)
if err == nil {
target = &pid
} else if !errors.Is(err, pgx.ErrNoRows) {
return "", err
}
}
payload, err := json.Marshal(in)
if err != nil {
return "", err
}
var id string
err = s.pool.QueryRow(ctx, `
INSERT INTO submission (gtin, name, payload, target_product_id, submitter_name, submitter_contact, note, remote_ip)
VALUES ($1,$2,$3,$4,$5,$6,$7,$8) RETURNING id`,
in.GTIN, in.Name, payload, target, in.SubmitterName, in.SubmitterContact, in.Note, remoteIP).Scan(&id)
return id, err
}
// ListSubmissions returns submissions filtered by status (empty = all).
func (s *Store) ListSubmissions(ctx context.Context, status string, limit, offset int) ([]SubmissionRow, int, error) {
args := []any{}
where := "WHERE 1=1"
if status != "" {
args = append(args, status)
where += " AND status = $1"
}
var total int
if err := s.pool.QueryRow(ctx, "SELECT count(*) FROM submission "+where, args...).Scan(&total); err != nil {
return nil, 0, err
}
args = append(args, limit, offset)
sql := `
SELECT id, gtin, name, status, submitter_name, NULLIF(payload->>'source',''),
(target_product_id IS NOT NULL), created_at, reviewed_at
FROM submission ` + where +
" ORDER BY (status='pending') DESC, created_at DESC LIMIT $" +
strconv.Itoa(len(args)-1) + " OFFSET $" + strconv.Itoa(len(args))
rows, err := s.pool.Query(ctx, sql, args...)
if err != nil {
return nil, 0, err
}
defer rows.Close()
out := []SubmissionRow{}
for rows.Next() {
var r SubmissionRow
var created time.Time
var reviewed *time.Time
if err := rows.Scan(&r.ID, &r.GTIN, &r.Name, &r.Status, &r.SubmitterName, &r.Source, &r.Matched, &created, &reviewed); err != nil {
return nil, 0, err
}
r.CreatedAt = created.Format(time.RFC3339)
if reviewed != nil {
t := reviewed.Format(time.RFC3339)
r.ReviewedAt = &t
}
out = append(out, r)
}
return out, total, rows.Err()
}
// PendingSubmissionCount returns the number of submissions awaiting review.
func (s *Store) PendingSubmissionCount(ctx context.Context) (int, error) {
var n int
err := s.pool.QueryRow(ctx, "SELECT count(*) FROM submission WHERE status='pending'").Scan(&n)
return n, err
}
// GetSubmission returns the full review detail for one submission.
func (s *Store) GetSubmission(ctx context.Context, id string) (*SubmissionDetail, error) {
var d SubmissionDetail
var payload []byte
var created time.Time
var reviewed *time.Time
err := s.pool.QueryRow(ctx, `
SELECT id, status, gtin, name, submitter_name, submitter_contact, note,
review_note, reviewed_by, reviewed_at, created_at, target_product_id, result_product_id, payload
FROM submission WHERE id = $1`, id).Scan(
&d.ID, &d.Status, &d.GTIN, &d.Name, &d.SubmitterName, &d.SubmitterContact, &d.Note,
&d.ReviewNote, &d.ReviewedBy, &reviewed, &created, &d.TargetProductID, &d.ResultProductID, &payload,
)
if errors.Is(err, pgx.ErrNoRows) {
return nil, ErrNotFound
}
if err != nil {
return nil, err
}
d.CreatedAt = created.Format(time.RFC3339)
if reviewed != nil {
t := reviewed.Format(time.RFC3339)
d.ReviewedAt = &t
}
if len(payload) > 0 {
_ = json.Unmarshal(payload, &d.Payload)
}
if d.TargetProductID != nil {
if ep, err := s.GetProduct(ctx, *d.TargetProductID); err == nil {
d.ExistingProduct = ep
}
}
return &d, nil
}
// RejectSubmission marks a pending submission as rejected with a reviewer note.
func (s *Store) RejectSubmission(ctx context.Context, id, actor, note string) error {
ct, err := s.pool.Exec(ctx, `
UPDATE submission SET status='rejected', review_note=$2, reviewed_by=$3, reviewed_at=now()
WHERE id=$1 AND status='pending'`, id, note, actor)
if err != nil {
return err
}
if ct.RowsAffected() == 0 {
// Distinguish missing vs already-reviewed.
var st string
if e := s.pool.QueryRow(ctx, "SELECT status FROM submission WHERE id=$1", id).Scan(&st); errors.Is(e, pgx.ErrNoRows) {
return ErrNotFound
}
return ErrConflict
}
_ = s.writeAudit(ctx, actor, "reject_submission", "submission", &id, []string{}, nil, map[string]string{"review_note": note})
return nil
}
// ApproveSubmission applies a pending contribution to the product store
// (creating or supplementing a product), records community provenance + audit,
// recomputes quality, and marks the submission approved.
func (s *Store) ApproveSubmission(ctx context.Context, id, actor string) (*ProductDetail, error) {
sub, err := s.GetSubmission(ctx, id)
if err != nil {
return nil, err
}
if sub.Status != "pending" {
return nil, ErrConflict
}
in := sub.Payload
tx, err := s.pool.Begin(ctx)
if err != nil {
return nil, err
}
defer tx.Rollback(ctx)
communityID, err := s.sourceIDTx(ctx, tx, "community")
if err != nil {
return nil, err
}
// Resolve the target product (existing supplement vs new create).
productID := ""
if sub.TargetProductID != nil {
productID = *sub.TargetProductID
} else if in.GTIN != nil {
var pid string
if e := tx.QueryRow(ctx, "SELECT id FROM product WHERE gtin=$1", *in.GTIN).Scan(&pid); e == nil {
productID = pid
} else if !errors.Is(e, pgx.ErrNoRows) {
return nil, e
}
}
var brandID *string
if in.BrandName != nil && strings.TrimSpace(*in.BrandName) != "" {
bid, err := s.ensureBrand(ctx, tx, strings.TrimSpace(*in.BrandName))
if err != nil {
return nil, err
}
brandID = &bid
}
var gpc *string
if in.CategoryID != nil && *in.CategoryID != "" {
if err := tx.QueryRow(ctx, "SELECT gpc_brick_code FROM category WHERE id=$1", *in.CategoryID).Scan(&gpc); err != nil && !errors.Is(err, pgx.ErrNoRows) {
return nil, err
}
}
canonical, err := s.netCanonical(ctx, tx, in.NetContentValue, in.NetContentUnit)
if err != nil {
return nil, err
}
fields := submissionFields(in)
var attrJSON []byte
if len(in.Attributes) > 0 {
attrJSON, _ = json.Marshal(in.Attributes)
}
if productID == "" {
// Create a new product from the contribution.
err = tx.QueryRow(ctx, `
INSERT INTO product (gtin, name, brand_id, category_id, gpc_brick_code,
net_content_value, net_content_unit, net_content_canonical, country_of_origin, attributes, status)
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,COALESCE($10::jsonb,'{}'::jsonb),'active') RETURNING id`,
in.GTIN, in.Name, brandID, in.CategoryID, gpc,
in.NetContentValue, in.NetContentUnit, canonical, in.CountryOfOrigin, attrJSON).Scan(&productID)
if err != nil {
return nil, err
}
} else {
// Supplement an existing product: only overwrite fields the
// contribution actually provides (COALESCE keeps current values).
_, err = tx.Exec(ctx, `
UPDATE product SET
name=COALESCE(NULLIF($2,''), name),
brand_id=COALESCE($3, brand_id),
category_id=COALESCE($4, category_id),
gpc_brick_code=COALESCE($5, gpc_brick_code),
net_content_value=COALESCE($6, net_content_value),
net_content_unit=COALESCE($7, net_content_unit),
net_content_canonical=COALESCE($8, net_content_canonical),
country_of_origin=COALESCE($9, country_of_origin),
gtin=COALESCE($10, gtin),
attributes=product.attributes || COALESCE($11::jsonb,'{}'::jsonb)
WHERE id=$1`,
productID, in.Name, brandID, in.CategoryID, gpc,
in.NetContentValue, in.NetContentUnit, canonical, in.CountryOfOrigin, in.GTIN, attrJSON)
if err != nil {
return nil, err
}
}
// food_detail: upsert only when the contribution carries food data, so a
// non-food submission (drug/3C/generic) doesn't create an empty row.
var nutriJSON []byte
if len(in.Nutriments) > 0 {
nutriJSON, _ = json.Marshal(in.Nutriments)
}
foodPresent := len(in.Nutriments) > 0 ||
(in.IngredientsText != nil && *in.IngredientsText != "") ||
(in.NutritionBasis != nil && *in.NutritionBasis != "") ||
(in.ServingSize != nil && *in.ServingSize != "") ||
(in.NutriScore != nil && *in.NutriScore != "")
if foodPresent {
_, err = tx.Exec(ctx, `
INSERT INTO food_detail (product_id, ingredients_text, nutriments, nutrition_basis, serving_size, nutri_score)
VALUES ($1,$2,$3,$4,$5,$6)
ON CONFLICT (product_id) DO UPDATE SET
ingredients_text=COALESCE(EXCLUDED.ingredients_text, food_detail.ingredients_text),
nutriments=COALESCE(EXCLUDED.nutriments, food_detail.nutriments),
nutrition_basis=COALESCE(EXCLUDED.nutrition_basis, food_detail.nutrition_basis),
serving_size=COALESCE(EXCLUDED.serving_size, food_detail.serving_size),
nutri_score=COALESCE(EXCLUDED.nutri_score, food_detail.nutri_score)`,
productID, in.IngredientsText, nutriJSON, in.NutritionBasis, in.ServingSize, in.NutriScore)
if err != nil {
return nil, err
}
}
for _, im := range in.Images {
url := strings.TrimSpace(im.URL)
if url == "" {
continue
}
kind := im.Kind
if kind != "front" && kind != "ingredients" && kind != "nutrition" {
kind = "other"
}
if _, err := tx.Exec(ctx, `
INSERT INTO product_image (product_id, url, kind, source_id) VALUES ($1,$2,$3,$4)`,
productID, url, kind, communityID); err != nil {
return nil, err
}
}
for _, m := range in.MSRP {
if m.Amount <= 0 {
continue
}
cur := m.Currency
if cur == "" {
cur = "CNY"
}
region := m.Region
if region == "" {
region = "CN"
}
if _, err := tx.Exec(ctx, `
INSERT INTO product_msrp (product_id, amount, currency, region, source_id, source_url, effective_date, note)
VALUES ($1,$2,$3,$4,$5,$6,$7,$8)`,
productID, m.Amount, cur, region, communityID, m.SourceURL, m.EffectiveDate, m.Note); err != nil {
return nil, err
}
}
if _, err := s.recomputeQualityTx(ctx, tx, productID); err != nil {
return nil, err
}
if _, err := tx.Exec(ctx, `
UPDATE submission SET status='approved', reviewed_by=$2, reviewed_at=now(), result_product_id=$3
WHERE id=$1`, id, actor, productID); err != nil {
return nil, err
}
// Field-level provenance for the contributed fields (community source).
if len(fields) > 0 {
if _, err := tx.Exec(ctx, `
INSERT INTO product_source (product_id, source_id, url, fields, fetched_at, raw)
VALUES ($1,$2,NULL,$3,now(),NULL)`, productID, communityID, fields); err != nil {
return nil, err
}
}
if err := tx.Commit(ctx); err != nil {
return nil, err
}
_ = s.writeAudit(ctx, actor, "approve_submission", "product", &productID, fields,
map[string]string{"submission_id": id}, map[string]string{"product_id": productID})
return s.GetProduct(ctx, productID)
}
func (s *Store) sourceIDTx(ctx context.Context, tx pgx.Tx, name string) (string, error) {
var id string
err := tx.QueryRow(ctx, "SELECT id FROM source WHERE name=$1", name).Scan(&id)
return id, err
}
// submissionFields lists the product fields a contribution provides values for.
func submissionFields(in SubmissionInput) []string {
fields := []string{"name"}
add := func(name string, present bool) {
if present {
fields = append(fields, name)
}
}
add("gtin", in.GTIN != nil && *in.GTIN != "")
add("brand", in.BrandName != nil && strings.TrimSpace(*in.BrandName) != "")
add("category", in.CategoryID != nil && *in.CategoryID != "")
add("net_content", in.NetContentValue != nil)
add("country_of_origin", in.CountryOfOrigin != nil && *in.CountryOfOrigin != "")
add("ingredients", in.IngredientsText != nil && *in.IngredientsText != "")
add("nutriments", len(in.Nutriments) > 0)
add("image", len(in.Images) > 0)
for k, v := range in.Attributes {
if v == nil {
continue
}
if s, ok := v.(string); ok && s == "" {
continue
}
fields = append(fields, k)
}
return fields
}
+559
View File
@@ -0,0 +1,559 @@
package adminstore
import (
"context"
"encoding/json"
"errors"
"strings"
"github.com/jackc/pgx/v5"
)
// ProductInput is the editable payload accepted from the admin UI.
type ProductInput struct {
GTIN *string `json:"gtin"`
Name string `json:"name"`
BrandID *string `json:"brand_id"`
BrandName *string `json:"brand_name"`
CategoryID *string `json:"category_id"`
NetContentValue *float64 `json:"net_content_value"`
NetContentUnit *string `json:"net_content_unit"`
CountryOfOrigin *string `json:"country_of_origin"`
Status string `json:"status"`
IngredientsText *string `json:"ingredients_text"`
Allergens []string `json:"allergens"`
Additives []string `json:"additives"`
Nutriments map[string]any `json:"nutriments"`
NutritionBasis *string `json:"nutrition_basis"`
ServingSize *string `json:"serving_size"`
NutriScore *string `json:"nutri_score"`
// Attributes carries non-food spec values (driven by kind_field) for the
// generic archive kinds. Nil means "leave unchanged".
Attributes map[string]any `json:"attributes"`
}
func normBrand(name string) string { return strings.Join(strings.Fields(strings.ToLower(name)), " ") }
func (s *Store) ensureBrand(ctx context.Context, tx pgx.Tx, name string) (string, error) {
var id string
err := tx.QueryRow(ctx, `
INSERT INTO brand (name, normalized_name) VALUES ($1, $2)
ON CONFLICT (normalized_name) DO UPDATE SET name = brand.name
RETURNING id`, name, normBrand(name)).Scan(&id)
return id, err
}
func (s *Store) manualSourceID(ctx context.Context, tx pgx.Tx) (string, error) {
var id string
err := tx.QueryRow(ctx, "SELECT id FROM source WHERE name = 'manual'").Scan(&id)
return id, err
}
// netCanonical converts value+unit to the canonical base unit via the unit table.
func (s *Store) netCanonical(ctx context.Context, tx pgx.Tx, value *float64, unit *string) (*float64, error) {
if value == nil || unit == nil || *unit == "" {
return nil, nil
}
var factor *float64
err := tx.QueryRow(ctx, "SELECT to_canonical_factor FROM unit WHERE code = $1", *unit).Scan(&factor)
if errors.Is(err, pgx.ErrNoRows) || factor == nil {
return nil, nil
}
if err != nil {
return nil, err
}
c := *value * *factor
return &c, nil
}
// UpdateProduct applies an edit, records provenance + audit, and recomputes quality.
func (s *Store) UpdateProduct(ctx context.Context, id, actor string, in ProductInput) (*ProductDetail, error) {
before, err := s.GetProduct(ctx, id)
if err != nil {
return nil, err
}
tx, err := s.pool.Begin(ctx)
if err != nil {
return nil, err
}
defer tx.Rollback(ctx)
// Resolve brand (create-by-name takes precedence over id).
brandID := in.BrandID
if in.BrandName != nil && strings.TrimSpace(*in.BrandName) != "" {
bid, err := s.ensureBrand(ctx, tx, strings.TrimSpace(*in.BrandName))
if err != nil {
return nil, err
}
brandID = &bid
}
// Resolve category gpc brick code + archive kind.
var gpc *string
kind := DefaultKind
if in.CategoryID != nil && *in.CategoryID != "" {
if err := tx.QueryRow(ctx, "SELECT gpc_brick_code, archive_kind FROM category WHERE id = $1", *in.CategoryID).Scan(&gpc, &kind); err != nil && !errors.Is(err, pgx.ErrNoRows) {
return nil, err
}
}
canonical, err := s.netCanonical(ctx, tx, in.NetContentValue, in.NetContentUnit)
if err != nil {
return nil, err
}
status := in.Status
if status == "" {
status = before.Status
}
_, err = tx.Exec(ctx, `
UPDATE product SET gtin=$1, name=$2, brand_id=$3, category_id=$4, gpc_brick_code=$5,
net_content_value=$6, net_content_unit=$7, net_content_canonical=$8,
country_of_origin=$9, status=$10
WHERE id=$11`,
in.GTIN, in.Name, brandID, in.CategoryID, gpc,
in.NetContentValue, in.NetContentUnit, canonical,
in.CountryOfOrigin, status, id)
if err != nil {
return nil, err
}
// Non-food spec values live in product.attributes (nil means unchanged).
if in.Attributes != nil {
attrJSON, _ := json.Marshal(in.Attributes)
if _, err = tx.Exec(ctx, "UPDATE product SET attributes=$1 WHERE id=$2", attrJSON, id); err != nil {
return nil, err
}
}
if kind == FoodKind {
var nutriJSON []byte
if in.Nutriments != nil {
nutriJSON, _ = json.Marshal(in.Nutriments)
}
allergens := in.Allergens
if allergens == nil {
allergens = []string{}
}
additives := in.Additives
if additives == nil {
additives = []string{}
}
_, err = tx.Exec(ctx, `
INSERT INTO food_detail (product_id, ingredients_text, allergens, additives,
nutriments, nutrition_basis, serving_size, nutri_score)
VALUES ($1,$2,$3,$4,$5,$6,$7,$8)
ON CONFLICT (product_id) DO UPDATE SET
ingredients_text=EXCLUDED.ingredients_text,
allergens=EXCLUDED.allergens,
additives=EXCLUDED.additives,
nutriments=EXCLUDED.nutriments,
nutrition_basis=EXCLUDED.nutrition_basis,
serving_size=EXCLUDED.serving_size,
nutri_score=EXCLUDED.nutri_score`,
id, in.IngredientsText, allergens, additives,
nutriJSON, in.NutritionBasis, in.ServingSize, in.NutriScore)
if err != nil {
return nil, err
}
}
if _, err := s.recomputeQualityTx(ctx, tx, id); err != nil {
return nil, err
}
if err := tx.Commit(ctx); err != nil {
return nil, err
}
after, err := s.GetProduct(ctx, id)
if err != nil {
return nil, err
}
changed := diffFields(before, after)
if len(changed) > 0 {
if err := s.recordProvenance(ctx, id, changed); err != nil {
return nil, err
}
}
if err := s.writeAudit(ctx, actor, "update", "product", &id, changed, before, after); err != nil {
return nil, err
}
return after, nil
}
// ErrDuplicateGTIN is returned when a product GTIN already exists.
var ErrDuplicateGTIN = errors.New("duplicate gtin")
// CreateProduct inserts a new product from the admin UI. Only the core fields
// are required; the operator completes the rest in the detail editor.
func (s *Store) CreateProduct(ctx context.Context, actor string, in ProductInput) (*ProductDetail, error) {
if strings.TrimSpace(in.Name) == "" {
return nil, errors.New("name required")
}
tx, err := s.pool.Begin(ctx)
if err != nil {
return nil, err
}
defer tx.Rollback(ctx)
brandID := in.BrandID
if in.BrandName != nil && strings.TrimSpace(*in.BrandName) != "" {
bid, err := s.ensureBrand(ctx, tx, strings.TrimSpace(*in.BrandName))
if err != nil {
return nil, err
}
brandID = &bid
}
var gpc *string
if in.CategoryID != nil && *in.CategoryID != "" {
if err := tx.QueryRow(ctx, "SELECT gpc_brick_code FROM category WHERE id = $1", *in.CategoryID).Scan(&gpc); err != nil && !errors.Is(err, pgx.ErrNoRows) {
return nil, err
}
}
canonical, err := s.netCanonical(ctx, tx, in.NetContentValue, in.NetContentUnit)
if err != nil {
return nil, err
}
status := in.Status
if status == "" {
status = "active"
}
var id string
err = tx.QueryRow(ctx, `
INSERT INTO product (gtin, name, brand_id, category_id, gpc_brick_code,
net_content_value, net_content_unit, net_content_canonical,
country_of_origin, status)
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10)
RETURNING id`,
in.GTIN, in.Name, brandID, in.CategoryID, gpc,
in.NetContentValue, in.NetContentUnit, canonical,
in.CountryOfOrigin, status).Scan(&id)
if isUniqueViolation(err) {
return nil, ErrDuplicateGTIN
}
if err != nil {
return nil, err
}
if _, err := s.recomputeQualityTx(ctx, tx, id); err != nil {
return nil, err
}
if err := tx.Commit(ctx); err != nil {
return nil, err
}
after, err := s.GetProduct(ctx, id)
if err != nil {
return nil, err
}
_ = s.writeAudit(ctx, actor, "create", "product", &id, []string{"name"}, nil, after)
return after, nil
}
func strEq(a, b *string) bool {
if a == nil && b == nil {
return true
}
if a == nil || b == nil {
return false
}
return *a == *b
}
func floatEq(a, b *float64) bool {
if a == nil && b == nil {
return true
}
if a == nil || b == nil {
return false
}
return *a == *b
}
func diffFields(a, b *ProductDetail) []string {
changed := []string{}
add := func(name string, eq bool) {
if !eq {
changed = append(changed, name)
}
}
add("gtin", strEq(a.GTIN, b.GTIN))
add("name", a.Name == b.Name)
add("brand", strEq(a.BrandID, b.BrandID))
add("category", strEq(a.CategoryID, b.CategoryID))
add("net_content", floatEq(a.NetContentValue, b.NetContentValue) && strEq(a.NetContentUnit, b.NetContentUnit))
add("country_of_origin", strEq(a.CountryOfOrigin, b.CountryOfOrigin))
add("status", a.Status == b.Status)
add("ingredients", strEq(a.IngredientsText, b.IngredientsText))
ja, _ := json.Marshal(a.Nutriments)
jb, _ := json.Marshal(b.Nutriments)
add("nutriments", string(ja) == string(jb))
add("nutrition_basis", strEq(a.NutritionBasis, b.NutritionBasis))
add("serving_size", strEq(a.ServingSize, b.ServingSize))
add("nutri_score", strEq(a.NutriScore, b.NutriScore))
aa, _ := json.Marshal(a.Attributes)
ab, _ := json.Marshal(b.Attributes)
add("attributes", string(aa) == string(ab))
return changed
}
func (s *Store) recordProvenance(ctx context.Context, productID string, fields []string) error {
var srcID string
if err := s.pool.QueryRow(ctx, "SELECT id FROM source WHERE name = 'manual'").Scan(&srcID); err != nil {
return err
}
_, err := s.pool.Exec(ctx, `
INSERT INTO product_source (product_id, source_id, url, fields, fetched_at, raw)
VALUES ($1, $2, NULL, $3, now(), NULL)`, productID, srcID, fields)
return err
}
func (s *Store) writeAudit(ctx context.Context, actor, action, entity string, entityID *string, fields []string, before, after any) error {
bj, _ := json.Marshal(before)
aj, _ := json.Marshal(after)
if fields == nil {
fields = []string{}
}
_, err := s.pool.Exec(ctx, `
INSERT INTO audit_log (actor, action, entity, entity_id, fields, before, after)
VALUES ($1,$2,$3,$4,$5,$6,$7)`, actor, action, entity, entityID, fields, bj, aj)
return err
}
// ---------- images ----------
// AddImage inserts an image URL (manual source) and recomputes quality.
func (s *Store) AddImage(ctx context.Context, productID, actor, url, kind string) (*ProductImage, error) {
if kind == "" {
kind = "other"
}
var srcID string
if err := s.pool.QueryRow(ctx, "SELECT id FROM source WHERE name = 'manual'").Scan(&srcID); err != nil {
return nil, err
}
var im ProductImage
err := s.pool.QueryRow(ctx, `
INSERT INTO product_image (product_id, url, kind, license, source_id)
VALUES ($1,$2,$3,NULL,$4) RETURNING id, url, kind, license`,
productID, url, kind, srcID).Scan(&im.ID, &im.URL, &im.Kind, &im.License)
if err != nil {
return nil, err
}
if _, err := s.recomputeQuality(ctx, productID); err != nil {
return nil, err
}
_ = s.writeAudit(ctx, actor, "add_image", "product", &productID, []string{"image"}, nil, im)
return &im, nil
}
// DeleteImage removes an image and recomputes quality.
func (s *Store) DeleteImage(ctx context.Context, productID, imageID, actor string) error {
ct, err := s.pool.Exec(ctx, "DELETE FROM product_image WHERE id=$1 AND product_id=$2", imageID, productID)
if err != nil {
return err
}
if ct.RowsAffected() == 0 {
return ErrNotFound
}
if _, err := s.recomputeQuality(ctx, productID); err != nil {
return err
}
_ = s.writeAudit(ctx, actor, "delete_image", "product", &productID, []string{"image"}, map[string]string{"image_id": imageID}, nil)
return nil
}
// ---------- msrp ----------
// MSRPInput is the payload for adding an MSRP snapshot.
type MSRPInput struct {
Amount float64 `json:"amount"`
Currency string `json:"currency"`
Region string `json:"region"`
EffectiveDate *string `json:"effective_date"`
SourceURL *string `json:"source_url"`
Note *string `json:"note"`
}
// AddMSRP inserts a suggested-retail-price snapshot.
func (s *Store) AddMSRP(ctx context.Context, productID, actor string, in MSRPInput) (*MSRP, error) {
if in.Currency == "" {
in.Currency = "CNY"
}
if in.Region == "" {
in.Region = "CN"
}
var srcID string
_ = s.pool.QueryRow(ctx, "SELECT id FROM source WHERE name = 'manual'").Scan(&srcID)
var m MSRP
err := s.pool.QueryRow(ctx, `
INSERT INTO product_msrp (product_id, amount, currency, region, source_id, source_url, effective_date, note)
VALUES ($1,$2,$3,$4,$5,$6,$7,$8)
RETURNING id, amount, currency, region, effective_date::text, source_url, note`,
productID, in.Amount, in.Currency, in.Region, srcID, in.SourceURL, in.EffectiveDate, in.Note).
Scan(&m.ID, &m.Amount, &m.Currency, &m.Region, &m.EffectiveDate, &m.SourceURL, &m.Note)
if err != nil {
return nil, err
}
_ = s.writeAudit(ctx, actor, "add_msrp", "product", &productID, []string{"msrp"}, nil, m)
return &m, nil
}
// DeleteMSRP removes an MSRP snapshot.
func (s *Store) DeleteMSRP(ctx context.Context, productID, msrpID, actor string) error {
ct, err := s.pool.Exec(ctx, "DELETE FROM product_msrp WHERE id=$1 AND product_id=$2", msrpID, productID)
if err != nil {
return err
}
if ct.RowsAffected() == 0 {
return ErrNotFound
}
_ = s.writeAudit(ctx, actor, "delete_msrp", "product", &productID, []string{"msrp"}, map[string]string{"msrp_id": msrpID}, nil)
return nil
}
// ---------- dictionaries ----------
// Brand is a brand option for the edit form and the management view.
type Brand struct {
ID string `json:"id"`
Name string `json:"name"`
ProductCount int `json:"product_count"`
}
// ListBrands returns all brands ordered by name, with the number of products
// referencing each one.
func (s *Store) ListBrands(ctx context.Context) ([]Brand, error) {
rows, err := s.pool.Query(ctx, `
SELECT b.id, b.name,
(SELECT count(*) FROM product p WHERE p.brand_id = b.id) AS product_count
FROM brand b
ORDER BY b.name`)
if err != nil {
return nil, err
}
defer rows.Close()
out := []Brand{}
for rows.Next() {
var b Brand
if err := rows.Scan(&b.ID, &b.Name, &b.ProductCount); err != nil {
return nil, err
}
out = append(out, b)
}
return out, rows.Err()
}
// Category is a category option for the edit form and the management view.
type Category struct {
ID string `json:"id"`
NameZH string `json:"name_zh"`
NameEN *string `json:"name_en"`
Path string `json:"path"`
Level int `json:"level"`
ParentID *string `json:"parent_id"`
GPCBrickCode *string `json:"gpc_brick_code"`
ArchiveKind string `json:"archive_kind"`
ProductCount int `json:"product_count"`
}
// ListCategories returns the full category tree (path order) with the number of
// products directly assigned to each node.
func (s *Store) ListCategories(ctx context.Context) ([]Category, error) {
rows, err := s.pool.Query(ctx, `
SELECT c.id, c.name_zh, c.name_en, c.path::text, c.level, c.parent_id::text,
c.gpc_brick_code, c.archive_kind,
(SELECT count(*) FROM product p WHERE p.category_id = c.id) AS product_count
FROM category c
ORDER BY c.path`)
if err != nil {
return nil, err
}
defer rows.Close()
out := []Category{}
for rows.Next() {
var c Category
if err := rows.Scan(&c.ID, &c.NameZH, &c.NameEN, &c.Path, &c.Level,
&c.ParentID, &c.GPCBrickCode, &c.ArchiveKind, &c.ProductCount); err != nil {
return nil, err
}
out = append(out, c)
}
return out, rows.Err()
}
// ---------- audit ----------
// AuditEntry is one audit-log row for the history view.
type AuditEntry struct {
ID string `json:"id"`
Actor string `json:"actor"`
Action string `json:"action"`
Fields []string `json:"fields"`
CreatedAt string `json:"created_at"`
}
// AuditLogRow is one global audit-log row for the operations log view.
type AuditLogRow struct {
ID string `json:"id"`
Actor string `json:"actor"`
Action string `json:"action"`
Entity string `json:"entity"`
EntityID *string `json:"entity_id"`
Fields []string `json:"fields"`
CreatedAt string `json:"created_at"`
}
// ListAllAudit returns a page of the global audit log, newest first, along with
// the total row count.
func (s *Store) ListAllAudit(ctx context.Context, limit, offset int) ([]AuditLogRow, int, error) {
var total int
if err := s.pool.QueryRow(ctx, "SELECT count(*) FROM audit_log").Scan(&total); err != nil {
return nil, 0, err
}
rows, err := s.pool.Query(ctx, `
SELECT id, actor, action, entity, entity_id::text, fields, created_at::text
FROM audit_log
ORDER BY created_at DESC
LIMIT $1 OFFSET $2`, limit, offset)
if err != nil {
return nil, 0, err
}
defer rows.Close()
out := []AuditLogRow{}
for rows.Next() {
var e AuditLogRow
if err := rows.Scan(&e.ID, &e.Actor, &e.Action, &e.Entity, &e.EntityID, &e.Fields, &e.CreatedAt); err != nil {
return nil, 0, err
}
out = append(out, e)
}
return out, total, rows.Err()
}
// ListAudit returns audit history for one product, newest first.
func (s *Store) ListAudit(ctx context.Context, productID string, limit int) ([]AuditEntry, error) {
rows, err := s.pool.Query(ctx, `
SELECT id, actor, action, fields, created_at::text
FROM audit_log WHERE entity='product' AND entity_id=$1
ORDER BY created_at DESC LIMIT $2`, productID, limit)
if err != nil {
return nil, err
}
defer rows.Close()
out := []AuditEntry{}
for rows.Next() {
var e AuditEntry
if err := rows.Scan(&e.ID, &e.Actor, &e.Action, &e.Fields, &e.CreatedAt); err != nil {
return nil, err
}
out = append(out, e)
}
return out, rows.Err()
}
+7
View File
@@ -0,0 +1,7 @@
<!doctype html>
<html lang="zh">
<head><meta charset="utf-8" /><title>OpenGoods 管理后台</title></head>
<body>
<p>管理后台前端尚未构建。Docker 构建会在此处放入真正的前端产物。</p>
</body>
</html>
+21
View File
@@ -0,0 +1,21 @@
// Package adminweb embeds the built admin SPA (Vite dist). During Docker builds
// the real dist/ is produced by the node stage and copied in before go build;
// the committed placeholder keeps the package compilable for `go build ./...`.
package adminweb
import (
"embed"
"io/fs"
)
//go:embed all:dist
var distFS embed.FS
// Dist returns the embedded SPA filesystem rooted at dist/.
func Dist() fs.FS {
sub, err := fs.Sub(distFS, "dist")
if err != nil {
panic(err)
}
return sub
}
+49
View File
@@ -0,0 +1,49 @@
// Package apikey handles generation and hashing of public-API keys.
//
// A key looks like "og_live_<random>". Only the SHA-256 hash is ever persisted;
// the plaintext is returned once at creation time and cannot be recovered.
package apikey
import (
"crypto/rand"
"crypto/sha256"
"encoding/base64"
"encoding/hex"
"strings"
)
// Prefix is the human-readable scheme prefix every key carries.
const Prefix = "og_live_"
// prefixLen is how many leading characters (including Prefix) are stored in
// api_key.key_prefix for identifying a key without revealing its secret.
const prefixLen = 12
// Generate returns a new random key (plaintext), its SHA-256 hash, and a short
// display prefix. The plaintext must be shown to the caller exactly once.
func Generate() (key, hash, displayPrefix string, err error) {
buf := make([]byte, 24)
if _, err = rand.Read(buf); err != nil {
return "", "", "", err
}
// URL-safe, no padding => stable, copy-pasteable token body.
body := base64.RawURLEncoding.EncodeToString(buf)
key = Prefix + body
hash = Hash(key)
displayPrefix = key
if len(displayPrefix) > prefixLen {
displayPrefix = displayPrefix[:prefixLen]
}
return key, hash, displayPrefix, nil
}
// Hash returns the hex-encoded SHA-256 of a key, used for storage and lookup.
func Hash(key string) string {
sum := sha256.Sum256([]byte(strings.TrimSpace(key)))
return hex.EncodeToString(sum[:])
}
// Looks like a key issued by this service (cheap pre-check before hashing).
func IsWellFormed(key string) bool {
return strings.HasPrefix(key, Prefix) && len(key) > len(Prefix)+8
}
+60
View File
@@ -0,0 +1,60 @@
package apikey
import "testing"
func TestGenerate(t *testing.T) {
key, hash, prefix, err := Generate()
if err != nil {
t.Fatalf("Generate: %v", err)
}
if !IsWellFormed(key) {
t.Fatalf("generated key not well-formed: %q", key)
}
if Hash(key) != hash {
t.Fatalf("Hash(key) != returned hash")
}
if len(prefix) != prefixLen || key[:prefixLen] != prefix {
t.Fatalf("prefix %q not a %d-char prefix of key %q", prefix, prefixLen, key)
}
if len(hash) != 64 {
t.Fatalf("hash not hex sha-256: %q", hash)
}
}
func TestGenerateUnique(t *testing.T) {
seen := map[string]bool{}
for i := 0; i < 100; i++ {
k, _, _, err := Generate()
if err != nil {
t.Fatal(err)
}
if seen[k] {
t.Fatalf("duplicate key generated: %q", k)
}
seen[k] = true
}
}
func TestHashStableAndTrimmed(t *testing.T) {
if Hash("og_live_abc") != Hash(" og_live_abc ") {
t.Fatal("Hash should ignore surrounding whitespace")
}
if Hash("a") == Hash("b") {
t.Fatal("distinct inputs must hash differently")
}
}
func TestIsWellFormed(t *testing.T) {
cases := map[string]bool{
"og_live_abcdefghijkl": true, // body longer than 8 chars
"og_live_": false, // empty body
"og_live_abc": false, // body too short
"nope_abcdefghijkl": false, // wrong prefix
"": false,
}
for in, want := range cases {
if got := IsWellFormed(in); got != want {
t.Errorf("IsWellFormed(%q) = %v, want %v", in, got, want)
}
}
}
+126
View File
@@ -0,0 +1,126 @@
// Package auth provides minimal single-account authentication for the admin
// console: a bcrypt-verified login and a stdlib HMAC-SHA256 signed token
// (JWT-compatible) plus a chi middleware that guards write routes.
package auth
import (
"context"
"crypto/hmac"
"crypto/sha256"
"encoding/base64"
"encoding/json"
"errors"
"net/http"
"strings"
"time"
"golang.org/x/crypto/bcrypt"
)
// Authenticator holds the single admin credential and token signing secret.
type Authenticator struct {
username string
passwordHash []byte
secret []byte
ttl time.Duration
}
// New builds an Authenticator. passwordHash must be a bcrypt hash.
func New(username string, passwordHash, secret []byte, ttl time.Duration) *Authenticator {
return &Authenticator{username: username, passwordHash: passwordHash, secret: secret, ttl: ttl}
}
// ErrInvalidCredentials is returned when login fails.
var ErrInvalidCredentials = errors.New("invalid credentials")
// Login verifies the username/password and returns a signed token on success.
func (a *Authenticator) Login(username, password string) (string, error) {
if username != a.username {
// Still run bcrypt to keep timing roughly constant.
_ = bcrypt.CompareHashAndPassword(a.passwordHash, []byte(password))
return "", ErrInvalidCredentials
}
if err := bcrypt.CompareHashAndPassword(a.passwordHash, []byte(password)); err != nil {
return "", ErrInvalidCredentials
}
return a.issue(username)
}
type claims struct {
Sub string `json:"sub"`
Exp int64 `json:"exp"`
}
func b64(b []byte) string { return base64.RawURLEncoding.EncodeToString(b) }
func (a *Authenticator) sign(signingInput string) string {
mac := hmac.New(sha256.New, a.secret)
mac.Write([]byte(signingInput))
return b64(mac.Sum(nil))
}
func (a *Authenticator) issue(sub string) (string, error) {
header := b64([]byte(`{"alg":"HS256","typ":"JWT"}`))
payloadJSON, err := json.Marshal(claims{Sub: sub, Exp: time.Now().Add(a.ttl).Unix()})
if err != nil {
return "", err
}
payload := b64(payloadJSON)
signingInput := header + "." + payload
return signingInput + "." + a.sign(signingInput), nil
}
// Verify checks a token's signature and expiry, returning the subject.
func (a *Authenticator) Verify(token string) (string, error) {
parts := strings.Split(token, ".")
if len(parts) != 3 {
return "", errors.New("malformed token")
}
signingInput := parts[0] + "." + parts[1]
if !hmac.Equal([]byte(a.sign(signingInput)), []byte(parts[2])) {
return "", errors.New("bad signature")
}
payload, err := base64.RawURLEncoding.DecodeString(parts[1])
if err != nil {
return "", err
}
var c claims
if err := json.Unmarshal(payload, &c); err != nil {
return "", err
}
if time.Now().Unix() >= c.Exp {
return "", errors.New("token expired")
}
return c.Sub, nil
}
type ctxKey int
const userKey ctxKey = 0
// UserFrom returns the authenticated subject from the request context.
func UserFrom(ctx context.Context) string {
if v, ok := ctx.Value(userKey).(string); ok {
return v
}
return ""
}
// Middleware rejects requests without a valid Bearer token.
func (a *Authenticator) Middleware(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
h := r.Header.Get("Authorization")
token := strings.TrimPrefix(h, "Bearer ")
if token == h || token == "" {
http.Error(w, `{"error":{"code":"unauthorized","message":"missing token"}}`, http.StatusUnauthorized)
return
}
sub, err := a.Verify(token)
if err != nil {
http.Error(w, `{"error":{"code":"unauthorized","message":"invalid token"}}`, http.StatusUnauthorized)
return
}
ctx := context.WithValue(r.Context(), userKey, sub)
next.ServeHTTP(w, r.WithContext(ctx))
})
}
+62
View File
@@ -0,0 +1,62 @@
package auth
import (
"testing"
"time"
"golang.org/x/crypto/bcrypt"
)
func newTestAuth(t *testing.T, ttl time.Duration) *Authenticator {
t.Helper()
hash, err := bcrypt.GenerateFromPassword([]byte("s3cret"), bcrypt.MinCost)
if err != nil {
t.Fatalf("hash: %v", err)
}
return New("admin", hash, []byte("test-secret"), ttl)
}
func TestLoginAndVerify(t *testing.T) {
a := newTestAuth(t, time.Hour)
token, err := a.Login("admin", "s3cret")
if err != nil {
t.Fatalf("login: %v", err)
}
sub, err := a.Verify(token)
if err != nil {
t.Fatalf("verify: %v", err)
}
if sub != "admin" {
t.Fatalf("sub = %q, want admin", sub)
}
}
func TestLoginWrongCredentials(t *testing.T) {
a := newTestAuth(t, time.Hour)
if _, err := a.Login("admin", "nope"); err == nil {
t.Fatal("expected error for wrong password")
}
if _, err := a.Login("other", "s3cret"); err == nil {
t.Fatal("expected error for wrong username")
}
}
func TestVerifyRejectsTampered(t *testing.T) {
a := newTestAuth(t, time.Hour)
token, _ := a.Login("admin", "s3cret")
if _, err := a.Verify(token + "x"); err == nil {
t.Fatal("expected bad signature error")
}
if _, err := a.Verify("not.a.token"); err == nil {
t.Fatal("expected malformed/decoding error")
}
}
func TestVerifyRejectsExpired(t *testing.T) {
a := newTestAuth(t, -time.Minute)
token, _ := a.Login("admin", "s3cret")
if _, err := a.Verify(token); err == nil {
t.Fatal("expected expired token error")
}
}
+126
View File
@@ -0,0 +1,126 @@
// Package cache is a Redis-backed, fail-open read cache for the public API.
//
// It caches hot product details and search results so repeated reads avoid
// PostgreSQL. Like the ratelimit package, every operation fails open: if Redis
// is unavailable or misconfigured the caller simply falls back to the database,
// so the cache can never take the API down or serve stale data after Redis loss.
//
// Invalidation is global and O(1): keys are namespaced by an epoch counter
// stored in Redis (og:cache:epoch). The Python ingestion bumps that counter
// after a write run, which logically invalidates every cached entry at once
// while old keys age out via their TTL. The epoch is read at most once per
// refresh interval per process, so it adds no per-request round trip.
package cache
import (
"context"
"encoding/json"
"errors"
"log"
"strconv"
"sync"
"time"
"github.com/redis/go-redis/v9"
)
// epochKey is the Redis key holding the global cache generation counter.
const epochKey = "og:cache:epoch"
// epochRefresh bounds how often a process re-reads the epoch from Redis.
const epochRefresh = 10 * time.Second
// opTimeout caps any single Redis operation so a slow backend never blocks a
// request beyond this; on timeout the cache fails open.
const opTimeout = 150 * time.Millisecond
// Cache wraps a Redis client. A nil-backed Cache (Redis unconfigured) disables
// caching: every Get misses and every Set is a no-op.
type Cache struct {
rdb *redis.Client
mu sync.RWMutex
epoch int64
epochSetAt time.Time
epochOK bool
}
// New builds a Cache from a redis:// URL. On a parse error it logs and returns a
// disabled (fail-open) cache so the server still boots.
func New(redisURL string) *Cache {
opt, err := redis.ParseURL(redisURL)
if err != nil {
log.Printf("cache: invalid redis url %q: %v (caching disabled)", redisURL, err)
return &Cache{}
}
return &Cache{rdb: redis.NewClient(opt)}
}
// Enabled reports whether a Redis backend is configured.
func (c *Cache) Enabled() bool { return c != nil && c.rdb != nil }
// epochNow returns the current cache generation, reading it from Redis at most
// once per epochRefresh. On any Redis error it keeps the last known value and
// throttles re-reads so a down backend cannot slow the hot path.
func (c *Cache) epochNow(ctx context.Context) int64 {
c.mu.RLock()
if c.epochOK && time.Since(c.epochSetAt) < epochRefresh {
e := c.epoch
c.mu.RUnlock()
return e
}
c.mu.RUnlock()
cctx, cancel := context.WithTimeout(ctx, opTimeout)
defer cancel()
n, err := c.rdb.Get(cctx, epochKey).Int64()
c.mu.Lock()
defer c.mu.Unlock()
switch {
case err == nil:
c.epoch = n
case errors.Is(err, redis.Nil):
c.epoch = 0
}
c.epochSetAt = time.Now()
c.epochOK = true
return c.epoch
}
// key namespaces a logical suffix under the current epoch.
func (c *Cache) key(ctx context.Context, suffix string) string {
return "og:v" + strconv.FormatInt(c.epochNow(ctx), 10) + ":" + suffix
}
// GetJSON unmarshals the cached value for suffix into dest and reports a hit.
// Any miss, decode error, or Redis error returns false (fail-open).
func (c *Cache) GetJSON(ctx context.Context, suffix string, dest any) bool {
if !c.Enabled() {
return false
}
k := c.key(ctx, suffix)
cctx, cancel := context.WithTimeout(ctx, opTimeout)
defer cancel()
b, err := c.rdb.Get(cctx, k).Bytes()
if err != nil {
return false
}
return json.Unmarshal(b, dest) == nil
}
// SetJSON stores val (JSON-encoded) for suffix with the given TTL. Best effort:
// marshal or Redis errors are ignored.
func (c *Cache) SetJSON(ctx context.Context, suffix string, val any, ttl time.Duration) {
if !c.Enabled() {
return
}
b, err := json.Marshal(val)
if err != nil {
return
}
k := c.key(ctx, suffix)
cctx, cancel := context.WithTimeout(ctx, opTimeout)
defer cancel()
_ = c.rdb.Set(cctx, k, b, ttl).Err()
}
+84
View File
@@ -0,0 +1,84 @@
package cache
import (
"context"
"fmt"
"os"
"testing"
"time"
)
// TestDisabledFailsOpen verifies a Cache without a Redis backend never panics,
// always misses, and silently drops writes.
func TestDisabledFailsOpen(t *testing.T) {
c := New("not-a-valid-url") // parse error => disabled
if c.Enabled() {
t.Fatal("expected cache to be disabled for invalid url")
}
c.SetJSON(context.Background(), "k", map[string]int{"a": 1}, time.Minute)
var dst map[string]int
if c.GetJSON(context.Background(), "k", &dst) {
t.Fatalf("disabled cache must always miss, got %+v", dst)
}
}
func testCache(t *testing.T) *Cache {
t.Helper()
url := os.Getenv("OPENGOODS_REDIS_URL")
if url == "" {
url = "redis://localhost:6379/0"
}
c := New(url)
if !c.Enabled() {
t.Skip("redis not configured")
}
ctx, cancel := context.WithTimeout(context.Background(), time.Second)
defer cancel()
if err := c.rdb.Ping(ctx).Err(); err != nil {
t.Skipf("redis not reachable: %v", err)
}
return c
}
// TestRoundTrip stores then reads a value back.
func TestRoundTrip(t *testing.T) {
c := testCache(t)
ctx := context.Background()
suffix := fmt.Sprintf("test:rt:%d", time.Now().UnixNano())
c.SetJSON(ctx, suffix, map[string]any{"name": "foo", "n": float64(3)}, time.Minute)
got := map[string]any{}
if !c.GetJSON(ctx, suffix, &got) {
t.Fatal("expected cache hit after set")
}
if got["name"] != "foo" || got["n"] != float64(3) {
t.Fatalf("unexpected payload: %+v", got)
}
}
// TestEpochInvalidation verifies that bumping the epoch counter logically drops
// every previously cached entry.
func TestEpochInvalidation(t *testing.T) {
c := testCache(t)
ctx := context.Background()
suffix := fmt.Sprintf("test:epoch:%d", time.Now().UnixNano())
c.SetJSON(ctx, suffix, map[string]int{"v": 1}, time.Minute)
var dst map[string]int
if !c.GetJSON(ctx, suffix, &dst) {
t.Fatal("expected hit before epoch bump")
}
// Simulate an ingestion write bumping the global epoch.
if err := c.rdb.Incr(ctx, epochKey).Err(); err != nil {
t.Fatalf("incr epoch: %v", err)
}
// Force the process to re-read the epoch rather than use its cached value.
c.mu.Lock()
c.epochOK = false
c.mu.Unlock()
if c.GetJSON(ctx, suffix, &dst) {
t.Fatal("entry should be invisible after epoch bump")
}
}
+24 -6
View File
@@ -2,26 +2,44 @@ package config
import ( import (
"os" "os"
"strconv"
) )
// Config holds runtime configuration for the OpenGoods API server. // Config holds runtime configuration for the OpenGoods API server.
// Values are read from environment variables with sensible defaults so the // Values are read from environment variables with sensible defaults so the
// server can boot in a local Docker Compose setup without extra configuration. // server can boot in a local Docker Compose setup without extra configuration.
type Config struct { type Config struct {
Addr string Addr string
DatabaseURL string DatabaseURL string
RedisURL string RedisURL string
AnonRateLimitPerMin int
AnonTotalQuota int
RegisteredRateLimitPerMin int
RegisteredQuotaTotal int
} }
// Load reads configuration from the environment. // Load reads configuration from the environment.
func Load() Config { func Load() Config {
return Config{ return Config{
Addr: getenv("OPENGOODS_ADDR", ":8080"), Addr: getenv("OPENGOODS_ADDR", ":8080"),
DatabaseURL: getenv("OPENGOODS_DATABASE_URL", "postgres://opengoods:opengoods@localhost:5432/opengoods?sslmode=disable"), DatabaseURL: getenv("OPENGOODS_DATABASE_URL", "postgres://opengoods:opengoods@localhost:5432/opengoods?sslmode=disable"),
RedisURL: getenv("OPENGOODS_REDIS_URL", "redis://localhost:6379/0"), RedisURL: getenv("OPENGOODS_REDIS_URL", "redis://localhost:6379/0"),
AnonRateLimitPerMin: getenvInt("OPENGOODS_ANON_RATE_LIMIT_PER_MIN", 60),
AnonTotalQuota: getenvInt("OPENGOODS_ANON_TOTAL_QUOTA", 1000),
RegisteredRateLimitPerMin: getenvInt("OPENGOODS_REGISTERED_RATE_LIMIT_PER_MIN", 300),
RegisteredQuotaTotal: getenvInt("OPENGOODS_REGISTERED_QUOTA_TOTAL", 100000),
} }
} }
func getenvInt(key string, fallback int) int {
if v, ok := os.LookupEnv(key); ok && v != "" {
if n, err := strconv.Atoi(v); err == nil && n > 0 {
return n
}
}
return fallback
}
func getenv(key, fallback string) string { func getenv(key, fallback string) string {
if v, ok := os.LookupEnv(key); ok && v != "" { if v, ok := os.LookupEnv(key); ok && v != "" {
return v return v
+110
View File
@@ -0,0 +1,110 @@
// Package gtin validates and normalizes GS1 trade item numbers (GTIN-8/12/13/14).
// Only globally-unique GS1 codes are accepted: store-internal / variable-weight /
// coupon codes (which are not globally unique) are rejected on purpose.
package gtin
import (
"errors"
"strings"
)
// Validation errors.
var (
ErrEmpty = errors.New("条码不能为空")
ErrFormat = errors.New("条码必须为 8/12/13/14 位数字")
ErrCheck = errors.New("条码校验位不正确")
ErrRestricted = errors.New("店内码/变量重量码/优惠券码等非全球唯一码,不予收录")
)
// Normalize trims and validates a GTIN, returning the cleaned digit string.
// It enforces length, the GS1 mod-10 check digit, and rejects restricted
// (non-globally-unique) number ranges.
func Normalize(raw string) (string, error) {
s := strings.TrimSpace(raw)
if s == "" {
return "", ErrEmpty
}
for _, c := range s {
if c < '0' || c > '9' {
return "", ErrFormat
}
}
switch len(s) {
case 8, 12, 13, 14:
default:
return "", ErrFormat
}
if !validCheckDigit(s) {
return "", ErrCheck
}
if restricted(s) {
return "", ErrRestricted
}
return s, nil
}
// InferType returns the conventional GTIN type label for a normalized code.
func InferType(s string) string {
switch len(s) {
case 8:
return "EAN8"
case 12:
return "UPC"
case 14:
return "GTIN14"
default:
return "EAN13"
}
}
// validCheckDigit verifies the trailing GS1 mod-10 check digit. The digit
// immediately left of the check digit carries weight 3, then weights alternate.
func validCheckDigit(s string) bool {
n := len(s)
sum := 0
for i := 0; i < n-1; i++ {
d := int(s[i] - '0')
if (n-1-i)%2 == 1 {
sum += d * 3
} else {
sum += d
}
}
check := (10 - (sum % 10)) % 10
return check == int(s[n-1]-'0')
}
// restricted reports whether a (length/check-digit valid) code falls in a
// number range reserved for non-globally-unique use.
func restricted(s string) bool {
switch len(s) {
case 13:
p2 := s[:2]
switch {
case s[0] == '2': // 20-29 restricted distribution / in-store
return true
case p2 == "02": // 020-029 variable-measure within a store
return true
case p2 == "04": // 040-049 restricted circulation within a company
return true
case p2 == "05": // 050-059 coupons
return true
case p2 == "98" || p2 == "99": // 980-989/99 coupons & refund receipts
return true
}
case 12: // UPC-A: leading number-system digit
switch s[0] {
case '2': // in-store / random weight
return true
case '4': // unrestricted in-store use
return true
case '5': // coupons
return true
}
case 8: // EAN-8: 0/2 prefixes reserved for in-store use
if s[0] == '0' || s[0] == '2' {
return true
}
}
return false
}
+49
View File
@@ -0,0 +1,49 @@
package gtin
import "testing"
func TestNormalizeValid(t *testing.T) {
cases := []struct{ in, want, typ string }{
{" 5449000000996 ", "5449000000996", "EAN13"}, // Coca-Cola EAN-13
{"3017624010701", "3017624010701", "EAN13"}, // Nutella EAN-13
{"036000291452", "036000291452", "UPC"}, // UPC-A
{"96385074", "96385074", "EAN8"}, // EAN-8
{"00012345600012", "00012345600012", "GTIN14"},
{"6901234567892", "6901234567892", "EAN13"}, // China 690 prefix
}
for _, c := range cases {
got, err := Normalize(c.in)
if err != nil {
t.Errorf("Normalize(%q) unexpected error: %v", c.in, err)
continue
}
if got != c.want {
t.Errorf("Normalize(%q) = %q, want %q", c.in, got, c.want)
}
if InferType(got) != c.typ {
t.Errorf("InferType(%q) = %q, want %q", got, InferType(got), c.typ)
}
}
}
func TestNormalizeRejects(t *testing.T) {
cases := []struct {
in string
want error
}{
{"", ErrEmpty},
{"12ab5678", ErrFormat},
{"12345", ErrFormat},
{"5449000000997", ErrCheck}, // bad check digit
{"2012345678903", ErrRestricted}, // 20-29 in-store EAN-13
{"0212345678909", ErrRestricted}, // 02x variable measure
{"212345678909", ErrRestricted}, // UPC number system 2
{"02345673", ErrRestricted}, // EAN-8 in-store
}
for _, c := range cases {
_, err := Normalize(c.in)
if err != c.want {
t.Errorf("Normalize(%q) error = %v, want %v", c.in, err, c.want)
}
}
}
+128
View File
@@ -0,0 +1,128 @@
package handler
import (
"encoding/json"
"errors"
"net/http"
"regexp"
"strings"
"github.com/baicai2026-baicai/goods/api/internal/store"
)
// emailRe is a deliberately permissive sanity check; real validation is the
// unique constraint plus the user being able to receive their own key.
var emailRe = regexp.MustCompile(`^[^@\s]+@[^@\s]+\.[^@\s]+$`)
const minPasswordLen = 8
type credentials struct {
Email string `json:"email"`
Password string `json:"password"`
}
func decodeCredentials(w http.ResponseWriter, r *http.Request) (credentials, bool) {
var c credentials
if err := json.NewDecoder(http.MaxBytesReader(w, r.Body, 4096)).Decode(&c); err != nil {
writeError(w, r, http.StatusBadRequest, "invalid_body", "请求格式无效")
return credentials{}, false
}
c.Email = strings.TrimSpace(c.Email)
if !emailRe.MatchString(c.Email) {
writeError(w, r, http.StatusBadRequest, "invalid_email", "邮箱格式无效")
return credentials{}, false
}
if len(c.Password) < minPasswordLen {
writeError(w, r, http.StatusBadRequest, "weak_password", "密码至少需要 8 位")
return credentials{}, false
}
return c, true
}
// keyResponse is returned whenever a fresh plaintext key is issued; the key is
// shown exactly once and cannot be recovered afterwards.
type keyResponse struct {
Email string `json:"email"`
APIKey string `json:"api_key"`
KeyPrefix string `json:"key_prefix"`
RateLimitPerMin int `json:"rate_limit_per_min"`
QuotaTotal int64 `json:"quota_total"`
}
// Register creates an account and issues its first API key. POST {email, password}.
func (h *Handler) Register(w http.ResponseWriter, r *http.Request) {
c, ok := decodeCredentials(w, r)
if !ok {
return
}
key, acct, err := h.store.RegisterUser(r.Context(), c.Email, c.Password, h.regRatePerMin, h.regQuotaTotal)
if errors.Is(err, store.ErrEmailTaken) {
writeError(w, r, http.StatusConflict, "email_taken", "该邮箱已注册,请直接登录查看或重置密钥")
return
}
if h.handleErr(w, r, err) {
return
}
writeJSON(w, http.StatusCreated, keyResponse{
Email: acct.Email,
APIKey: key,
KeyPrefix: acct.KeyPrefix,
RateLimitPerMin: acct.RateLimitPerMin,
QuotaTotal: acct.QuotaTotal,
})
}
// AccountInfo verifies credentials and returns the account's key metadata plus
// cumulative usage. POST {email, password}. The plaintext key is not returned.
func (h *Handler) AccountInfo(w http.ResponseWriter, r *http.Request) {
c, ok := decodeCredentials(w, r)
if !ok {
return
}
acct, err := h.store.Authenticate(r.Context(), c.Email, c.Password)
if errors.Is(err, store.ErrNotFound) {
writeError(w, r, http.StatusUnauthorized, "invalid_credentials", "邮箱或密码错误")
return
}
if h.handleErr(w, r, err) {
return
}
used := h.limiter.TotalUsed(r.Context(), acct.KeyID)
remaining := acct.QuotaTotal - used
if remaining < 0 {
remaining = 0
}
writeJSON(w, http.StatusOK, map[string]any{
"email": acct.Email,
"key_prefix": acct.KeyPrefix,
"rate_limit_per_min": acct.RateLimitPerMin,
"quota_total": acct.QuotaTotal,
"quota_used": used,
"quota_remaining": remaining,
})
}
// RegenerateKey revokes the account's current key and issues a new one, carrying
// over cumulative usage so the quota cannot be reset. POST {email, password}.
func (h *Handler) RegenerateKey(w http.ResponseWriter, r *http.Request) {
c, ok := decodeCredentials(w, r)
if !ok {
return
}
key, acct, oldKeyID, err := h.store.RegenerateKey(r.Context(), c.Email, c.Password, h.regRatePerMin, h.regQuotaTotal)
if errors.Is(err, store.ErrNotFound) {
writeError(w, r, http.StatusUnauthorized, "invalid_credentials", "邮箱或密码错误")
return
}
if h.handleErr(w, r, err) {
return
}
h.limiter.CopyTotal(r.Context(), oldKeyID, acct.KeyID)
writeJSON(w, http.StatusOK, keyResponse{
Email: acct.Email,
APIKey: key,
KeyPrefix: acct.KeyPrefix,
RateLimitPerMin: acct.RateLimitPerMin,
QuotaTotal: acct.QuotaTotal,
})
}
+149
View File
@@ -0,0 +1,149 @@
package handler
import (
"context"
"encoding/json"
"fmt"
"net/http"
"net/http/httptest"
"os"
"strings"
"testing"
"time"
"github.com/jackc/pgx/v5/pgxpool"
"github.com/redis/go-redis/v9"
"github.com/baicai2026-baicai/goods/api/internal/ratelimit"
"github.com/baicai2026-baicai/goods/api/internal/store"
)
func cleanupCounter(t *testing.T, subject string) {
t.Helper()
redisURL := os.Getenv("OPENGOODS_REDIS_URL")
if redisURL == "" {
redisURL = "redis://localhost:6379/0"
}
opt, err := redis.ParseURL(redisURL)
if err != nil {
return
}
rdb := redis.NewClient(opt)
defer rdb.Close()
rdb.Del(context.Background(), "usage:total:"+subject)
}
// newQuotaHandler builds a handler backed by the test DB and a live Redis
// limiter, with a small anonymous quota so exhaustion is cheap to exercise.
func newQuotaHandler(t *testing.T, anonQuota int) *Handler {
t.Helper()
dsn := os.Getenv("OPENGOODS_DATABASE_URL")
if dsn == "" {
dsn = "postgres://opengoods:opengoods@localhost:5432/opengoods?sslmode=disable"
}
ctx, cancel := context.WithTimeout(context.Background(), 3*time.Second)
defer cancel()
pool, err := pgxpool.New(ctx, dsn)
if err != nil {
t.Skipf("no database: %v", err)
}
if err := pool.Ping(ctx); err != nil {
pool.Close()
t.Skipf("database not reachable: %v", err)
}
var hasUser bool
if err := pool.QueryRow(ctx, "SELECT to_regclass('public.app_user') IS NOT NULL").Scan(&hasUser); err != nil || !hasUser {
pool.Close()
t.Skip("migrations not applied")
}
redisURL := os.Getenv("OPENGOODS_REDIS_URL")
if redisURL == "" {
redisURL = "redis://localhost:6379/0"
}
limiter := ratelimit.New(redisURL)
pingCtx, pingCancel := context.WithTimeout(context.Background(), time.Second)
defer pingCancel()
if err := limiter.Ping(pingCtx); err != nil {
pool.Close()
t.Skipf("redis not reachable: %v", err)
}
t.Cleanup(pool.Close)
return New(store.New(pool), nil).
WithRateLimit(limiter, 1000).
WithQuotas(anonQuota, 300, 100000)
}
func cleanupAccount(t *testing.T, email string) {
t.Helper()
dsn := os.Getenv("OPENGOODS_DATABASE_URL")
if dsn == "" {
dsn = "postgres://opengoods:opengoods@localhost:5432/opengoods?sslmode=disable"
}
ctx := context.Background()
pool, err := pgxpool.New(ctx, dsn)
if err != nil {
return
}
defer pool.Close()
_, _ = pool.Exec(ctx, "DELETE FROM app_user WHERE lower(email)=lower($1)", email)
_, _ = pool.Exec(ctx, "DELETE FROM api_key WHERE owner_email=$1", email)
}
func TestAnonTotalQuotaExhausts(t *testing.T) {
h := newQuotaHandler(t, 3)
// Unique client IP so the lifetime counter starts fresh for this test; the
// counter never expires, so drop it afterwards to keep runs independent.
n := time.Now().UnixNano()
ip := fmt.Sprintf("203.%d.%d.%d", n/65536%256, n/256%256, n%256)
t.Cleanup(func() { cleanupCounter(t, "ip:"+ip) })
call := func() *httptest.ResponseRecorder {
req := httptest.NewRequest(http.MethodGet, "/api/"+APIVersion+"/stats", nil)
req.RemoteAddr = ip + ":12345"
rec := httptest.NewRecorder()
h.Router().ServeHTTP(rec, req)
return rec
}
for i := 1; i <= 3; i++ {
if rec := call(); rec.Code != http.StatusOK {
t.Fatalf("call %d should be allowed, got %d (%s)", i, rec.Code, rec.Body.String())
}
}
rec := call()
if rec.Code != http.StatusForbidden {
t.Fatalf("4th call should be 403 quota_exhausted, got %d (%s)", rec.Code, rec.Body.String())
}
if !strings.Contains(rec.Body.String(), "quota_exhausted") {
t.Fatalf("expected quota_exhausted error, got %s", rec.Body.String())
}
}
func TestRegisterIssuesHigherQuotaKey(t *testing.T) {
h := newQuotaHandler(t, 1000)
email := fmt.Sprintf("h-user-%d@example.com", time.Now().UnixNano())
t.Cleanup(func() { cleanupAccount(t, email) })
body := fmt.Sprintf(`{"email":%q,"password":"supersecret"}`, email)
req := httptest.NewRequest(http.MethodPost, "/api/"+APIVersion+"/register", strings.NewReader(body))
req.RemoteAddr = "198.51.100.7:9999"
rec := httptest.NewRecorder()
h.Router().ServeHTTP(rec, req)
if rec.Code != http.StatusCreated {
t.Fatalf("register status = %d (%s)", rec.Code, rec.Body.String())
}
var resp keyResponse
if err := json.NewDecoder(rec.Body).Decode(&resp); err != nil {
t.Fatal(err)
}
if resp.APIKey == "" || resp.QuotaTotal != 100000 || resp.RateLimitPerMin != 300 {
t.Fatalf("unexpected register response: %+v", resp)
}
// A second registration with the same email conflicts.
req2 := httptest.NewRequest(http.MethodPost, "/api/"+APIVersion+"/register", strings.NewReader(body))
req2.RemoteAddr = "198.51.100.7:9999"
rec2 := httptest.NewRecorder()
h.Router().ServeHTTP(rec2, req2)
if rec2.Code != http.StatusConflict {
t.Fatalf("duplicate register status = %d (%s)", rec2.Code, rec2.Body.String())
}
}
+166 -17
View File
@@ -5,33 +5,102 @@
package handler package handler
import ( import (
_ "embed"
"encoding/json" "encoding/json"
"errors" "errors"
"io/fs"
"net/http" "net/http"
"strconv" "strconv"
"strings"
"github.com/go-chi/chi/v5" "github.com/go-chi/chi/v5"
"github.com/go-chi/chi/v5/middleware" "github.com/go-chi/chi/v5/middleware"
"github.com/baicai2026-baicai/goods/api/internal/ratelimit"
"github.com/baicai2026-baicai/goods/api/internal/store" "github.com/baicai2026-baicai/goods/api/internal/store"
) )
//go:embed openapi.json
var openAPISpec []byte
// APIVersion is the current public API version prefix. // APIVersion is the current public API version prefix.
const APIVersion = "v1" const APIVersion = "v1"
// QualifiedMinScore is the quality_score threshold at or above which a product
// record is considered "qualified" (合格) for public stats.
const QualifiedMinScore = 0.6
const ( const (
defaultPageSize = 20 defaultPageSize = 20
maxPageSize = 100 maxPageSize = 100
// defaultAnonLimit is the per-minute request budget for unauthenticated
// callers (identified by client IP) when none is configured.
defaultAnonLimit = 60
// defaultAnonTotalQuota is the lifetime number of calls an anonymous caller
// (by IP) may make before being asked to register for a higher quota.
defaultAnonTotalQuota = 1000
// defaultRegRatePerMin / defaultRegQuotaTotal are the per-minute budget and
// cumulative quota granted to a self-registered API key.
defaultRegRatePerMin = 300
defaultRegQuotaTotal = 100000
// registerRatePerMin caps account registration/login attempts per IP to
// curb abuse; these endpoints sit outside the metered quota group.
registerRatePerMin = 10
) )
// Handler holds dependencies shared by the HTTP routes. // Handler holds dependencies shared by the HTTP routes.
type Handler struct { type Handler struct {
store *store.Store store *store.Store
spa fs.FS
limiter *ratelimit.Limiter
anonLimit int
anonTotalQuota int64
regRatePerMin int
regQuotaTotal int64
} }
// New constructs a Handler backed by the given store. // New constructs a Handler backed by the given store. spa may be nil (JSON-only).
func New(s *store.Store) *Handler { // Rate limiting is disabled until WithRateLimit is called.
return &Handler{store: s} func New(s *store.Store, spa fs.FS) *Handler {
return &Handler{
store: s,
spa: spa,
anonLimit: defaultAnonLimit,
anonTotalQuota: defaultAnonTotalQuota,
regRatePerMin: defaultRegRatePerMin,
regQuotaTotal: defaultRegQuotaTotal,
}
}
// WithRateLimit attaches a Redis-backed limiter and the anonymous per-minute
// budget, enabling rate limiting + usage tracking on the public API routes.
// A non-positive anonPerMin keeps the default.
func (h *Handler) WithRateLimit(l *ratelimit.Limiter, anonPerMin int) *Handler {
h.limiter = l
if anonPerMin > 0 {
h.anonLimit = anonPerMin
}
return h
}
// WithQuotas configures the cumulative free quota for anonymous callers and the
// per-minute rate + cumulative quota self-registered keys receive. Non-positive
// values keep the defaults.
func (h *Handler) WithQuotas(anonTotal, regPerMin, regTotal int) *Handler {
if anonTotal > 0 {
h.anonTotalQuota = int64(anonTotal)
}
if regPerMin > 0 {
h.regRatePerMin = regPerMin
}
if regTotal > 0 {
h.regQuotaTotal = int64(regTotal)
}
return h
} }
// Router builds the top-level HTTP handler with middleware and routes mounted. // Router builds the top-level HTTP handler with middleware and routes mounted.
@@ -44,26 +113,85 @@ func (h *Handler) Router() http.Handler {
r.Get("/healthz", h.Healthz) r.Get("/healthz", h.Healthz)
r.Route("/api/"+APIVersion, func(r chi.Router) { r.Route("/api/"+APIVersion, func(r chi.Router) {
r.Route("/products", func(r chi.Router) { // Machine-readable spec; not rate limited so tooling can always fetch it.
r.Get("/barcode/{gtin}", h.ProductByBarcode) r.Get("/openapi.json", h.OpenAPI)
r.Get("/search", h.SearchProducts)
r.Get("/{id}", h.ProductByID) r.Group(func(r chi.Router) {
r.Get("/{id}/nutriments", h.ProductNutriments) r.Use(h.rateLimit)
r.Get("/{id}/msrp", h.ProductMSRP) r.Route("/products", func(r chi.Router) {
r.Get("/barcode/{gtin}", h.ProductByBarcode)
r.Get("/search", h.SearchProducts)
r.Get("/{id}", h.ProductByID)
r.Get("/{id}/nutriments", h.ProductNutriments)
r.Get("/{id}/msrp", h.ProductMSRP)
})
r.Get("/brands", h.ListBrands)
r.Get("/categories", h.ListCategories)
r.Get("/kind-fields", h.ListKindFields)
r.Get("/sources/{id}", h.SourceByID)
r.Get("/stats", h.Stats)
})
// Self-service account routes. Lightly IP-throttled to curb abuse but
// outside the metered quota group so a user can always register or
// check their key even after exhausting the free anonymous quota.
r.Group(func(r chi.Router) {
r.Use(h.registerLimit)
r.Post("/register", h.Register)
r.Post("/account", h.AccountInfo)
r.Post("/account/regenerate", h.RegenerateKey)
}) })
r.Get("/brands", h.ListBrands)
r.Get("/categories", h.ListCategories)
r.Get("/sources/{id}", h.SourceByID)
}) })
// Public SPA (homepage + search + contribute). API routes above take
// precedence; everything else falls back to the embedded single-page app.
if h.spa != nil {
r.Handle("/*", http.HandlerFunc(h.serveSPA))
}
return r return r
} }
func (h *Handler) serveSPA(w http.ResponseWriter, r *http.Request) {
rel := strings.TrimPrefix(r.URL.Path, "/")
if rel == "" {
rel = "index.html"
}
if f, err := h.spa.Open(rel); err == nil {
f.Close()
http.FileServer(http.FS(h.spa)).ServeHTTP(w, r)
return
}
// SPA fallback: serve index.html for client-side routes.
data, err := fs.ReadFile(h.spa, "index.html")
if err != nil {
http.NotFound(w, r)
return
}
w.Header().Set("Content-Type", "text/html; charset=utf-8")
_, _ = w.Write(data)
}
// Healthz reports liveness of the service. // Healthz reports liveness of the service.
func (h *Handler) Healthz(w http.ResponseWriter, r *http.Request) { func (h *Handler) Healthz(w http.ResponseWriter, r *http.Request) {
writeJSON(w, http.StatusOK, map[string]string{"status": "ok"}) writeJSON(w, http.StatusOK, map[string]string{"status": "ok"})
} }
// Stats returns catalog totals and the count of qualified records.
func (h *Handler) Stats(w http.ResponseWriter, r *http.Request) {
st, err := h.store.Stats(r.Context(), QualifiedMinScore)
if h.handleErr(w, r, err) {
return
}
writeJSON(w, http.StatusOK, st)
}
// OpenAPI serves the embedded OpenAPI 3 specification for the public API.
func (h *Handler) OpenAPI(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json; charset=utf-8")
_, _ = w.Write(openAPISpec)
}
// ProductByBarcode returns a product by its GTIN. // ProductByBarcode returns a product by its GTIN.
func (h *Handler) ProductByBarcode(w http.ResponseWriter, r *http.Request) { func (h *Handler) ProductByBarcode(w http.ResponseWriter, r *http.Request) {
p, err := h.store.ProductByGTIN(r.Context(), chi.URLParam(r, "gtin")) p, err := h.store.ProductByGTIN(r.Context(), chi.URLParam(r, "gtin"))
@@ -82,13 +210,19 @@ func (h *Handler) ProductByID(w http.ResponseWriter, r *http.Request) {
writeJSON(w, http.StatusOK, p) writeJSON(w, http.StatusOK, p)
} }
// SearchProducts runs a fuzzy name search with optional category filter + paging. // SearchProducts runs a trigram-fuzzy name search with optional
// category/brand/country filters, ranked by relevance, plus paging.
func (h *Handler) SearchProducts(w http.ResponseWriter, r *http.Request) { func (h *Handler) SearchProducts(w http.ResponseWriter, r *http.Request) {
q := r.URL.Query().Get("q") qv := r.URL.Query()
category := r.URL.Query().Get("category") filters := store.SearchFilters{
Query: strings.TrimSpace(qv.Get("q")),
Category: strings.TrimSpace(qv.Get("category")),
Brand: strings.TrimSpace(qv.Get("brand")),
Country: strings.TrimSpace(qv.Get("country")),
}
page, size := pageParams(r) page, size := pageParams(r)
items, total, err := h.store.SearchProducts(r.Context(), q, category, size, (page-1)*size) items, total, err := h.store.SearchProducts(r.Context(), filters, size, (page-1)*size)
if h.handleErr(w, r, err) { if h.handleErr(w, r, err) {
return return
} }
@@ -142,6 +276,21 @@ func (h *Handler) ListCategories(w http.ResponseWriter, r *http.Request) {
writeJSON(w, http.StatusOK, map[string]any{"items": items}) writeJSON(w, http.StatusOK, map[string]any{"items": items})
} }
// ListKindFields returns the read-only spec field template for an archive kind,
// used by the contribution form to render kind-specific inputs.
func (h *Handler) ListKindFields(w http.ResponseWriter, r *http.Request) {
kind := strings.TrimSpace(r.URL.Query().Get("kind"))
if kind == "" {
writeError(w, r, http.StatusBadRequest, "bad_request", "missing kind")
return
}
items, err := h.store.ListKindFields(r.Context(), kind)
if h.handleErr(w, r, err) {
return
}
writeJSON(w, http.StatusOK, map[string]any{"items": items, "kind": kind})
}
// SourceByID returns a single data source. // SourceByID returns a single data source.
func (h *Handler) SourceByID(w http.ResponseWriter, r *http.Request) { func (h *Handler) SourceByID(w http.ResponseWriter, r *http.Request) {
src, err := h.store.SourceByID(r.Context(), chi.URLParam(r, "id")) src, err := h.store.SourceByID(r.Context(), chi.URLParam(r, "id"))
+96 -1
View File
@@ -61,7 +61,7 @@ func newTestHandler(t *testing.T) (*Handler, string) {
_, _ = pool.Exec(context.Background(), "DELETE FROM product WHERE gtin=$1", gtin) _, _ = pool.Exec(context.Background(), "DELETE FROM product WHERE gtin=$1", gtin)
pool.Close() pool.Close()
}) })
return New(store.New(pool)), gtin return New(store.New(pool), nil), gtin
} }
func doGET(t *testing.T, h *Handler, path string) *httptest.ResponseRecorder { func doGET(t *testing.T, h *Handler, path string) *httptest.ResponseRecorder {
@@ -116,6 +116,101 @@ func TestSearchProducts(t *testing.T) {
} }
} }
func TestSearchFuzzyAndFilters(t *testing.T) {
h, _ := newTestHandler(t)
ctx := context.Background()
dsn := os.Getenv("OPENGOODS_DATABASE_URL")
if dsn == "" {
dsn = "postgres://opengoods:opengoods@localhost:5432/opengoods?sslmode=disable"
}
pool, err := pgxpool.New(ctx, dsn)
if err != nil {
t.Skipf("no database: %v", err)
}
defer pool.Close()
_, err = pool.Exec(ctx,
"INSERT INTO brand (name, normalized_name) VALUES ('ZZ Test Brand','zz test brand') ON CONFLICT DO NOTHING")
if err != nil {
t.Fatalf("seed brand: %v", err)
}
_, err = pool.Exec(ctx, `
INSERT INTO product (name, brand_id, country_of_origin, quality_score, status)
VALUES ('ZZ Hazelnut Chocolate', (SELECT id FROM brand WHERE name='ZZ Test Brand'), 'Testland', 0.5, 'active')`)
if err != nil {
t.Fatalf("seed product: %v", err)
}
t.Cleanup(func() {
_, _ = pool.Exec(ctx, "DELETE FROM product WHERE name='ZZ Hazelnut Chocolate'")
_, _ = pool.Exec(ctx, "DELETE FROM brand WHERE name='ZZ Test Brand'")
})
decode := func(path string) []store.ProductSummary {
rec := doGET(t, h, path)
if rec.Code != http.StatusOK {
t.Fatalf("%s -> status %d", path, rec.Code)
}
var body struct {
Items []store.ProductSummary `json:"items"`
}
if err := json.NewDecoder(rec.Body).Decode(&body); err != nil {
t.Fatal(err)
}
return body.Items
}
has := func(items []store.ProductSummary, name string) *store.ProductSummary {
for i := range items {
if items[i].Name == name {
return &items[i]
}
}
return nil
}
// Typo "choclate" should fuzzy-match via word_similarity and carry a score.
got := has(decode("/api/"+APIVersion+"/products/search?q=choclate"), "ZZ Hazelnut Chocolate")
if got == nil {
t.Fatal("fuzzy query 'choclate' did not match 'ZZ Hazelnut Chocolate'")
}
if got.Score == nil || *got.Score <= 0 {
t.Fatalf("expected positive fuzzy score, got %v", got.Score)
}
// Brand filter.
if has(decode("/api/"+APIVersion+"/products/search?brand=ZZ+Test+Brand"), "ZZ Hazelnut Chocolate") == nil {
t.Fatal("brand filter did not return the product")
}
// Country filter (case-insensitive prefix).
if has(decode("/api/"+APIVersion+"/products/search?country=test"), "ZZ Hazelnut Chocolate") == nil {
t.Fatal("country filter did not return the product")
}
// Non-matching country excludes it.
if has(decode("/api/"+APIVersion+"/products/search?country=france"), "ZZ Hazelnut Chocolate") != nil {
t.Fatal("country filter 'france' should not return the product")
}
}
func TestOpenAPISpec(t *testing.T) {
h, _ := newTestHandler(t)
rec := doGET(t, h, "/api/"+APIVersion+"/openapi.json")
if rec.Code != http.StatusOK {
t.Fatalf("status = %d", rec.Code)
}
var spec struct {
OpenAPI string `json:"openapi"`
Paths map[string]any `json:"paths"`
}
if err := json.NewDecoder(rec.Body).Decode(&spec); err != nil {
t.Fatalf("openapi.json is not valid JSON: %v", err)
}
if spec.OpenAPI == "" || len(spec.Paths) == 0 {
t.Fatalf("unexpected spec: %+v", spec)
}
if _, ok := spec.Paths["/products/search"]; !ok {
t.Fatal("spec missing /products/search path")
}
}
func TestListCategories(t *testing.T) { func TestListCategories(t *testing.T) {
h, _ := newTestHandler(t) h, _ := newTestHandler(t)
rec := doGET(t, h, "/api/"+APIVersion+"/categories") rec := doGET(t, h, "/api/"+APIVersion+"/categories")
+1 -1
View File
@@ -11,7 +11,7 @@ func TestHealthz(t *testing.T) {
req := httptest.NewRequest(http.MethodGet, "/healthz", nil) req := httptest.NewRequest(http.MethodGet, "/healthz", nil)
rec := httptest.NewRecorder() rec := httptest.NewRecorder()
New(nil).Router().ServeHTTP(rec, req) New(nil, nil).Router().ServeHTTP(rec, req)
if rec.Code != http.StatusOK { if rec.Code != http.StatusOK {
t.Fatalf("expected status %d, got %d", http.StatusOK, rec.Code) t.Fatalf("expected status %d, got %d", http.StatusOK, rec.Code)
+142
View File
@@ -0,0 +1,142 @@
package handler
import (
"context"
"errors"
"net"
"net/http"
"strconv"
"strings"
"time"
"github.com/baicai2026-baicai/goods/api/internal/apikey"
"github.com/baicai2026-baicai/goods/api/internal/store"
)
type ctxKey int
const apiKeyIDKey ctxKey = 0
// rateLimit authenticates an optional API key and enforces a per-minute budget
// on the public API. Anonymous callers are limited by client IP at a lower
// budget; a valid key raises the budget and attributes usage. An API key that
// is present but invalid or revoked is rejected with 401. Rate-limit headers
// are set on every response; over-budget callers get 429 + Retry-After.
func (h *Handler) rateLimit(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
ip := clientIP(r)
id := "ip:" + ip
subject := "ip:" + ip // cumulative-quota counter subject
limit := h.anonLimit
quota := h.anonTotalQuota
keyID := ""
if raw := presentedKey(r); raw != "" {
if !apikey.IsWellFormed(raw) {
writeError(w, r, http.StatusUnauthorized, "invalid_api_key", "API key 格式无效")
return
}
k, err := h.store.APIKeyByHash(r.Context(), apikey.Hash(raw))
if errors.Is(err, store.ErrNotFound) {
writeError(w, r, http.StatusUnauthorized, "invalid_api_key", "API key 无效或已吊销")
return
}
if err != nil {
writeError(w, r, http.StatusInternalServerError, "internal_error", "internal server error")
return
}
keyID = k.ID
limit = k.RateLimitPerMin
id = "key:" + k.ID
subject = k.ID
quota = k.QuotaTotal
}
res := h.limiter.Allow(r.Context(), id, limit, time.Minute)
w.Header().Set("X-RateLimit-Limit", strconv.Itoa(res.Limit))
w.Header().Set("X-RateLimit-Remaining", strconv.Itoa(res.Remaining))
w.Header().Set("X-RateLimit-Reset", strconv.FormatInt(res.ResetUnix, 10))
if !res.Allowed {
retry := res.ResetUnix - time.Now().Unix()
if retry < 1 {
retry = 1
}
w.Header().Set("Retry-After", strconv.FormatInt(retry, 10))
writeError(w, r, http.StatusTooManyRequests, "rate_limited", "请求过于频繁,请稍后再试")
return
}
// Attribute one call to the caller's lifetime counter, then enforce the
// cumulative quota (quota <= 0 means unlimited). Keys also get daily and
// last-used stats recorded for the admin console.
var used int64
if keyID != "" {
h.limiter.RecordUsage(r.Context(), keyID)
used = h.limiter.TotalUsed(r.Context(), keyID)
} else {
used = h.limiter.IncrTotal(r.Context(), subject)
}
if quota > 0 {
remaining := quota - used
if remaining < 0 {
remaining = 0
}
w.Header().Set("X-Quota-Limit", strconv.FormatInt(quota, 10))
w.Header().Set("X-Quota-Used", strconv.FormatInt(used, 10))
w.Header().Set("X-Quota-Remaining", strconv.FormatInt(remaining, 10))
if used > quota {
if keyID == "" {
writeError(w, r, http.StatusForbidden, "quota_exhausted",
"免费额度(共 "+strconv.FormatInt(quota, 10)+" 次)已用尽,请注册账号获取更高配额的 API 密钥")
} else {
writeError(w, r, http.StatusForbidden, "quota_exhausted", "API 密钥配额已用尽")
}
return
}
}
if keyID != "" {
next.ServeHTTP(w, r.WithContext(context.WithValue(r.Context(), apiKeyIDKey, keyID)))
return
}
next.ServeHTTP(w, r)
})
}
// registerLimit throttles self-service account endpoints per client IP without
// consuming the metered free quota, so a caller can still register or recover
// their key after exhausting the anonymous quota.
func (h *Handler) registerLimit(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
res := h.limiter.Allow(r.Context(), "register:"+clientIP(r), h.regRatePerMin, time.Minute)
if !res.Allowed {
retry := res.ResetUnix - time.Now().Unix()
if retry < 1 {
retry = 1
}
w.Header().Set("Retry-After", strconv.FormatInt(retry, 10))
writeError(w, r, http.StatusTooManyRequests, "rate_limited", "操作过于频繁,请稍后再试")
return
}
next.ServeHTTP(w, r)
})
}
// presentedKey extracts an API key from the X-API-Key header or a Bearer token.
func presentedKey(r *http.Request) string {
if v := strings.TrimSpace(r.Header.Get("X-API-Key")); v != "" {
return v
}
if v := r.Header.Get("Authorization"); strings.HasPrefix(v, "Bearer ") {
return strings.TrimSpace(strings.TrimPrefix(v, "Bearer "))
}
return ""
}
// clientIP returns the caller IP, preferring chi's RealIP-normalized RemoteAddr.
func clientIP(r *http.Request) string {
if host, _, err := net.SplitHostPort(r.RemoteAddr); err == nil {
return host
}
return r.RemoteAddr
}
+120
View File
@@ -0,0 +1,120 @@
package handler
import (
"context"
"fmt"
"net/http"
"net/http/httptest"
"os"
"testing"
"time"
"github.com/jackc/pgx/v5/pgxpool"
"github.com/baicai2026-baicai/goods/api/internal/apikey"
"github.com/baicai2026-baicai/goods/api/internal/ratelimit"
"github.com/baicai2026-baicai/goods/api/internal/store"
)
// newRateLimitedHandler builds a handler backed by the test DB and a live Redis
// limiter, plus a freshly issued API key with the given per-minute limit. It
// skips when either backend is unavailable.
func newRateLimitedHandler(t *testing.T, keyLimit int) (h *Handler, plaintextKey string) {
t.Helper()
dsn := os.Getenv("OPENGOODS_DATABASE_URL")
if dsn == "" {
dsn = "postgres://opengoods:opengoods@localhost:5432/opengoods?sslmode=disable"
}
ctx, cancel := context.WithTimeout(context.Background(), 3*time.Second)
defer cancel()
pool, err := pgxpool.New(ctx, dsn)
if err != nil {
t.Skipf("no database: %v", err)
}
if err := pool.Ping(ctx); err != nil {
pool.Close()
t.Skipf("database not reachable: %v", err)
}
var hasTable bool
if err := pool.QueryRow(ctx, "SELECT to_regclass('public.api_key') IS NOT NULL").Scan(&hasTable); err != nil || !hasTable {
pool.Close()
t.Skip("migrations not applied (api_key missing)")
}
redisURL := os.Getenv("OPENGOODS_REDIS_URL")
if redisURL == "" {
redisURL = "redis://localhost:6379/0"
}
limiter := ratelimit.New(redisURL)
pingCtx, pingCancel := context.WithTimeout(context.Background(), time.Second)
defer pingCancel()
if err := limiter.Ping(pingCtx); err != nil {
pool.Close()
t.Skipf("redis not reachable: %v", err)
}
key, hash, prefix, err := apikey.Generate()
if err != nil {
pool.Close()
t.Fatal(err)
}
name := fmt.Sprintf("test-key-%d", time.Now().UnixNano())
if _, err := pool.Exec(context.Background(),
`INSERT INTO api_key (name, key_prefix, key_hash, rate_limit_per_min) VALUES ($1,$2,$3,$4)`,
name, prefix, hash, keyLimit); err != nil {
pool.Close()
t.Fatalf("insert api_key: %v", err)
}
t.Cleanup(func() {
_, _ = pool.Exec(context.Background(), "DELETE FROM api_key WHERE key_hash=$1", hash)
pool.Close()
})
return New(store.New(pool), nil).WithRateLimit(limiter, 60), key
}
func TestRateLimitHeadersAndKeyAuth(t *testing.T) {
h, key := newRateLimitedHandler(t, 100)
req := httptest.NewRequest(http.MethodGet, "/api/"+APIVersion+"/categories", nil)
req.Header.Set("X-API-Key", key)
rec := httptest.NewRecorder()
h.Router().ServeHTTP(rec, req)
if rec.Code != http.StatusOK {
t.Fatalf("status = %d, body = %s", rec.Code, rec.Body.String())
}
if got := rec.Header().Get("X-RateLimit-Limit"); got != "100" {
t.Fatalf("X-RateLimit-Limit = %q, want 100 (key limit)", got)
}
if rec.Header().Get("X-RateLimit-Remaining") == "" {
t.Fatal("missing X-RateLimit-Remaining header")
}
}
func TestInvalidKeyRejected(t *testing.T) {
h, _ := newRateLimitedHandler(t, 100)
req := httptest.NewRequest(http.MethodGet, "/api/"+APIVersion+"/categories", nil)
req.Header.Set("X-API-Key", "og_live_thiskeydoesnotexist123456")
rec := httptest.NewRecorder()
h.Router().ServeHTTP(rec, req)
if rec.Code != http.StatusUnauthorized {
t.Fatalf("status = %d, want 401; body = %s", rec.Code, rec.Body.String())
}
}
func TestRateLimitExceeded(t *testing.T) {
h, key := newRateLimitedHandler(t, 1)
do := func() int {
req := httptest.NewRequest(http.MethodGet, "/api/"+APIVersion+"/categories", nil)
req.Header.Set("X-API-Key", key)
rec := httptest.NewRecorder()
h.Router().ServeHTTP(rec, req)
return rec.Code
}
if code := do(); code != http.StatusOK {
t.Fatalf("first request status = %d, want 200", code)
}
if code := do(); code != http.StatusTooManyRequests {
t.Fatalf("second request status = %d, want 429", code)
}
}
+231
View File
@@ -0,0 +1,231 @@
{
"openapi": "3.0.3",
"info": {
"title": "OpenGoods / 天工商品档案公共仓 API",
"version": "1.0.0",
"description": "Public, read-only product-facts REST API. Anonymous access is allowed at a lower per-minute rate; an optional API key grants a higher rate limit and attributes usage. No purchase or commerce endpoints by design.",
"license": { "name": "Data under each source's license (e.g. ODbL)" }
},
"servers": [{ "url": "https://goods.tangshasha.com/api/v1" }],
"tags": [
{ "name": "products" },
{ "name": "catalog" },
{ "name": "meta" },
{ "name": "account" }
],
"security": [{ "ApiKeyHeader": [] }, { "BearerKey": [] }, {}],
"paths": {
"/products/search": {
"get": {
"tags": ["products"],
"summary": "Search products",
"description": "Trigram-fuzzy name search (typo-tolerant) with optional category/brand/country filters, ranked by name similarity blended with data quality_score.",
"parameters": [
{ "name": "q", "in": "query", "schema": { "type": "string" }, "description": "Keyword (name or barcode); fuzzy-matched. Empty returns all, ordered by quality_score." },
{ "name": "category", "in": "query", "schema": { "type": "string" }, "description": "Category code (matches the subtree), e.g. food.beverages." },
{ "name": "brand", "in": "query", "schema": { "type": "string" }, "description": "Brand name (fuzzy)." },
{ "name": "country", "in": "query", "schema": { "type": "string" }, "description": "Country of origin (case-insensitive prefix)." },
{ "name": "page", "in": "query", "schema": { "type": "integer", "default": 1, "minimum": 1 } },
{ "name": "size", "in": "query", "schema": { "type": "integer", "default": 20, "maximum": 100 } }
],
"responses": {
"200": {
"description": "Paged search results.",
"headers": {
"X-RateLimit-Limit": { "schema": { "type": "integer" }, "description": "Max requests in the current window." },
"X-RateLimit-Remaining": { "schema": { "type": "integer" }, "description": "Remaining requests in the window." },
"X-RateLimit-Reset": { "schema": { "type": "integer" }, "description": "Unix timestamp when the window resets." }
},
"content": {
"application/json": {
"schema": {
"type": "object",
"properties": {
"items": { "type": "array", "items": { "$ref": "#/components/schemas/ProductSummary" } },
"page": { "type": "integer" },
"size": { "type": "integer" },
"total": { "type": "integer" }
}
}
}
}
},
"401": { "$ref": "#/components/responses/InvalidApiKey" },
"429": { "$ref": "#/components/responses/RateLimited" }
}
}
},
"/products/barcode/{gtin}": {
"get": {
"tags": ["products"],
"summary": "Get product by barcode (GTIN)",
"parameters": [{ "name": "gtin", "in": "path", "required": true, "schema": { "type": "string" } }],
"responses": {
"200": { "description": "Product", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/Product" } } } },
"404": { "$ref": "#/components/responses/NotFound" },
"429": { "$ref": "#/components/responses/RateLimited" }
}
}
},
"/products/{id}": {
"get": {
"tags": ["products"],
"summary": "Get product detail by UUID",
"parameters": [{ "name": "id", "in": "path", "required": true, "schema": { "type": "string", "format": "uuid" } }],
"responses": {
"200": { "description": "Product", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/Product" } } } },
"404": { "$ref": "#/components/responses/NotFound" }
}
}
},
"/products/{id}/nutriments": {
"get": {
"tags": ["products"],
"summary": "Get product nutriments",
"parameters": [{ "name": "id", "in": "path", "required": true, "schema": { "type": "string", "format": "uuid" } }],
"responses": { "200": { "description": "Nutriments" }, "404": { "$ref": "#/components/responses/NotFound" } }
}
},
"/products/{id}/msrp": {
"get": {
"tags": ["products"],
"summary": "Get manufacturer suggested retail price snapshots (reference only)",
"parameters": [{ "name": "id", "in": "path", "required": true, "schema": { "type": "string", "format": "uuid" } }],
"responses": { "200": { "description": "MSRP snapshots" } }
}
},
"/brands": {
"get": {
"tags": ["catalog"],
"summary": "List brands",
"parameters": [
{ "name": "page", "in": "query", "schema": { "type": "integer", "default": 1 } },
{ "name": "size", "in": "query", "schema": { "type": "integer", "default": 20, "maximum": 100 } }
],
"responses": { "200": { "description": "Paged brands" } }
}
},
"/categories": {
"get": { "tags": ["catalog"], "summary": "List the category tree", "responses": { "200": { "description": "Category tree" } } }
},
"/sources/{id}": {
"get": {
"tags": ["meta"],
"summary": "Get a data source",
"parameters": [{ "name": "id", "in": "path", "required": true, "schema": { "type": "string", "format": "uuid" } }],
"responses": { "200": { "description": "Source" }, "404": { "$ref": "#/components/responses/NotFound" } }
}
},
"/register": {
"post": {
"tags": ["account"],
"summary": "Register an account and issue an API key",
"description": "Self-service registration; returns the plaintext API key exactly once.",
"security": [],
"requestBody": { "required": true, "content": { "application/json": { "schema": { "type": "object", "required": ["email", "password"], "properties": { "email": { "type": "string", "format": "email" }, "password": { "type": "string", "minLength": 8 } } } } } },
"responses": { "201": { "description": "Account created; plaintext key returned once" }, "400": { "description": "Invalid email or weak password" }, "409": { "description": "Email already registered" } }
}
},
"/account": {
"post": {
"tags": ["account"],
"summary": "View account key metadata and cumulative quota usage",
"security": [],
"requestBody": { "required": true, "content": { "application/json": { "schema": { "type": "object", "required": ["email", "password"], "properties": { "email": { "type": "string", "format": "email" }, "password": { "type": "string" } } } } } },
"responses": { "200": { "description": "Account info with quota usage" }, "401": { "description": "Invalid credentials" } }
}
},
"/account/regenerate": {
"post": {
"tags": ["account"],
"summary": "Revoke the current key and issue a new one",
"description": "Cumulative usage carries over; returns the plaintext key exactly once.",
"security": [],
"requestBody": { "required": true, "content": { "application/json": { "schema": { "type": "object", "required": ["email", "password"], "properties": { "email": { "type": "string", "format": "email" }, "password": { "type": "string" } } } } } },
"responses": { "200": { "description": "New plaintext key returned once" }, "401": { "description": "Invalid credentials" } }
}
}
},
"components": {
"securitySchemes": {
"ApiKeyHeader": { "type": "apiKey", "in": "header", "name": "X-API-Key", "description": "API key, e.g. og_live_xxx. Optional." },
"BearerKey": { "type": "http", "scheme": "bearer", "description": "Authorization: Bearer og_live_xxx. Optional." }
},
"responses": {
"NotFound": {
"description": "Resource not found.",
"content": { "application/json": { "schema": { "$ref": "#/components/schemas/Error" } } }
},
"RateLimited": {
"description": "Rate limit exceeded.",
"headers": { "Retry-After": { "schema": { "type": "integer" }, "description": "Seconds to wait before retrying." } },
"content": { "application/json": { "schema": { "$ref": "#/components/schemas/Error" } } }
},
"InvalidApiKey": {
"description": "API key invalid or revoked.",
"content": { "application/json": { "schema": { "$ref": "#/components/schemas/Error" } } }
}
},
"schemas": {
"Error": {
"type": "object",
"properties": {
"error": {
"type": "object",
"properties": {
"code": { "type": "string" },
"message": { "type": "string" },
"request_id": { "type": "string" }
}
}
}
},
"ProductSummary": {
"type": "object",
"properties": {
"id": { "type": "string", "format": "uuid" },
"gtin": { "type": "string", "nullable": true },
"name": { "type": "string" },
"brand": { "type": "string", "nullable": true },
"category_path": { "type": "string", "nullable": true },
"country_of_origin": { "type": "string", "nullable": true },
"quality_score": { "type": "number", "format": "float" },
"score": { "type": "number", "format": "float", "nullable": true, "description": "Relevance (name word-similarity) when q is provided; null otherwise." }
}
},
"Product": {
"type": "object",
"properties": {
"id": { "type": "string", "format": "uuid" },
"gtin": { "type": "string", "nullable": true },
"name": { "type": "string" },
"brand": { "type": "string", "nullable": true },
"category_path": { "type": "string", "nullable": true },
"net_content_value": { "type": "number", "nullable": true },
"net_content_unit": { "type": "string", "nullable": true },
"country_of_origin": { "type": "string", "nullable": true },
"quality_score": { "type": "number", "format": "float" },
"nutriments": { "type": "object", "additionalProperties": true, "nullable": true },
"nutrition_basis": { "type": "string", "nullable": true },
"nutri_score": { "type": "string", "nullable": true },
"ingredients_text": { "type": "string", "nullable": true },
"msrp": {
"type": "array",
"description": "Manufacturer suggested retail price snapshots (reference only, newest first; zero-amount entries omitted).",
"items": {
"type": "object",
"properties": {
"amount": { "type": "number" },
"currency": { "type": "string" },
"region": { "type": "string" },
"effective_date": { "type": "string", "nullable": true },
"source_url": { "type": "string", "nullable": true },
"note": { "type": "string", "nullable": true }
}
}
}
}
}
}
}
}
+10
View File
@@ -0,0 +1,10 @@
<!doctype html>
<html lang="zh-CN">
<head>
<meta charset="UTF-8" />
<title>OpenGoods</title>
</head>
<body>
<div id="root">OpenGoods public site placeholder. Built assets are injected during Docker build.</div>
</body>
</html>
+22
View File
@@ -0,0 +1,22 @@
// Package publicweb embeds the built public SPA (Vite dist). During Docker
// builds the real dist/ is produced by the node stage and copied in before go
// build; the committed placeholder keeps the package compilable for
// `go build ./...`.
package publicweb
import (
"embed"
"io/fs"
)
//go:embed all:dist
var distFS embed.FS
// Dist returns the embedded SPA filesystem rooted at dist/.
func Dist() fs.FS {
sub, err := fs.Sub(distFS, "dist")
if err != nil {
panic(err)
}
return sub
}
+171
View File
@@ -0,0 +1,171 @@
// Package ratelimit provides a Redis-backed fixed-window rate limiter and
// lightweight per-key usage counters for the public API.
//
// All state lives in Redis so it is shared across API replicas and visible to
// the admin console, and so the public server keeps its read-only contract
// against PostgreSQL. Every operation fails open: if Redis is unavailable the
// limiter allows the request rather than taking the API down.
package ratelimit
import (
"context"
"fmt"
"log"
"time"
"github.com/redis/go-redis/v9"
)
// Limiter throttles callers and records usage. A nil-backed Limiter (when Redis
// could not be configured) disables limiting and usage tracking.
type Limiter struct {
rdb *redis.Client
}
// Result describes the outcome of an Allow check and the headers to surface.
type Result struct {
Allowed bool
Limit int
Remaining int
ResetUnix int64
}
// UsageStat is the aggregated usage for a single API key.
type UsageStat struct {
Total int64 `json:"total"`
Today int64 `json:"today"`
LastUsedAt *int64 `json:"last_used_at,omitempty"`
}
// New builds a Limiter from a redis:// URL. On a parse error it logs and returns
// a fail-open limiter (Redis disabled) so the server still boots.
func New(redisURL string) *Limiter {
opt, err := redis.ParseURL(redisURL)
if err != nil {
log.Printf("ratelimit: invalid redis url %q: %v (rate limiting disabled)", redisURL, err)
return &Limiter{}
}
return &Limiter{rdb: redis.NewClient(opt)}
}
// Enabled reports whether a Redis backend is configured.
func (l *Limiter) Enabled() bool { return l != nil && l.rdb != nil }
// Ping verifies the Redis backend is reachable. Returns an error if disabled or
// unreachable.
func (l *Limiter) Ping(ctx context.Context) error {
if !l.Enabled() {
return redis.ErrClosed
}
return l.rdb.Ping(ctx).Err()
}
// Allow records a hit for id within a fixed window and reports whether the
// caller is under limit. Fails open (Allowed=true) on any Redis error.
func (l *Limiter) Allow(ctx context.Context, id string, limit int, window time.Duration) Result {
reset := func() int64 {
win := int64(window / time.Second)
if win < 1 {
win = 1
}
return (time.Now().Unix()/win + 1) * win
}
if !l.Enabled() {
return Result{Allowed: true, Limit: limit, Remaining: limit, ResetUnix: reset()}
}
win := int64(window / time.Second)
if win < 1 {
win = 1
}
bucket := time.Now().Unix() / win
key := fmt.Sprintf("rl:%s:%d", id, bucket)
n, err := l.rdb.Incr(ctx, key).Result()
if err != nil {
return Result{Allowed: true, Limit: limit, Remaining: limit, ResetUnix: (bucket + 1) * win}
}
if n == 1 {
l.rdb.Expire(ctx, key, time.Duration(win)*time.Second)
}
remaining := limit - int(n)
if remaining < 0 {
remaining = 0
}
return Result{
Allowed: int(n) <= limit,
Limit: limit,
Remaining: remaining,
ResetUnix: (bucket + 1) * win,
}
}
// RecordUsage increments total/daily counters and stamps last-used for a key.
// Best-effort: errors are ignored.
func (l *Limiter) RecordUsage(ctx context.Context, keyID string) {
if !l.Enabled() || keyID == "" {
return
}
now := time.Now()
day := now.Format("20060102")
pipe := l.rdb.Pipeline()
pipe.Incr(ctx, "usage:total:"+keyID)
dayKey := "usage:day:" + keyID + ":" + day
pipe.Incr(ctx, dayKey)
pipe.Expire(ctx, dayKey, 90*24*time.Hour)
pipe.Set(ctx, "usage:last:"+keyID, now.Unix(), 0)
_, _ = pipe.Exec(ctx)
}
// IncrTotal increments the lifetime call counter for subject and returns the
// new total. The counter never expires; it is the cumulative number of calls
// attributed to a caller (an API key id, or "ip:<addr>" for anonymous callers).
// Fails open returning 0 on any error so quota enforcement never takes the API
// down.
func (l *Limiter) IncrTotal(ctx context.Context, subject string) int64 {
if !l.Enabled() || subject == "" {
return 0
}
n, err := l.rdb.Incr(ctx, "usage:total:"+subject).Result()
if err != nil {
return 0
}
return n
}
// CopyTotal carries a lifetime counter from one subject to another, used when a
// key is regenerated so a caller cannot reset their cumulative quota. Best
// effort: a missing or zero source counter is a no-op.
func (l *Limiter) CopyTotal(ctx context.Context, from, to string) {
if !l.Enabled() || from == "" || to == "" {
return
}
n, err := l.rdb.Get(ctx, "usage:total:"+from).Int64()
if err != nil || n == 0 {
return
}
l.rdb.Set(ctx, "usage:total:"+to, n, 0)
}
// TotalUsed reads the lifetime call counter for subject without incrementing.
func (l *Limiter) TotalUsed(ctx context.Context, subject string) int64 {
if !l.Enabled() || subject == "" {
return 0
}
n, _ := l.rdb.Get(ctx, "usage:total:"+subject).Int64()
return n
}
// Usage reads aggregated usage for a key. Returns a zero-value stat on error.
func (l *Limiter) Usage(ctx context.Context, keyID string) UsageStat {
var st UsageStat
if !l.Enabled() || keyID == "" {
return st
}
day := time.Now().Format("20060102")
st.Total, _ = l.rdb.Get(ctx, "usage:total:"+keyID).Int64()
st.Today, _ = l.rdb.Get(ctx, "usage:day:"+keyID+":"+day).Int64()
if v, err := l.rdb.Get(ctx, "usage:last:"+keyID).Int64(); err == nil {
st.LastUsedAt = &v
}
return st
}
+94
View File
@@ -0,0 +1,94 @@
package ratelimit
import (
"context"
"fmt"
"os"
"testing"
"time"
)
// TestDisabledFailsOpen verifies that a Limiter without a Redis backend allows
// all requests and reports usage as zero rather than erroring.
func TestDisabledFailsOpen(t *testing.T) {
l := New("not-a-valid-url") // parse error => disabled
if l.Enabled() {
t.Fatal("expected limiter to be disabled for invalid url")
}
res := l.Allow(context.Background(), "x", 1, time.Minute)
if !res.Allowed || res.Remaining != 1 {
t.Fatalf("disabled limiter must fail open: %+v", res)
}
// Must not panic and must return zero usage.
l.RecordUsage(context.Background(), "k1")
if u := l.Usage(context.Background(), "k1"); u.Total != 0 {
t.Fatalf("disabled usage should be zero, got %+v", u)
}
}
// TestNilReceiverSafe ensures a nil *Limiter is safe to use (handler default).
func TestNilReceiverSafe(t *testing.T) {
var l *Limiter
if l.Enabled() {
t.Fatal("nil limiter must report disabled")
}
res := l.Allow(context.Background(), "x", 5, time.Minute)
if !res.Allowed {
t.Fatal("nil limiter must fail open")
}
l.RecordUsage(context.Background(), "k")
_ = l.Usage(context.Background(), "k")
}
func testLimiter(t *testing.T) *Limiter {
t.Helper()
url := os.Getenv("OPENGOODS_REDIS_URL")
if url == "" {
url = "redis://localhost:6379/0"
}
l := New(url)
if !l.Enabled() {
t.Skip("redis not configured")
}
ctx, cancel := context.WithTimeout(context.Background(), time.Second)
defer cancel()
if err := l.rdb.Ping(ctx).Err(); err != nil {
t.Skipf("redis not reachable: %v", err)
}
return l
}
func TestAllowFixedWindow(t *testing.T) {
l := testLimiter(t)
ctx := context.Background()
id := fmt.Sprintf("test:%d", time.Now().UnixNano())
for i := 1; i <= 2; i++ {
if res := l.Allow(ctx, id, 2, time.Minute); !res.Allowed {
t.Fatalf("request %d should be allowed: %+v", i, res)
}
}
res := l.Allow(ctx, id, 2, time.Minute)
if res.Allowed {
t.Fatalf("3rd request over limit 2 should be denied: %+v", res)
}
if res.Remaining != 0 {
t.Fatalf("remaining should be 0 when over limit, got %d", res.Remaining)
}
}
func TestRecordAndReadUsage(t *testing.T) {
l := testLimiter(t)
ctx := context.Background()
key := fmt.Sprintf("usagekey:%d", time.Now().UnixNano())
l.RecordUsage(ctx, key)
l.RecordUsage(ctx, key)
u := l.Usage(ctx, key)
if u.Total != 2 || u.Today != 2 {
t.Fatalf("expected total=2 today=2, got %+v", u)
}
if u.LastUsedAt == nil {
t.Fatal("expected last-used timestamp to be set")
}
}
+169
View File
@@ -0,0 +1,169 @@
package store
import (
"context"
"errors"
"strings"
"github.com/jackc/pgx/v5"
"github.com/jackc/pgx/v5/pgconn"
"golang.org/x/crypto/bcrypt"
"github.com/baicai2026-baicai/goods/api/internal/apikey"
)
// ErrEmailTaken is returned when registering an email that already exists.
var ErrEmailTaken = errors.New("email already registered")
// Account is a self-registered public-API user and its current key metadata.
type Account struct {
ID string `json:"id"`
Email string `json:"email"`
KeyID string `json:"-"`
KeyPrefix string `json:"key_prefix"`
RateLimitPerMin int `json:"rate_limit_per_min"`
QuotaTotal int64 `json:"quota_total"`
}
// bcryptDummyHash is compared against on unknown-email logins to keep timing
// roughly constant and avoid leaking which emails are registered.
const bcryptDummyHash = "$2a$10$N9qo8uLOickgx2ZMRZoMyeIjZAgcfl7p92ldGxad68LJZdL17lhWy"
func isUniqueViolation(err error) bool {
var pgErr *pgconn.PgError
return errors.As(err, &pgErr) && pgErr.Code == "23505"
}
// RegisterUser creates an account plus a self-issued API key with the given
// per-minute rate and cumulative quota, returning the plaintext key (shown
// once). Email uniqueness is case-insensitive; ErrEmailTaken signals a dupe.
func (s *Store) RegisterUser(ctx context.Context, email, password string, ratePerMin int, quotaTotal int64) (plaintext string, acct Account, err error) {
email = strings.TrimSpace(email)
pwHash, err := bcrypt.GenerateFromPassword([]byte(password), bcrypt.DefaultCost)
if err != nil {
return "", Account{}, err
}
key, keyHash, prefix, err := apikey.Generate()
if err != nil {
return "", Account{}, err
}
tx, err := s.pool.Begin(ctx)
if err != nil {
return "", Account{}, err
}
defer func() { _ = tx.Rollback(ctx) }()
var keyID string
if err = tx.QueryRow(ctx,
`INSERT INTO api_key (name, key_prefix, key_hash, owner_email, tier, rate_limit_per_min, quota_total, created_by)
VALUES ($1,$2,$3,$4,'registered',$5,$6,'self-register') RETURNING id`,
"user:"+strings.ToLower(email), prefix, keyHash, email, ratePerMin, quotaTotal,
).Scan(&keyID); err != nil {
return "", Account{}, err
}
var userID string
if err = tx.QueryRow(ctx,
`INSERT INTO app_user (email, password_hash, api_key_id) VALUES ($1,$2,$3) RETURNING id`,
email, string(pwHash), keyID,
).Scan(&userID); err != nil {
if isUniqueViolation(err) {
return "", Account{}, ErrEmailTaken
}
return "", Account{}, err
}
if err = tx.Commit(ctx); err != nil {
return "", Account{}, err
}
return key, Account{
ID: userID, Email: email, KeyID: keyID, KeyPrefix: prefix,
RateLimitPerMin: ratePerMin, QuotaTotal: quotaTotal,
}, nil
}
// Authenticate verifies an email/password pair and returns the account with its
// current (non-revoked) key metadata. Returns ErrNotFound on unknown email or
// wrong password.
func (s *Store) Authenticate(ctx context.Context, email, password string) (Account, error) {
email = strings.TrimSpace(email)
var (
userID, pwHash string
keyID *string
)
err := s.pool.QueryRow(ctx,
`SELECT id, password_hash, api_key_id FROM app_user WHERE lower(email) = lower($1)`, email,
).Scan(&userID, &pwHash, &keyID)
if errors.Is(err, pgx.ErrNoRows) {
_ = bcrypt.CompareHashAndPassword([]byte(bcryptDummyHash), []byte(password))
return Account{}, ErrNotFound
}
if err != nil {
return Account{}, err
}
if err := bcrypt.CompareHashAndPassword([]byte(pwHash), []byte(password)); err != nil {
return Account{}, ErrNotFound
}
acct := Account{ID: userID, Email: email}
if keyID != nil {
acct.KeyID = *keyID
_ = s.pool.QueryRow(ctx,
`SELECT key_prefix, rate_limit_per_min, quota_total
FROM api_key WHERE id = $1 AND revoked_at IS NULL`, *keyID,
).Scan(&acct.KeyPrefix, &acct.RateLimitPerMin, &acct.QuotaTotal)
}
return acct, nil
}
// RegenerateKey verifies credentials, revokes the account's current key, and
// issues a fresh one with the same rate/quota, returning the plaintext key and
// the previous key id (so cumulative usage can be carried over). Returns
// ErrNotFound on bad credentials.
func (s *Store) RegenerateKey(ctx context.Context, email, password string, ratePerMin int, quotaTotal int64) (plaintext string, acct Account, oldKeyID string, err error) {
cur, err := s.Authenticate(ctx, email, password)
if err != nil {
return "", Account{}, "", err
}
key, keyHash, prefix, err := apikey.Generate()
if err != nil {
return "", Account{}, "", err
}
oldKeyID = cur.KeyID
tx, err := s.pool.Begin(ctx)
if err != nil {
return "", Account{}, "", err
}
defer func() { _ = tx.Rollback(ctx) }()
if oldKeyID != "" {
if _, err = tx.Exec(ctx,
`UPDATE api_key SET revoked_at = now() WHERE id = $1`, oldKeyID); err != nil {
return "", Account{}, "", err
}
}
var newKeyID string
if err = tx.QueryRow(ctx,
`INSERT INTO api_key (name, key_prefix, key_hash, owner_email, tier, rate_limit_per_min, quota_total, created_by)
VALUES ($1,$2,$3,$4,'registered',$5,$6,'self-register') RETURNING id`,
"user:"+strings.ToLower(cur.Email), prefix, keyHash, cur.Email, ratePerMin, quotaTotal,
).Scan(&newKeyID); err != nil {
return "", Account{}, "", err
}
if _, err = tx.Exec(ctx,
`UPDATE app_user SET api_key_id = $1 WHERE id = $2`, newKeyID, cur.ID); err != nil {
return "", Account{}, "", err
}
if err = tx.Commit(ctx); err != nil {
return "", Account{}, "", err
}
cur.KeyID = newKeyID
cur.KeyPrefix = prefix
cur.RateLimitPerMin = ratePerMin
cur.QuotaTotal = quotaTotal
return key, cur, oldKeyID, nil
}
+99
View File
@@ -0,0 +1,99 @@
package store
import (
"context"
"errors"
"fmt"
"os"
"testing"
"time"
"github.com/jackc/pgx/v5/pgxpool"
"github.com/baicai2026-baicai/goods/api/internal/apikey"
)
func testStore(t *testing.T) *Store {
t.Helper()
dsn := os.Getenv("OPENGOODS_DATABASE_URL")
if dsn == "" {
dsn = "postgres://opengoods:opengoods@localhost:5432/opengoods?sslmode=disable"
}
ctx, cancel := context.WithTimeout(context.Background(), 3*time.Second)
defer cancel()
pool, err := pgxpool.New(ctx, dsn)
if err != nil {
t.Skipf("no database: %v", err)
}
if err := pool.Ping(ctx); err != nil {
pool.Close()
t.Skipf("database not reachable: %v", err)
}
var hasUser bool
if err := pool.QueryRow(ctx, "SELECT to_regclass('public.app_user') IS NOT NULL").Scan(&hasUser); err != nil || !hasUser {
pool.Close()
t.Skip("migrations not applied")
}
t.Cleanup(pool.Close)
return New(pool)
}
func TestRegisterAuthenticateRegenerate(t *testing.T) {
s := testStore(t)
ctx := context.Background()
email := fmt.Sprintf("user-%d@example.com", time.Now().UnixNano())
t.Cleanup(func() {
_, _ = s.pool.Exec(ctx, "DELETE FROM app_user WHERE lower(email)=lower($1)", email)
_, _ = s.pool.Exec(ctx, "DELETE FROM api_key WHERE owner_email=$1", email)
})
key, acct, err := s.RegisterUser(ctx, email, "supersecret", 300, 100000)
if err != nil {
t.Fatalf("register: %v", err)
}
if key == "" || acct.KeyPrefix == "" || acct.QuotaTotal != 100000 || acct.RateLimitPerMin != 300 {
t.Fatalf("unexpected account: %+v key=%q", acct, key)
}
// The issued key resolves via the public auth path with its quota attached.
k, err := s.APIKeyByHash(ctx, apikey.Hash(key))
if err != nil {
t.Fatalf("APIKeyByHash: %v", err)
}
if k.QuotaTotal != 100000 || k.RateLimitPerMin != 300 {
t.Fatalf("key metadata mismatch: %+v", k)
}
// Duplicate email is rejected.
if _, _, err := s.RegisterUser(ctx, email, "anotherpass", 300, 100000); !errors.Is(err, ErrEmailTaken) {
t.Fatalf("expected ErrEmailTaken, got %v", err)
}
// Wrong password fails; correct password authenticates.
if _, err := s.Authenticate(ctx, email, "wrong"); !errors.Is(err, ErrNotFound) {
t.Fatalf("expected ErrNotFound for bad password, got %v", err)
}
got, err := s.Authenticate(ctx, email, "supersecret")
if err != nil {
t.Fatalf("authenticate: %v", err)
}
if got.KeyPrefix != acct.KeyPrefix {
t.Fatalf("authenticate key prefix = %q want %q", got.KeyPrefix, acct.KeyPrefix)
}
// Regeneration revokes the old key and issues a new one.
newKey, regen, oldKeyID, err := s.RegenerateKey(ctx, email, "supersecret", 300, 100000)
if err != nil {
t.Fatalf("regenerate: %v", err)
}
if newKey == key || oldKeyID != acct.KeyID || regen.KeyID == oldKeyID {
t.Fatalf("regenerate did not rotate key: old=%s new=%+v", oldKeyID, regen)
}
if _, err := s.APIKeyByHash(ctx, apikey.Hash(key)); !errors.Is(err, ErrNotFound) {
t.Fatalf("old key should be revoked, got %v", err)
}
if _, err := s.APIKeyByHash(ctx, apikey.Hash(newKey)); err != nil {
t.Fatalf("new key should be active: %v", err)
}
}
+381 -39
View File
@@ -4,19 +4,35 @@ package store
import ( import (
"context" "context"
"crypto/sha1"
"encoding/hex"
"encoding/json"
"errors" "errors"
"strconv" "strconv"
"strings"
"time"
"github.com/jackc/pgx/v5" "github.com/jackc/pgx/v5"
"github.com/jackc/pgx/v5/pgxpool" "github.com/jackc/pgx/v5/pgxpool"
"github.com/baicai2026-baicai/goods/api/internal/cache"
)
// Cache TTLs for the public read cache. Product details are far less volatile
// than search result sets, so they live longer; both are also invalidated
// wholesale whenever ingestion bumps the cache epoch.
const (
productCacheTTL = 24 * time.Hour
searchCacheTTL = time.Hour
) )
// ErrNotFound is returned when a requested row does not exist. // ErrNotFound is returned when a requested row does not exist.
var ErrNotFound = errors.New("not found") var ErrNotFound = errors.New("not found")
// Store wraps a PostgreSQL connection pool. // Store wraps a PostgreSQL connection pool and an optional read cache.
type Store struct { type Store struct {
pool *pgxpool.Pool pool *pgxpool.Pool
cache *cache.Cache
} }
// New constructs a Store from an existing pgx pool. // New constructs a Store from an existing pgx pool.
@@ -24,11 +40,71 @@ func New(pool *pgxpool.Pool) *Store {
return &Store{pool: pool} return &Store{pool: pool}
} }
// WithCache attaches a Redis-backed read cache. A nil or disabled cache leaves
// the Store reading straight from PostgreSQL.
func (s *Store) WithCache(c *cache.Cache) *Store {
s.cache = c
return s
}
// cacheGet reads a cached JSON value into dest, reporting a hit. It is a no-op
// miss when no cache is attached.
func (s *Store) cacheGet(ctx context.Context, suffix string, dest any) bool {
if s.cache == nil {
return false
}
return s.cache.GetJSON(ctx, suffix, dest)
}
// cacheSet stores a JSON value when a cache is attached.
func (s *Store) cacheSet(ctx context.Context, suffix string, val any, ttl time.Duration) {
if s.cache == nil {
return
}
s.cache.SetJSON(ctx, suffix, val, ttl)
}
// Ping verifies database connectivity. // Ping verifies database connectivity.
func (s *Store) Ping(ctx context.Context) error { func (s *Store) Ping(ctx context.Context) error {
return s.pool.Ping(ctx) return s.pool.Ping(ctx)
} }
// PublicStats summarizes the public catalog for the homepage.
type PublicStats struct {
Total int `json:"total"`
Qualified int `json:"qualified"`
MinScore float64 `json:"min_score"`
}
// Stats returns active-product totals and the number of qualified records whose
// quality_score meets minScore.
func (s *Store) Stats(ctx context.Context, minScore float64) (PublicStats, error) {
st := PublicStats{MinScore: minScore}
err := s.pool.QueryRow(ctx, `
SELECT count(*) FILTER (WHERE status = 'active'),
count(*) FILTER (WHERE status = 'active' AND quality_score >= $1)
FROM product`, minScore).Scan(&st.Total, &st.Qualified)
return st, err
}
// Barcode is one GS1 trade item number attached to a product.
type Barcode struct {
GTIN string `json:"gtin"`
GTINType string `json:"gtin_type"`
PackLevel string `json:"pack_level"`
Region *string `json:"region"`
IsPrimary bool `json:"is_primary"`
}
// ProductSpec is one labeled spec line for a non-food product, rendered from
// the product's attributes JSONB against its archive kind's field template.
type ProductSpec struct {
Key string `json:"key"`
Label string `json:"label"`
Value string `json:"value"`
Unit string `json:"unit,omitempty"`
}
// Product is the full public view of a product. // Product is the full public view of a product.
type Product struct { type Product struct {
ID string `json:"id"` ID string `json:"id"`
@@ -37,29 +113,66 @@ type Product struct {
Brand *string `json:"brand"` Brand *string `json:"brand"`
CategoryPath *string `json:"category_path"` CategoryPath *string `json:"category_path"`
GPCBrickCode *string `json:"gpc_brick_code"` GPCBrickCode *string `json:"gpc_brick_code"`
ArchiveKind string `json:"archive_kind"`
NetContentValue *float64 `json:"net_content_value"` NetContentValue *float64 `json:"net_content_value"`
NetContentUnit *string `json:"net_content_unit"` NetContentUnit *string `json:"net_content_unit"`
CountryOfOrigin *string `json:"country_of_origin"` CountryOfOrigin *string `json:"country_of_origin"`
QualityScore float64 `json:"quality_score"` QualityScore float64 `json:"quality_score"`
Barcodes []Barcode `json:"barcodes"`
Specs []ProductSpec `json:"specs,omitempty"`
Nutriments map[string]any `json:"nutriments,omitempty"` Nutriments map[string]any `json:"nutriments,omitempty"`
NutritionBasis *string `json:"nutrition_basis,omitempty"` NutritionBasis *string `json:"nutrition_basis,omitempty"`
NutriScore *string `json:"nutri_score,omitempty"` NutriScore *string `json:"nutri_score,omitempty"`
Ingredients *string `json:"ingredients_text,omitempty"` Ingredients *string `json:"ingredients_text,omitempty"`
Allergens []string `json:"allergens,omitempty"` Allergens []string `json:"allergens,omitempty"`
Additives []string `json:"additives,omitempty"` Additives []string `json:"additives,omitempty"`
MSRP []MSRP `json:"msrp,omitempty"`
}
// ProductBarcodes returns every barcode attached to a product, primary first.
func (s *Store) ProductBarcodes(ctx context.Context, productID string) ([]Barcode, error) {
rows, err := s.pool.Query(ctx,
`SELECT gtin, gtin_type, pack_level, region, is_primary
FROM product_barcode WHERE product_id = $1
ORDER BY is_primary DESC, gtin`, productID)
if err != nil {
return nil, err
}
defer rows.Close()
out := []Barcode{}
for rows.Next() {
var b Barcode
if err := rows.Scan(&b.GTIN, &b.GTINType, &b.PackLevel, &b.Region, &b.IsPrimary); err != nil {
return nil, err
}
out = append(out, b)
}
return out, rows.Err()
} }
// ProductSummary is a lightweight row used in search/listing responses. // ProductSummary is a lightweight row used in search/listing responses.
type ProductSummary struct { type ProductSummary struct {
ID string `json:"id"` ID string `json:"id"`
GTIN *string `json:"gtin"` GTIN *string `json:"gtin"`
Name string `json:"name"` Name string `json:"name"`
Brand *string `json:"brand"` Brand *string `json:"brand"`
CategoryPath *string `json:"category_path"` CategoryPath *string `json:"category_path"`
Country *string `json:"country_of_origin"`
QualityScore float64 `json:"quality_score"`
Score *float64 `json:"score,omitempty"`
}
// SearchFilters bundles the optional filters accepted by SearchProducts.
type SearchFilters struct {
Query string // fuzzy name / barcode query
Category string // ltree path; matches the subtree
Brand string // fuzzy brand name
Country string // country_of_origin prefix (case-insensitive)
} }
const productSelect = ` const productSelect = `
SELECT p.id, p.gtin, p.name, b.name, c.path::text, p.gpc_brick_code, SELECT p.id, p.gtin, p.name, b.name, c.path::text, p.gpc_brick_code,
COALESCE(c.archive_kind, 'generic'), p.attributes,
p.net_content_value, p.net_content_unit, p.country_of_origin, p.quality_score, p.net_content_value, p.net_content_unit, p.country_of_origin, p.quality_score,
f.nutriments, f.nutrition_basis, f.nutri_score, f.ingredients_text, f.nutriments, f.nutrition_basis, f.nutri_score, f.ingredients_text,
f.allergens, f.additives f.allergens, f.additives
@@ -69,61 +182,202 @@ LEFT JOIN category c ON c.id = p.category_id
LEFT JOIN food_detail f ON f.product_id = p.id LEFT JOIN food_detail f ON f.product_id = p.id
` `
func scanProduct(row pgx.Row) (*Product, error) { func scanProduct(row pgx.Row) (*Product, []byte, error) {
var p Product var p Product
var attributes []byte
err := row.Scan( err := row.Scan(
&p.ID, &p.GTIN, &p.Name, &p.Brand, &p.CategoryPath, &p.GPCBrickCode, &p.ID, &p.GTIN, &p.Name, &p.Brand, &p.CategoryPath, &p.GPCBrickCode,
&p.ArchiveKind, &attributes,
&p.NetContentValue, &p.NetContentUnit, &p.CountryOfOrigin, &p.QualityScore, &p.NetContentValue, &p.NetContentUnit, &p.CountryOfOrigin, &p.QualityScore,
&p.Nutriments, &p.NutritionBasis, &p.NutriScore, &p.Ingredients, &p.Nutriments, &p.NutritionBasis, &p.NutriScore, &p.Ingredients,
&p.Allergens, &p.Additives, &p.Allergens, &p.Additives,
) )
if errors.Is(err, pgx.ErrNoRows) { if errors.Is(err, pgx.ErrNoRows) {
return nil, ErrNotFound return nil, nil, ErrNotFound
} }
if err != nil {
return nil, nil, err
}
return &p, attributes, nil
}
// buildSpecs renders the labeled, ordered spec list for a non-food product from
// its attributes JSONB against its archive kind's field template.
func (s *Store) buildSpecs(ctx context.Context, kind string, attributes []byte) ([]ProductSpec, error) {
if kind == "" || kind == "food" || len(attributes) == 0 {
return nil, nil
}
attrs := map[string]any{}
if err := json.Unmarshal(attributes, &attrs); err != nil || len(attrs) == 0 {
return nil, nil
}
rows, err := s.pool.Query(ctx,
"SELECT field_key, label_zh, COALESCE(unit, '') FROM kind_field WHERE kind = $1 ORDER BY sort_order, field_key", kind)
if err != nil { if err != nil {
return nil, err return nil, err
} }
return &p, nil defer rows.Close()
specs := []ProductSpec{}
for rows.Next() {
var key, label, unit string
if err := rows.Scan(&key, &label, &unit); err != nil {
return nil, err
}
v, ok := attrs[key]
if !ok || v == nil {
continue
}
val := stringifyAttr(v)
if val == "" {
continue
}
specs = append(specs, ProductSpec{Key: key, Label: label, Value: val, Unit: unit})
}
return specs, rows.Err()
} }
// ProductByGTIN looks up an active product by its barcode. // stringifyAttr renders a JSON attribute value as display text.
func stringifyAttr(v any) string {
switch t := v.(type) {
case string:
return t
case float64:
return strconv.FormatFloat(t, 'f', -1, 64)
case bool:
if t {
return "是"
}
return "否"
case []any:
parts := make([]string, 0, len(t))
for _, e := range t {
parts = append(parts, stringifyAttr(e))
}
return strings.Join(parts, "、")
default:
return ""
}
}
// ProductByGTIN looks up an active product by any of its barcodes.
func (s *Store) ProductByGTIN(ctx context.Context, gtin string) (*Product, error) { func (s *Store) ProductByGTIN(ctx context.Context, gtin string) (*Product, error) {
row := s.pool.QueryRow(ctx, productSelect+" WHERE p.gtin = $1 AND p.status = 'active'", gtin) const suffix = "prod:gtin:"
return scanProduct(row) if cached := new(Product); s.cacheGet(ctx, suffix+gtin, cached) {
return cached, nil
}
row := s.pool.QueryRow(ctx, productSelect+`
WHERE p.status = 'active'
AND (p.gtin = $1 OR EXISTS (
SELECT 1 FROM product_barcode pb
WHERE pb.product_id = p.id AND pb.gtin = $1))
LIMIT 1`, gtin)
p, attrs, err := scanProduct(row)
if err != nil {
return nil, err
}
if p.Barcodes, err = s.ProductBarcodes(ctx, p.ID); err != nil {
return nil, err
}
if p.Specs, err = s.buildSpecs(ctx, p.ArchiveKind, attrs); err != nil {
return nil, err
}
if p.MSRP, err = s.ListMSRP(ctx, p.ID); err != nil {
return nil, err
}
s.cacheSet(ctx, suffix+gtin, p, productCacheTTL)
return p, nil
} }
// ProductByID looks up a product by its UUID. // ProductByID looks up a product by its UUID.
func (s *Store) ProductByID(ctx context.Context, id string) (*Product, error) { func (s *Store) ProductByID(ctx context.Context, id string) (*Product, error) {
const suffix = "prod:id:"
if cached := new(Product); s.cacheGet(ctx, suffix+id, cached) {
return cached, nil
}
row := s.pool.QueryRow(ctx, productSelect+" WHERE p.id = $1", id) row := s.pool.QueryRow(ctx, productSelect+" WHERE p.id = $1", id)
return scanProduct(row) p, attrs, err := scanProduct(row)
if err != nil {
return nil, err
}
if p.Barcodes, err = s.ProductBarcodes(ctx, p.ID); err != nil {
return nil, err
}
if p.Specs, err = s.buildSpecs(ctx, p.ArchiveKind, attrs); err != nil {
return nil, err
}
if p.MSRP, err = s.ListMSRP(ctx, p.ID); err != nil {
return nil, err
}
s.cacheSet(ctx, suffix+id, p, productCacheTTL)
return p, nil
} }
// SearchProducts performs a fuzzy name search with optional category subtree filter. // fuzzyThreshold is the minimum word_similarity for a name to be considered a
func (s *Store) SearchProducts(ctx context.Context, q, category string, limit, offset int) ([]ProductSummary, int, error) { // fuzzy match. ~0.42 tolerates common typos (e.g. "choclate"→"Chocolate")
args := []any{} // without returning unrelated products.
where := "WHERE p.status = 'active'" const fuzzyThreshold = "0.42"
if q != "" {
args = append(args, q) // SearchProducts runs a trigram-fuzzy name search with optional category /
where += " AND p.name ILIKE '%' || $1 || '%'" // brand / country filters. When a query is present, matching is inclusive
} // (substring OR trigram-similar OR barcode), and results are ranked by name
if category != "" { // similarity blended with quality_score so the best, most-complete records
args = append(args, category) // surface first. Without a query, results are ordered by quality_score.
where += " AND c.path <@ $" + strconv.Itoa(len(args)) + "::ltree" func (s *Store) SearchProducts(ctx context.Context, f SearchFilters, limit, offset int) ([]ProductSummary, int, error) {
suffix := searchCacheSuffix(f, limit, offset)
if entry := new(searchCacheEntry); s.cacheGet(ctx, suffix, entry) {
return entry.Items, entry.Total, nil
} }
countSQL := "SELECT count(*) FROM product p LEFT JOIN category c ON c.id = p.category_id " + where args := []any{}
where := "WHERE p.status = 'active'"
qIdx := 0
if f.Query != "" {
args = append(args, f.Query)
qIdx = len(args)
q := "$" + strconv.Itoa(qIdx)
where += ` AND (p.name ILIKE '%' || ` + q + ` || '%'
OR word_similarity(` + q + `, p.name) >= ` + fuzzyThreshold + `
OR EXISTS (SELECT 1 FROM product_barcode pb
WHERE pb.product_id = p.id AND pb.gtin ILIKE '%' || ` + q + ` || '%'))`
}
if f.Category != "" {
args = append(args, f.Category)
where += " AND c.path <@ $" + strconv.Itoa(len(args)) + "::ltree"
}
if f.Brand != "" {
args = append(args, f.Brand)
where += " AND b.name ILIKE '%' || $" + strconv.Itoa(len(args)) + " || '%'"
}
if f.Country != "" {
args = append(args, f.Country)
where += " AND p.country_of_origin ILIKE $" + strconv.Itoa(len(args)) + " || '%'"
}
from := `FROM product p
LEFT JOIN brand b ON b.id = p.brand_id
LEFT JOIN category c ON c.id = p.category_id `
var total int var total int
if err := s.pool.QueryRow(ctx, countSQL, args...).Scan(&total); err != nil { if err := s.pool.QueryRow(ctx, "SELECT count(*) "+from+where, args...).Scan(&total); err != nil {
return nil, 0, err return nil, 0, err
} }
// Ranking: when querying, similarity drives order, multiplied by a
// quality factor floored at 0.5 so low-quality records aren't zeroed out.
scoreExpr := "NULL::real"
orderBy := "p.quality_score DESC, p.name"
if f.Query != "" {
q := "$" + strconv.Itoa(qIdx)
scoreExpr = "word_similarity(" + q + ", p.name)"
orderBy = scoreExpr + " * (0.5 + p.quality_score) DESC, p.quality_score DESC, p.name"
}
args = append(args, limit, offset) args = append(args, limit, offset)
listSQL := ` listSQL := "SELECT p.id, p.gtin, p.name, b.name, c.path::text, p.country_of_origin, p.quality_score, " +
SELECT p.id, p.gtin, p.name, b.name, c.path::text scoreExpr + " AS score " + from + where +
FROM product p " ORDER BY " + orderBy +
LEFT JOIN brand b ON b.id = p.brand_id " LIMIT $" + strconv.Itoa(len(args)-1) + " OFFSET $" + strconv.Itoa(len(args))
LEFT JOIN category c ON c.id = p.category_id ` + where +
" ORDER BY p.name LIMIT $" + strconv.Itoa(len(args)-1) + " OFFSET $" + strconv.Itoa(len(args))
rows, err := s.pool.Query(ctx, listSQL, args...) rows, err := s.pool.Query(ctx, listSQL, args...)
if err != nil { if err != nil {
@@ -134,12 +388,34 @@ LEFT JOIN category c ON c.id = p.category_id ` + where +
out := []ProductSummary{} out := []ProductSummary{}
for rows.Next() { for rows.Next() {
var ps ProductSummary var ps ProductSummary
if err := rows.Scan(&ps.ID, &ps.GTIN, &ps.Name, &ps.Brand, &ps.CategoryPath); err != nil { if err := rows.Scan(&ps.ID, &ps.GTIN, &ps.Name, &ps.Brand, &ps.CategoryPath,
&ps.Country, &ps.QualityScore, &ps.Score); err != nil {
return nil, 0, err return nil, 0, err
} }
out = append(out, ps) out = append(out, ps)
} }
return out, total, rows.Err() if err := rows.Err(); err != nil {
return nil, 0, err
}
s.cacheSet(ctx, suffix, searchCacheEntry{Items: out, Total: total}, searchCacheTTL)
return out, total, nil
}
// searchCacheEntry is the cached payload for a SearchProducts call.
type searchCacheEntry struct {
Items []ProductSummary `json:"items"`
Total int `json:"total"`
}
// searchCacheSuffix derives a stable cache key from the full filter set and
// paging window so distinct queries never collide.
func searchCacheSuffix(f SearchFilters, limit, offset int) string {
raw := strings.Join([]string{
f.Query, f.Category, f.Brand, f.Country,
strconv.Itoa(limit), strconv.Itoa(offset),
}, "\x1f")
sum := sha1.Sum([]byte(raw))
return "search:" + hex.EncodeToString(sum[:])
} }
// Nutriments returns just the nutrition payload for a product. // Nutriments returns just the nutrition payload for a product.
@@ -176,11 +452,12 @@ type MSRP struct {
Note *string `json:"note"` Note *string `json:"note"`
} }
// ListMSRP returns all MSRP snapshots for a product. // ListMSRP returns a product's suggested-retail-price snapshots, newest first.
// Zero-amount entries are excluded as they carry no price information.
func (s *Store) ListMSRP(ctx context.Context, id string) ([]MSRP, error) { func (s *Store) ListMSRP(ctx context.Context, id string) ([]MSRP, error) {
rows, err := s.pool.Query(ctx, rows, err := s.pool.Query(ctx,
`SELECT amount, currency, region, effective_date::text, source_url, note `SELECT amount, currency, region, effective_date::text, source_url, note
FROM product_msrp WHERE product_id = $1 ORDER BY effective_date DESC NULLS LAST`, id) FROM product_msrp WHERE product_id = $1 AND amount > 0 ORDER BY effective_date DESC NULLS LAST`, id)
if err != nil { if err != nil {
return nil, err return nil, err
} }
@@ -232,12 +509,13 @@ type Category struct {
Path string `json:"path"` Path string `json:"path"`
GPCBrickCode *string `json:"gpc_brick_code"` GPCBrickCode *string `json:"gpc_brick_code"`
Level int `json:"level"` Level int `json:"level"`
ArchiveKind string `json:"archive_kind"`
} }
// ListCategories returns the full category tree ordered by path. // ListCategories returns the full category tree ordered by path.
func (s *Store) ListCategories(ctx context.Context) ([]Category, error) { func (s *Store) ListCategories(ctx context.Context) ([]Category, error) {
rows, err := s.pool.Query(ctx, rows, err := s.pool.Query(ctx,
"SELECT id, name_zh, name_en, path::text, gpc_brick_code, level FROM category ORDER BY path") "SELECT id, name_zh, name_en, path::text, gpc_brick_code, level, COALESCE(archive_kind, 'generic') FROM category ORDER BY path")
if err != nil { if err != nil {
return nil, err return nil, err
} }
@@ -245,7 +523,7 @@ func (s *Store) ListCategories(ctx context.Context) ([]Category, error) {
out := []Category{} out := []Category{}
for rows.Next() { for rows.Next() {
var c Category var c Category
if err := rows.Scan(&c.ID, &c.NameZH, &c.NameEN, &c.Path, &c.GPCBrickCode, &c.Level); err != nil { if err := rows.Scan(&c.ID, &c.NameZH, &c.NameEN, &c.Path, &c.GPCBrickCode, &c.Level, &c.ArchiveKind); err != nil {
return nil, err return nil, err
} }
out = append(out, c) out = append(out, c)
@@ -253,6 +531,70 @@ func (s *Store) ListCategories(ctx context.Context) ([]Category, error) {
return out, rows.Err() return out, rows.Err()
} }
// KindField describes one editable spec field for an archive kind. It drives
// the dynamic contribution form (public, read-only view of the template).
type KindField struct {
Kind string `json:"kind"`
FieldKey string `json:"field_key"`
GroupLabel string `json:"group_label"`
LabelZH string `json:"label_zh"`
FieldType string `json:"field_type"`
Unit *string `json:"unit"`
Options []string `json:"options"`
Placeholder *string `json:"placeholder"`
SortOrder int `json:"sort_order"`
Qualified bool `json:"qualified"`
}
// ListKindFields returns the ordered field template for one archive kind so the
// public contribution form can render kind-specific inputs.
func (s *Store) ListKindFields(ctx context.Context, kind string) ([]KindField, error) {
rows, err := s.pool.Query(ctx, `
SELECT kind, field_key, group_label, label_zh, field_type, unit, options,
placeholder, sort_order, qualified
FROM kind_field WHERE kind = $1 ORDER BY sort_order, field_key`, kind)
if err != nil {
return nil, err
}
defer rows.Close()
out := []KindField{}
for rows.Next() {
var f KindField
if err := rows.Scan(&f.Kind, &f.FieldKey, &f.GroupLabel, &f.LabelZH,
&f.FieldType, &f.Unit, &f.Options, &f.Placeholder, &f.SortOrder,
&f.Qualified); err != nil {
return nil, err
}
out = append(out, f)
}
return out, rows.Err()
}
// APIKey is the minimal metadata the public API needs to authorize a caller.
type APIKey struct {
ID string
Name string
RateLimitPerMin int
QuotaTotal int64
}
// APIKeyByHash returns the active (non-revoked) key matching a SHA-256 hash,
// or ErrNotFound if no such active key exists.
func (s *Store) APIKeyByHash(ctx context.Context, hash string) (*APIKey, error) {
var k APIKey
err := s.pool.QueryRow(ctx,
`SELECT id, name, rate_limit_per_min, quota_total
FROM api_key WHERE key_hash = $1 AND revoked_at IS NULL`, hash,
).Scan(&k.ID, &k.Name, &k.RateLimitPerMin, &k.QuotaTotal)
if errors.Is(err, pgx.ErrNoRows) {
return nil, ErrNotFound
}
if err != nil {
return nil, err
}
return &k, nil
}
// Source describes a data source with its license and trust weight. // Source describes a data source with its license and trust weight.
type Source struct { type Source struct {
ID string `json:"id"` ID string `json:"id"`
+79
View File
@@ -0,0 +1,79 @@
name: goods
services:
postgres:
image: postgres:16-alpine
restart: unless-stopped
environment:
POSTGRES_USER: ${POSTGRES_USER}
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
POSTGRES_DB: ${POSTGRES_DB}
volumes:
- pgdata:/var/lib/postgresql/data
healthcheck:
test: ["CMD-SHELL", "pg_isready -U ${POSTGRES_USER}"]
interval: 5s
timeout: 5s
retries: 10
redis:
image: redis:7-alpine
restart: unless-stopped
healthcheck:
test: ["CMD", "redis-cli", "ping"]
interval: 5s
timeout: 5s
retries: 10
minio:
image: minio/minio:latest
restart: unless-stopped
command: server /data --console-address ":9001"
environment:
MINIO_ROOT_USER: ${MINIO_ROOT_USER}
MINIO_ROOT_PASSWORD: ${MINIO_ROOT_PASSWORD}
volumes:
- miniodata:/data
healthcheck:
test: ["CMD", "mc", "ready", "local"]
interval: 5s
timeout: 5s
retries: 10
api:
build:
context: .
dockerfile: api/Dockerfile.prod
restart: unless-stopped
depends_on:
postgres:
condition: service_healthy
redis:
condition: service_healthy
environment:
OPENGOODS_ADDR: ":8080"
OPENGOODS_DATABASE_URL: "postgres://${POSTGRES_USER}:${POSTGRES_PASSWORD}@postgres:5432/${POSTGRES_DB}?sslmode=disable"
OPENGOODS_REDIS_URL: "redis://redis:6379/0"
ports:
- "127.0.0.1:8120:8080"
admin:
build:
context: .
dockerfile: api/Dockerfile.admin
restart: unless-stopped
depends_on:
postgres:
condition: service_healthy
environment:
GOODS_ADMIN_ADDR: ":8080"
OPENGOODS_DATABASE_URL: "postgres://${POSTGRES_USER}:${POSTGRES_PASSWORD}@postgres:5432/${POSTGRES_DB}?sslmode=disable"
GOODS_ADMIN_BASE_PATH: "/ping"
GOODS_ADMIN_USER: "${GOODS_ADMIN_USER}"
GOODS_ADMIN_PASSWORD: "${GOODS_ADMIN_PASSWORD}"
GOODS_ADMIN_JWT_SECRET: "${GOODS_ADMIN_JWT_SECRET}"
ports:
- "127.0.0.1:8121:8080"
volumes:
pgdata:
miniodata:
+175
View File
@@ -0,0 +1,175 @@
# OpenGoods 公共 API 开发者文档
天工商品档案公共仓(OpenGoods)提供**公开、只读**的商品事实 REST API:按条码/名称查询商品的客观资料(品牌、品类、净含量、产地、配料、营养成分、Nutri-Score、厂商建议零售价快照等)。返回均为 JSON(UTF-8)。**本服务不含任何购买/交易接口。**
- 基础地址:`https://goods.tangshasha.com/api/v1`
- 机器可读规范(OpenAPI 3):`GET /api/v1/openapi.json`
- 交互式文档:站点「API 调用说明」页
## 鉴权
API 默认**匿名可用**,无需任何凭证即可调用。匿名请求按来源 IP 计入一个较低的默认每分钟额度。
如需更高额度并让用量归属到你,可向运营方申请一枚 **API Key**(形如 `og_live_xxxxxxxx`),请求时二选一携带:
```bash
curl -H "X-API-Key: og_live_xxxxxxxx" \
"https://goods.tangshasha.com/api/v1/products/search?q=牛奶"
# 或
curl -H "Authorization: Bearer og_live_xxxxxxxx" \
"https://goods.tangshasha.com/api/v1/products/search?q=牛奶"
```
> 仅在创建时返回一次明文 Key,请妥善保存。服务端只存储其 SHA-256 哈希。
## 限流
采用**固定窗口**限流(每分钟)。每个响应都会回写以下响应头:
| 响应头 | 含义 |
| --- | --- |
| `X-RateLimit-Limit` | 当前窗口允许的最大请求数 |
| `X-RateLimit-Remaining` | 当前窗口剩余可用次数 |
| `X-RateLimit-Reset` | 窗口重置的 Unix 时间戳(秒) |
| `Retry-After` | 仅在超额(429)时返回,建议等待的秒数 |
- 超过额度:`429 Too Many Requests`,错误码 `rate_limited`
- Key 无效或已吊销:`401 Unauthorized`,错误码 `invalid_api_key`
## 错误格式
非 2xx 响应体统一为:
```json
{ "error": { "code": "not_found", "message": "…", "request_id": "…" } }
```
## 分页
列表类接口支持 `page`(默认 `1`)与 `size`(默认 `20`,最大 `100`),响应含 `page`/`size`/`total`
## 端点
### `GET /products/search` — 搜索商品
按名称做三元组(trigram)模糊搜索,**可容忍错别字**;支持品类/品牌/产地过滤;结果按相关度(名称相似度 × 数据质量分)排序。
| 参数 | 必填 | 说明 |
| --- | --- | --- |
| `q` | 否 | 关键词(名称/条码),模糊匹配;留空则按质量分返回全部 |
| `category` | 否 | 品类编码(含子树),如 `food.beverages` |
| `brand` | 否 | 品牌名(模糊匹配),如 `Ferrero` |
| `country` | 否 | 产地前缀(不区分大小写),如 `China` |
| `page` | 否 | 页码,默认 1 |
| `size` | 否 | 每页条数,默认 20,最大 100 |
```bash
curl "https://goods.tangshasha.com/api/v1/products/search?q=nutela&country=Italy"
```
```json
{
"items": [
{
"id": "…",
"gtin": "3017624010701",
"name": "Nutella",
"brand": "Ferrero",
"category_path": "food.snacks.chocolate",
"country_of_origin": "Italy",
"quality_score": 0.81,
"score": 0.71
}
],
"page": 1,
"size": 20,
"total": 1
}
```
`score` 为名称相关度(提供 `q` 时返回,01),未提供 `q` 时为 `null`
### `GET /products/barcode/{gtin}` — 按条码查询
```bash
curl "https://goods.tangshasha.com/api/v1/products/barcode/5449000000996"
```
### `GET /products/{id}` — 商品详情
按商品 UUID 获取完整档案(含配料、营养、添加剂、图片、MSRP 等)。
### `GET /products/{id}/nutriments` — 商品营养成分
### `GET /products/{id}/msrp` — 厂商建议零售价快照
官方建议零售价历史快照,仅供参考,不含任何购买入口。
### `GET /brands` — 品牌列表(分页)
### `GET /categories` — 品类树
### `GET /sources/{id}` — 数据来源
## 档案回流(写接口,需 API Key)
> 仅供进销存等机器调用方使用:把档案里**尚未收录**的商品批量回流到站点,进入人工审核队列,审核通过后才会收录。**必须携带 API Key**(与上文同一类 `og_live_` 密钥),不会直接写入商品。
### `POST /api/public/backflow` — 批量回流未收录商品
- 鉴权:请求头携带 `X-API-Key: og_live_xxxxxxxx`(或 `Authorization: Bearer og_live_xxxxxxxx`)。缺失/无效/已吊销返回 `401`
- 请求体:商品对象**数组**(与公众投稿同结构),单次最多 `1000` 条。常用字段:
| 字段 | 必填 | 说明 |
| --- | --- | --- |
| `name` | 是 | 商品名称 |
| `gtin` | 否 | 条码(GTIN)。强烈建议提供,用于去重 |
| `brand_name` | 否 | 品牌名 |
| `category_id` | 否 | 品类编码,如 `food.beverages` |
| `net_content_value` / `net_content_unit` | 否 | 净含量数值 / 单位 |
| `country_of_origin` | 否 | 产地 |
| `ingredients_text` | 否 | 配料表 |
| `nutriments` | 否 | 营养成分对象 |
| `msrp` | 否 | 零售价快照数组,元素含 `amount`/`currency`/`region`/`effective_date` |
| `note` | 否 | 备注 |
> 来源会自动标记为 `source="backflow"`,在后台审核队列中与公众投稿区分,无需调用方提供。
- 去重(按 `gtin` 逐条判断,互不影响):
- 该条码已收录为商品 → `exists`,跳过;
- 已存在同条码的待审核回流 → `duplicate`,跳过(避免反复刷队列);
- 否则入队 → `queued`status=`pending`,等待后台审核);
- 名称为空等 → `invalid`
```bash
curl -X POST "https://goods.tangshasha.com/api/public/backflow" \
-H "X-API-Key: og_live_xxxxxxxx" \
-H "Content-Type: application/json" \
-d '[
{"name":"某某牛奶 250ml","gtin":"6901234567890","brand_name":"某品牌",
"net_content_value":250,"net_content_unit":"ml",
"msrp":[{"amount":3.5,"currency":"CNY","region":"CN"}]},
{"name":"已收录商品","gtin":"5449000000996"}
]'
```
```json
{
"total": 2,
"queued": 1,
"exists": 1,
"duplicate": 0,
"invalid": 0,
"results": [
{ "gtin": "6901234567890", "name": "某某牛奶 250ml", "status": "queued", "id": "<submission-id>" },
{ "gtin": "5449000000996", "name": "已收录商品", "status": "exists", "reason": "该条码商品已收录" }
]
}
```
审核通过后,系统按提交内容**新建商品**;若审核时该条码已存在商品,则**补全**到已有商品(逻辑与公众投稿一致)。
## 免责声明
数据可能存在误差或滞后,按「现状」提供,不构成医疗/购买建议。商品资料版权归各原始来源所有,请遵循其许可(如 OpenFoodFacts 的 ODbL),引用时请注明天工商品档案公共仓及原始来源。
+34
View File
@@ -0,0 +1,34 @@
# 生产部署 (Docker)
`docker-compose.prod.yml` 部署,与本地 `docker-compose.yml` 的区别:
-`api` 映射宿主端口,且绑定 `127.0.0.1:8120`(由外层 nginx 反代 + HTTPS);`postgres`/`redis`/`minio` 不对外暴露端口,仅容器内网互通。
- 所有服务 `restart: unless-stopped`
- 凭据从 `.env` 注入(见 `.env.example`),不写入仓库。
- `api` 使用 `api/Dockerfile.prod`:运行镜像用 `scratch`(从构建镜像拷贝 ca-certs),适用于 `gcr.io/distroless` 不可达的环境;Go 模块走 `goproxy.cn`
- `admin`(运营后台):带登录的写入服务 + 内嵌前端,绑定 `127.0.0.1:8121`,由 nginx 反代到公开站点的 `/ping` 路径。镜像 `api/Dockerfile.admin`(node 构建前端 → 内嵌进 Go 二进制 → scratch 运行)。仅 `admin` 可写库(人工编辑以 `source=manual` 记录字段级溯源 + `audit_log` 留痕),公开 `api` 仍只读。
## 步骤
```bash
cp .env.example .env # 填入真实随机密码
docker compose -f docker-compose.prod.yml up -d --build
# 迁移(migrate 容器接入同一网络,DSN 指向 postgres 服务)
set -a; . ./.env; set +a
DBURL="postgres://${POSTGRES_USER}:${POSTGRES_PASSWORD}@postgres:5432/${POSTGRES_DB}?sslmode=disable"
docker run --rm --network goods_default -v "$PWD/migrations:/migrations" \
migrate/migrate -path=/migrations -database "$DBURL" up
curl -s http://127.0.0.1:8120/healthz # {"status":"ok"}
```
nginx 反代(子域 + HTTPS):80 端口 301 跳转到 443443 `proxy_pass http://127.0.0.1:8120`,证书用 acme.sh 签发并配 `--reloadcmd "nginx -s reload"` 自动续期。
运营后台 `/ping`(同域复用证书):在 443 server 块内加一段
```nginx
location /ping { proxy_pass http://127.0.0.1:8121; }
```
后台凭据见 `.env``GOODS_ADMIN_USER` / `GOODS_ADMIN_PASSWORD` / `GOODS_ADMIN_JWT_SECRET`。新增迁移 `0005_admin``audit_log` 表 + `manual` 来源)随 `migrate ... up` 自动应用。
+76
View File
@@ -0,0 +1,76 @@
# 采集管理 (M4)
M4 在 M2Open Food Facts 首次导入)基础上,补齐"持续运营"所需的采集能力:
增量更新、第二数据源补全(GS1)、去重合并与字段级冲突解决、数据质量评分,
以及把这些串起来的定时调度。全部为 Python 侧(`ingestion/`),只写库、可单测。
## 组成
| 能力 | 模块 | 说明 |
|------|------|------|
| 增量采集 | `adapters/openfoodfacts.py: fetch_modified_since()` | 按 `last_modified_t` 拉取自上次水位后变更的商品 |
| 采集水位 | `etl/state.py` + `ingest_state` 表 | 每个源持久化 `last_modified_t`,只前进不回退 |
| GS1 补全 | `adapters/gs1.py` + `etl/supplement.py` | 用权威条码源补**缺失**字段(品牌/厂商/GPC/产地/净含量),不覆盖已有值 |
| 去重合并 | `etl/dedup.py` | 非 GTIN 重复(同名+品牌+净含量)合并到质量最高的主记录 |
| 冲突解决 | `etl/merge.py` | 多源同字段按"源权重 > 新鲜度"择优,保留字段级溯源 |
| 质量评分 | `etl/quality.py` | 0~1 分,落到 `product.quality_score` |
| 定时调度 | `jobs/schedule.py` | 固定周期跑"增量 + 去重"一轮,零额外依赖 |
## 质量评分
锁定公式(各分量均归一到 0~1):
```
quality = 0.4 * 完整度 + 0.3 * 源权重 + 0.2 * 多源一致 + 0.1 * 新鲜度
```
- **完整度**`name/gtin/brand/category/net_content/country/nutriments/ingredients/image` 9 项的命中比例。
- **源权重**:贡献该商品的源中最高 `source.trust_weight`OFF=0.7GS1=0.9)。
- **多源一致**:源数量代理——单源 0.5、两源 0.8、三源及以上 1.0(单源无法互证)。
- **新鲜度**:最近一次 `product_source.fetched_at` 的时间衰减(≤30d=1.0 … >730d=0.2)。
`load_record()``merge_products()` 写入后都会调 `update_quality()` 重算。
## 增量水位
`ingest_state`(迁移 `0004`)每源一行,记录 `last_modified_t``last_run_at``stats`
`set_watermark()``GREATEST(...)` 保证水位只前进,避免乱序/中断的运行回退进度。
## 运行
前置:`docker compose up -d postgres` 且迁移已 `up`(含 `0004`)。DSN 默认读 `OPENGOODS_DATABASE_URL`
```bash
# 增量更新 OFF(从持久化水位开始;--since 可覆盖)
python -m opengoods.jobs.update_off --max-pages 5
python -m opengoods.jobs.update_off --since 1700000000
# 去重合并(--dry-run 只报告不写库)
python -m opengoods.jobs.dedup --dry-run
python -m opengoods.jobs.dedup --actor nightly
# 定时调度:单轮 / 周期循环(增量 + 去重)
python -m opengoods.jobs.schedule --once
python -m opengoods.jobs.schedule --interval 3600
```
## GS1 补全
GS1 为付费、分区域的授权数据,适配器支持两种模式:
- **离线**(默认):从本地 JSON 映射 `{gtin: {...}}` 查(`GS1Adapter.from_file(path)`),
供测试与内网环境使用。
- **在线**:传 `base_url` + `client`+ `api_key`),`GET {base_url}/{gtin}`,按
Verified-by-GS1 风格字段解析。
补全只填**空缺**字段并在 `product_source` 记字段级溯源,源标记为 `gs1`
## 测试
```bash
cd ingestion && pip install -e ".[dev]"
ruff check . && ruff format --check . && pytest -q
```
纯函数测试(质量/冲突/增量分页)始终运行;依赖库的测试(水位/质量落库/去重/GS1 补全)
在无数据库或未应用 M4 迁移时自动跳过。
+112
View File
@@ -0,0 +1,112 @@
# 可扩展性设计与路线图 (Scalability Roadmap)
本文档回答一个长期问题:随着商品越来越多、品类越来越杂(食品 / 电子 3C / 药品 /
……),**检索与新增会不会压垮数据库?要不要按品类「分表」?**
> 结论先行:**现阶段不要按品类手动分表。** 现有「单表 + JSONB + archive_kind 框架」
> 的设计方向是对的。扩展应当靠 **分区(非分表) + 读副本 + 缓存 + 专用搜索引擎**,
> 按数据量分阶段推进,避免提前过度设计。
---
## 1. 现状盘点
### 1.1 数据模型
- 所有商品落在**一张 `product` 表**;非食品的领域字段存 `product.attributes`(JSONB)。
- 食品有独立明细表 `food_detail`(配料 / 营养 / 过敏原等结构化字段)。
- `0010_archive_kinds` 引入 **archive_kind 框架**:每个品类带一个 `archive_kind`
(`food` / `electronics` / `generic`),`kind_field` 表按 kind 定义字段模板,
驱动后台动态表单与合格度评分。
- **加新品类(如药品)不需要新建表**:只要新增一组 `kind_field` 行 + 一棵品类子树;
仅当某品类有大量需被独立筛选/排序的结构化字段时,才考虑像 `food_detail` 那样补一张
明细表。
### 1.2 已有索引(检索性能的基础)
| 对象 | 索引 | 用途 |
|------|------|------|
| `product.gtin` | 唯一索引 | 条码精确查 |
| `product.name` | trigram GIN (`pg_trgm`) | 名称模糊/相似匹配 |
| `product.search_tsv` | 全文 GIN (`tsvector`) | 全文检索 |
| `product.attributes` | JSONB GIN | 属性过滤 |
| `brand.name` | trigram GIN | 品牌模糊匹配 |
| `product.country_of_origin` | btree | 产地精确/前缀过滤 |
| category / brand / updated_at | btree | 关联与增量 |
### 1.3 检索方式
- 有关键词时:`name ILIKE` `word_similarity ≥ 阈值(~0.42)` 条码匹配,
排序按 `相似度 × (0.5 + quality_score)`
- 无关键词时:按 `quality_score` 排序。
- 翻页:`LIMIT / OFFSET`
### 1.4 写入特征
- 写库**只有** Python ingestion 一条路径(批量 ETL),不是高并发 OLTP。
- 插入瓶颈极低;`search_tsv` 由触发器逐行重算,正常增量下开销可忽略。
---
## 2. 为什么不建议按品类「分表」
1. **核心场景是全局检索**:用户通常不知道商品属于哪个品类,搜索要跨所有品类。
按品类拆成多表后,一次搜索得 `UNION ALL` 所有表,**更慢、代码更复杂、排序更难统一**。
2. **单表足够能打**:Postgres 单表配好索引,**几千万行**量级的检索完全可承载。
"表大"很少是真正瓶颈,"搜索方式"和"读并发"才是。
3. **分表会侵蚀框架优势**:archive_kind 框架的价值就是"加品类零建表";手动分表等于
把这套通用能力又拆碎。
> 区分两个概念:**分表(sharding,应用层拆多张表)** ≠ **分区(Postgres 原生
> declarative partitioning,对上层透明的一张逻辑表)**。后者在超大规模时才有意义,
> 见第 3 阶段。
---
## 3. 分阶段路线图(按数据量触发,不提前做)
### 阶段 0 — 现在 ~ 数百万条:维持现状 + 低成本优化
触发:当前规模。改动小、收益稳,建议尽早做:
- **深翻页改 keyset 分页**:`OFFSET` 越翻越慢(需扫描并丢弃前 N 行);改用
`WHERE (score, id) < (:last_score, :last_id)` 形式的游标分页。
- **部分索引**:绝大多数查询限定 `status='active'`,可建
`CREATE INDEX ... WHERE status='active'` 缩小索引、提速。
- **常用筛选复合索引**:如 `(category_id, quality_score DESC)`
`(archive_kind, quality_score DESC)` 配合域内列表。
- **Redis 缓存**(已在技术栈内):缓存热门搜索结果与商品详情,挡住重复读。
### 阶段 1 — 千万级以上:读扩展 + 调优
触发:单库读 QPS 升高、P99 变慢。
- **只读副本(read replica)**:本服务是**只读公益 API**,天然适合一主多从,
把检索/详情读流量分到副本,主库只承接 ingestion 写入。
- **索引与查询调优**:按慢查询日志补/删索引,`EXPLAIN ANALYZE` 校核计划。
- **(可选)Postgres 原生分区**:若多数检索"限定单一域"(只搜药品 / 只搜食品),
可按 `archive_kind`**LIST 分区**(对上层透明,仍是一张逻辑表)。主要利好
**维护**(分区级 vacuum / 归档)与**域内查询裁剪**;对真正的全局搜索帮助有限。
### 阶段 2 — 搜索相关性/规模成为痛点:引入专用搜索引擎
触发:`pg_trgm`/`tsvector` 在相关性排序、跨字段检索、规模上吃力。
- 把**检索**迁到专用倒排引擎:**OpenSearch / Meilisearch / Typesense**,
或 Postgres 内的 **ParadeDB(`pg_search`,BM25)**
- **Postgres 仍是唯一事实来源**;搜索引擎只做索引,由 ingestion 在写库后同步。
- 这才是"搜索量大"的正解,**比分表有效得多**。
---
## 4. 大批量导入的建议
- 海量初始化/回填用 `COPY` 而非逐行 `INSERT`
- 超大批量时可"先停建二级索引 → COPY → 重建索引",比边插边维护索引快得多。
- ETL 控制并发与批大小,避免与在线读争抢。
---
## 5. 药品档案怎么落地(回到最初的问题)
在上述设计下,加"药品"属于**阶段 0 的常规扩展**,不触动架构:
1. 新增 `drug``kind_field` 模板(批准文号 / 通用名 / 商品名 / 剂型 / 规格 /
生产企业 / OTC 分类 / 适应症 / 用法用量 / 不良反应 / 禁忌 / 注意事项 / 贮藏 /
有效期 等),`qualified` 标记关键字段参与合格度评分。
2. 加一棵药品品类子树,并把这些品类的 `archive_kind` 置为 `drug`
3. 仅当药品需要**被独立筛选/排序的强结构化字段**(如按批准文号精确查、按 OTC 分类
过滤)时,才考虑补一张 `drug_detail` 明细表;否则继续走 `attributes` JSONB。
---
## 6. 版本
- 本路线图随规模演进更新;任何落地改动需同步:迁移(SQL) + 本文档 +(涉及对外字段时)
`docs/data-contract.md` / `docs/openapi.yaml`
+116
View File
@@ -0,0 +1,116 @@
"""Adapter for the bypos-collector output (central product library zc.bypos.net).
The ``bypos-collector`` tool (see ``tools/bypos-collector``) queries the same
central product library that the 云店 POS uses when adding a product by barcode,
and writes one JSON object per line (JSONL) with these fields::
barcode name spec unit area manufacturer license
in_price sell_price status retmsg fetched_at source
This module turns one such record into the internal product shape consumed by
:func:`opengoods.etl.load.load_bypos_record`. It is a pure function (no DB, no
network) so it is easy to unit-test.
The central library is a Chinese retail catalogue: it provides 品名/规格/单位/
产地/厂商/建议价 but **not** ingredients or nutrition, so no ``food`` block is
produced.
"""
from __future__ import annotations
from decimal import Decimal, InvalidOperation
from opengoods.etl.transform import _clamp, is_valid_gtin, parse_quantity
from opengoods.units import UnitError, normalize
SOURCE_NAME = "bypos中心库"
SOURCE_HOMEPAGE = "https://zc.bypos.net"
# Vendor catalogue data; not an open-licensed dataset. Display facts only.
SOURCE_LICENSE = "proprietary"
SOURCE_TRUST = 0.6
def _to_price(text: str | None) -> Decimal | None:
"""Parse a price string to a positive Decimal, or None for empty/zero."""
if not text:
return None
try:
amount = Decimal(str(text).strip())
except (InvalidOperation, ValueError):
return None
if amount <= 0:
return None
return amount
def _net_content(spec: str | None) -> tuple[Decimal | None, str | None, Decimal | None]:
"""Best-effort parse a spec like '500mL'/'5kg' to (value, unit, canonical).
Packaging-style specs ('20支', '1X24', '') have no mass/volume unit and
yield ``(None, None, None)`` — the raw spec is kept in attributes instead.
"""
parsed = parse_quantity(spec or "")
if not parsed:
return None, None, None
value, unit = parsed
try:
norm = normalize(value, unit)
except UnitError:
return None, None, None
return norm.value, norm.unit, norm.canonical_value
def transform_bypos(rec: dict) -> dict | None:
"""Transform one bypos-collector JSONL record into an internal product dict.
Returns ``None`` for non-hit rows or rows without a usable name.
"""
if rec.get("status") != "hit":
return None
name = (rec.get("name") or "").strip()
if not name:
return None
barcode = str(rec.get("barcode") or "").strip()
gtin = barcode if barcode and is_valid_gtin(barcode) else None
net_value, net_unit, net_canonical = _net_content(rec.get("spec"))
spec = (rec.get("spec") or "").strip()
pack_unit = (rec.get("unit") or "").strip()
area = (rec.get("area") or "").strip()
manufacturer = (rec.get("manufacturer") or "").strip() or None
license_no = (rec.get("license") or "").strip()
in_price = _to_price(rec.get("in_price"))
sell_price = _to_price(rec.get("sell_price"))
attributes: dict[str, object] = {}
if spec:
attributes["spec"] = spec
if pack_unit:
attributes["pack_unit"] = pack_unit
if area:
attributes["origin_area"] = area
if license_no:
attributes["production_license"] = license_no
if in_price is not None:
attributes["suggested_in_price"] = float(in_price)
if sell_price is not None:
attributes["suggested_retail_price"] = float(sell_price)
# Domestic GS1-China barcodes (69x) are China-made; area is a province/city,
# kept separately in attributes.origin_area.
country = "中国" if gtin and gtin.startswith("69") else None
return {
"gtin": gtin,
"name": name,
"manufacturer": manufacturer,
"net_content_value": net_value,
"net_content_unit": _clamp(net_unit, 16),
"net_content_canonical": net_canonical,
"country_of_origin": country,
"attributes": attributes,
"msrp": sell_price,
"fetched_at": (rec.get("fetched_at") or "").strip() or None,
}
+124
View File
@@ -0,0 +1,124 @@
"""GS1 barcode supplement adapter.
GS1 (e.g. *Verified by GS1* / GS1 China) is the authoritative registry that maps
a GTIN to its brand owner, product description and GPC category. We use it to
*supplement* — fill gaps in — records gathered from crowd sources like Open Food
Facts, never to overwrite existing values.
Real GS1 access is credentialed and region-specific, so this adapter supports
two modes:
* **offline** (default): look barcodes up in a local JSON mapping file. This is
what tests and air-gapped runs use.
* **online**: GET ``{base_url}/{gtin}`` with an API key header, then normalize
the response. Enabled by passing ``base_url`` + ``client``.
Either way :meth:`fetch_barcode` returns a normalized *supplement* dict (or
``None``); :mod:`opengoods.etl.supplement` applies it to the database.
"""
from __future__ import annotations
import json
from collections.abc import Iterator
from pathlib import Path
import httpx
SOURCE_NAME = "gs1"
GS1_HOMEPAGE = "https://www.gs1.org"
GS1_LICENSE = "proprietary"
# GS1 is the authoritative barcode registry -> high trust.
GS1_TRUST = 0.9
# Keys of a normalized supplement record.
_SUPPLEMENT_KEYS = (
"gtin",
"name",
"brand",
"manufacturer",
"gpc_brick_code",
"country_of_origin",
"net_content_value",
"net_content_unit",
)
def _normalize(code: str, data: dict) -> dict:
"""Project a raw mapping/record onto the supplement schema (non-empty only)."""
rec: dict = {"gtin": code}
for key in _SUPPLEMENT_KEYS:
if key == "gtin":
continue
value = data.get(key)
if value not in (None, "", []):
rec[key] = value
return rec
def _parse_api(code: str, payload: dict) -> dict:
"""Best-effort mapping of a Verified-by-GS1 style payload to our schema."""
item = payload
if isinstance(payload.get("gtinRecords"), list) and payload["gtinRecords"]:
item = payload["gtinRecords"][0]
return _normalize(
code,
{
"name": item.get("productDescription") or item.get("description"),
"brand": item.get("brandName"),
"manufacturer": item.get("companyName") or item.get("licenseeName"),
"gpc_brick_code": item.get("gpcCategoryCode"),
"country_of_origin": item.get("countryOfSaleCode") or item.get("countryCode"),
"net_content_value": item.get("netContent"),
"net_content_unit": item.get("netContentUnit"),
},
)
class GS1Adapter:
"""Look up GTIN supplements from a local mapping or a GS1-style API."""
source_name = SOURCE_NAME
def __init__(
self,
mapping: dict | None = None,
*,
client: httpx.Client | None = None,
base_url: str | None = None,
api_key: str | None = None,
) -> None:
self._mapping = mapping or {}
self._client = client
self._base_url = base_url.rstrip("/") if base_url else None
self._api_key = api_key
@classmethod
def from_file(cls, path: str | Path) -> GS1Adapter:
"""Build an offline adapter from a JSON ``{gtin: {...}}`` mapping file."""
data = json.loads(Path(path).read_text(encoding="utf-8"))
return cls(mapping=data)
def fetch_barcode(self, code: str) -> dict | None:
"""Return a normalized supplement dict for ``code`` (or ``None``)."""
if self._base_url and self._client is not None:
headers = {"apikey": self._api_key} if self._api_key else {}
resp = self._client.get(f"{self._base_url}/{code}", headers=headers)
if resp.status_code == 404:
return None
resp.raise_for_status()
rec = _parse_api(code, resp.json())
else:
data = self._mapping.get(code)
if not data:
return None
rec = _normalize(code, data)
# A record with only the GTIN carries no supplement.
return rec if len(rec) > 1 else None
def fetch(self, barcodes: list[str]) -> Iterator[dict]:
"""Yield supplement records for the given barcodes."""
for code in barcodes:
rec = self.fetch_barcode(code)
if rec is not None:
yield rec
+149 -3
View File
@@ -25,6 +25,19 @@ USER_AGENT = "OpenGoods/0.1 (+https://github.com/baicai2026-baicai/goods) public
# Conservative client-side spacing between API calls (seconds). # Conservative client-side spacing between API calls (seconds).
_DEFAULT_MIN_INTERVAL = 4.0 _DEFAULT_MIN_INTERVAL = 4.0
_API_URL = "https://world.openfoodfacts.org/api/v2/product/{barcode}.json" _API_URL = "https://world.openfoodfacts.org/api/v2/product/{barcode}.json"
_SEARCH_URL = "https://world.openfoodfacts.org/api/v2/search"
# HTTP statuses worth retrying: rate limiting and transient server errors.
_RETRY_STATUS = frozenset({429, 500, 502, 503, 504})
# Fields requested from the search API so a returned product can be transformed
# without an extra per-barcode round trip.
_SEARCH_FIELDS = (
"code,product_name,product_name_en,product_name_zh,brands,quantity,"
"categories,categories_tags,countries,ingredients_text,allergens_tags,"
"additives_tags,nutriments,nutriscore_grade,serving_size,"
"image_front_url,image_url,last_modified_t"
)
class OpenFoodFactsAdapter: class OpenFoodFactsAdapter:
@@ -36,9 +49,13 @@ class OpenFoodFactsAdapter:
self, self,
client: httpx.Client | None = None, client: httpx.Client | None = None,
min_interval: float = _DEFAULT_MIN_INTERVAL, min_interval: float = _DEFAULT_MIN_INTERVAL,
max_retries: int = 4,
backoff_base: float = 2.0,
) -> None: ) -> None:
self._client = client or httpx.Client(headers={"User-Agent": USER_AGENT}, timeout=30.0) self._client = client or httpx.Client(headers={"User-Agent": USER_AGENT}, timeout=30.0)
self._min_interval = min_interval self._min_interval = min_interval
self._max_retries = max_retries
self._backoff_base = backoff_base
self._last_call = 0.0 self._last_call = 0.0
def _throttle(self) -> None: def _throttle(self) -> None:
@@ -48,11 +65,49 @@ class OpenFoodFactsAdapter:
time.sleep(wait) time.sleep(wait)
self._last_call = time.monotonic() self._last_call = time.monotonic()
def _get(self, url: str, params: dict | None = None) -> httpx.Response:
"""GET with throttling and retry/backoff on transient errors.
Retries on connection/timeout errors and on retryable HTTP statuses
(429 and 5xx, which OFF returns intermittently when overloaded), using
exponential backoff that honours a ``Retry-After`` header when present.
"""
last_exc: Exception | None = None
for attempt in range(self._max_retries + 1):
self._throttle()
try:
resp = self._client.get(url, params=params)
except httpx.TransportError as exc:
last_exc = exc
else:
if resp.status_code < 400 or resp.status_code not in _RETRY_STATUS:
resp.raise_for_status()
return resp
last_exc = httpx.HTTPStatusError(
f"retryable status {resp.status_code}", request=resp.request, response=resp
)
if attempt < self._max_retries:
retry_after = self._retry_after(last_exc)
time.sleep(retry_after if retry_after is not None else self._backoff_base**attempt)
assert last_exc is not None
raise last_exc
@staticmethod
def _retry_after(exc: Exception | None) -> float | None:
resp = getattr(exc, "response", None)
if resp is None:
return None
value = resp.headers.get("Retry-After")
if not value:
return None
try:
return float(value)
except ValueError:
return None
def fetch_barcode(self, barcode: str) -> dict | None: def fetch_barcode(self, barcode: str) -> dict | None:
"""Fetch a single product by barcode; return the raw `product` dict.""" """Fetch a single product by barcode; return the raw `product` dict."""
self._throttle() resp = self._get(_API_URL.format(barcode=barcode))
resp = self._client.get(_API_URL.format(barcode=barcode))
resp.raise_for_status()
payload = resp.json() payload = resp.json()
if payload.get("status") != 1: if payload.get("status") != 1:
return None return None
@@ -65,6 +120,97 @@ class OpenFoodFactsAdapter:
if record is not None: if record is not None:
yield record yield record
def fetch_modified_since(
self,
since_t: int,
*,
page_size: int = 100,
max_pages: int = 10,
) -> Iterator[dict]:
"""Yield products modified after ``since_t`` (unix ``last_modified_t``).
Uses the OFF search API sorted by ``last_modified_t`` (most recent
first) and paginates until it reaches products at or before the
watermark, an empty/short page, or ``max_pages``. This is the
incremental ingestion path: callers persist the highest
``last_modified_t`` they processed as the next watermark.
"""
for page in range(1, max_pages + 1):
resp = self._get(
_SEARCH_URL,
params={
"fields": _SEARCH_FIELDS,
"sort_by": "last_modified_t",
"page": page,
"page_size": page_size,
},
)
products = resp.json().get("products") or []
if not products:
return
reached_old = False
for prod in products:
if int(prod.get("last_modified_t") or 0) <= since_t:
reached_old = True
break
yield prod
if reached_old or len(products) < page_size:
return
def fetch_by_country(
self,
country: str,
*,
page_size: int = 100,
max_pages: int = 10,
sort_by: str = "unique_scans_n",
) -> Iterator[dict]:
"""Yield products sold in ``country`` (an OFF ``countries_tags_en`` slug).
Used to seed a market-specific catalogue (e.g. ``china``). Results are
sorted by ``sort_by`` (default ``unique_scans_n`` so the most-scanned,
best-known products come first) and de-duplicated across pages, since
OFF's popularity ordering is not stable between page requests.
"""
seen: set[str] = set()
for page in range(1, max_pages + 1):
resp = self._get(
_SEARCH_URL,
params={
"fields": _SEARCH_FIELDS,
"countries_tags_en": country,
"sort_by": sort_by,
"page": page,
"page_size": page_size,
},
)
products = resp.json().get("products") or []
if not products:
return
new_on_page = 0
for prod in products:
code = str(prod.get("code") or "")
if code and code in seen:
continue
if code:
seen.add(code)
new_on_page += 1
yield prod
if len(products) < page_size or new_on_page == 0:
return
def is_cn_gs1(code: str | None) -> bool:
"""Return True for a GS1 China company prefix (barcodes starting 690-699).
These identify products registered with GS1 China, i.e. genuinely domestic
items, as opposed to imported goods merely tagged as sold in China.
"""
if not code:
return False
code = code.strip()
return len(code) >= 3 and code[:2] == "69" and code[2].isdigit()
def read_dump(path: str | Path) -> Iterator[dict]: def read_dump(path: str | Path) -> Iterator[dict]:
"""Yield raw product records from an OFF JSONL dump file. """Yield raw product records from an OFF JSONL dump file.
+52
View File
@@ -0,0 +1,52 @@
"""Read-cache invalidation signal for the public API.
The Go API caches hot product details and search results in Redis, namespacing
every key by a global generation counter (``og:cache:epoch``). Bumping that
counter logically invalidates the entire cache in one O(1) operation while the
old keys age out via their TTL.
Ingestion is the only writer to the database, so after a run that changed data
it calls :func:`bump_cache_epoch` to make those changes visible immediately
instead of waiting for per-key TTLs to expire.
Like the API's cache, this is strictly best effort and fails open: if Redis is
unconfigured or unreachable the ingestion run still succeeds, and stale entries
simply expire on their own.
"""
from __future__ import annotations
import logging
import os
logger = logging.getLogger(__name__)
EPOCH_KEY = "og:cache:epoch"
def default_redis_url() -> str | None:
"""Return the configured Redis URL, or ``None`` when caching is disabled."""
return os.environ.get("OPENGOODS_REDIS_URL") or None
def bump_cache_epoch(url: str | None = None) -> bool:
"""Increment the API cache generation counter.
Returns ``True`` if the counter was bumped, ``False`` if caching is disabled
or Redis was unreachable. Never raises: invalidation failures must not fail
an ingestion run.
"""
url = url or default_redis_url()
if not url:
return False
try:
import redis # imported lazily so the dependency is optional at runtime
client = redis.Redis.from_url(url, socket_timeout=2, socket_connect_timeout=2)
new_epoch = client.incr(EPOCH_KEY)
client.close()
logger.info("bumped API cache epoch to %s", new_epoch)
return True
except Exception as exc: # noqa: BLE001 - invalidation is best effort
logger.warning("cache epoch bump skipped: %s", exc)
return False
+139
View File
@@ -0,0 +1,139 @@
"""Duplicate detection and product merging.
Barcodes (GTIN) are already unique at the schema level, so duplicates here are
non-GTIN records that describe the same product (same normalized name + brand +
net content). For each duplicate group we keep the highest-quality product as
canonical and merge the rest into it: child rows (provenance, images, MSRP) are
re-pointed to the canonical product, the merged product is marked ``merged``
with ``canonical_id`` set, and a row is written to ``merge_log``.
"""
from __future__ import annotations
from typing import Any
import psycopg
from opengoods.etl.quality import update_quality
def _norm(text: str | None) -> str:
return " ".join((text or "").lower().split())
def product_signature(name: str | None, brand: str | None, net_canonical: Any | None) -> str | None:
"""Stable signature for non-GTIN dedup, or ``None`` if too sparse to match."""
n = _norm(name)
if not n:
return None
net = "" if net_canonical is None else str(net_canonical)
return f"{n}|{_norm(brand)}|{net}"
def choose_canonical(members: list[dict]) -> dict:
"""Pick the canonical product: best quality, then oldest, then lowest id."""
return min(
members,
key=lambda m: (
-float(m.get("quality_score") or 0.0),
m.get("created_at"),
str(m.get("id")),
),
)
def find_duplicate_groups(conn: psycopg.Connection) -> list[list[dict]]:
"""Return groups (size >= 2) of active products sharing a signature."""
rows = conn.execute(
"""
SELECT p.id, p.name, b.normalized_name, p.net_content_canonical,
p.quality_score, p.created_at
FROM product p
LEFT JOIN brand b ON b.id = p.brand_id
WHERE p.status = 'active'
"""
).fetchall()
groups: dict[str, list[dict]] = {}
for r in rows:
sig = product_signature(r[1], r[2], r[3])
if sig is None:
continue
member = {
"id": r[0],
"name": r[1],
"quality_score": r[4],
"created_at": r[5],
}
groups.setdefault(sig, []).append(member)
return [m for m in groups.values() if len(m) >= 2]
def merge_products(
conn: psycopg.Connection,
kept_id: str,
merged_id: str,
reason: str = "auto-dedup",
actor: str = "ingestion",
) -> None:
"""Merge ``merged_id`` into ``kept_id`` (re-point children, mark merged)."""
if kept_id == merged_id:
return
# Re-point provenance, images and MSRP to the canonical product.
conn.execute(
"UPDATE product_source SET product_id = %s WHERE product_id = %s",
(kept_id, merged_id),
)
conn.execute(
"UPDATE product_image SET product_id = %s WHERE product_id = %s",
(kept_id, merged_id),
)
conn.execute(
"UPDATE product_msrp SET product_id = %s WHERE product_id = %s",
(kept_id, merged_id),
)
# food_detail has product_id as PK, so it can only move if the canonical
# product does not already have one.
kept_has_food = conn.execute(
"SELECT 1 FROM food_detail WHERE product_id = %s", (kept_id,)
).fetchone()
if not kept_has_food:
conn.execute(
"UPDATE food_detail SET product_id = %s WHERE product_id = %s",
(kept_id, merged_id),
)
conn.execute(
"UPDATE product SET status = 'merged', canonical_id = %s WHERE id = %s",
(kept_id, merged_id),
)
conn.execute(
"""
INSERT INTO merge_log (kept_id, merged_id, reason, actor)
VALUES (%s, %s, %s, %s)
""",
(kept_id, merged_id, reason, actor),
)
# The canonical product gained sources, so its quality may have changed.
update_quality(conn, kept_id)
def dedup_all(
conn: psycopg.Connection, actor: str = "ingestion", dry_run: bool = False
) -> dict[str, int]:
"""Merge every duplicate group. Returns counts of groups and merges."""
groups = find_duplicate_groups(conn)
merged = 0
for members in groups:
canonical = choose_canonical(members)
for m in members:
if m["id"] == canonical["id"]:
continue
if not dry_run:
merge_products(conn, canonical["id"], m["id"], actor=actor)
merged += 1
return {"groups": len(groups), "merged": merged}
+195 -3
View File
@@ -7,16 +7,24 @@ source with field-level provenance in `product_source`.
from __future__ import annotations from __future__ import annotations
import json import json
import logging
import os import os
from typing import Any from typing import Any
import psycopg import psycopg
from psycopg.types.json import Jsonb from psycopg.types.json import Jsonb
from opengoods.adapters.bypos import SOURCE_HOMEPAGE as SOURCE_HOMEPAGE_BYPOS
from opengoods.adapters.bypos import SOURCE_LICENSE as SOURCE_LICENSE_BYPOS
from opengoods.adapters.bypos import SOURCE_NAME as SOURCE_NAME_BYPOS
from opengoods.adapters.bypos import SOURCE_TRUST as SOURCE_TRUST_BYPOS
from opengoods.adapters.openfoodfacts import OFF_LICENSE, SOURCE_NAME from opengoods.adapters.openfoodfacts import OFF_LICENSE, SOURCE_NAME
from opengoods.etl.quality import update_quality
OFF_HOMEPAGE = "https://world.openfoodfacts.org" OFF_HOMEPAGE = "https://world.openfoodfacts.org"
logger = logging.getLogger(__name__)
def default_dsn() -> str: def default_dsn() -> str:
return os.environ.get( return os.environ.get(
@@ -29,8 +37,14 @@ def _normalize_brand(name: str) -> str:
return " ".join(name.lower().split()) return " ".join(name.lower().split())
def ensure_source(conn: psycopg.Connection) -> str: def ensure_source_named(
"""Upsert the Open Food Facts source row and return its id.""" conn: psycopg.Connection,
name: str,
homepage: str,
license: str,
trust_weight: float,
) -> str:
"""Upsert a source row by name and return its id."""
row = conn.execute( row = conn.execute(
""" """
INSERT INTO source (name, homepage, license, trust_weight) INSERT INTO source (name, homepage, license, trust_weight)
@@ -38,11 +52,16 @@ def ensure_source(conn: psycopg.Connection) -> str:
ON CONFLICT (name) DO UPDATE SET homepage = EXCLUDED.homepage ON CONFLICT (name) DO UPDATE SET homepage = EXCLUDED.homepage
RETURNING id RETURNING id
""", """,
(SOURCE_NAME, OFF_HOMEPAGE, OFF_LICENSE, 0.7), (name, homepage, license, trust_weight),
).fetchone() ).fetchone()
return row[0] return row[0]
def ensure_source(conn: psycopg.Connection) -> str:
"""Upsert the Open Food Facts source row and return its id."""
return ensure_source_named(conn, SOURCE_NAME, OFF_HOMEPAGE, OFF_LICENSE, 0.7)
def _ensure_brand(conn: psycopg.Connection, name: str | None) -> str | None: def _ensure_brand(conn: psycopg.Connection, name: str | None) -> str | None:
if not name: if not name:
return None return None
@@ -58,6 +77,23 @@ def _ensure_brand(conn: psycopg.Connection, name: str | None) -> str | None:
return row[0] return row[0]
def _ensure_manufacturer(
conn: psycopg.Connection, name: str | None, country: str | None = None
) -> str | None:
if not name:
return None
row = conn.execute(
"""
INSERT INTO manufacturer (name, normalized_name, country)
VALUES (%s, %s, %s)
ON CONFLICT (normalized_name) DO UPDATE SET name = manufacturer.name
RETURNING id
""",
(name, _normalize_brand(name), country),
).fetchone()
return row[0]
def _category_id(conn: psycopg.Connection, path: str | None) -> tuple[str | None, str | None]: def _category_id(conn: psycopg.Connection, path: str | None) -> tuple[str | None, str | None]:
if not path: if not path:
return None, None return None, None
@@ -178,9 +214,165 @@ def load_record(conn: psycopg.Connection, rec: dict[str, Any], source_id: str, r
Jsonb(_jsonable(raw)), Jsonb(_jsonable(raw)),
), ),
) )
# Recompute the data-quality score now that all facts + provenance exist.
update_quality(conn, product_id)
return product_id return product_id
def load_record_safe(
conn: psycopg.Connection, rec: dict[str, Any], source_id: str, raw: dict
) -> bool:
"""Load one record inside a savepoint.
On success the record's writes stay in the surrounding transaction. On any
error, only this record's writes are rolled back (to the savepoint) and the
batch continues, so a single malformed source record cannot abort a large
import. Returns True if loaded, False if skipped due to an error.
"""
try:
with conn.transaction():
load_record(conn, rec, source_id, raw)
return True
except Exception as exc: # noqa: BLE001 - per-record isolation is intentional
logger.warning("skipping record gtin=%s: %s", rec.get("gtin"), exc)
return False
def ensure_bypos_source(conn: psycopg.Connection) -> str:
"""Upsert the bypos central-library source row and return its id."""
return ensure_source_named(
conn,
SOURCE_NAME_BYPOS,
SOURCE_HOMEPAGE_BYPOS,
SOURCE_LICENSE_BYPOS,
SOURCE_TRUST_BYPOS,
)
def load_bypos_record(
conn: psycopg.Connection, rec: dict[str, Any], source_id: str, raw: dict
) -> str:
"""Upsert one transformed bypos record; return the product id.
Unlike OFF records these have no ingredients/nutrition, so no ``food_detail``
row is written. The suggested retail price (if any) is stored as a CNY MSRP
snapshot, and provenance/MSRP rows are keyed by source so a re-import
refreshes rather than duplicates them.
"""
manufacturer_id = _ensure_manufacturer(
conn, rec.get("manufacturer"), rec.get("country_of_origin")
)
attrs = rec.get("attributes") or {}
fields = ["name", "net_content", "country_of_origin"]
if manufacturer_id:
fields.append("manufacturer")
if attrs:
fields.append("attributes")
if rec.get("gtin"):
fields.append("gtin")
prod = conn.execute(
"""
INSERT INTO product (gtin, name, manufacturer_id,
net_content_value, net_content_unit, net_content_canonical,
country_of_origin, attributes)
VALUES (%s,%s,%s,%s,%s,%s,%s,%s)
ON CONFLICT (gtin) WHERE gtin IS NOT NULL DO UPDATE SET
name = EXCLUDED.name,
manufacturer_id = COALESCE(EXCLUDED.manufacturer_id, product.manufacturer_id),
net_content_value = COALESCE(EXCLUDED.net_content_value, product.net_content_value),
net_content_unit = COALESCE(EXCLUDED.net_content_unit, product.net_content_unit),
net_content_canonical = COALESCE(
EXCLUDED.net_content_canonical, product.net_content_canonical),
country_of_origin = COALESCE(EXCLUDED.country_of_origin, product.country_of_origin),
attributes = product.attributes || EXCLUDED.attributes
RETURNING id
""",
(
rec["gtin"],
rec["name"],
manufacturer_id,
rec.get("net_content_value"),
rec.get("net_content_unit"),
rec.get("net_content_canonical"),
rec.get("country_of_origin"),
Jsonb(attrs),
),
).fetchone()
else:
prod = conn.execute(
"""
INSERT INTO product (name, manufacturer_id,
net_content_value, net_content_unit, net_content_canonical,
country_of_origin, attributes)
VALUES (%s,%s,%s,%s,%s,%s,%s)
RETURNING id
""",
(
rec["name"],
manufacturer_id,
rec.get("net_content_value"),
rec.get("net_content_unit"),
rec.get("net_content_canonical"),
rec.get("country_of_origin"),
Jsonb(attrs),
),
).fetchone()
product_id = prod[0]
# Refresh this source's MSRP snapshot (suggested retail price, CNY).
conn.execute(
"DELETE FROM product_msrp WHERE product_id = %s AND source_id = %s",
(product_id, source_id),
)
if rec.get("msrp") is not None:
conn.execute(
"""
INSERT INTO product_msrp (product_id, amount, currency, region, source_id, source_url)
VALUES (%s,%s,'CNY','CN',%s,%s)
""",
(product_id, rec["msrp"], source_id, SOURCE_HOMEPAGE_BYPOS),
)
fields.append("msrp")
# Refresh this source's provenance row (one per source for idempotency).
conn.execute(
"DELETE FROM product_source WHERE product_id = %s AND source_id = %s",
(product_id, source_id),
)
conn.execute(
"""
INSERT INTO product_source (product_id, source_id, url, fields, fetched_at, raw)
VALUES (%s,%s,%s,%s, COALESCE(%s::timestamptz, now()), %s)
""",
(
product_id,
source_id,
SOURCE_HOMEPAGE_BYPOS,
fields,
rec.get("fetched_at"),
Jsonb(_jsonable(raw)),
),
)
update_quality(conn, product_id)
return product_id
def load_bypos_record_safe(
conn: psycopg.Connection, rec: dict[str, Any], source_id: str, raw: dict
) -> bool:
"""Load one bypos record inside a savepoint (see :func:`load_record_safe`)."""
try:
with conn.transaction():
load_bypos_record(conn, rec, source_id, raw)
return True
except Exception as exc: # noqa: BLE001 - per-record isolation is intentional
logger.warning("skipping bypos record gtin=%s: %s", rec.get("gtin"), exc)
return False
def _jsonable(raw: dict) -> dict: def _jsonable(raw: dict) -> dict:
"""Drop values that are not JSON-serializable from a raw record.""" """Drop values that are not JSON-serializable from a raw record."""
try: try:
+104
View File
@@ -0,0 +1,104 @@
"""Field-level conflict resolution for multi-source records.
When more than one source describes the same product, each field may have
several candidate values. We pick a winner per field by source trust first,
then recency, ignoring empty values, and keep a provenance trail of which
source won each field.
These are pure functions (no DB / no network) so they are easy to unit-test;
the DB-level record merge lives in :mod:`opengoods.etl.dedup`.
"""
from __future__ import annotations
from dataclasses import dataclass, field
from datetime import datetime
@dataclass(frozen=True)
class Candidate:
"""One source's proposed value for a field."""
value: object
source: str
trust: float = 0.5
fetched_at: datetime | None = None
@dataclass
class FieldResolution:
"""The winning value for a field plus the source it came from."""
value: object
source: str | None = None
@dataclass
class MergedRecord:
"""A merged record with per-field provenance (field name -> source)."""
values: dict[str, object] = field(default_factory=dict)
provenance: dict[str, str] = field(default_factory=dict)
def _is_empty(value: object) -> bool:
if value is None:
return True
if isinstance(value, str):
return value.strip() == ""
if isinstance(value, (list, dict, tuple, set)):
return len(value) == 0
return False
def _sort_key(c: Candidate) -> tuple[float, float]:
ts = c.fetched_at.timestamp() if c.fetched_at is not None else float("-inf")
return (c.trust, ts)
def resolve_field(candidates: list[Candidate]) -> FieldResolution | None:
"""Pick the best non-empty candidate for one field.
Ranking: highest source trust, then most recent ``fetched_at``. Returns
``None`` when there is no usable (non-empty) candidate.
"""
usable = [c for c in candidates if not _is_empty(c.value)]
if not usable:
return None
winner = max(usable, key=_sort_key)
return FieldResolution(value=winner.value, source=winner.source)
def merge_records(records: list[dict], *, fields: list[str] | None = None) -> MergedRecord:
"""Merge several ``{field: Candidate|value}`` records into one.
Each input record maps field name -> :class:`Candidate` (preferred) or a
bare value (treated as trust 0.5, no timestamp). The result keeps, for each
field, the winning value and the name of the source that supplied it.
"""
keys: list[str]
if fields is not None:
keys = list(fields)
else:
seen: dict[str, None] = {}
for rec in records:
for k in rec:
seen.setdefault(k, None)
keys = list(seen)
merged = MergedRecord()
for key in keys:
candidates: list[Candidate] = []
for rec in records:
if key not in rec:
continue
cand = rec[key]
if not isinstance(cand, Candidate):
cand = Candidate(value=cand, source="unknown")
candidates.append(cand)
resolution = resolve_field(candidates)
if resolution is not None:
merged.values[key] = resolution.value
if resolution.source is not None:
merged.provenance[key] = resolution.source
return merged
+176
View File
@@ -0,0 +1,176 @@
"""Product data-quality scoring.
The quality score is a 0..1 number combining four signals, per the locked
project decision:
quality = 0.4 * completeness
+ 0.3 * source_trust
+ 0.2 * multi_source_agreement
+ 0.1 * freshness
Each component is itself normalized to 0..1. The pure helpers below are
unit-testable; :func:`compute_quality` / :func:`update_quality` read the signals
for a product out of the database and persist the result on ``product``.
"""
from __future__ import annotations
from datetime import UTC, datetime
import psycopg
W_COMPLETENESS = 0.4
W_SOURCE_TRUST = 0.3
W_AGREEMENT = 0.2
W_FRESHNESS = 0.1
# Fields that count towards completeness (weighted equally).
COMPLETENESS_FIELDS = (
"name",
"gtin",
"brand",
"category",
"net_content",
"country_of_origin",
"nutriments",
"ingredients",
"image",
)
def completeness(present: set[str]) -> float:
"""Fraction of :data:`COMPLETENESS_FIELDS` that are present for a product."""
if not COMPLETENESS_FIELDS:
return 0.0
hits = sum(1 for f in COMPLETENESS_FIELDS if f in present)
return hits / len(COMPLETENESS_FIELDS)
def agreement_from_sources(source_count: int) -> float:
"""Multi-source corroboration proxy from the number of distinct sources.
A single source cannot be corroborated, so it scores a neutral 0.5; more
independent sources that describe the same product raise confidence.
"""
if source_count <= 1:
return 0.5
if source_count == 2:
return 0.8
return 1.0
def freshness_from_age(age_days: float | None) -> float:
"""Recency score from the age (in days) of the most recent source fetch."""
if age_days is None:
return 0.5
if age_days <= 30:
return 1.0
if age_days <= 180:
return 0.8
if age_days <= 365:
return 0.6
if age_days <= 730:
return 0.4
return 0.2
def score(
*,
completeness_score: float,
source_trust: float,
agreement: float,
freshness: float,
) -> float:
"""Combine the four normalized components into a 0..1 quality score."""
raw = (
W_COMPLETENESS * completeness_score
+ W_SOURCE_TRUST * source_trust
+ W_AGREEMENT * agreement
+ W_FRESHNESS * freshness
)
return round(max(0.0, min(1.0, raw)), 3)
def _present_fields(prod: dict, has_image: bool) -> set[str]:
present: set[str] = set()
if prod.get("name"):
present.add("name")
if prod.get("gtin"):
present.add("gtin")
if prod.get("brand_id"):
present.add("brand")
if prod.get("category_id"):
present.add("category")
if prod.get("net_content_canonical") is not None:
present.add("net_content")
if prod.get("country_of_origin"):
present.add("country_of_origin")
if prod.get("nutriments"):
present.add("nutriments")
if prod.get("ingredients_text"):
present.add("ingredients")
if has_image:
present.add("image")
return present
def compute_quality(conn: psycopg.Connection, product_id: str) -> float:
"""Compute (but do not persist) the quality score for one product."""
row = conn.execute(
"""
SELECT p.name, p.gtin, p.brand_id, p.category_id, p.net_content_canonical,
p.country_of_origin, f.nutriments, f.ingredients_text,
EXISTS (SELECT 1 FROM product_image pi WHERE pi.product_id = p.id)
FROM product p
LEFT JOIN food_detail f ON f.product_id = p.id
WHERE p.id = %s
""",
(product_id,),
).fetchone()
if row is None:
return 0.0
prod = {
"name": row[0],
"gtin": row[1],
"brand_id": row[2],
"category_id": row[3],
"net_content_canonical": row[4],
"country_of_origin": row[5],
"nutriments": row[6],
"ingredients_text": row[7],
}
has_image = bool(row[8])
src = conn.execute(
"""
SELECT count(DISTINCT ps.source_id), COALESCE(max(s.trust_weight), 0), max(ps.fetched_at)
FROM product_source ps
LEFT JOIN source s ON s.id = ps.source_id
WHERE ps.product_id = %s
""",
(product_id,),
).fetchone()
source_count = int(src[0] or 0)
source_trust = float(src[1] or 0.0)
last_fetched: datetime | None = src[2]
age_days: float | None = None
if last_fetched is not None:
now = datetime.now(UTC)
if last_fetched.tzinfo is None:
last_fetched = last_fetched.replace(tzinfo=UTC)
age_days = max(0.0, (now - last_fetched).total_seconds() / 86400.0)
return score(
completeness_score=completeness(_present_fields(prod, has_image)),
source_trust=source_trust,
agreement=agreement_from_sources(source_count),
freshness=freshness_from_age(age_days),
)
def update_quality(conn: psycopg.Connection, product_id: str) -> float:
"""Compute the quality score and write it to ``product.quality_score``."""
value = compute_quality(conn, product_id)
conn.execute("UPDATE product SET quality_score = %s WHERE id = %s", (value, product_id))
return value
+45
View File
@@ -0,0 +1,45 @@
"""Persistent ingestion watermark stored in the ``ingest_state`` table.
The incremental updater uses this to remember how far it got for each source
(e.g. Open Food Facts exposes a ``last_modified_t`` unix timestamp on every
product) so repeated runs only fetch what changed.
"""
from __future__ import annotations
from typing import Any
import psycopg
from psycopg.types.json import Jsonb
def get_watermark(conn: psycopg.Connection, source: str) -> int:
"""Return the last processed ``last_modified_t`` for *source* (0 if none)."""
row = conn.execute(
"SELECT last_modified_t FROM ingest_state WHERE source = %s", (source,)
).fetchone()
return int(row[0]) if row else 0
def set_watermark(
conn: psycopg.Connection,
source: str,
last_modified_t: int,
stats: dict[str, Any] | None = None,
) -> None:
"""Upsert the watermark and run metadata for *source*.
The watermark only ever moves forward: a lower ``last_modified_t`` is
ignored so an out-of-order or partial run cannot rewind progress.
"""
conn.execute(
"""
INSERT INTO ingest_state (source, last_modified_t, last_run_at, stats)
VALUES (%s, %s, now(), %s)
ON CONFLICT (source) DO UPDATE SET
last_modified_t = GREATEST(ingest_state.last_modified_t, EXCLUDED.last_modified_t),
last_run_at = now(),
stats = EXCLUDED.stats
""",
(source, int(last_modified_t), Jsonb(stats or {})),
)
+133
View File
@@ -0,0 +1,133 @@
"""Apply GS1 (or other authoritative) supplements to existing products.
A supplement only fills *gaps*: a field is written only when the product does
not already have a value. Each applied supplement records field-level provenance
in ``product_source`` and refreshes the product's quality score.
"""
from __future__ import annotations
from decimal import Decimal, InvalidOperation
from typing import Any
import psycopg
from psycopg.types.json import Jsonb
from opengoods import units
from opengoods.adapters.gs1 import GS1_HOMEPAGE, GS1_LICENSE, GS1_TRUST, SOURCE_NAME
from opengoods.etl.load import _ensure_brand, _normalize_brand, ensure_source_named
from opengoods.etl.quality import update_quality
def ensure_gs1_source(conn: psycopg.Connection) -> str:
"""Upsert the GS1 source row and return its id."""
return ensure_source_named(conn, SOURCE_NAME, GS1_HOMEPAGE, GS1_LICENSE, GS1_TRUST)
def _ensure_manufacturer(conn: psycopg.Connection, name: str | None) -> str | None:
if not name:
return None
row = conn.execute(
"""
INSERT INTO manufacturer (name, normalized_name)
VALUES (%s, %s)
ON CONFLICT (normalized_name) DO UPDATE SET name = manufacturer.name
RETURNING id
""",
(name, _normalize_brand(name)),
).fetchone()
return row[0]
def _net_content(rec: dict) -> tuple[Decimal, str, Decimal | None] | None:
raw_value = rec.get("net_content_value")
unit = rec.get("net_content_unit")
if raw_value is None or not unit:
return None
try:
value = Decimal(str(raw_value))
except (InvalidOperation, ValueError):
return None
try:
canonical = units.normalize(value, unit).canonical_value
except units.UnitError:
canonical = None
return value, unit, canonical
def apply_supplement(conn: psycopg.Connection, rec: dict[str, Any], source_id: str) -> list[str]:
"""Fill missing fields of the GTIN-matched product from ``rec``.
Returns the list of field names actually filled (empty if the product is
unknown or already complete for the supplied fields).
"""
gtin = rec.get("gtin")
if not gtin:
return []
prod = conn.execute(
"""
SELECT id, brand_id, manufacturer_id, gpc_brick_code, country_of_origin,
net_content_value
FROM product
WHERE gtin = %s AND status = 'active'
""",
(gtin,),
).fetchone()
if prod is None:
return []
product_id, brand_id, manufacturer_id, gpc, country, net_value = prod
sets: list[str] = []
params: list[Any] = []
filled: list[str] = []
if brand_id is None and rec.get("brand"):
new_brand_id = _ensure_brand(conn, rec["brand"])
if new_brand_id is not None:
sets.append("brand_id = %s")
params.append(new_brand_id)
filled.append("brand")
if manufacturer_id is None and rec.get("manufacturer"):
new_mfr_id = _ensure_manufacturer(conn, rec["manufacturer"])
if new_mfr_id is not None:
sets.append("manufacturer_id = %s")
params.append(new_mfr_id)
filled.append("manufacturer")
if gpc is None and rec.get("gpc_brick_code"):
sets.append("gpc_brick_code = %s")
params.append(rec["gpc_brick_code"])
filled.append("gpc_brick_code")
if country is None and rec.get("country_of_origin"):
sets.append("country_of_origin = %s")
params.append(rec["country_of_origin"])
filled.append("country_of_origin")
if net_value is None:
net = _net_content(rec)
if net is not None:
value, unit, canonical = net
sets += [
"net_content_value = %s",
"net_content_unit = %s",
"net_content_canonical = %s",
]
params += [value, unit, canonical]
filled.append("net_content")
if not filled:
return []
params.append(product_id)
conn.execute(f"UPDATE product SET {', '.join(sets)} WHERE id = %s", params)
conn.execute(
"""
INSERT INTO product_source (product_id, source_id, url, fields, fetched_at, raw)
VALUES (%s, %s, %s, %s, now(), %s)
""",
(product_id, source_id, GS1_HOMEPAGE, filled, Jsonb(rec)),
)
update_quality(conn, product_id)
return filled
+11 -3
View File
@@ -78,6 +78,14 @@ def map_category(raw: dict) -> str | None:
return None return None
def _clamp(value: str | None, max_len: int) -> str | None:
"""Trim a string to fit a bounded DB column; external data length varies."""
if value is None:
return None
value = value.strip()
return value[:max_len] or None
def _clean_tags(tags: list[str] | None, prefix: str = "") -> list[str]: def _clean_tags(tags: list[str] | None, prefix: str = "") -> list[str]:
out: list[str] = [] out: list[str] = []
for t in tags or []: for t in tags or []:
@@ -143,16 +151,16 @@ def transform(raw: dict) -> dict | None:
"brand": brand, "brand": brand,
"category_path": map_category(raw), "category_path": map_category(raw),
"net_content_value": net_value, "net_content_value": net_value,
"net_content_unit": net_unit, "net_content_unit": _clamp(net_unit, 16),
"net_content_canonical": net_canonical, "net_content_canonical": net_canonical,
"country_of_origin": (raw.get("countries") or "").split(",")[0].strip() or None, "country_of_origin": _clamp((raw.get("countries") or "").split(",")[0].strip() or None, 64),
"food": { "food": {
"ingredients_text": raw.get("ingredients_text") or None, "ingredients_text": raw.get("ingredients_text") or None,
"allergens": _clean_tags(raw.get("allergens_tags")), "allergens": _clean_tags(raw.get("allergens_tags")),
"additives": _clean_tags(raw.get("additives_tags")), "additives": _clean_tags(raw.get("additives_tags")),
"nutriments": transform_nutriments(raw.get("nutriments") or {}), "nutriments": transform_nutriments(raw.get("nutriments") or {}),
"nutrition_basis": "per_100g", "nutrition_basis": "per_100g",
"serving_size": raw.get("serving_size") or None, "serving_size": _clamp(raw.get("serving_size") or None, 32),
"nutri_score": (raw.get("nutriscore_grade") or "").upper()[:1] or None, "nutri_score": (raw.get("nutriscore_grade") or "").upper()[:1] or None,
}, },
"image_url": raw.get("image_front_url") or raw.get("image_url") or None, "image_url": raw.get("image_front_url") or raw.get("image_url") or None,
+43
View File
@@ -0,0 +1,43 @@
"""Deduplicate products: merge non-GTIN duplicates into a canonical record.
Usage:
python -m opengoods.jobs.dedup --dry-run
python -m opengoods.jobs.dedup --actor nightly
"""
from __future__ import annotations
import argparse
import sys
import psycopg
from opengoods.cache import bump_cache_epoch
from opengoods.etl.dedup import dedup_all
from opengoods.etl.load import default_dsn
def run(args: argparse.Namespace) -> int:
with psycopg.connect(args.dsn, autocommit=False) as conn:
summary = dedup_all(conn, actor=args.actor, dry_run=args.dry_run)
if args.dry_run:
conn.rollback()
else:
conn.commit()
if not args.dry_run and summary["merged"]:
bump_cache_epoch()
mode = "dry-run" if args.dry_run else "applied"
print(f"{mode} groups={summary['groups']} merged={summary['merged']}")
return 0
def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser(description="Deduplicate OpenGoods products")
parser.add_argument("--actor", default="ingestion", help="merge_log actor label")
parser.add_argument("--dry-run", action="store_true", help="report only, do not write")
parser.add_argument("--dsn", default=default_dsn(), help="PostgreSQL DSN")
return run(parser.parse_args(argv))
if __name__ == "__main__":
sys.exit(main())
+80
View File
@@ -0,0 +1,80 @@
"""Import products collected by the bypos-collector tool into OpenGoods.
The ``tools/bypos-collector`` program writes one product per line (JSONL). This
job reads such a file, transforms each ``hit`` record into the internal product
shape, and upserts it into the database under the ``bypos中心库`` source with
field-level provenance.
Usage::
python -m opengoods.jobs.import_bypos --input products.jsonl
python -m opengoods.jobs.import_bypos --input products.jsonl --limit 500
Re-running is safe: products are upserted by GTIN and the source's MSRP/
provenance rows are refreshed rather than duplicated.
"""
from __future__ import annotations
import argparse
import gzip
import json
import sys
from collections.abc import Iterator
import psycopg
from opengoods.adapters.bypos import transform_bypos
from opengoods.cache import bump_cache_epoch
from opengoods.etl.load import default_dsn, ensure_bypos_source, load_bypos_record_safe
def _read_jsonl(path: str) -> Iterator[dict]:
opener = gzip.open if path.endswith(".gz") else open
with opener(path, "rt", encoding="utf-8") as fh:
for line in fh:
line = line.strip()
if not line:
continue
try:
yield json.loads(line)
except json.JSONDecodeError:
continue
def run(args: argparse.Namespace) -> int:
loaded = skipped = errored = 0
with psycopg.connect(args.dsn, autocommit=False) as conn:
source_id = ensure_bypos_source(conn)
yielded = 0
for raw in _read_jsonl(args.input):
if args.limit and yielded >= args.limit:
break
rec = transform_bypos(raw)
if rec is None:
skipped += 1
continue
yielded += 1
if load_bypos_record_safe(conn, rec, source_id, raw):
loaded += 1
else:
errored += 1
conn.commit()
if loaded:
bump_cache_epoch()
print(f"loaded={loaded} skipped={skipped} errored={errored}")
return 0
def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser(description="Import bypos-collector JSONL into OpenGoods")
parser.add_argument(
"--input", required=True, help="path to a bypos-collector JSONL (.jsonl or .jsonl.gz)"
)
parser.add_argument("--limit", type=int, default=0, help="max hit records to load (0 = all)")
parser.add_argument("--dsn", default=default_dsn(), help="PostgreSQL DSN")
return run(parser.parse_args(argv))
if __name__ == "__main__":
sys.exit(main())
+66
View File
@@ -0,0 +1,66 @@
"""Lightweight recurring ingestion scheduler.
Runs one ingestion cycle (incremental OFF update, then dedup) on a fixed
interval. Dependency-free: a plain sleep loop rather than a cron/APScheduler
dependency, so it is trivial to run in a container or under systemd/supervisor.
Usage:
python -m opengoods.jobs.schedule --once # single cycle, then exit
python -m opengoods.jobs.schedule --interval 3600 # every hour
"""
from __future__ import annotations
import argparse
import sys
import time
from datetime import UTC, datetime
from opengoods.etl.load import default_dsn
from opengoods.jobs import dedup as dedup_job
from opengoods.jobs import update_off as update_job
def _cycle(args: argparse.Namespace) -> None:
ts = datetime.now(UTC).isoformat(timespec="seconds")
print(f"[{ts}] cycle start")
update_job.run(
argparse.Namespace(
since=None,
page_size=args.page_size,
max_pages=args.max_pages,
min_interval=args.min_interval,
dsn=args.dsn,
)
)
if not args.skip_dedup:
dedup_job.run(argparse.Namespace(actor="scheduler", dry_run=False, dsn=args.dsn))
def run(args: argparse.Namespace) -> int:
_cycle(args)
if args.once:
return 0
while True:
time.sleep(args.interval)
try:
_cycle(args)
except Exception as exc: # noqa: BLE001 - keep the loop alive across failures
print(f"cycle error: {exc}", file=sys.stderr)
return 0
def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser(description="Recurring OpenGoods ingestion")
parser.add_argument("--interval", type=int, default=3600, help="seconds between cycles")
parser.add_argument("--once", action="store_true", help="run a single cycle and exit")
parser.add_argument("--skip-dedup", action="store_true", help="run update only")
parser.add_argument("--page-size", type=int, default=100, help="search page size")
parser.add_argument("--max-pages", type=int, default=10, help="max pages to scan")
parser.add_argument("--min-interval", type=float, default=4.0, help="API throttle seconds")
parser.add_argument("--dsn", default=default_dsn(), help="PostgreSQL DSN")
return run(parser.parse_args(argv))
if __name__ == "__main__":
sys.exit(main())

Some files were not shown because too many files have changed in this diff Show More