Compare commits

...

42 Commits

Author SHA1 Message Date
lixu 9ee83e3535 feat(public): 首页改为极简谷歌风格
CI / Go (api) (pull_request) Successful in 54s
CI / Python (ingestion) (pull_request) Successful in 16s
CI / Migrations (postgres) (pull_request) Successful in 27s
2026-06-25 10:08:56 +00:00
lixu 313bc84cb0 feat(public): 首页搜索居中、移动端自适应,移除下方三卡片
CI / Go (api) (pull_request) Successful in 52s
CI / Python (ingestion) (pull_request) Successful in 25s
CI / Migrations (postgres) (pull_request) Successful in 29s
2026-06-25 07:25:52 +00:00
lixu 8c8df2b8a2 Merge pull request 'fix(admin): 回流接口校验/规范化 GTIN' (#36) from devin/1782352663-backflow-gtin-validate into main
CI / Go (api) (push) Successful in 1m7s
CI / Python (ingestion) (push) Successful in 24s
CI / Migrations (postgres) (push) Successful in 25s
2026-06-25 10:00:29 +08:00
busenalbantoglu51424 f04891d0b9 fix(admin): 回流接口校验/规范化 GTIN,非法条码标记 invalid 而非中断批次
CI / Go (api) (pull_request) Successful in 56s
CI / Python (ingestion) (pull_request) Successful in 28s
CI / Migrations (postgres) (pull_request) Successful in 29s
2026-06-25 01:57:43 +00:00
lixu 3a5575f51b Merge pull request 'feat(admin): 新增档案回流接口 POST /api/public/backflow' (#35) from devin/1782350840-backflow-api into main
CI / Go (api) (push) Successful in 53s
CI / Python (ingestion) (push) Successful in 28s
CI / Migrations (postgres) (push) Successful in 47s
2026-06-25 09:52:43 +08:00
busenalbantoglu51424 a61d1c558c docs: 补充档案回流接口调用说明(docs/api.md + 前台 API 文档页)
CI / Go (api) (pull_request) Successful in 1m2s
CI / Python (ingestion) (pull_request) Successful in 27s
CI / Migrations (postgres) (pull_request) Successful in 24s
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-25 01:41:39 +00:00
busenalbantoglu51424 e7e5ce22ab feat(admin): 新增档案回流接口 POST /api/public/backflow
CI / Go (api) (pull_request) Successful in 55s
CI / Python (ingestion) (pull_request) Successful in 16s
CI / Migrations (postgres) (pull_request) Successful in 25s
进销存软件可用公开 API Key(og_live_) 批量回流未收录商品,进入现有审核
队列,审核通过后收录。按 GTIN 去重(已收录跳过 exists,已有待审跳过
duplicate),来源标记 source=backflow,在后台队列与公众投稿区分。

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-25 01:27:31 +00:00
lixu 856b81125c Merge PR #34: public MSRP display
CI / Go (api) (push) Successful in 54s
CI / Python (ingestion) (push) Successful in 16s
CI / Migrations (postgres) (push) Successful in 26s
feat: embed MSRP in public product detail (hide zero amounts)
2026-06-25 08:59:21 +08:00
busenalbantoglu51424 5fe5774251 feat(public): 商品详情页展示厂商建议零售价(MSRP)
CI / Go (api) (pull_request) Successful in 1m59s
CI / Python (ingestion) (pull_request) Successful in 1m13s
CI / Migrations (postgres) (pull_request) Successful in 27s
后端 ProductByID/ProductByGTIN 在商品详情 JSON 中内嵌 msrp 字段(按 effective_date 倒序,过滤 amount=0 的无效快照);公开前台详情页新增「建议零售价」展示。

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-25 00:42:18 +00:00
lixu 0da57a2fb3 Merge pull request 'feat(admin): 新增 /api/import/bypos 导入接口(修复采集器导入失败)' (#33) from devin/1782307670-bypos-import-endpoint into main
CI / Go (api) (push) Successful in 53s
CI / Python (ingestion) (push) Successful in 16s
CI / Migrations (postgres) (push) Successful in 28s
2026-06-24 21:33:02 +08:00
rosemariejebbjtxbfp 85c58fbd52 feat(admin): add /api/import/bypos endpoint for bypos-collector import
CI / Go (api) (pull_request) Successful in 58s
CI / Python (ingestion) (pull_request) Successful in 16s
CI / Migrations (postgres) (pull_request) Successful in 27s
Brings the bypos JSONL import backend (handler + store) into main so the
bypos-collector tool's import feature works end-to-end. Records are upserted
on GTIN with manufacturer/MSRP/barcode/source provenance, quality recomputed
per product. Only status=hit records with a name are imported.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-24 13:30:27 +00:00
lixu 3aec0b84fe Merge pull request 'feat(web+api): 贡献表单按 archive_kind 动态生成字段' (#32) from devin/1782295354-contribution-form-dynamic into main
CI / Go (api) (push) Successful in 55s
CI / Python (ingestion) (push) Successful in 32s
CI / Migrations (postgres) (push) Successful in 26s
2026-06-24 18:07:05 +08:00
rosemariejebbjtxbfp bd1d3bde7c feat(web+api): 贡献表单按 archive_kind 动态生成字段
CI / Go (api) (pull_request) Successful in 1m3s
CI / Python (ingestion) (pull_request) Successful in 23s
CI / Migrations (postgres) (pull_request) Successful in 26s
- categories 接口返回 archive_kind
- 新增公开只读接口 /api/v1/kind-fields?kind=X 返回字段模板
- 投稿 payload 支持通用 attributes,审核通过写入 product.attributes (JSONB 合并)
- food_detail 仅在含食品数据时才 upsert (药品/3C/通用不再产生空行)
- 前端 Contribute 按所选品类 archive_kind 动态渲染字段
  (食品营养 / 药品 18 字段 / 3C / 通用),除商品名外均选填

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-24 10:02:46 +00:00
lixu 998471f6df Merge pull request 'feat: 新增药品(drug)商品档案格式' (#31) from devin/1782286467-drug-archive-kind into main
CI / Go (api) (push) Successful in 47s
CI / Python (ingestion) (push) Successful in 18s
CI / Migrations (postgres) (push) Successful in 27s
2026-06-24 15:39:07 +08:00
rosemariejebbjtxbfp 1f9b1ecd30 feat(migrations): add drug (药品) archive kind template + category subtree
CI / Go (api) (pull_request) Successful in 55s
CI / Python (ingestion) (pull_request) Successful in 23s
CI / Migrations (postgres) (pull_request) Successful in 27s
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-24 07:36:43 +00:00
lixu 1d71617f71 Merge pull request 'chore(web): 添加微信域名校验文件' (#30) from devin/1782285002-wechat-domain-verify into main
CI / Go (api) (push) Successful in 51s
CI / Python (ingestion) (push) Successful in 30s
CI / Migrations (postgres) (push) Successful in 26s
2026-06-24 15:11:58 +08:00
rosemariejebbjtxbfp d18fc2470b chore(web): add WeChat domain verification file
CI / Go (api) (pull_request) Successful in 54s
CI / Python (ingestion) (pull_request) Successful in 26s
CI / Migrations (postgres) (pull_request) Successful in 27s
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-24 07:10:07 +00:00
lixu f29696607a Merge pull request 'feat(api): Redis read cache for product details and search (stage 0)' (#29) from devin/1782283345-redis-read-cache into main
CI / Go (api) (push) Successful in 50s
CI / Python (ingestion) (push) Successful in 37s
CI / Migrations (postgres) (push) Successful in 27s
2026-06-24 14:54:59 +08:00
rosemariejebbjtxbfp a75318b811 feat(api): Redis read cache for product details and search
CI / Go (api) (pull_request) Successful in 54s
CI / Python (ingestion) (pull_request) Successful in 19s
CI / Migrations (postgres) (pull_request) Successful in 27s
Stage-0 caching from docs/scalability.md. The Go API now caches hot
product-detail and search-result reads in Redis with a fail-open,
epoch-versioned scheme; Python ingestion bumps the epoch after a write
run to invalidate the cache globally in O(1).

- api/internal/cache: fail-open Cache (GetJSON/SetJSON) namespaced by an
  epoch counter (og:cache:epoch); disabled when Redis is unconfigured.
- store: ProductByID/ProductByGTIN (24h TTL) and SearchProducts (1h TTL)
  read-through the cache via WithCache.
- ingestion: bump_cache_epoch() called after update_off/seed_off/
  import_bypos/dedup commits when rows changed; best-effort, never fails
  a run. Adds redis dependency.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-24 06:52:56 +00:00
lixu 20b9eb0fc9 Merge pull request 'ci: 修复 main 上 Python CI 历史失败(在 python3.12+node 容器内运行)' (#28) from devin/1782280936-fix-python-ci into main
CI / Go (api) (push) Successful in 54s
CI / Python (ingestion) (push) Successful in 23s
CI / Migrations (postgres) (push) Successful in 27s
2026-06-24 14:32:51 +08:00
rosemariejebbjtxbfp 17bc0ed680 ci: remove GitHub-hosted actions, manual checkout + Go from CN mirror
CI / Go (api) (pull_request) Successful in 51s
CI / Python (ingestion) (pull_request) Successful in 21s
CI / Migrations (postgres) (pull_request) Successful in 26s
Self-hosted Gitea runner has flaky/blocked access to github.com, causing
actions/checkout and actions/setup-go to time out intermittently across all
jobs. Replace them with:
- manual git checkout against $GITHUB_SERVER_URL (the Gitea host, reachable
  from job containers)
- Go installed from mirrors.aliyun.com

Python job stays on the python3.12-nodejs20 container (fixes the original
PEP 660 editable-install failure). No more github.com network dependency.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-24 06:30:03 +00:00
rosemariejebbjtxbfp 5020fdcc19 ci: re-trigger with public gitea actions url
CI / Go (api) (pull_request) Failing after 11s
CI / Python (ingestion) (pull_request) Failing after 31s
CI / Migrations (postgres) (pull_request) Failing after 11s
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-24 06:24:52 +00:00
rosemariejebbjtxbfp 885f6c2b01 ci: re-trigger with self-hosted actions
CI / Go (api) (pull_request) Failing after 12s
CI / Python (ingestion) (pull_request) Failing after 1m31s
CI / Migrations (postgres) (pull_request) Failing after 32s
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-24 06:21:01 +00:00
rosemariejebbjtxbfp 2aff6287d2 ci: re-trigger after configuring action proxy
CI / Go (api) (pull_request) Failing after 13s
CI / Python (ingestion) (pull_request) Failing after 31s
CI / Migrations (postgres) (pull_request) Failing after 1m32s
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-24 06:10:22 +00:00
rosemariejebbjtxbfp 3364a50c68 ci: run Python job in python3.12+node container
CI / Go (api) (pull_request) Failing after 1m32s
CI / Python (ingestion) (pull_request) Failing after 31s
CI / Migrations (postgres) (pull_request) Failing after 19m16s
setup-python@v5 can't fetch CPython 3.12 on the self-hosted Gitea runner
(it queries the Gitea API for actions/python-versions, which 404s), so the
job silently fell back to the image's system Python 3.10 + pip 22.0.2. That
old pip lacks PEP 660 editable support, so 'pip install -e' failed with
'build backend is missing the build_editable hook', and 3.10 is below the
project's requires-python>=3.11 (code uses datetime.UTC).

Run the job inside nikolaik/python-nodejs:python3.12-nodejs20 which bundles
Python 3.12, Node 20 (for actions/checkout) and modern pip, removing the
GitHub download dependency entirely.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-24 06:02:28 +00:00
lixu 936828abae Merge pull request 'perf(search): 浏览路径部分复合索引 (0012)' (#27) from devin/1782279211-search-indexes into main
CI / Go (api) (push) Successful in 49s
CI / Python (ingestion) (push) Failing after 15s
CI / Migrations (postgres) (push) Successful in 31s
2026-06-24 13:35:50 +08:00
rosemariejebbjtxbfp ab7a964934 perf(search): 为浏览路径补部分复合索引 (0012)
CI / Go (api) (pull_request) Successful in 58s
CI / Python (ingestion) (pull_request) Failing after 21s
CI / Migrations (postgres) (pull_request) Successful in 32s
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-24 05:33:31 +00:00
lixu 649fcc711c Merge pull request 'docs: 可扩展性设计与路线图 (scalability roadmap)' (#26) from devin/1782278890-scalability-doc into main
CI / Go (api) (push) Successful in 52s
CI / Python (ingestion) (push) Failing after 16s
CI / Migrations (postgres) (push) Successful in 26s
2026-06-24 13:31:21 +08:00
rosemariejebbjtxbfp 3b62f61288 docs: 新增可扩展性设计与路线图 (scalability roadmap)
CI / Go (api) (pull_request) Successful in 59s
CI / Python (ingestion) (pull_request) Failing after 20s
CI / Migrations (postgres) (pull_request) Successful in 34s
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-24 05:29:08 +00:00
lixu ba36e3ff4c Merge pull request '新增联系我们页面' (#25) from devin/1782277231-contact-us into main
CI / Go (api) (push) Failing after 32s
CI / Python (ingestion) (push) Failing after 31s
CI / Migrations (postgres) (push) Successful in 35s
2026-06-24 13:02:07 +08:00
rosemariejebbjtxbfp d837dd38bf feat(public-frontend): 新增联系我们页面
CI / Go (api) (pull_request) Successful in 56s
CI / Python (ingestion) (pull_request) Failing after 1m31s
CI / Migrations (postgres) (pull_request) Failing after 32s
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-24 05:00:31 +00:00
lixu 53ce705572 Merge pull request '移除首页 API 调用说明入口' (#23) from devin/1782276245-home-remove-api-cta into main
CI / Go (api) (push) Failing after 16m4s
CI / Python (ingestion) (push) Failing after 17s
CI / Migrations (postgres) (push) Successful in 29s
2026-06-24 12:46:03 +08:00
rosemariejebbjtxbfp 836ee73d73 feat(public-frontend): 移除首页 API 调用说明入口
CI / Go (api) (pull_request) Failing after 1m31s
CI / Python (ingestion) (pull_request) Failing after 16s
CI / Migrations (postgres) (pull_request) Successful in 25s
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-24 04:44:05 +00:00
lixu dbbad274b6 Merge pull request 'feat(public-frontend): 公开前端 UI 视觉升级' (#22) from devin/1782270985-public-ui-redesign into main
CI / Go (api) (push) Successful in 55s
CI / Python (ingestion) (push) Failing after 14s
CI / Migrations (postgres) (push) Successful in 23s
2026-06-24 12:36:17 +08:00
rosemariejebbjtxbfp d58f46bc80 feat(public-frontend): 公开前端 UI 视觉升级
CI / Go (api) (pull_request) Failing after 32s
CI / Python (ingestion) (pull_request) Failing after 31s
CI / Migrations (postgres) (pull_request) Failing after 32s
引入统一品牌主题(色阶/字体/阴影/动效),重做首页 hero、卡片、导航与页脚,统一各页面的卡片/输入框/按钮样式。

- tailwind: 新增 brand 色阶、Inter 字体、card/glow 阴影与 fade-up 动效
- index.html: 引入 Inter 字体与 theme-color/description meta
- index.css: 双径向渐变背景 + @layer 组件类(.card/.input/.btn-primary 等)
- App/Home/ProductView/Contribute/ApiDocs/Account: 套用新设计系统

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-24 03:16:41 +00:00
lixu 8f9a03a929 Merge pull request 'feat(ingestion): import bypos-collector JSONL into goods' (#20) from devin/1782268418-bypos-importer into main
CI / Python (ingestion) (push) Successful in 11s
CI / Migrations (postgres) (push) Successful in 24s
CI / Go (api) (push) Successful in 49s
2026-06-24 10:35:27 +08:00
novaalphastrikeomegaz663 f04da0a135 feat(ingestion): import bypos-collector JSONL into goods
CI / Python (ingestion) (pull_request) Successful in 16s
CI / Migrations (postgres) (pull_request) Successful in 24s
CI / Go (api) (pull_request) Successful in 51s
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-24 02:33:45 +00:00
lixu a2ef7319e9 Merge pull request 'chore(tools): add bypos-collector (条码批量采集器源码备份)' (#19) from devin/1782267629-add-bypos-collector into main
CI / Python (ingestion) (push) Successful in 11s
CI / Migrations (postgres) (push) Successful in 22s
CI / Go (api) (push) Successful in 48s
2026-06-24 10:27:15 +08:00
novaalphastrikeomegaz663 e746b9cd31 chore(tools): add bypos-collector (条码批量采集器源码备份)
CI / Python (ingestion) (pull_request) Successful in 12s
CI / Migrations (postgres) (pull_request) Successful in 23s
CI / Go (api) (pull_request) Successful in 48s
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-24 02:20:34 +00:00
lixu ddda61252b Merge pull request '后台商品档案:点击表头排序' (#18) from devin/1782028836-product-sort into main
CI / Go (api) (push) Successful in 9s
CI / Python (ingestion) (push) Successful in 10s
CI / Migrations (postgres) (push) Successful in 15s
2026-06-21 16:01:56 +08:00
sulaimaannaasif6866 241fd38a56 feat(admin): sortable product list column headers
CI / Go (api) (pull_request) Successful in 11s
CI / Python (ingestion) (pull_request) Successful in 9s
CI / Migrations (postgres) (pull_request) Successful in 14s
Click a column header (名称/品牌/条码/品类/状态/质量分) to sort asc, click
again for desc, and a third time to clear back to the default
most-recently-updated order. Sort key/direction are whitelisted server-side.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-21 08:00:36 +00:00
lixu 3178f8a85a Merge pull request 'API 配额分级:免费累计 1000 次 + 公开注册自助领取更高配额密钥' (#17) from devin/1782026795-api-quota-registration into main
CI / Go (api) (push) Successful in 10s
CI / Python (ingestion) (push) Successful in 9s
CI / Migrations (postgres) (push) Successful in 17s
2026-06-21 15:28:59 +08:00
52 changed files with 3539 additions and 238 deletions
+45 -13
View File
@@ -5,6 +5,10 @@ on:
branches: [main]
pull_request:
env:
GO_VERSION: "1.23.12"
GOPROXY: "https://goproxy.cn,direct"
jobs:
go:
name: Go (api)
@@ -25,11 +29,22 @@ jobs:
env:
OPENGOODS_DATABASE_URL: postgres://opengoods:opengoods@postgres:5432/opengoods?sslmode=disable
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
with:
go-version: "1.23"
cache-dependency-path: api/go.sum
- name: Checkout
working-directory: ${{ github.workspace }}
run: |
git config --global --add safe.directory '*'
git init -q .
git remote add origin "${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}.git"
git -c protocol.version=2 fetch -q --no-tags --depth 1 origin "${GITHUB_REF}"
git checkout -q --force FETCH_HEAD
- name: Setup Go
working-directory: ${{ github.workspace }}
run: |
curl -fsSL -o /tmp/go.tgz "https://mirrors.aliyun.com/golang/go${GO_VERSION}.linux-amd64.tar.gz"
rm -rf /usr/local/go
tar -C /usr/local -xzf /tmp/go.tgz
echo "/usr/local/go/bin" >> "$GITHUB_PATH"
echo "$HOME/go/bin" >> "$GITHUB_PATH"
- name: Apply migrations
working-directory: .
run: |
@@ -44,14 +59,20 @@ jobs:
python:
name: Python (ingestion)
runs-on: ubuntu-latest
container:
image: nikolaik/python-nodejs:python3.12-nodejs20
defaults:
run:
working-directory: ingestion
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.12"
- name: Checkout
working-directory: ${{ github.workspace }}
run: |
git config --global --add safe.directory '*'
git init -q .
git remote add origin "${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}.git"
git -c protocol.version=2 fetch -q --no-tags --depth 1 origin "${GITHUB_REF}"
git checkout -q --force FETCH_HEAD
- name: Install
run: pip install -e ".[dev]"
- name: Ruff lint
@@ -77,10 +98,21 @@ jobs:
env:
DBURL: postgres://opengoods:opengoods@postgres:5432/opengoods?sslmode=disable
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
with:
go-version: "1.23"
- name: Checkout
working-directory: ${{ github.workspace }}
run: |
git config --global --add safe.directory '*'
git init -q .
git remote add origin "${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}.git"
git -c protocol.version=2 fetch -q --no-tags --depth 1 origin "${GITHUB_REF}"
git checkout -q --force FETCH_HEAD
- name: Setup Go
run: |
curl -fsSL -o /tmp/go.tgz "https://mirrors.aliyun.com/golang/go${GO_VERSION}.linux-amd64.tar.gz"
rm -rf /usr/local/go
tar -C /usr/local -xzf /tmp/go.tgz
echo "/usr/local/go/bin" >> "$GITHUB_PATH"
echo "$HOME/go/bin" >> "$GITHUB_PATH"
- name: Install golang-migrate
run: go install -tags 'postgres' github.com/golang-migrate/migrate/v4/cmd/migrate@v4.18.1
- name: Migrate up
+11 -2
View File
@@ -51,14 +51,23 @@ export const api = {
body: JSON.stringify({ username, password }),
}),
me: () => request<{ username: string }>("/me"),
listProducts: (q: string, page: number, size: number) =>
listProducts: (
q: string,
page: number,
size: number,
sort?: string,
order?: string,
) =>
request<{
items: import("./types").ProductRow[];
page: number;
size: number;
total: number;
completeness_fields: string[];
}>(`/products?q=${encodeURIComponent(q)}&page=${page}&size=${size}`),
}>(
`/products?q=${encodeURIComponent(q)}&page=${page}&size=${size}` +
(sort ? `&sort=${sort}&order=${order || "asc"}` : ""),
),
getProduct: (id: string) =>
request<import("./types").ProductDetail>(`/products/${id}`),
createProduct: (body: unknown) =>
+68 -9
View File
@@ -1,7 +1,15 @@
import { useEffect, useState } from "react";
import { api, ApiError } from "../api";
import { Brand, Category, FIELD_LABELS, ProductRow } from "../types";
import { Search, AlertCircle, Plus } from "lucide-react";
import { Search, AlertCircle, Plus, ChevronUp, ChevronDown, ChevronsUpDown } from "lucide-react";
type SortKey =
| "name"
| "brand"
| "gtin"
| "category_path"
| "status"
| "quality_score";
const STATUS_LABEL: Record<string, string> = {
active: "在用",
@@ -24,6 +32,42 @@ function QualityBadge({ score }: { score: number }) {
);
}
function SortableTh({
label,
sortKey,
sort,
order,
onSort,
}: {
label: string;
sortKey: SortKey;
sort: SortKey | "";
order: "asc" | "desc";
onSort: (key: SortKey) => void;
}) {
const active = sort === sortKey;
return (
<th className="px-4 py-3">
<button
type="button"
onClick={() => onSort(sortKey)}
className={`flex items-center gap-1 uppercase hover:text-gray-700 ${
active ? "text-emerald-600" : ""
}`}
>
{label}
{!active ? (
<ChevronsUpDown className="h-3.5 w-3.5 text-gray-300" />
) : order === "asc" ? (
<ChevronUp className="h-3.5 w-3.5" />
) : (
<ChevronDown className="h-3.5 w-3.5" />
)}
</button>
</th>
);
}
export default function ProductList({
onOpen,
}: {
@@ -33,6 +77,8 @@ export default function ProductList({
const [input, setInput] = useState("");
const [page, setPage] = useState(1);
const [size, setSize] = useState(20);
const [sort, setSort] = useState<SortKey | "">("");
const [order, setOrder] = useState<"asc" | "desc">("asc");
const [jump, setJump] = useState("");
const [rows, setRows] = useState<ProductRow[]>([]);
const [total, setTotal] = useState(0);
@@ -49,7 +95,7 @@ export default function ProductList({
setLoading(true);
setError("");
api
.listProducts(q, page, size)
.listProducts(q, page, size, sort || undefined, order)
.then((r) => {
setRows(r.items);
setTotal(r.total);
@@ -61,7 +107,20 @@ export default function ProductList({
useEffect(() => {
setSelected(new Set());
reload();
}, [q, page, size]);
}, [q, page, size, sort, order]);
function toggleSort(key: SortKey) {
setPage(1);
if (sort !== key) {
setSort(key);
setOrder("asc");
} else if (order === "asc") {
setOrder("desc");
} else {
setSort("");
setOrder("asc");
}
}
useEffect(() => {
api.listCategories().then((r) => setCategories(r.items)).catch(() => {});
@@ -245,12 +304,12 @@ export default function ProductList({
aria-label="全选"
/>
</th>
<th className="px-4 py-3"></th>
<th className="px-4 py-3"></th>
<th className="px-4 py-3"></th>
<th className="px-4 py-3"></th>
<th className="px-4 py-3"></th>
<th className="px-4 py-3"></th>
<SortableTh label="名称" sortKey="name" sort={sort} order={order} onSort={toggleSort} />
<SortableTh label="品牌" sortKey="brand" sort={sort} order={order} onSort={toggleSort} />
<SortableTh label="条码" sortKey="gtin" sort={sort} order={order} onSort={toggleSort} />
<SortableTh label="品类" sortKey="category_path" sort={sort} order={order} onSort={toggleSort} />
<SortableTh label="状态" sortKey="status" sort={sort} order={order} onSort={toggleSort} />
<SortableTh label="质量分" sortKey="quality_score" sort={sort} order={order} onSort={toggleSort} />
<th className="px-4 py-3"></th>
</tr>
</thead>
+7 -1
View File
@@ -9,6 +9,7 @@ import (
"github.com/jackc/pgx/v5/pgxpool"
"github.com/baicai2026-baicai/goods/api/internal/cache"
"github.com/baicai2026-baicai/goods/api/internal/config"
"github.com/baicai2026-baicai/goods/api/internal/handler"
"github.com/baicai2026-baicai/goods/api/internal/publicweb"
@@ -36,7 +37,12 @@ func main() {
if !limiter.Enabled() {
log.Print("warning: Redis not configured; public API rate limiting disabled")
}
h := handler.New(store.New(pool), publicweb.Dist()).
readCache := cache.New(cfg.RedisURL)
if !readCache.Enabled() {
log.Print("warning: Redis not configured; public API read cache disabled")
}
h := handler.New(store.New(pool).WithCache(readCache), publicweb.Dist()).
WithRateLimit(limiter, cfg.AnonRateLimitPerMin).
WithQuotas(cfg.AnonTotalQuota, cfg.RegisteredRateLimitPerMin, cfg.RegisteredQuotaTotal)
+67 -1
View File
@@ -14,6 +14,7 @@ import (
"github.com/go-chi/chi/v5/middleware"
"github.com/baicai2026-baicai/goods/api/internal/adminstore"
"github.com/baicai2026-baicai/goods/api/internal/apikey"
"github.com/baicai2026-baicai/goods/api/internal/auth"
"github.com/baicai2026-baicai/goods/api/internal/gtin"
"github.com/baicai2026-baicai/goods/api/internal/ratelimit"
@@ -90,6 +91,8 @@ func (h *Handler) Router() http.Handler {
r.Put("/api/categories/{id}", h.UpdateCategory)
r.Delete("/api/categories/{id}", h.DeleteCategory)
r.Post("/api/import/bypos", h.ImportBypos)
r.Get("/api/submissions", h.ListSubmissions)
r.Get("/api/submissions/{id}", h.GetSubmission)
r.Post("/api/submissions/{id}/approve", h.ApproveSubmission)
@@ -108,6 +111,11 @@ func (h *Handler) Router() http.Handler {
// admin approves them.
r.Post("/api/public/submissions", h.CreateSubmission)
// Archive backflow: inventory-management software pushes products missing
// from the archive. Authenticated with a public API key; records enter the
// same moderation queue and are archived only after admin approval.
r.Post("/api/public/backflow", h.Backflow)
return r
}
@@ -164,8 +172,10 @@ func (h *Handler) Me(w http.ResponseWriter, r *http.Request) {
// ListProducts returns a paginated product list.
func (h *Handler) ListProducts(w http.ResponseWriter, r *http.Request) {
q := r.URL.Query().Get("q")
sort := r.URL.Query().Get("sort")
order := r.URL.Query().Get("order")
page, size := pageParams(r)
items, total, err := h.store.ListProducts(r.Context(), q, size, (page-1)*size)
items, total, err := h.store.ListProducts(r.Context(), q, sort, order, size, (page-1)*size)
if h.handleErr(w, err) {
return
}
@@ -630,6 +640,62 @@ func (h *Handler) CreateSubmission(w http.ResponseWriter, r *http.Request) {
writeJSON(w, http.StatusCreated, map[string]string{"id": id, "status": "pending"})
}
// Backflow accepts a batch of products pushed by inventory-management software.
// It authenticates with a public API key (X-API-Key or Bearer), enqueues each
// item for admin review (deduplicating by GTIN), and returns a per-item summary.
func (h *Handler) Backflow(w http.ResponseWriter, r *http.Request) {
raw := presentedAPIKey(r)
if raw == "" {
writeError(w, http.StatusUnauthorized, "missing_api_key", "缺少 API key(请在 X-API-Key 或 Authorization: Bearer 中提供)")
return
}
if !apikey.IsWellFormed(raw) {
writeError(w, http.StatusUnauthorized, "invalid_api_key", "API key 格式无效")
return
}
if _, err := h.store.APIKeyByHash(r.Context(), apikey.Hash(raw)); err != nil {
if errors.Is(err, adminstore.ErrNotFound) {
writeError(w, http.StatusUnauthorized, "invalid_api_key", "API key 无效或已吊销")
return
}
writeError(w, http.StatusInternalServerError, "internal_error", err.Error())
return
}
var items []adminstore.SubmissionInput
if err := json.NewDecoder(http.MaxBytesReader(w, r.Body, 16<<20)).Decode(&items); err != nil {
writeError(w, http.StatusBadRequest, "bad_request", "请求体应为商品数组 (JSON array)")
return
}
if len(items) == 0 {
writeError(w, http.StatusBadRequest, "bad_request", "回流列表为空")
return
}
if len(items) > 1000 {
writeError(w, http.StatusBadRequest, "too_many", "单次回流最多 1000 条")
return
}
sum, err := h.store.CreateBackflowSubmissions(r.Context(), items, realIP(r))
if err != nil {
writeError(w, http.StatusInternalServerError, "internal_error", err.Error())
return
}
writeJSON(w, http.StatusOK, sum)
}
// presentedAPIKey extracts a public API key from X-API-Key or a Bearer token.
func presentedAPIKey(r *http.Request) string {
if v := strings.TrimSpace(r.Header.Get("X-API-Key")); v != "" {
return v
}
if v := r.Header.Get("Authorization"); strings.HasPrefix(v, "Bearer ") {
return strings.TrimSpace(strings.TrimPrefix(v, "Bearer "))
}
return ""
}
// ListSubmissions returns the moderation queue (admin).
func (h *Handler) ListSubmissions(w http.ResponseWriter, r *http.Request) {
status := r.URL.Query().Get("status")
+30
View File
@@ -0,0 +1,30 @@
package adminhandler
import (
"encoding/json"
"net/http"
"github.com/baicai2026-baicai/goods/api/internal/adminstore"
)
func (h *Handler) ImportBypos(w http.ResponseWriter, r *http.Request) {
var records []adminstore.ByposRecord
dec := json.NewDecoder(http.MaxBytesReader(w, r.Body, 50<<20))
for dec.More() {
var rec adminstore.ByposRecord
if err := dec.Decode(&rec); err != nil {
continue
}
records = append(records, rec)
}
if len(records) == 0 {
writeError(w, http.StatusBadRequest, "empty", "\u6ca1\u6709\u53ef\u5bfc\u5165\u7684\u8bb0\u5f55")
return
}
result, err := h.store.ImportByposRecords(r.Context(), records)
if err != nil {
writeError(w, http.StatusInternalServerError, "import_error", err.Error())
return
}
writeJSON(w, http.StatusOK, result)
}
+30 -2
View File
@@ -9,6 +9,7 @@ import (
"encoding/json"
"errors"
"strconv"
"strings"
"time"
"github.com/jackc/pgx/v5"
@@ -50,8 +51,34 @@ type ProductRow struct {
UpdatedAt string `json:"updated_at"`
}
// productSortColumns whitelists the sortable list columns, mapping the API sort
// key to a SQL expression. NULLs sort last regardless of direction.
var productSortColumns = map[string]string{
"name": "p.name",
"brand": "b.name",
"gtin": "p.gtin",
"category_path": "c.path",
"status": "p.status",
"quality_score": "p.quality_score",
"updated_at": "p.updated_at",
}
// productOrderBy returns a safe ORDER BY clause for the given sort key/direction,
// falling back to the default (most recently updated first) for unknown keys.
func productOrderBy(sort, order string) string {
col, ok := productSortColumns[sort]
if !ok {
return "p.updated_at DESC"
}
dir := "ASC"
if strings.EqualFold(order, "desc") {
dir = "DESC"
}
return col + " " + dir + " NULLS LAST, p.updated_at DESC"
}
// ListProducts returns a paginated, optionally name/gtin-filtered list.
func (s *Store) ListProducts(ctx context.Context, q string, limit, offset int) ([]ProductRow, int, error) {
func (s *Store) ListProducts(ctx context.Context, q, sort, order string, limit, offset int) ([]ProductRow, int, error) {
args := []any{}
where := "WHERE 1=1"
if q != "" {
@@ -84,7 +111,8 @@ FROM product p
LEFT JOIN brand b ON b.id = p.brand_id
LEFT JOIN category c ON c.id = p.category_id
LEFT JOIN food_detail f ON f.product_id = p.id ` + where +
" ORDER BY p.updated_at DESC LIMIT $" + strconv.Itoa(len(args)-1) + " OFFSET $" + strconv.Itoa(len(args))
" ORDER BY " + productOrderBy(sort, order) +
" LIMIT $" + strconv.Itoa(len(args)-1) + " OFFSET $" + strconv.Itoa(len(args))
rows, err := s.pool.Query(ctx, sql, args...)
if err != nil {
+158
View File
@@ -0,0 +1,158 @@
package adminstore
import (
"context"
"encoding/json"
"errors"
"strings"
"github.com/baicai2026-baicai/goods/api/internal/gtin"
"github.com/jackc/pgx/v5"
)
// backflowSource is the value stored in submission.payload->>'source' for
// records pushed by inventory-management software via the backflow API.
const backflowSource = "backflow"
// APIKeyAuth is the minimal key metadata needed to authenticate a backflow
// caller. Only active (non-revoked) keys resolve.
type APIKeyAuth struct {
ID string
Name string
}
// APIKeyByHash returns the active key matching a SHA-256 hash, or ErrNotFound
// when no such active key exists. Used to authenticate machine callers (e.g.
// the backflow endpoint) with the same public API keys issued to API users.
func (s *Store) APIKeyByHash(ctx context.Context, hash string) (*APIKeyAuth, error) {
var k APIKeyAuth
err := s.pool.QueryRow(ctx,
"SELECT id, name FROM api_key WHERE key_hash = $1 AND revoked_at IS NULL", hash,
).Scan(&k.ID, &k.Name)
if errors.Is(err, pgx.ErrNoRows) {
return nil, ErrNotFound
}
if err != nil {
return nil, err
}
return &k, nil
}
// BackflowResult reports the outcome of one item in a backflow batch.
type BackflowResult struct {
GTIN *string `json:"gtin"`
Name string `json:"name"`
Status string `json:"status"` // queued | exists | duplicate | invalid
ID string `json:"id,omitempty"` // submission id when queued
Reason string `json:"reason,omitempty"`
}
// BackflowSummary aggregates a backflow batch outcome.
type BackflowSummary struct {
Total int `json:"total"`
Queued int `json:"queued"`
Exists int `json:"exists"`
Duplicate int `json:"duplicate"`
Invalid int `json:"invalid"`
Results []BackflowResult `json:"results"`
}
// CreateBackflowSubmissions enqueues products pushed by inventory software for
// admin review. Each item is deduplicated by GTIN: items whose barcode already
// matches an archived product are skipped ("exists"), and items that duplicate
// a pending submission are skipped ("duplicate"). Accepted items are tagged
// with source="backflow" and enter the same moderation queue as public
// contributions; approval creates the product exactly as ApproveSubmission does.
//
// Items are processed independently: a bad item never rolls back accepted ones.
func (s *Store) CreateBackflowSubmissions(ctx context.Context, items []SubmissionInput, remoteIP string) (BackflowSummary, error) {
sum := BackflowSummary{Total: len(items), Results: make([]BackflowResult, 0, len(items))}
for _, in := range items {
in.Name = strings.TrimSpace(in.Name)
res := BackflowResult{Name: in.Name}
if in.GTIN != nil {
g := strings.TrimSpace(*in.GTIN)
if g == "" {
in.GTIN = nil
} else {
// Validate/normalize so a malformed barcode is reported as an
// invalid item instead of aborting the batch (the gtin column
// is varchar(14) and only GS1 codes are archived).
norm, err := gtin.Normalize(g)
if err != nil {
res.GTIN = &g
res.Status = "invalid"
res.Reason = err.Error()
sum.Invalid++
sum.Results = append(sum.Results, res)
continue
}
in.GTIN = &norm
res.GTIN = &norm
}
}
if in.Name == "" {
res.Status = "invalid"
res.Reason = "商品名称不能为空"
sum.Invalid++
sum.Results = append(sum.Results, res)
continue
}
if in.GTIN != nil {
// Already archived: backflow only carries products we don't have yet.
var pid string
err := s.pool.QueryRow(ctx, "SELECT id FROM product WHERE gtin = $1", *in.GTIN).Scan(&pid)
if err == nil {
res.Status = "exists"
res.Reason = "该条码商品已收录"
sum.Exists++
sum.Results = append(sum.Results, res)
continue
} else if !errors.Is(err, pgx.ErrNoRows) {
return sum, err
}
// Collapse repeated auto-pushes of the same barcode in the queue.
var sid string
err = s.pool.QueryRow(ctx,
"SELECT id FROM submission WHERE gtin = $1 AND status = 'pending' LIMIT 1", *in.GTIN).Scan(&sid)
if err == nil {
res.Status = "duplicate"
res.Reason = "已有待审核的同条码回流记录"
res.ID = sid
sum.Duplicate++
sum.Results = append(sum.Results, res)
continue
} else if !errors.Is(err, pgx.ErrNoRows) {
return sum, err
}
}
src := backflowSource
in.Source = &src
payload, err := json.Marshal(in)
if err != nil {
return sum, err
}
var id string
err = s.pool.QueryRow(ctx, `
INSERT INTO submission (gtin, name, payload, submitter_name, submitter_contact, note, remote_ip)
VALUES ($1,$2,$3,$4,$5,$6,$7) RETURNING id`,
in.GTIN, in.Name, payload, in.SubmitterName, in.SubmitterContact, in.Note, remoteIP).Scan(&id)
if err != nil {
return sum, err
}
res.Status = "queued"
res.ID = id
sum.Queued++
sum.Results = append(sum.Results, res)
}
return sum, nil
}
+217
View File
@@ -0,0 +1,217 @@
package adminstore
import (
"context"
"encoding/json"
"strings"
"github.com/jackc/pgx/v5"
)
type ByposRecord struct {
Barcode string `json:"barcode"`
Name string `json:"name"`
Spec string `json:"spec"`
Unit string `json:"unit"`
Area string `json:"area"`
Manufacturer string `json:"manufacturer"`
License string `json:"license"`
InPrice string `json:"in_price"`
SellPrice string `json:"sell_price"`
Status string `json:"status"`
RetMsg string `json:"retmsg"`
FetchedAt string `json:"fetched_at"`
Source string `json:"source"`
}
type ImportByposResult struct {
Loaded int `json:"loaded"`
Skipped int `json:"skipped"`
Errored int `json:"errored"`
}
const byposSourceName = "bypos\u4e2d\u5fc3\u5e93"
const byposSourceURL = "https://zc.bypos.net"
func (s *Store) ensureByposSource(ctx context.Context, tx pgx.Tx) (string, error) {
var id string
err := tx.QueryRow(ctx, `
INSERT INTO source (name, homepage, license, trust_weight)
VALUES ($1, $2, 'proprietary', 0.6)
ON CONFLICT (name) DO UPDATE SET homepage = EXCLUDED.homepage
RETURNING id`, byposSourceName, byposSourceURL).Scan(&id)
return id, err
}
func (s *Store) ensureManufacturer(ctx context.Context, tx pgx.Tx, name string, country *string) (string, error) {
norm := strings.Join(strings.Fields(strings.ToLower(name)), " ")
var id string
err := tx.QueryRow(ctx, `
INSERT INTO manufacturer (name, normalized_name, country)
VALUES ($1, $2, $3)
ON CONFLICT (normalized_name) DO UPDATE SET name = manufacturer.name
RETURNING id`, name, norm, country).Scan(&id)
return id, err
}
func (s *Store) importByposRecord(ctx context.Context, tx pgx.Tx, rec ByposRecord, sourceID string) error {
if rec.Status != "hit" || strings.TrimSpace(rec.Name) == "" {
return nil
}
barcode := strings.TrimSpace(rec.Barcode)
name := strings.TrimSpace(rec.Name)
var country *string
if strings.HasPrefix(barcode, "69") {
c := "\u4e2d\u56fd"
country = &c
}
var manufacturerID *string
if mfr := strings.TrimSpace(rec.Manufacturer); mfr != "" {
mid, err := s.ensureManufacturer(ctx, tx, mfr, country)
if err != nil {
return err
}
manufacturerID = &mid
}
attrs := map[string]interface{}{}
if v := strings.TrimSpace(rec.Spec); v != "" {
attrs["spec"] = v
}
if v := strings.TrimSpace(rec.Unit); v != "" {
attrs["pack_unit"] = v
}
if v := strings.TrimSpace(rec.Area); v != "" {
attrs["origin_area"] = v
}
if v := strings.TrimSpace(rec.License); v != "" {
attrs["production_license"] = v
}
attrsJSON, _ := json.Marshal(attrs)
var productID string
if barcode != "" {
err := tx.QueryRow(ctx, `
INSERT INTO product (gtin, name, manufacturer_id, country_of_origin, attributes, status)
VALUES ($1, $2, $3, $4, $5, 'active')
ON CONFLICT (gtin) WHERE gtin IS NOT NULL DO UPDATE SET
name = EXCLUDED.name,
manufacturer_id = COALESCE(EXCLUDED.manufacturer_id, product.manufacturer_id),
country_of_origin = COALESCE(EXCLUDED.country_of_origin, product.country_of_origin),
attributes = product.attributes || EXCLUDED.attributes
RETURNING id`, barcode, name, manufacturerID, country, attrsJSON).Scan(&productID)
if err != nil {
return err
}
} else {
err := tx.QueryRow(ctx, `
INSERT INTO product (name, manufacturer_id, country_of_origin, attributes, status)
VALUES ($1, $2, $3, $4, 'active')
RETURNING id`, name, manufacturerID, country, attrsJSON).Scan(&productID)
if err != nil {
return err
}
}
_, _ = tx.Exec(ctx, "DELETE FROM product_msrp WHERE product_id = $1 AND source_id = $2", productID, sourceID)
if sp := strings.TrimSpace(rec.SellPrice); sp != "" && sp != "0" {
_, _ = tx.Exec(ctx, `
INSERT INTO product_msrp (product_id, amount, currency, region, source_id, source_url)
VALUES ($1, $2::numeric, 'CNY', 'CN', $3, $4)`, productID, sp, sourceID, byposSourceURL)
}
_, _ = tx.Exec(ctx, "DELETE FROM product_source WHERE product_id = $1 AND source_id = $2", productID, sourceID)
fields := []string{"name", "country_of_origin"}
if manufacturerID != nil {
fields = append(fields, "manufacturer")
}
if len(attrs) > 0 {
fields = append(fields, "attributes")
}
if barcode != "" {
fields = append(fields, "gtin")
}
rawJSON, _ := json.Marshal(rec)
fetchedAt := strings.TrimSpace(rec.FetchedAt)
if fetchedAt == "" {
fetchedAt = ""
}
if fetchedAt != "" {
_, err := tx.Exec(ctx, `
INSERT INTO product_source (product_id, source_id, url, fields, fetched_at, raw)
VALUES ($1, $2, $3, $4, $5::timestamptz, $6)`,
productID, sourceID, byposSourceURL, fields, fetchedAt, rawJSON)
if err != nil {
return err
}
} else {
_, err := tx.Exec(ctx, `
INSERT INTO product_source (product_id, source_id, url, fields, fetched_at, raw)
VALUES ($1, $2, $3, $4, now(), $5)`,
productID, sourceID, byposSourceURL, fields, rawJSON)
if err != nil {
return err
}
}
if barcode != "" {
gtinType := "EAN13"
switch len(barcode) {
case 8:
gtinType = "EAN8"
case 12:
gtinType = "UPC"
case 14:
gtinType = "GTIN14"
}
_, _ = tx.Exec(ctx, `
INSERT INTO product_barcode (product_id, gtin, gtin_type, pack_level, is_primary, source_id)
VALUES ($1, $2, $3, 'each', true, $4)
ON CONFLICT (gtin) DO NOTHING`, productID, barcode, gtinType, sourceID)
}
_, err := s.recomputeQualityTx(ctx, tx, productID)
return err
}
func (s *Store) ImportByposRecords(ctx context.Context, records []ByposRecord) (*ImportByposResult, error) {
result := &ImportByposResult{}
tx, err := s.pool.Begin(ctx)
if err != nil {
return nil, err
}
defer tx.Rollback(ctx)
sourceID, err := s.ensureByposSource(ctx, tx)
if err != nil {
return nil, err
}
for _, rec := range records {
if rec.Status != "hit" || strings.TrimSpace(rec.Name) == "" {
result.Skipped++
continue
}
sp, spErr := tx.Begin(ctx)
if spErr != nil {
result.Errored++
continue
}
if err := s.importByposRecord(ctx, sp, rec, sourceID); err != nil {
sp.Rollback(ctx)
result.Errored++
} else {
sp.Commit(ctx)
result.Loaded++
}
}
if err := tx.Commit(ctx); err != nil {
return nil, err
}
return result, nil
}
+42 -13
View File
@@ -31,6 +31,7 @@ type SubmissionInput struct {
CountryOfOrigin *string `json:"country_of_origin"`
IngredientsText *string `json:"ingredients_text"`
Nutriments map[string]any `json:"nutriments"`
Attributes map[string]any `json:"attributes"`
NutritionBasis *string `json:"nutrition_basis"`
ServingSize *string `json:"serving_size"`
NutriScore *string `json:"nutri_score"`
@@ -39,6 +40,10 @@ type SubmissionInput struct {
SubmitterName *string `json:"submitter_name"`
SubmitterContact *string `json:"submitter_contact"`
Note *string `json:"note"`
// Source tags the origin of the submission, stored inside the payload so no
// schema change is needed. Empty means the default public contribution
// ("community"); "backflow" marks records pushed by inventory software.
Source *string `json:"source,omitempty"`
}
// SubmissionRow is a queue-list row for the admin review table.
@@ -48,6 +53,7 @@ type SubmissionRow struct {
Name string `json:"name"`
Status string `json:"status"`
SubmitterName *string `json:"submitter_name"`
Source *string `json:"source"`
Matched bool `json:"matched"`
CreatedAt string `json:"created_at"`
ReviewedAt *string `json:"reviewed_at"`
@@ -128,8 +134,8 @@ func (s *Store) ListSubmissions(ctx context.Context, status string, limit, offse
args = append(args, limit, offset)
sql := `
SELECT id, gtin, name, status, submitter_name, (target_product_id IS NOT NULL),
created_at, reviewed_at
SELECT id, gtin, name, status, submitter_name, NULLIF(payload->>'source',''),
(target_product_id IS NOT NULL), created_at, reviewed_at
FROM submission ` + where +
" ORDER BY (status='pending') DESC, created_at DESC LIMIT $" +
strconv.Itoa(len(args)-1) + " OFFSET $" + strconv.Itoa(len(args))
@@ -145,7 +151,7 @@ FROM submission ` + where +
var r SubmissionRow
var created time.Time
var reviewed *time.Time
if err := rows.Scan(&r.ID, &r.GTIN, &r.Name, &r.Status, &r.SubmitterName, &r.Matched, &created, &reviewed); err != nil {
if err := rows.Scan(&r.ID, &r.GTIN, &r.Name, &r.Status, &r.SubmitterName, &r.Source, &r.Matched, &created, &reviewed); err != nil {
return nil, 0, err
}
r.CreatedAt = created.Format(time.RFC3339)
@@ -278,14 +284,19 @@ func (s *Store) ApproveSubmission(ctx context.Context, id, actor string) (*Produ
fields := submissionFields(in)
var attrJSON []byte
if len(in.Attributes) > 0 {
attrJSON, _ = json.Marshal(in.Attributes)
}
if productID == "" {
// Create a new product from the contribution.
err = tx.QueryRow(ctx, `
INSERT INTO product (gtin, name, brand_id, category_id, gpc_brick_code,
net_content_value, net_content_unit, net_content_canonical, country_of_origin, status)
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,'active') RETURNING id`,
net_content_value, net_content_unit, net_content_canonical, country_of_origin, attributes, status)
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,COALESCE($10::jsonb,'{}'::jsonb),'active') RETURNING id`,
in.GTIN, in.Name, brandID, in.CategoryID, gpc,
in.NetContentValue, in.NetContentUnit, canonical, in.CountryOfOrigin).Scan(&productID)
in.NetContentValue, in.NetContentUnit, canonical, in.CountryOfOrigin, attrJSON).Scan(&productID)
if err != nil {
return nil, err
}
@@ -302,21 +313,29 @@ UPDATE product SET
net_content_unit=COALESCE($7, net_content_unit),
net_content_canonical=COALESCE($8, net_content_canonical),
country_of_origin=COALESCE($9, country_of_origin),
gtin=COALESCE($10, gtin)
gtin=COALESCE($10, gtin),
attributes=product.attributes || COALESCE($11::jsonb,'{}'::jsonb)
WHERE id=$1`,
productID, in.Name, brandID, in.CategoryID, gpc,
in.NetContentValue, in.NetContentUnit, canonical, in.CountryOfOrigin, in.GTIN)
in.NetContentValue, in.NetContentUnit, canonical, in.CountryOfOrigin, in.GTIN, attrJSON)
if err != nil {
return nil, err
}
}
// food_detail: upsert, preserving existing values where not provided.
// food_detail: upsert only when the contribution carries food data, so a
// non-food submission (drug/3C/generic) doesn't create an empty row.
var nutriJSON []byte
if len(in.Nutriments) > 0 {
nutriJSON, _ = json.Marshal(in.Nutriments)
}
_, err = tx.Exec(ctx, `
foodPresent := len(in.Nutriments) > 0 ||
(in.IngredientsText != nil && *in.IngredientsText != "") ||
(in.NutritionBasis != nil && *in.NutritionBasis != "") ||
(in.ServingSize != nil && *in.ServingSize != "") ||
(in.NutriScore != nil && *in.NutriScore != "")
if foodPresent {
_, err = tx.Exec(ctx, `
INSERT INTO food_detail (product_id, ingredients_text, nutriments, nutrition_basis, serving_size, nutri_score)
VALUES ($1,$2,$3,$4,$5,$6)
ON CONFLICT (product_id) DO UPDATE SET
@@ -325,9 +344,10 @@ ON CONFLICT (product_id) DO UPDATE SET
nutrition_basis=COALESCE(EXCLUDED.nutrition_basis, food_detail.nutrition_basis),
serving_size=COALESCE(EXCLUDED.serving_size, food_detail.serving_size),
nutri_score=COALESCE(EXCLUDED.nutri_score, food_detail.nutri_score)`,
productID, in.IngredientsText, nutriJSON, in.NutritionBasis, in.ServingSize, in.NutriScore)
if err != nil {
return nil, err
productID, in.IngredientsText, nutriJSON, in.NutritionBasis, in.ServingSize, in.NutriScore)
if err != nil {
return nil, err
}
}
for _, im := range in.Images {
@@ -417,5 +437,14 @@ func submissionFields(in SubmissionInput) []string {
add("ingredients", in.IngredientsText != nil && *in.IngredientsText != "")
add("nutriments", len(in.Nutriments) > 0)
add("image", len(in.Images) > 0)
for k, v := range in.Attributes {
if v == nil {
continue
}
if s, ok := v.(string); ok && s == "" {
continue
}
fields = append(fields, k)
}
return fields
}
+126
View File
@@ -0,0 +1,126 @@
// Package cache is a Redis-backed, fail-open read cache for the public API.
//
// It caches hot product details and search results so repeated reads avoid
// PostgreSQL. Like the ratelimit package, every operation fails open: if Redis
// is unavailable or misconfigured the caller simply falls back to the database,
// so the cache can never take the API down or serve stale data after Redis loss.
//
// Invalidation is global and O(1): keys are namespaced by an epoch counter
// stored in Redis (og:cache:epoch). The Python ingestion bumps that counter
// after a write run, which logically invalidates every cached entry at once
// while old keys age out via their TTL. The epoch is read at most once per
// refresh interval per process, so it adds no per-request round trip.
package cache
import (
"context"
"encoding/json"
"errors"
"log"
"strconv"
"sync"
"time"
"github.com/redis/go-redis/v9"
)
// epochKey is the Redis key holding the global cache generation counter.
const epochKey = "og:cache:epoch"
// epochRefresh bounds how often a process re-reads the epoch from Redis.
const epochRefresh = 10 * time.Second
// opTimeout caps any single Redis operation so a slow backend never blocks a
// request beyond this; on timeout the cache fails open.
const opTimeout = 150 * time.Millisecond
// Cache wraps a Redis client. A nil-backed Cache (Redis unconfigured) disables
// caching: every Get misses and every Set is a no-op.
type Cache struct {
rdb *redis.Client
mu sync.RWMutex
epoch int64
epochSetAt time.Time
epochOK bool
}
// New builds a Cache from a redis:// URL. On a parse error it logs and returns a
// disabled (fail-open) cache so the server still boots.
func New(redisURL string) *Cache {
opt, err := redis.ParseURL(redisURL)
if err != nil {
log.Printf("cache: invalid redis url %q: %v (caching disabled)", redisURL, err)
return &Cache{}
}
return &Cache{rdb: redis.NewClient(opt)}
}
// Enabled reports whether a Redis backend is configured.
func (c *Cache) Enabled() bool { return c != nil && c.rdb != nil }
// epochNow returns the current cache generation, reading it from Redis at most
// once per epochRefresh. On any Redis error it keeps the last known value and
// throttles re-reads so a down backend cannot slow the hot path.
func (c *Cache) epochNow(ctx context.Context) int64 {
c.mu.RLock()
if c.epochOK && time.Since(c.epochSetAt) < epochRefresh {
e := c.epoch
c.mu.RUnlock()
return e
}
c.mu.RUnlock()
cctx, cancel := context.WithTimeout(ctx, opTimeout)
defer cancel()
n, err := c.rdb.Get(cctx, epochKey).Int64()
c.mu.Lock()
defer c.mu.Unlock()
switch {
case err == nil:
c.epoch = n
case errors.Is(err, redis.Nil):
c.epoch = 0
}
c.epochSetAt = time.Now()
c.epochOK = true
return c.epoch
}
// key namespaces a logical suffix under the current epoch.
func (c *Cache) key(ctx context.Context, suffix string) string {
return "og:v" + strconv.FormatInt(c.epochNow(ctx), 10) + ":" + suffix
}
// GetJSON unmarshals the cached value for suffix into dest and reports a hit.
// Any miss, decode error, or Redis error returns false (fail-open).
func (c *Cache) GetJSON(ctx context.Context, suffix string, dest any) bool {
if !c.Enabled() {
return false
}
k := c.key(ctx, suffix)
cctx, cancel := context.WithTimeout(ctx, opTimeout)
defer cancel()
b, err := c.rdb.Get(cctx, k).Bytes()
if err != nil {
return false
}
return json.Unmarshal(b, dest) == nil
}
// SetJSON stores val (JSON-encoded) for suffix with the given TTL. Best effort:
// marshal or Redis errors are ignored.
func (c *Cache) SetJSON(ctx context.Context, suffix string, val any, ttl time.Duration) {
if !c.Enabled() {
return
}
b, err := json.Marshal(val)
if err != nil {
return
}
k := c.key(ctx, suffix)
cctx, cancel := context.WithTimeout(ctx, opTimeout)
defer cancel()
_ = c.rdb.Set(cctx, k, b, ttl).Err()
}
+84
View File
@@ -0,0 +1,84 @@
package cache
import (
"context"
"fmt"
"os"
"testing"
"time"
)
// TestDisabledFailsOpen verifies a Cache without a Redis backend never panics,
// always misses, and silently drops writes.
func TestDisabledFailsOpen(t *testing.T) {
c := New("not-a-valid-url") // parse error => disabled
if c.Enabled() {
t.Fatal("expected cache to be disabled for invalid url")
}
c.SetJSON(context.Background(), "k", map[string]int{"a": 1}, time.Minute)
var dst map[string]int
if c.GetJSON(context.Background(), "k", &dst) {
t.Fatalf("disabled cache must always miss, got %+v", dst)
}
}
func testCache(t *testing.T) *Cache {
t.Helper()
url := os.Getenv("OPENGOODS_REDIS_URL")
if url == "" {
url = "redis://localhost:6379/0"
}
c := New(url)
if !c.Enabled() {
t.Skip("redis not configured")
}
ctx, cancel := context.WithTimeout(context.Background(), time.Second)
defer cancel()
if err := c.rdb.Ping(ctx).Err(); err != nil {
t.Skipf("redis not reachable: %v", err)
}
return c
}
// TestRoundTrip stores then reads a value back.
func TestRoundTrip(t *testing.T) {
c := testCache(t)
ctx := context.Background()
suffix := fmt.Sprintf("test:rt:%d", time.Now().UnixNano())
c.SetJSON(ctx, suffix, map[string]any{"name": "foo", "n": float64(3)}, time.Minute)
got := map[string]any{}
if !c.GetJSON(ctx, suffix, &got) {
t.Fatal("expected cache hit after set")
}
if got["name"] != "foo" || got["n"] != float64(3) {
t.Fatalf("unexpected payload: %+v", got)
}
}
// TestEpochInvalidation verifies that bumping the epoch counter logically drops
// every previously cached entry.
func TestEpochInvalidation(t *testing.T) {
c := testCache(t)
ctx := context.Background()
suffix := fmt.Sprintf("test:epoch:%d", time.Now().UnixNano())
c.SetJSON(ctx, suffix, map[string]int{"v": 1}, time.Minute)
var dst map[string]int
if !c.GetJSON(ctx, suffix, &dst) {
t.Fatal("expected hit before epoch bump")
}
// Simulate an ingestion write bumping the global epoch.
if err := c.rdb.Incr(ctx, epochKey).Err(); err != nil {
t.Fatalf("incr epoch: %v", err)
}
// Force the process to re-read the epoch rather than use its cached value.
c.mu.Lock()
c.epochOK = false
c.mu.Unlock()
if c.GetJSON(ctx, suffix, &dst) {
t.Fatal("entry should be invisible after epoch bump")
}
}
+16
View File
@@ -127,6 +127,7 @@ func (h *Handler) Router() http.Handler {
})
r.Get("/brands", h.ListBrands)
r.Get("/categories", h.ListCategories)
r.Get("/kind-fields", h.ListKindFields)
r.Get("/sources/{id}", h.SourceByID)
r.Get("/stats", h.Stats)
})
@@ -275,6 +276,21 @@ func (h *Handler) ListCategories(w http.ResponseWriter, r *http.Request) {
writeJSON(w, http.StatusOK, map[string]any{"items": items})
}
// ListKindFields returns the read-only spec field template for an archive kind,
// used by the contribution form to render kind-specific inputs.
func (h *Handler) ListKindFields(w http.ResponseWriter, r *http.Request) {
kind := strings.TrimSpace(r.URL.Query().Get("kind"))
if kind == "" {
writeError(w, r, http.StatusBadRequest, "bad_request", "missing kind")
return
}
items, err := h.store.ListKindFields(r.Context(), kind)
if h.handleErr(w, r, err) {
return
}
writeJSON(w, http.StatusOK, map[string]any{"items": items, "kind": kind})
}
// SourceByID returns a single data source.
func (h *Handler) SourceByID(w http.ResponseWriter, r *http.Request) {
src, err := h.store.SourceByID(r.Context(), chi.URLParam(r, "id"))
+16 -1
View File
@@ -208,7 +208,22 @@
"nutriments": { "type": "object", "additionalProperties": true, "nullable": true },
"nutrition_basis": { "type": "string", "nullable": true },
"nutri_score": { "type": "string", "nullable": true },
"ingredients_text": { "type": "string", "nullable": true }
"ingredients_text": { "type": "string", "nullable": true },
"msrp": {
"type": "array",
"description": "Manufacturer suggested retail price snapshots (reference only, newest first; zero-amount entries omitted).",
"items": {
"type": "object",
"properties": {
"amount": { "type": "number" },
"currency": { "type": "string" },
"region": { "type": "string" },
"effective_date": { "type": "string", "nullable": true },
"source_url": { "type": "string", "nullable": true },
"note": { "type": "string", "nullable": true }
}
}
}
}
}
}
+129 -7
View File
@@ -4,21 +4,35 @@ package store
import (
"context"
"crypto/sha1"
"encoding/hex"
"encoding/json"
"errors"
"strconv"
"strings"
"time"
"github.com/jackc/pgx/v5"
"github.com/jackc/pgx/v5/pgxpool"
"github.com/baicai2026-baicai/goods/api/internal/cache"
)
// Cache TTLs for the public read cache. Product details are far less volatile
// than search result sets, so they live longer; both are also invalidated
// wholesale whenever ingestion bumps the cache epoch.
const (
productCacheTTL = 24 * time.Hour
searchCacheTTL = time.Hour
)
// ErrNotFound is returned when a requested row does not exist.
var ErrNotFound = errors.New("not found")
// Store wraps a PostgreSQL connection pool.
// Store wraps a PostgreSQL connection pool and an optional read cache.
type Store struct {
pool *pgxpool.Pool
pool *pgxpool.Pool
cache *cache.Cache
}
// New constructs a Store from an existing pgx pool.
@@ -26,6 +40,30 @@ func New(pool *pgxpool.Pool) *Store {
return &Store{pool: pool}
}
// WithCache attaches a Redis-backed read cache. A nil or disabled cache leaves
// the Store reading straight from PostgreSQL.
func (s *Store) WithCache(c *cache.Cache) *Store {
s.cache = c
return s
}
// cacheGet reads a cached JSON value into dest, reporting a hit. It is a no-op
// miss when no cache is attached.
func (s *Store) cacheGet(ctx context.Context, suffix string, dest any) bool {
if s.cache == nil {
return false
}
return s.cache.GetJSON(ctx, suffix, dest)
}
// cacheSet stores a JSON value when a cache is attached.
func (s *Store) cacheSet(ctx context.Context, suffix string, val any, ttl time.Duration) {
if s.cache == nil {
return
}
s.cache.SetJSON(ctx, suffix, val, ttl)
}
// Ping verifies database connectivity.
func (s *Store) Ping(ctx context.Context) error {
return s.pool.Ping(ctx)
@@ -88,6 +126,7 @@ type Product struct {
Ingredients *string `json:"ingredients_text,omitempty"`
Allergens []string `json:"allergens,omitempty"`
Additives []string `json:"additives,omitempty"`
MSRP []MSRP `json:"msrp,omitempty"`
}
// ProductBarcodes returns every barcode attached to a product, primary first.
@@ -222,6 +261,10 @@ func stringifyAttr(v any) string {
// ProductByGTIN looks up an active product by any of its barcodes.
func (s *Store) ProductByGTIN(ctx context.Context, gtin string) (*Product, error) {
const suffix = "prod:gtin:"
if cached := new(Product); s.cacheGet(ctx, suffix+gtin, cached) {
return cached, nil
}
row := s.pool.QueryRow(ctx, productSelect+`
WHERE p.status = 'active'
AND (p.gtin = $1 OR EXISTS (
@@ -238,11 +281,19 @@ func (s *Store) ProductByGTIN(ctx context.Context, gtin string) (*Product, error
if p.Specs, err = s.buildSpecs(ctx, p.ArchiveKind, attrs); err != nil {
return nil, err
}
if p.MSRP, err = s.ListMSRP(ctx, p.ID); err != nil {
return nil, err
}
s.cacheSet(ctx, suffix+gtin, p, productCacheTTL)
return p, nil
}
// ProductByID looks up a product by its UUID.
func (s *Store) ProductByID(ctx context.Context, id string) (*Product, error) {
const suffix = "prod:id:"
if cached := new(Product); s.cacheGet(ctx, suffix+id, cached) {
return cached, nil
}
row := s.pool.QueryRow(ctx, productSelect+" WHERE p.id = $1", id)
p, attrs, err := scanProduct(row)
if err != nil {
@@ -254,6 +305,10 @@ func (s *Store) ProductByID(ctx context.Context, id string) (*Product, error) {
if p.Specs, err = s.buildSpecs(ctx, p.ArchiveKind, attrs); err != nil {
return nil, err
}
if p.MSRP, err = s.ListMSRP(ctx, p.ID); err != nil {
return nil, err
}
s.cacheSet(ctx, suffix+id, p, productCacheTTL)
return p, nil
}
@@ -268,6 +323,11 @@ const fuzzyThreshold = "0.42"
// similarity blended with quality_score so the best, most-complete records
// surface first. Without a query, results are ordered by quality_score.
func (s *Store) SearchProducts(ctx context.Context, f SearchFilters, limit, offset int) ([]ProductSummary, int, error) {
suffix := searchCacheSuffix(f, limit, offset)
if entry := new(searchCacheEntry); s.cacheGet(ctx, suffix, entry) {
return entry.Items, entry.Total, nil
}
args := []any{}
where := "WHERE p.status = 'active'"
@@ -334,7 +394,28 @@ LEFT JOIN category c ON c.id = p.category_id `
}
out = append(out, ps)
}
return out, total, rows.Err()
if err := rows.Err(); err != nil {
return nil, 0, err
}
s.cacheSet(ctx, suffix, searchCacheEntry{Items: out, Total: total}, searchCacheTTL)
return out, total, nil
}
// searchCacheEntry is the cached payload for a SearchProducts call.
type searchCacheEntry struct {
Items []ProductSummary `json:"items"`
Total int `json:"total"`
}
// searchCacheSuffix derives a stable cache key from the full filter set and
// paging window so distinct queries never collide.
func searchCacheSuffix(f SearchFilters, limit, offset int) string {
raw := strings.Join([]string{
f.Query, f.Category, f.Brand, f.Country,
strconv.Itoa(limit), strconv.Itoa(offset),
}, "\x1f")
sum := sha1.Sum([]byte(raw))
return "search:" + hex.EncodeToString(sum[:])
}
// Nutriments returns just the nutrition payload for a product.
@@ -371,11 +452,12 @@ type MSRP struct {
Note *string `json:"note"`
}
// ListMSRP returns all MSRP snapshots for a product.
// ListMSRP returns a product's suggested-retail-price snapshots, newest first.
// Zero-amount entries are excluded as they carry no price information.
func (s *Store) ListMSRP(ctx context.Context, id string) ([]MSRP, error) {
rows, err := s.pool.Query(ctx,
`SELECT amount, currency, region, effective_date::text, source_url, note
FROM product_msrp WHERE product_id = $1 ORDER BY effective_date DESC NULLS LAST`, id)
FROM product_msrp WHERE product_id = $1 AND amount > 0 ORDER BY effective_date DESC NULLS LAST`, id)
if err != nil {
return nil, err
}
@@ -427,12 +509,13 @@ type Category struct {
Path string `json:"path"`
GPCBrickCode *string `json:"gpc_brick_code"`
Level int `json:"level"`
ArchiveKind string `json:"archive_kind"`
}
// ListCategories returns the full category tree ordered by path.
func (s *Store) ListCategories(ctx context.Context) ([]Category, error) {
rows, err := s.pool.Query(ctx,
"SELECT id, name_zh, name_en, path::text, gpc_brick_code, level FROM category ORDER BY path")
"SELECT id, name_zh, name_en, path::text, gpc_brick_code, level, COALESCE(archive_kind, 'generic') FROM category ORDER BY path")
if err != nil {
return nil, err
}
@@ -440,7 +523,7 @@ func (s *Store) ListCategories(ctx context.Context) ([]Category, error) {
out := []Category{}
for rows.Next() {
var c Category
if err := rows.Scan(&c.ID, &c.NameZH, &c.NameEN, &c.Path, &c.GPCBrickCode, &c.Level); err != nil {
if err := rows.Scan(&c.ID, &c.NameZH, &c.NameEN, &c.Path, &c.GPCBrickCode, &c.Level, &c.ArchiveKind); err != nil {
return nil, err
}
out = append(out, c)
@@ -448,6 +531,45 @@ func (s *Store) ListCategories(ctx context.Context) ([]Category, error) {
return out, rows.Err()
}
// KindField describes one editable spec field for an archive kind. It drives
// the dynamic contribution form (public, read-only view of the template).
type KindField struct {
Kind string `json:"kind"`
FieldKey string `json:"field_key"`
GroupLabel string `json:"group_label"`
LabelZH string `json:"label_zh"`
FieldType string `json:"field_type"`
Unit *string `json:"unit"`
Options []string `json:"options"`
Placeholder *string `json:"placeholder"`
SortOrder int `json:"sort_order"`
Qualified bool `json:"qualified"`
}
// ListKindFields returns the ordered field template for one archive kind so the
// public contribution form can render kind-specific inputs.
func (s *Store) ListKindFields(ctx context.Context, kind string) ([]KindField, error) {
rows, err := s.pool.Query(ctx, `
SELECT kind, field_key, group_label, label_zh, field_type, unit, options,
placeholder, sort_order, qualified
FROM kind_field WHERE kind = $1 ORDER BY sort_order, field_key`, kind)
if err != nil {
return nil, err
}
defer rows.Close()
out := []KindField{}
for rows.Next() {
var f KindField
if err := rows.Scan(&f.Kind, &f.FieldKey, &f.GroupLabel, &f.LabelZH,
&f.FieldType, &f.Unit, &f.Options, &f.Placeholder, &f.SortOrder,
&f.Qualified); err != nil {
return nil, err
}
out = append(out, f)
}
return out, rows.Err()
}
// APIKey is the minimal metadata the public API needs to authorize a caller.
type APIKey struct {
ID string
+58
View File
@@ -112,6 +112,64 @@ curl "https://goods.tangshasha.com/api/v1/products/barcode/5449000000996"
### `GET /sources/{id}` — 数据来源
## 档案回流(写接口,需 API Key)
> 仅供进销存等机器调用方使用:把档案里**尚未收录**的商品批量回流到站点,进入人工审核队列,审核通过后才会收录。**必须携带 API Key**(与上文同一类 `og_live_` 密钥),不会直接写入商品。
### `POST /api/public/backflow` — 批量回流未收录商品
- 鉴权:请求头携带 `X-API-Key: og_live_xxxxxxxx`(或 `Authorization: Bearer og_live_xxxxxxxx`)。缺失/无效/已吊销返回 `401`
- 请求体:商品对象**数组**(与公众投稿同结构),单次最多 `1000` 条。常用字段:
| 字段 | 必填 | 说明 |
| --- | --- | --- |
| `name` | 是 | 商品名称 |
| `gtin` | 否 | 条码(GTIN)。强烈建议提供,用于去重 |
| `brand_name` | 否 | 品牌名 |
| `category_id` | 否 | 品类编码,如 `food.beverages` |
| `net_content_value` / `net_content_unit` | 否 | 净含量数值 / 单位 |
| `country_of_origin` | 否 | 产地 |
| `ingredients_text` | 否 | 配料表 |
| `nutriments` | 否 | 营养成分对象 |
| `msrp` | 否 | 零售价快照数组,元素含 `amount`/`currency`/`region`/`effective_date` |
| `note` | 否 | 备注 |
> 来源会自动标记为 `source="backflow"`,在后台审核队列中与公众投稿区分,无需调用方提供。
- 去重(按 `gtin` 逐条判断,互不影响):
- 该条码已收录为商品 → `exists`,跳过;
- 已存在同条码的待审核回流 → `duplicate`,跳过(避免反复刷队列);
- 否则入队 → `queued`status=`pending`,等待后台审核);
- 名称为空等 → `invalid`
```bash
curl -X POST "https://goods.tangshasha.com/api/public/backflow" \
-H "X-API-Key: og_live_xxxxxxxx" \
-H "Content-Type: application/json" \
-d '[
{"name":"某某牛奶 250ml","gtin":"6901234567890","brand_name":"某品牌",
"net_content_value":250,"net_content_unit":"ml",
"msrp":[{"amount":3.5,"currency":"CNY","region":"CN"}]},
{"name":"已收录商品","gtin":"5449000000996"}
]'
```
```json
{
"total": 2,
"queued": 1,
"exists": 1,
"duplicate": 0,
"invalid": 0,
"results": [
{ "gtin": "6901234567890", "name": "某某牛奶 250ml", "status": "queued", "id": "<submission-id>" },
{ "gtin": "5449000000996", "name": "已收录商品", "status": "exists", "reason": "该条码商品已收录" }
]
}
```
审核通过后,系统按提交内容**新建商品**;若审核时该条码已存在商品,则**补全**到已有商品(逻辑与公众投稿一致)。
## 免责声明
数据可能存在误差或滞后,按「现状」提供,不构成医疗/购买建议。商品资料版权归各原始来源所有,请遵循其许可(如 OpenFoodFacts 的 ODbL),引用时请注明天工商品档案公共仓及原始来源。
+112
View File
@@ -0,0 +1,112 @@
# 可扩展性设计与路线图 (Scalability Roadmap)
本文档回答一个长期问题:随着商品越来越多、品类越来越杂(食品 / 电子 3C / 药品 /
……),**检索与新增会不会压垮数据库?要不要按品类「分表」?**
> 结论先行:**现阶段不要按品类手动分表。** 现有「单表 + JSONB + archive_kind 框架」
> 的设计方向是对的。扩展应当靠 **分区(非分表) + 读副本 + 缓存 + 专用搜索引擎**,
> 按数据量分阶段推进,避免提前过度设计。
---
## 1. 现状盘点
### 1.1 数据模型
- 所有商品落在**一张 `product` 表**;非食品的领域字段存 `product.attributes`(JSONB)。
- 食品有独立明细表 `food_detail`(配料 / 营养 / 过敏原等结构化字段)。
- `0010_archive_kinds` 引入 **archive_kind 框架**:每个品类带一个 `archive_kind`
(`food` / `electronics` / `generic`),`kind_field` 表按 kind 定义字段模板,
驱动后台动态表单与合格度评分。
- **加新品类(如药品)不需要新建表**:只要新增一组 `kind_field` 行 + 一棵品类子树;
仅当某品类有大量需被独立筛选/排序的结构化字段时,才考虑像 `food_detail` 那样补一张
明细表。
### 1.2 已有索引(检索性能的基础)
| 对象 | 索引 | 用途 |
|------|------|------|
| `product.gtin` | 唯一索引 | 条码精确查 |
| `product.name` | trigram GIN (`pg_trgm`) | 名称模糊/相似匹配 |
| `product.search_tsv` | 全文 GIN (`tsvector`) | 全文检索 |
| `product.attributes` | JSONB GIN | 属性过滤 |
| `brand.name` | trigram GIN | 品牌模糊匹配 |
| `product.country_of_origin` | btree | 产地精确/前缀过滤 |
| category / brand / updated_at | btree | 关联与增量 |
### 1.3 检索方式
- 有关键词时:`name ILIKE` `word_similarity ≥ 阈值(~0.42)` 条码匹配,
排序按 `相似度 × (0.5 + quality_score)`
- 无关键词时:按 `quality_score` 排序。
- 翻页:`LIMIT / OFFSET`
### 1.4 写入特征
- 写库**只有** Python ingestion 一条路径(批量 ETL),不是高并发 OLTP。
- 插入瓶颈极低;`search_tsv` 由触发器逐行重算,正常增量下开销可忽略。
---
## 2. 为什么不建议按品类「分表」
1. **核心场景是全局检索**:用户通常不知道商品属于哪个品类,搜索要跨所有品类。
按品类拆成多表后,一次搜索得 `UNION ALL` 所有表,**更慢、代码更复杂、排序更难统一**。
2. **单表足够能打**:Postgres 单表配好索引,**几千万行**量级的检索完全可承载。
"表大"很少是真正瓶颈,"搜索方式"和"读并发"才是。
3. **分表会侵蚀框架优势**:archive_kind 框架的价值就是"加品类零建表";手动分表等于
把这套通用能力又拆碎。
> 区分两个概念:**分表(sharding,应用层拆多张表)** ≠ **分区(Postgres 原生
> declarative partitioning,对上层透明的一张逻辑表)**。后者在超大规模时才有意义,
> 见第 3 阶段。
---
## 3. 分阶段路线图(按数据量触发,不提前做)
### 阶段 0 — 现在 ~ 数百万条:维持现状 + 低成本优化
触发:当前规模。改动小、收益稳,建议尽早做:
- **深翻页改 keyset 分页**:`OFFSET` 越翻越慢(需扫描并丢弃前 N 行);改用
`WHERE (score, id) < (:last_score, :last_id)` 形式的游标分页。
- **部分索引**:绝大多数查询限定 `status='active'`,可建
`CREATE INDEX ... WHERE status='active'` 缩小索引、提速。
- **常用筛选复合索引**:如 `(category_id, quality_score DESC)`
`(archive_kind, quality_score DESC)` 配合域内列表。
- **Redis 缓存**(已在技术栈内):缓存热门搜索结果与商品详情,挡住重复读。
### 阶段 1 — 千万级以上:读扩展 + 调优
触发:单库读 QPS 升高、P99 变慢。
- **只读副本(read replica)**:本服务是**只读公益 API**,天然适合一主多从,
把检索/详情读流量分到副本,主库只承接 ingestion 写入。
- **索引与查询调优**:按慢查询日志补/删索引,`EXPLAIN ANALYZE` 校核计划。
- **(可选)Postgres 原生分区**:若多数检索"限定单一域"(只搜药品 / 只搜食品),
可按 `archive_kind`**LIST 分区**(对上层透明,仍是一张逻辑表)。主要利好
**维护**(分区级 vacuum / 归档)与**域内查询裁剪**;对真正的全局搜索帮助有限。
### 阶段 2 — 搜索相关性/规模成为痛点:引入专用搜索引擎
触发:`pg_trgm`/`tsvector` 在相关性排序、跨字段检索、规模上吃力。
- 把**检索**迁到专用倒排引擎:**OpenSearch / Meilisearch / Typesense**,
或 Postgres 内的 **ParadeDB(`pg_search`,BM25)**
- **Postgres 仍是唯一事实来源**;搜索引擎只做索引,由 ingestion 在写库后同步。
- 这才是"搜索量大"的正解,**比分表有效得多**。
---
## 4. 大批量导入的建议
- 海量初始化/回填用 `COPY` 而非逐行 `INSERT`
- 超大批量时可"先停建二级索引 → COPY → 重建索引",比边插边维护索引快得多。
- ETL 控制并发与批大小,避免与在线读争抢。
---
## 5. 药品档案怎么落地(回到最初的问题)
在上述设计下,加"药品"属于**阶段 0 的常规扩展**,不触动架构:
1. 新增 `drug``kind_field` 模板(批准文号 / 通用名 / 商品名 / 剂型 / 规格 /
生产企业 / OTC 分类 / 适应症 / 用法用量 / 不良反应 / 禁忌 / 注意事项 / 贮藏 /
有效期 等),`qualified` 标记关键字段参与合格度评分。
2. 加一棵药品品类子树,并把这些品类的 `archive_kind` 置为 `drug`
3. 仅当药品需要**被独立筛选/排序的强结构化字段**(如按批准文号精确查、按 OTC 分类
过滤)时,才考虑补一张 `drug_detail` 明细表;否则继续走 `attributes` JSONB。
---
## 6. 版本
- 本路线图随规模演进更新;任何落地改动需同步:迁移(SQL) + 本文档 +(涉及对外字段时)
`docs/data-contract.md` / `docs/openapi.yaml`
+116
View File
@@ -0,0 +1,116 @@
"""Adapter for the bypos-collector output (central product library zc.bypos.net).
The ``bypos-collector`` tool (see ``tools/bypos-collector``) queries the same
central product library that the 云店 POS uses when adding a product by barcode,
and writes one JSON object per line (JSONL) with these fields::
barcode name spec unit area manufacturer license
in_price sell_price status retmsg fetched_at source
This module turns one such record into the internal product shape consumed by
:func:`opengoods.etl.load.load_bypos_record`. It is a pure function (no DB, no
network) so it is easy to unit-test.
The central library is a Chinese retail catalogue: it provides 品名/规格/单位/
产地/厂商/建议价 but **not** ingredients or nutrition, so no ``food`` block is
produced.
"""
from __future__ import annotations
from decimal import Decimal, InvalidOperation
from opengoods.etl.transform import _clamp, is_valid_gtin, parse_quantity
from opengoods.units import UnitError, normalize
SOURCE_NAME = "bypos中心库"
SOURCE_HOMEPAGE = "https://zc.bypos.net"
# Vendor catalogue data; not an open-licensed dataset. Display facts only.
SOURCE_LICENSE = "proprietary"
SOURCE_TRUST = 0.6
def _to_price(text: str | None) -> Decimal | None:
"""Parse a price string to a positive Decimal, or None for empty/zero."""
if not text:
return None
try:
amount = Decimal(str(text).strip())
except (InvalidOperation, ValueError):
return None
if amount <= 0:
return None
return amount
def _net_content(spec: str | None) -> tuple[Decimal | None, str | None, Decimal | None]:
"""Best-effort parse a spec like '500mL'/'5kg' to (value, unit, canonical).
Packaging-style specs ('20支', '1X24', '') have no mass/volume unit and
yield ``(None, None, None)`` — the raw spec is kept in attributes instead.
"""
parsed = parse_quantity(spec or "")
if not parsed:
return None, None, None
value, unit = parsed
try:
norm = normalize(value, unit)
except UnitError:
return None, None, None
return norm.value, norm.unit, norm.canonical_value
def transform_bypos(rec: dict) -> dict | None:
"""Transform one bypos-collector JSONL record into an internal product dict.
Returns ``None`` for non-hit rows or rows without a usable name.
"""
if rec.get("status") != "hit":
return None
name = (rec.get("name") or "").strip()
if not name:
return None
barcode = str(rec.get("barcode") or "").strip()
gtin = barcode if barcode and is_valid_gtin(barcode) else None
net_value, net_unit, net_canonical = _net_content(rec.get("spec"))
spec = (rec.get("spec") or "").strip()
pack_unit = (rec.get("unit") or "").strip()
area = (rec.get("area") or "").strip()
manufacturer = (rec.get("manufacturer") or "").strip() or None
license_no = (rec.get("license") or "").strip()
in_price = _to_price(rec.get("in_price"))
sell_price = _to_price(rec.get("sell_price"))
attributes: dict[str, object] = {}
if spec:
attributes["spec"] = spec
if pack_unit:
attributes["pack_unit"] = pack_unit
if area:
attributes["origin_area"] = area
if license_no:
attributes["production_license"] = license_no
if in_price is not None:
attributes["suggested_in_price"] = float(in_price)
if sell_price is not None:
attributes["suggested_retail_price"] = float(sell_price)
# Domestic GS1-China barcodes (69x) are China-made; area is a province/city,
# kept separately in attributes.origin_area.
country = "中国" if gtin and gtin.startswith("69") else None
return {
"gtin": gtin,
"name": name,
"manufacturer": manufacturer,
"net_content_value": net_value,
"net_content_unit": _clamp(net_unit, 16),
"net_content_canonical": net_canonical,
"country_of_origin": country,
"attributes": attributes,
"msrp": sell_price,
"fetched_at": (rec.get("fetched_at") or "").strip() or None,
}
+52
View File
@@ -0,0 +1,52 @@
"""Read-cache invalidation signal for the public API.
The Go API caches hot product details and search results in Redis, namespacing
every key by a global generation counter (``og:cache:epoch``). Bumping that
counter logically invalidates the entire cache in one O(1) operation while the
old keys age out via their TTL.
Ingestion is the only writer to the database, so after a run that changed data
it calls :func:`bump_cache_epoch` to make those changes visible immediately
instead of waiting for per-key TTLs to expire.
Like the API's cache, this is strictly best effort and fails open: if Redis is
unconfigured or unreachable the ingestion run still succeeds, and stale entries
simply expire on their own.
"""
from __future__ import annotations
import logging
import os
logger = logging.getLogger(__name__)
EPOCH_KEY = "og:cache:epoch"
def default_redis_url() -> str | None:
"""Return the configured Redis URL, or ``None`` when caching is disabled."""
return os.environ.get("OPENGOODS_REDIS_URL") or None
def bump_cache_epoch(url: str | None = None) -> bool:
"""Increment the API cache generation counter.
Returns ``True`` if the counter was bumped, ``False`` if caching is disabled
or Redis was unreachable. Never raises: invalidation failures must not fail
an ingestion run.
"""
url = url or default_redis_url()
if not url:
return False
try:
import redis # imported lazily so the dependency is optional at runtime
client = redis.Redis.from_url(url, socket_timeout=2, socket_connect_timeout=2)
new_epoch = client.incr(EPOCH_KEY)
client.close()
logger.info("bumped API cache epoch to %s", new_epoch)
return True
except Exception as exc: # noqa: BLE001 - invalidation is best effort
logger.warning("cache epoch bump skipped: %s", exc)
return False
+155
View File
@@ -14,6 +14,10 @@ from typing import Any
import psycopg
from psycopg.types.json import Jsonb
from opengoods.adapters.bypos import SOURCE_HOMEPAGE as SOURCE_HOMEPAGE_BYPOS
from opengoods.adapters.bypos import SOURCE_LICENSE as SOURCE_LICENSE_BYPOS
from opengoods.adapters.bypos import SOURCE_NAME as SOURCE_NAME_BYPOS
from opengoods.adapters.bypos import SOURCE_TRUST as SOURCE_TRUST_BYPOS
from opengoods.adapters.openfoodfacts import OFF_LICENSE, SOURCE_NAME
from opengoods.etl.quality import update_quality
@@ -73,6 +77,23 @@ def _ensure_brand(conn: psycopg.Connection, name: str | None) -> str | None:
return row[0]
def _ensure_manufacturer(
conn: psycopg.Connection, name: str | None, country: str | None = None
) -> str | None:
if not name:
return None
row = conn.execute(
"""
INSERT INTO manufacturer (name, normalized_name, country)
VALUES (%s, %s, %s)
ON CONFLICT (normalized_name) DO UPDATE SET name = manufacturer.name
RETURNING id
""",
(name, _normalize_brand(name), country),
).fetchone()
return row[0]
def _category_id(conn: psycopg.Connection, path: str | None) -> tuple[str | None, str | None]:
if not path:
return None, None
@@ -218,6 +239,140 @@ def load_record_safe(
return False
def ensure_bypos_source(conn: psycopg.Connection) -> str:
"""Upsert the bypos central-library source row and return its id."""
return ensure_source_named(
conn,
SOURCE_NAME_BYPOS,
SOURCE_HOMEPAGE_BYPOS,
SOURCE_LICENSE_BYPOS,
SOURCE_TRUST_BYPOS,
)
def load_bypos_record(
conn: psycopg.Connection, rec: dict[str, Any], source_id: str, raw: dict
) -> str:
"""Upsert one transformed bypos record; return the product id.
Unlike OFF records these have no ingredients/nutrition, so no ``food_detail``
row is written. The suggested retail price (if any) is stored as a CNY MSRP
snapshot, and provenance/MSRP rows are keyed by source so a re-import
refreshes rather than duplicates them.
"""
manufacturer_id = _ensure_manufacturer(
conn, rec.get("manufacturer"), rec.get("country_of_origin")
)
attrs = rec.get("attributes") or {}
fields = ["name", "net_content", "country_of_origin"]
if manufacturer_id:
fields.append("manufacturer")
if attrs:
fields.append("attributes")
if rec.get("gtin"):
fields.append("gtin")
prod = conn.execute(
"""
INSERT INTO product (gtin, name, manufacturer_id,
net_content_value, net_content_unit, net_content_canonical,
country_of_origin, attributes)
VALUES (%s,%s,%s,%s,%s,%s,%s,%s)
ON CONFLICT (gtin) WHERE gtin IS NOT NULL DO UPDATE SET
name = EXCLUDED.name,
manufacturer_id = COALESCE(EXCLUDED.manufacturer_id, product.manufacturer_id),
net_content_value = COALESCE(EXCLUDED.net_content_value, product.net_content_value),
net_content_unit = COALESCE(EXCLUDED.net_content_unit, product.net_content_unit),
net_content_canonical = COALESCE(
EXCLUDED.net_content_canonical, product.net_content_canonical),
country_of_origin = COALESCE(EXCLUDED.country_of_origin, product.country_of_origin),
attributes = product.attributes || EXCLUDED.attributes
RETURNING id
""",
(
rec["gtin"],
rec["name"],
manufacturer_id,
rec.get("net_content_value"),
rec.get("net_content_unit"),
rec.get("net_content_canonical"),
rec.get("country_of_origin"),
Jsonb(attrs),
),
).fetchone()
else:
prod = conn.execute(
"""
INSERT INTO product (name, manufacturer_id,
net_content_value, net_content_unit, net_content_canonical,
country_of_origin, attributes)
VALUES (%s,%s,%s,%s,%s,%s,%s)
RETURNING id
""",
(
rec["name"],
manufacturer_id,
rec.get("net_content_value"),
rec.get("net_content_unit"),
rec.get("net_content_canonical"),
rec.get("country_of_origin"),
Jsonb(attrs),
),
).fetchone()
product_id = prod[0]
# Refresh this source's MSRP snapshot (suggested retail price, CNY).
conn.execute(
"DELETE FROM product_msrp WHERE product_id = %s AND source_id = %s",
(product_id, source_id),
)
if rec.get("msrp") is not None:
conn.execute(
"""
INSERT INTO product_msrp (product_id, amount, currency, region, source_id, source_url)
VALUES (%s,%s,'CNY','CN',%s,%s)
""",
(product_id, rec["msrp"], source_id, SOURCE_HOMEPAGE_BYPOS),
)
fields.append("msrp")
# Refresh this source's provenance row (one per source for idempotency).
conn.execute(
"DELETE FROM product_source WHERE product_id = %s AND source_id = %s",
(product_id, source_id),
)
conn.execute(
"""
INSERT INTO product_source (product_id, source_id, url, fields, fetched_at, raw)
VALUES (%s,%s,%s,%s, COALESCE(%s::timestamptz, now()), %s)
""",
(
product_id,
source_id,
SOURCE_HOMEPAGE_BYPOS,
fields,
rec.get("fetched_at"),
Jsonb(_jsonable(raw)),
),
)
update_quality(conn, product_id)
return product_id
def load_bypos_record_safe(
conn: psycopg.Connection, rec: dict[str, Any], source_id: str, raw: dict
) -> bool:
"""Load one bypos record inside a savepoint (see :func:`load_record_safe`)."""
try:
with conn.transaction():
load_bypos_record(conn, rec, source_id, raw)
return True
except Exception as exc: # noqa: BLE001 - per-record isolation is intentional
logger.warning("skipping bypos record gtin=%s: %s", rec.get("gtin"), exc)
return False
def _jsonable(raw: dict) -> dict:
"""Drop values that are not JSON-serializable from a raw record."""
try:
+3
View File
@@ -12,6 +12,7 @@ import sys
import psycopg
from opengoods.cache import bump_cache_epoch
from opengoods.etl.dedup import dedup_all
from opengoods.etl.load import default_dsn
@@ -23,6 +24,8 @@ def run(args: argparse.Namespace) -> int:
conn.rollback()
else:
conn.commit()
if not args.dry_run and summary["merged"]:
bump_cache_epoch()
mode = "dry-run" if args.dry_run else "applied"
print(f"{mode} groups={summary['groups']} merged={summary['merged']}")
return 0
+80
View File
@@ -0,0 +1,80 @@
"""Import products collected by the bypos-collector tool into OpenGoods.
The ``tools/bypos-collector`` program writes one product per line (JSONL). This
job reads such a file, transforms each ``hit`` record into the internal product
shape, and upserts it into the database under the ``bypos中心库`` source with
field-level provenance.
Usage::
python -m opengoods.jobs.import_bypos --input products.jsonl
python -m opengoods.jobs.import_bypos --input products.jsonl --limit 500
Re-running is safe: products are upserted by GTIN and the source's MSRP/
provenance rows are refreshed rather than duplicated.
"""
from __future__ import annotations
import argparse
import gzip
import json
import sys
from collections.abc import Iterator
import psycopg
from opengoods.adapters.bypos import transform_bypos
from opengoods.cache import bump_cache_epoch
from opengoods.etl.load import default_dsn, ensure_bypos_source, load_bypos_record_safe
def _read_jsonl(path: str) -> Iterator[dict]:
opener = gzip.open if path.endswith(".gz") else open
with opener(path, "rt", encoding="utf-8") as fh:
for line in fh:
line = line.strip()
if not line:
continue
try:
yield json.loads(line)
except json.JSONDecodeError:
continue
def run(args: argparse.Namespace) -> int:
loaded = skipped = errored = 0
with psycopg.connect(args.dsn, autocommit=False) as conn:
source_id = ensure_bypos_source(conn)
yielded = 0
for raw in _read_jsonl(args.input):
if args.limit and yielded >= args.limit:
break
rec = transform_bypos(raw)
if rec is None:
skipped += 1
continue
yielded += 1
if load_bypos_record_safe(conn, rec, source_id, raw):
loaded += 1
else:
errored += 1
conn.commit()
if loaded:
bump_cache_epoch()
print(f"loaded={loaded} skipped={skipped} errored={errored}")
return 0
def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser(description="Import bypos-collector JSONL into OpenGoods")
parser.add_argument(
"--input", required=True, help="path to a bypos-collector JSONL (.jsonl or .jsonl.gz)"
)
parser.add_argument("--limit", type=int, default=0, help="max hit records to load (0 = all)")
parser.add_argument("--dsn", default=default_dsn(), help="PostgreSQL DSN")
return run(parser.parse_args(argv))
if __name__ == "__main__":
sys.exit(main())
+3
View File
@@ -28,6 +28,7 @@ from opengoods.adapters.openfoodfacts import (
is_cn_gs1,
read_dump,
)
from opengoods.cache import bump_cache_epoch
from opengoods.etl.load import default_dsn, ensure_source, load_record_safe
from opengoods.etl.transform import transform
@@ -67,6 +68,8 @@ def run(args: argparse.Namespace) -> int:
else:
errored += 1
conn.commit()
if loaded:
bump_cache_epoch()
print(f"loaded={loaded} skipped={skipped} errored={errored}")
return 0
+3
View File
@@ -17,6 +17,7 @@ import sys
import psycopg
from opengoods.adapters.openfoodfacts import SOURCE_NAME, OpenFoodFactsAdapter
from opengoods.cache import bump_cache_epoch
from opengoods.etl.load import default_dsn, ensure_source, load_record_safe
from opengoods.etl.state import get_watermark, set_watermark
from opengoods.etl.transform import transform
@@ -49,6 +50,8 @@ def run(args: argparse.Namespace) -> int:
stats={"loaded": loaded, "skipped": skipped, "errored": errored, "since": since},
)
conn.commit()
if loaded:
bump_cache_epoch()
print(
f"since={since} loaded={loaded} skipped={skipped} "
f"errored={errored} watermark={high_watermark}"
+1
View File
@@ -6,6 +6,7 @@ requires-python = ">=3.11"
dependencies = [
"httpx>=0.27",
"psycopg[binary]>=3.2",
"redis>=5.0,<6",
]
[project.optional-dependencies]
+155
View File
@@ -0,0 +1,155 @@
"""Tests for the bypos-collector adapter/transform and DB loader.
The pure-function tests run anywhere; the DB roundtrip is skipped automatically
when no database is reachable or migrations are not applied.
"""
from __future__ import annotations
from decimal import Decimal
import pytest
from opengoods.adapters.bypos import transform_bypos
psycopg = pytest.importorskip("psycopg")
# A real central-library "hit" row as emitted by the collector.
HIT = {
"barcode": "6901028941068",
"name": "泰山合悦",
"spec": "20支",
"unit": "",
"area": "广东",
"manufacturer": "",
"license": "",
"in_price": "22.50",
"sell_price": "28.00",
"status": "hit",
"retmsg": "获取商品信息成功",
"fetched_at": "2026-06-24T02:08:34Z",
"source": "zc.bypos.net",
}
# A hit with a real mass/volume spec that should normalize to net content.
HIT_VOLUME = {
"barcode": "6920459905012",
"name": "康师傅冰红茶490ml",
"spec": "490毫升",
"unit": "",
"area": "浙江杭州",
"manufacturer": "",
"license": "",
"in_price": "2.20",
"sell_price": "3.00",
"status": "hit",
"fetched_at": "2026-06-24T02:08:34Z",
"source": "zc.bypos.net",
}
def test_transform_skips_non_hit():
assert transform_bypos({**HIT, "status": "miss", "name": ""}) is None
assert transform_bypos({**HIT, "status": "invalid"}) is None
def test_transform_skips_missing_name():
assert transform_bypos({**HIT, "name": " "}) is None
def test_transform_packaging_spec_kept_raw():
rec = transform_bypos(HIT)
assert rec is not None
assert rec["gtin"] == "6901028941068"
assert rec["name"] == "泰山合悦"
# '20支' is a count, not mass/volume -> no net_content, raw spec retained.
assert rec["net_content_value"] is None
assert rec["net_content_unit"] is None
assert rec["attributes"]["spec"] == "20支"
assert rec["attributes"]["pack_unit"] == ""
assert rec["attributes"]["origin_area"] == "广东"
assert rec["attributes"]["suggested_in_price"] == 22.5
assert rec["attributes"]["suggested_retail_price"] == 28.0
assert rec["msrp"] == Decimal("28.00")
assert rec["country_of_origin"] == "中国"
def test_transform_volume_spec_normalized():
rec = transform_bypos(HIT_VOLUME)
assert rec is not None
assert rec["net_content_value"] == Decimal("490")
assert rec["net_content_unit"] == "ml"
assert rec["net_content_canonical"] == Decimal("490")
assert rec["attributes"]["origin_area"] == "浙江杭州"
def test_transform_zero_price_dropped():
rec = transform_bypos({**HIT, "in_price": "0.00", "sell_price": "0.00"})
assert rec is not None
assert rec["msrp"] is None
assert "suggested_in_price" not in rec["attributes"]
assert "suggested_retail_price" not in rec["attributes"]
def test_transform_invalid_barcode_no_gtin():
rec = transform_bypos({**HIT, "barcode": "123"})
assert rec is not None
assert rec["gtin"] is None
assert rec["country_of_origin"] is None
# --- DB roundtrip (skipped without a database) ---------------------------------
@pytest.fixture()
def conn():
from opengoods.etl.load import default_dsn
try:
c = psycopg.connect(default_dsn(), connect_timeout=3)
except psycopg.OperationalError as exc: # pragma: no cover - env dependent
pytest.skip(f"no database available: {exc}")
has_product = c.execute("SELECT to_regclass('public.product') IS NOT NULL").fetchone()[0]
if not has_product:
c.close()
pytest.skip("migrations not applied")
yield c
c.rollback()
c.close()
def test_load_bypos_roundtrip(conn):
from opengoods.etl.load import ensure_bypos_source, load_bypos_record
source_id = ensure_bypos_source(conn)
rec = transform_bypos(HIT_VOLUME)
product_id = load_bypos_record(conn, rec, source_id, HIT_VOLUME)
row = conn.execute(
"SELECT name, gtin, net_content_unit, country_of_origin, attributes ->> 'pack_unit' "
"FROM product WHERE id = %s",
(product_id,),
).fetchone()
assert row[0] == "康师傅冰红茶490ml"
assert row[1] == "6920459905012"
assert row[2] == "ml"
assert row[3] == "中国"
assert row[4] == ""
msrp = conn.execute(
"SELECT amount, currency FROM product_msrp WHERE product_id = %s AND source_id = %s",
(product_id, source_id),
).fetchone()
assert msrp[0] == Decimal("3.00")
assert msrp[1] == "CNY"
# Re-import is idempotent: still one MSRP and one provenance row per source.
load_bypos_record(conn, rec, source_id, HIT_VOLUME)
counts = conn.execute(
"SELECT (SELECT count(*) FROM product_msrp WHERE product_id=%s AND source_id=%s), "
"(SELECT count(*) FROM product_source WHERE product_id=%s AND source_id=%s)",
(product_id, source_id, product_id, source_id),
).fetchone()
assert counts == (1, 1)
conn.rollback()
+44
View File
@@ -0,0 +1,44 @@
"""Tests for the best-effort API cache invalidation signal."""
from __future__ import annotations
import sys
import opengoods.cache as cache
def test_disabled_when_no_url(monkeypatch):
monkeypatch.delenv("OPENGOODS_REDIS_URL", raising=False)
assert cache.default_redis_url() is None
assert cache.bump_cache_epoch() is False
def test_bump_fails_open_when_unreachable(monkeypatch):
# An unroutable URL must not raise; the run still succeeds.
monkeypatch.setenv("OPENGOODS_REDIS_URL", "redis://127.0.0.1:1/0")
assert cache.bump_cache_epoch() is False
def test_bump_increments_epoch(monkeypatch):
calls = {}
class FakeClient:
def incr(self, key):
calls["key"] = key
return 7
def close(self):
calls["closed"] = True
class FakeRedis:
@staticmethod
def from_url(url, **kwargs):
calls["url"] = url
return FakeClient()
monkeypatch.setitem(sys.modules, "redis", type("M", (), {"Redis": FakeRedis}))
assert cache.bump_cache_epoch("redis://example:6379/0") is True
assert calls["key"] == "og:cache:epoch"
assert calls["url"] == "redis://example:6379/0"
assert calls["closed"] is True
+2
View File
@@ -0,0 +1,2 @@
DROP INDEX IF EXISTS idx_product_active_category;
DROP INDEX IF EXISTS idx_product_active_quality;
+15
View File
@@ -0,0 +1,15 @@
-- 检索优化(可扩展性路线图 阶段0):为最常见的"浏览"路径补部分/复合索引。
-- 搜索查询恒带 WHERE status = 'active';无关键词时按 quality_score DESC, name 排序。
-- 现有索引无法同时满足"过滤 active + 按 quality_score/name 排序",深翻页时需要对全部
-- active 行排序。下面的部分复合索引让规划器直接走索引顺序扫描,省掉排序、加速深翻页与
-- count(*)。索引只覆盖 active 行,体积更小。
-- 默认浏览(无关键词、无品类):ORDER BY quality_score DESC, name
CREATE INDEX IF NOT EXISTS idx_product_active_quality
ON product (quality_score DESC, name)
WHERE status = 'active';
-- 品类内浏览:先按 category_id 收敛,再按 quality_score 排序
CREATE INDEX IF NOT EXISTS idx_product_active_category
ON product (category_id, quality_score DESC)
WHERE status = 'active';
@@ -0,0 +1,8 @@
-- Reverse 0013: detach products from the seeded drug categories, drop the drug
-- category subtree, and remove the drug field template.
UPDATE product SET category_id = NULL
WHERE category_id IN (SELECT id FROM category WHERE path <@ 'drug');
DELETE FROM category WHERE path <@ 'drug';
DELETE FROM kind_field WHERE kind = 'drug';
+43
View File
@@ -0,0 +1,43 @@
-- Drug (药品) archive kind: a new domain on the generic archive_kind framework
-- (see 0010). No new table — drug spec values live in product.attributes JSONB,
-- driven by the kind_field template below, plus a 'drug' category subtree.
-- Field template for the drug domain. field_type is constrained to
-- text/number/textarea/select/list by kind_field_type_chk. The `qualified`
-- fields are the identifying / regulatory essentials that count toward the
-- kind-aware completeness score.
INSERT INTO kind_field (kind, field_key, group_label, label_zh, field_type, unit, options, sort_order, qualified) VALUES
('drug','approval_number', '基础信息','批准文号', 'text', NULL, '{}', 10, true),
('drug','generic_name', '基础信息','通用名', 'text', NULL, '{}', 20, true),
('drug','trade_name', '基础信息','商品名', 'text', NULL, '{}', 30, false),
('drug','dosage_form', '基础信息','剂型', 'select', NULL, '{片剂,胶囊剂,颗粒剂,散剂,丸剂,注射剂,口服液,糖浆剂,软膏剂,乳膏剂,凝胶剂,滴剂,喷雾剂,气雾剂,栓剂,贴剂,其他}', 40, true),
('drug','specification', '基础信息','规格', 'text', NULL, '{}', 50, true),
('drug','pack_spec', '基础信息','包装规格', 'text', NULL, '{}', 60, false),
('drug','rx_class', '分类与企业','处方分类', 'select', NULL, '{处方药,甲类OTC,乙类OTC}', 70, true),
('drug','manufacturer', '分类与企业','生产企业', 'text', NULL, '{}', 80, true),
('drug','mah', '分类与企业','上市许可持有人','text', NULL, '{}', 90, false),
('drug','indications', '说明书','适应症/功能主治', 'textarea', NULL, '{}', 100, true),
('drug','dosage_usage', '说明书','用法用量', 'textarea', NULL, '{}', 110, false),
('drug','adverse_reactions', '说明书','不良反应', 'textarea', NULL, '{}', 120, false),
('drug','contraindications', '说明书','禁忌', 'textarea', NULL, '{}', 130, false),
('drug','precautions', '说明书','注意事项', 'textarea', NULL, '{}', 140, false),
('drug','interactions', '说明书','药物相互作用', 'textarea', NULL, '{}', 150, false),
('drug','storage', '贮藏与有效期','贮藏', 'text', NULL, '{}', 160, false),
('drug','shelf_life', '贮藏与有效期','有效期', 'text', NULL, '{}', 170, false),
('drug','spec_source', '贮藏与有效期','说明书来源', 'text', NULL, '{}', 180, false);
-- Drug category tree. ltree labels are English slugs (no spaces/Chinese);
-- Chinese names live in category.name_zh.
INSERT INTO category (name_zh, name_en, parent_id, path, level, archive_kind)
VALUES ('药品', 'Drug', NULL, 'drug', 0, 'drug');
INSERT INTO category (name_zh, name_en, parent_id, path, level, archive_kind)
SELECT v.name_zh, v.name_en, c.id, v.path::ltree, 1, 'drug'
FROM (VALUES
('化学药品', 'Chemical drug', 'drug.chemical'),
('中成药', 'Chinese patent medicine', 'drug.tcm_patent'),
('中药饮片', 'Chinese herbal pieces', 'drug.tcm_herb'),
('生物制品', 'Biological product', 'drug.biological'),
('疫苗', 'Vaccine', 'drug.vaccine')
) AS v(name_zh, name_en, path)
JOIN category c ON c.path = 'drug';
+12 -1
View File
@@ -3,7 +3,18 @@
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>天工商品档案公共仓</title>
<meta name="theme-color" content="#059669" />
<meta
name="description"
content="天工商品档案公共仓——公益、开放、可溯源的商品事实查询与开放 API。"
/>
<title>天工 · 商品档案公共仓</title>
<link rel="preconnect" href="https://fonts.googleapis.com" />
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin />
<link
href="https://fonts.googleapis.com/css2?family=Inter:wght@400;500;600;700;800&display=swap"
rel="stylesheet"
/>
</head>
<body>
<div id="root"></div>
@@ -0,0 +1 @@
9ef4c7d0ea804c11bc637f4e82198e98af908894
+66 -52
View File
@@ -1,10 +1,11 @@
import { useEffect, useState } from "react";
import { Boxes, Search, PlusCircle, Code2, KeyRound } from "lucide-react";
import { Boxes, Search, PlusCircle, Code2, KeyRound, Headset } from "lucide-react";
import Home from "./components/Home";
import ProductView from "./components/ProductView";
import Contribute from "./components/Contribute";
import ApiDocs from "./components/ApiDocs";
import Account from "./components/Account";
import Contact from "./components/Contact";
import { api } from "./api";
type View =
@@ -12,7 +13,16 @@ type View =
| { name: "product"; id: string }
| { name: "contribute" }
| { name: "api" }
| { name: "account" };
| { name: "account" }
| { name: "contact" };
const NAV: { key: View["name"]; label: string; icon: typeof Search }[] = [
{ key: "home", label: "检索", icon: Search },
{ key: "contribute", label: "贡献档案", icon: PlusCircle },
{ key: "api", label: "API", icon: Code2 },
{ key: "account", label: "API 密钥", icon: KeyRound },
{ key: "contact", label: "联系我们", icon: Headset },
];
export default function App() {
const [view, setView] = useState<View>({ name: "home" });
@@ -27,58 +37,50 @@ export default function App() {
return (
<div className="min-h-full flex flex-col">
<header className="bg-white border-b">
<div className="max-w-5xl mx-auto px-4 h-14 flex items-center justify-between">
<header className="sticky top-0 z-30 border-b border-gray-200/70 bg-white/80 backdrop-blur-md">
<div className="max-w-5xl mx-auto px-4 h-16 flex items-center justify-between gap-4">
<button
className="flex items-center gap-2 font-semibold text-gray-800"
className="flex items-center gap-2.5 font-semibold text-gray-800 group"
onClick={() => setView({ name: "home" })}
>
<Boxes className="w-6 h-6 text-emerald-600" />
<span className="text-gray-400 font-normal text-sm"></span>
<span className="grid h-9 w-9 place-items-center rounded-xl bg-gradient-to-br from-brand-500 to-brand-700 text-white shadow-glow transition group-hover:scale-105">
<Boxes className="w-5 h-5" />
</span>
<span className="flex items-baseline gap-1.5">
<span className="text-lg tracking-tight"></span>
<span className="hidden sm:inline text-gray-400 font-normal text-xs">
</span>
</span>
</button>
<nav className="flex items-center gap-1 text-sm">
<button
className={`px-3 py-1.5 rounded-md flex items-center gap-1.5 ${
view.name === "home" ? "bg-emerald-50 text-emerald-700" : "text-gray-600 hover:bg-gray-100"
}`}
onClick={() => setView({ name: "home" })}
>
<Search className="w-4 h-4" />
</button>
<button
className={`px-3 py-1.5 rounded-md flex items-center gap-1.5 ${
view.name === "contribute" ? "bg-emerald-50 text-emerald-700" : "text-gray-600 hover:bg-gray-100"
}`}
onClick={() => setView({ name: "contribute" })}
>
<PlusCircle className="w-4 h-4" />
</button>
<button
className={`px-3 py-1.5 rounded-md flex items-center gap-1.5 ${
view.name === "api" ? "bg-emerald-50 text-emerald-700" : "text-gray-600 hover:bg-gray-100"
}`}
onClick={() => setView({ name: "api" })}
>
<Code2 className="w-4 h-4" /> API
</button>
<button
className={`px-3 py-1.5 rounded-md flex items-center gap-1.5 ${
view.name === "account" ? "bg-emerald-50 text-emerald-700" : "text-gray-600 hover:bg-gray-100"
}`}
onClick={() => setView({ name: "account" })}
>
<KeyRound className="w-4 h-4" /> API
</button>
<nav className="flex items-center gap-0.5 text-sm">
{NAV.map(({ key, label, icon: Icon }) => {
const active =
view.name === key || (key === "home" && view.name === "product");
return (
<button
key={key}
className={`px-3 py-1.5 rounded-lg flex items-center gap-1.5 font-medium transition ${
active
? "bg-brand-50 text-brand-700"
: "text-gray-600 hover:bg-gray-100"
}`}
onClick={() => setView({ name: key } as View)}
>
<Icon className="w-4 h-4" />
<span className="hidden sm:inline">{label}</span>
</button>
);
})}
</nav>
</div>
</header>
<main className="flex-1 max-w-5xl w-full mx-auto px-4 py-6">
<main className="flex-1 max-w-5xl w-full mx-auto px-4 py-8">
{view.name === "home" && (
<Home
onOpen={(id) => setView({ name: "product", id })}
onContribute={() => setView({ name: "contribute" })}
onApi={() => setView({ name: "api" })}
/>
)}
{view.name === "product" && (
@@ -89,24 +91,36 @@ export default function App() {
)}
{view.name === "api" && <ApiDocs onRegister={() => setView({ name: "account" })} />}
{view.name === "account" && <Account />}
{view.name === "contact" && <Contact />}
</main>
<footer className="border-t bg-white">
<div className="max-w-5xl mx-auto px-4 py-4 text-xs text-gray-400 leading-relaxed text-center">
<footer className="border-t border-gray-200/70 bg-white/60">
<div className="max-w-5xl mx-auto px-4 py-6 text-xs text-gray-400 leading-relaxed text-center">
{qualified != null && (
<div className="mb-2 text-gray-500">
<span className="mx-1 font-semibold text-emerald-600">
<div className="mb-3 inline-flex items-center gap-1.5 rounded-full border border-brand-100 bg-brand-50 px-3 py-1 text-gray-500">
<span className="font-semibold text-brand-600">
{qualified.toLocaleString()}
</span>
</div>
)}
<button onClick={() => setView({ name: "api" })} className="ml-1 text-emerald-600 hover:underline">
API
</button>
<div className="mt-2">
<p className="max-w-2xl mx-auto">
<button
onClick={() => setView({ name: "api" })}
className="ml-1 text-brand-600 font-medium hover:underline"
>
API
</button>
<button
onClick={() => setView({ name: "contact" })}
className="ml-1 text-brand-600 font-medium hover:underline"
>
</button>
</p>
<div className="mt-3">
<a
href="https://beian.miit.gov.cn/"
target="_blank"
+5 -1
View File
@@ -1,4 +1,4 @@
import type { Category, Product, ProductSummary, SubmissionInput } from "./types";
import type { Category, KindField, Product, ProductSummary, SubmissionInput } from "./types";
async function req<T>(path: string, init?: RequestInit): Promise<T> {
const res = await fetch(path, {
@@ -67,6 +67,10 @@ export const api = {
},
product: (id: string) => req<Product>(`/api/v1/products/${id}`),
categories: () => req<{ items: Category[] }>(`/api/v1/categories`),
kindFields: (kind: string) =>
req<{ items: KindField[]; kind: string }>(
`/api/v1/kind-fields?kind=${encodeURIComponent(kind)}`,
),
submit: (input: SubmissionInput) =>
req<{ id: string; status: string }>(`/api/public/submissions`, {
method: "POST",
+11 -11
View File
@@ -5,7 +5,7 @@ import { api, type AccountInfo, type KeyResponse } from "../api";
function KeyReveal({ data }: { data: KeyResponse }) {
const [copied, setCopied] = useState(false);
return (
<div className="mt-4 rounded-lg border border-emerald-200 bg-emerald-50 p-4">
<div className="mt-4 rounded-lg border border-brand-100 bg-brand-50 p-4">
<div className="flex items-start gap-2 text-amber-700 text-sm">
<AlertTriangle className="w-4 h-4 mt-0.5 shrink-0" />
<span></span>
@@ -27,7 +27,7 @@ function KeyReveal({ data }: { data: KeyResponse }) {
className="text-gray-400 hover:text-gray-600 shrink-0"
title="复制"
>
{copied ? <Check className="w-5 h-5 text-emerald-600" /> : <Copy className="w-5 h-5" />}
{copied ? <Check className="w-5 h-5 text-brand-600" /> : <Copy className="w-5 h-5" />}
</button>
</div>
<div className="mt-3 text-sm text-gray-600">
@@ -92,9 +92,9 @@ export default function Account() {
return (
<div className="max-w-xl mx-auto space-y-5">
<div className="bg-white border rounded-lg p-5">
<div className="card p-5">
<h1 className="flex items-center gap-2 text-2xl font-bold text-gray-800">
<KeyRound className="w-6 h-6 text-emerald-600" /> API
<KeyRound className="w-6 h-6 text-brand-600" /> API
</h1>
<p className="mt-2 text-sm text-gray-600 leading-relaxed">
IP <strong>1000</strong> API
@@ -104,7 +104,7 @@ export default function Account() {
<div className="mt-4 inline-flex rounded-md border bg-gray-50 p-0.5 text-sm">
<button
className={`px-4 py-1.5 rounded ${
mode === "register" ? "bg-white shadow-sm text-emerald-700" : "text-gray-500"
mode === "register" ? "bg-white shadow-sm text-brand-700" : "text-gray-500"
}`}
onClick={() => {
setMode("register");
@@ -115,7 +115,7 @@ export default function Account() {
</button>
<button
className={`px-4 py-1.5 rounded ${
mode === "manage" ? "bg-white shadow-sm text-emerald-700" : "text-gray-500"
mode === "manage" ? "bg-white shadow-sm text-brand-700" : "text-gray-500"
}`}
onClick={() => {
setMode("manage");
@@ -135,7 +135,7 @@ export default function Account() {
value={email}
onChange={(e) => setEmail(e.target.value)}
placeholder="you@example.com"
className="w-full border rounded-md px-3 py-2 text-sm focus:outline-none focus:ring-2 focus:ring-emerald-500"
className="w-full border rounded-md px-3 py-2 text-sm focus:outline-none focus:ring-2 focus:ring-brand-500"
/>
</div>
<div>
@@ -146,14 +146,14 @@ export default function Account() {
value={password}
onChange={(e) => setPassword(e.target.value)}
placeholder="••••••••"
className="w-full border rounded-md px-3 py-2 text-sm focus:outline-none focus:ring-2 focus:ring-emerald-500"
className="w-full border rounded-md px-3 py-2 text-sm focus:outline-none focus:ring-2 focus:ring-brand-500"
/>
</div>
{error && <div className="text-sm text-red-600">{error}</div>}
<button
type="submit"
disabled={loading}
className="w-full bg-emerald-600 text-white rounded-md py-2 text-sm font-medium hover:bg-emerald-700 disabled:opacity-50"
className="w-full bg-brand-600 text-white rounded-md py-2 text-sm font-medium hover:bg-brand-700 disabled:opacity-50"
>
{loading ? "处理中…" : mode === "register" ? "注册并领取密钥" : "查询账号"}
</button>
@@ -175,12 +175,12 @@ export default function Account() {
<div>
<strong>{info.quota_used.toLocaleString()}</strong> /{" "}
{info.quota_total.toLocaleString()} {" "}
<strong className="text-emerald-600">{info.quota_remaining.toLocaleString()}</strong>
<strong className="text-brand-600">{info.quota_remaining.toLocaleString()}</strong>
</div>
<button
onClick={regenerate}
disabled={loading}
className="mt-2 text-emerald-600 hover:underline disabled:opacity-50"
className="mt-2 text-brand-600 hover:underline disabled:opacity-50"
>
</button>
+42 -7
View File
@@ -20,7 +20,7 @@ function CopyBtn({ text }: { text: string }) {
className="text-gray-400 hover:text-gray-600"
title="复制"
>
{done ? <Check className="w-4 h-4 text-emerald-600" /> : <Copy className="w-4 h-4" />}
{done ? <Check className="w-4 h-4 text-brand-600" /> : <Copy className="w-4 h-4" />}
</button>
);
}
@@ -39,7 +39,7 @@ function Code({ children }: { children: string }) {
}
function Method({ m }: { m: string }) {
const color = m === "GET" ? "bg-sky-100 text-sky-700" : "bg-emerald-100 text-emerald-700";
const color = m === "GET" ? "bg-sky-100 text-sky-700" : "bg-brand-100 text-brand-700";
return <span className={`text-xs font-mono font-semibold rounded px-1.5 py-0.5 ${color}`}>{m}</span>;
}
@@ -63,7 +63,7 @@ function Endpoint({
response: string;
}) {
return (
<div className="bg-white border rounded-lg p-5">
<div className="card p-5">
<div className="flex items-center gap-2 flex-wrap">
<Method m={method} />
<code className="text-sm text-gray-800 font-mono break-all">{path}</code>
@@ -105,7 +105,7 @@ function Endpoint({
export default function ApiDocs({ onRegister }: { onRegister?: () => void }) {
return (
<div className="space-y-5">
<div className="bg-white border rounded-lg p-5">
<div className="card p-5">
<h1 className="text-2xl font-bold text-gray-800">API </h1>
<p className="mt-2 text-gray-600 text-sm leading-relaxed">
<strong></strong> REST API
@@ -119,7 +119,7 @@ export default function ApiDocs({ onRegister }: { onRegister?: () => void }) {
<ul className="mt-2 list-disc pl-5 text-gray-600 space-y-1">
<li>
API Key / Token GET IP <strong>1000</strong>
<button onClick={onRegister} className="text-emerald-600 hover:underline"></button>
<button onClick={onRegister} className="text-brand-600 hover:underline"></button>
</li>
<li>
@@ -135,13 +135,13 @@ export default function ApiDocs({ onRegister }: { onRegister?: () => void }) {
</div>
</div>
<div className="bg-white border rounded-lg p-5">
<div className="card p-5">
<h2 className="text-lg font-semibold text-gray-800"></h2>
<p className="mt-2 text-gray-600 text-sm leading-relaxed">
API <strong></strong> IP
<strong> 1000 </strong>
<code className="font-mono">403</code> <code className="font-mono">quota_exhausted</code>
<button onClick={onRegister} className="text-emerald-600 hover:underline"></button>
<button onClick={onRegister} className="text-brand-600 hover:underline"></button>
API Key
</p>
<div className="mt-3">
@@ -343,6 +343,41 @@ curl -H "Authorization: Bearer og_live_xxxxxxxx" ${BASE}/products/search?q=牛
}`}
/>
<Endpoint
method="POST"
path="/api/public/backflow"
title="档案回流(批量回流未收录商品,需 API Key)"
desc="供进销存等机器调用方使用:把档案里尚未收录的商品批量推送过来,进入人工审核队列,审核通过后才收录。必须携带 API Key(与上文同一类 og_live_ 密钥),不会直接写入商品。请求体为商品对象数组(与公众投稿同结构),单次最多 1000 条;按条码 GTIN 逐条去重,返回每条结果:queued(入队待审)/ exists(已收录跳过)/ duplicate(已有同条码待审跳过)/ invalid(无效,如名称为空)。"
params={[
{ name: "name", required: true, desc: "商品名称(数组中每个元素)" },
{ name: "gtin", desc: "条码(GTIN),强烈建议提供,用于去重" },
{ name: "brand_name", desc: "品牌名" },
{ name: "category_id", desc: "品类编码,如 food.beverages" },
{ name: "net_content_value / net_content_unit", desc: "净含量数值 / 单位" },
{ name: "country_of_origin", desc: "产地" },
{ name: "msrp", desc: "零售价快照数组,元素含 amount/currency/region/effective_date" },
{ name: "note", desc: "备注" },
]}
example={`curl -X POST ${ORIGIN}/api/public/backflow \\
-H "X-API-Key: og_live_xxxxxxxx" \\
-H "Content-Type: application/json" \\
-d '[
{"name":"某某牛奶 250ml","gtin":"6901234567890","brand_name":"某品牌",
"net_content_value":250,"net_content_unit":"ml",
"msrp":[{"amount":3.5,"currency":"CNY","region":"CN"}]},
{"name":"已收录商品","gtin":"5449000000996"}
]'`}
response={`{
"total": 2, "queued": 1, "exists": 1, "duplicate": 0, "invalid": 0,
"results": [
{ "gtin": "6901234567890", "name": "某某牛奶 250ml",
"status": "queued", "id": "<submission-id>" },
{ "gtin": "5449000000996", "name": "已收录商品",
"status": "exists", "reason": "该条码商品已收录" }
]
}`}
/>
<Endpoint
method="POST"
path="/api/v1/register"
+113
View File
@@ -0,0 +1,113 @@
import { useState } from "react";
import { Phone, Mail, Globe, MessageSquare, Copy, Check, Headset } from "lucide-react";
type Channel = {
key: string;
icon: typeof Phone;
label: string;
value: string;
href?: string;
copy: string;
};
const CHANNELS: Channel[] = [
{
key: "phone",
icon: Phone,
label: "电话",
value: "188 6595 7520",
href: "tel:18865957520",
copy: "18865957520",
},
{
key: "email",
icon: Mail,
label: "邮箱",
value: "1115084741@qq.com",
href: "mailto:1115084741@qq.com",
copy: "1115084741@qq.com",
},
{
key: "site",
icon: Globe,
label: "官网",
value: "www.wenyaoyu.com",
href: "https://www.wenyaoyu.com",
copy: "https://www.wenyaoyu.com",
},
{
key: "wechat",
icon: MessageSquare,
label: "微信",
value: "s-b-m-y",
copy: "s-b-m-y",
},
];
export default function Contact() {
const [copied, setCopied] = useState<string | null>(null);
async function copy(channel: Channel) {
try {
await navigator.clipboard.writeText(channel.copy);
setCopied(channel.key);
setTimeout(() => setCopied((k) => (k === channel.key ? null : k)), 1500);
} catch {
/* clipboard unavailable */
}
}
return (
<div className="max-w-3xl mx-auto animate-fade-up">
<div className="text-center">
<span className="mx-auto grid h-14 w-14 place-items-center rounded-2xl bg-gradient-to-br from-brand-500 to-brand-700 text-white shadow-glow">
<Headset className="w-7 h-7" />
</span>
<h1 className="mt-4 text-2xl font-semibold tracking-tight text-gray-900"></h1>
<p className="mx-auto mt-2 max-w-xl text-sm text-gray-500">
</p>
</div>
<div className="mt-8 grid grid-cols-1 sm:grid-cols-2 gap-4">
{CHANNELS.map((c) => {
const Icon = c.icon;
return (
<div key={c.key} className="card p-5 flex items-center gap-4">
<span className="grid h-11 w-11 shrink-0 place-items-center rounded-xl bg-brand-50 text-brand-600">
<Icon className="w-5 h-5" />
</span>
<div className="min-w-0 flex-1">
<div className="text-xs font-medium text-gray-400">{c.label}</div>
{c.href ? (
<a
href={c.href}
target={c.key === "site" ? "_blank" : undefined}
rel={c.key === "site" ? "noreferrer" : undefined}
className="block truncate font-medium text-gray-800 hover:text-brand-600 hover:underline"
>
{c.value}
</a>
) : (
<div className="truncate font-medium text-gray-800">{c.value}</div>
)}
</div>
<button
type="button"
onClick={() => copy(c)}
title="复制"
className="shrink-0 grid h-9 w-9 place-items-center rounded-lg border border-gray-200 text-gray-400 transition hover:bg-gray-50 hover:text-brand-600"
>
{copied === c.key ? (
<Check className="w-4 h-4 text-brand-600" />
) : (
<Copy className="w-4 h-4" />
)}
</button>
</div>
);
})}
</div>
</div>
);
}
+181 -58
View File
@@ -1,7 +1,7 @@
import { useEffect, useState } from "react";
import { useEffect, useMemo, useState } from "react";
import { CheckCircle2, PlusCircle, Trash2 } from "lucide-react";
import { api } from "../api";
import type { Category, SubmissionImage, SubmissionInput } from "../types";
import type { Category, KindField, SubmissionImage, SubmissionInput } from "../types";
const NUTRI_FIELDS: { key: string; label: string }[] = [
{ key: "energy_kcal", label: "能量 (kcal)" },
@@ -35,6 +35,8 @@ export default function Contribute({ onDone }: { onDone: () => void }) {
const [ingredients, setIngredients] = useState("");
const [basis, setBasis] = useState("");
const [nutri, setNutri] = useState<Record<string, string>>({});
const [kindFields, setKindFields] = useState<KindField[]>([]);
const [attrs, setAttrs] = useState<Record<string, string>>({});
const [images, setImages] = useState<SubmissionImage[]>([]);
const [imageURL, setImageURL] = useState("");
const [submitter, setSubmitter] = useState("");
@@ -45,6 +47,33 @@ export default function Contribute({ onDone }: { onDone: () => void }) {
api.categories().then((r) => setCategories(r.items)).catch(() => undefined);
}, []);
// Derive the archive kind from the selected category. No category => generic.
const kind = useMemo(() => {
const c = categories.find((x) => x.id === categoryID);
return c?.archive_kind || "generic";
}, [categories, categoryID]);
// Fetch the kind-specific field template for non-food kinds.
useEffect(() => {
setAttrs({});
if (kind === "food" || kind === "generic") {
setKindFields([]);
return;
}
let alive = true;
api
.kindFields(kind)
.then((r) => {
if (alive) setKindFields(r.items);
})
.catch(() => {
if (alive) setKindFields([]);
});
return () => {
alive = false;
};
}, [kind]);
async function submit(e: React.FormEvent) {
e.preventDefault();
if (name.trim() === "") {
@@ -55,9 +84,31 @@ export default function Contribute({ onDone }: { onDone: () => void }) {
setError("");
const nutriments: Record<string, number> = {};
for (const [k, v] of Object.entries(nutri)) {
const n = parseFloat(v);
if (!Number.isNaN(n)) nutriments[k] = n;
if (kind === "food") {
for (const [k, v] of Object.entries(nutri)) {
const n = parseFloat(v);
if (!Number.isNaN(n)) nutriments[k] = n;
}
}
const attributes: Record<string, unknown> = {};
if (kind !== "food" && kind !== "generic") {
for (const f of kindFields) {
const raw = (attrs[f.field_key] || "").trim();
if (raw === "") continue;
if (f.field_type === "number") {
const n = parseFloat(raw);
if (!Number.isNaN(n)) attributes[f.field_key] = n;
} else if (f.field_type === "list") {
const parts = raw
.split(/[\n,,、]/)
.map((s) => s.trim())
.filter((s) => s !== "");
if (parts.length) attributes[f.field_key] = parts;
} else {
attributes[f.field_key] = raw;
}
}
}
const input: SubmissionInput = {
@@ -68,9 +119,10 @@ export default function Contribute({ onDone }: { onDone: () => void }) {
net_content_value: field(netValue) ? parseFloat(netValue) : null,
net_content_unit: field(netUnit),
country_of_origin: field(country),
ingredients_text: field(ingredients),
ingredients_text: kind === "food" ? field(ingredients) : null,
nutriments: Object.keys(nutriments).length ? nutriments : null,
nutrition_basis: basis || null,
attributes: Object.keys(attributes).length ? attributes : null,
nutrition_basis: kind === "food" && basis ? basis : null,
images: images.length ? images : undefined,
submitter_name: field(submitter),
submitter_contact: field(contact),
@@ -89,38 +141,91 @@ export default function Contribute({ onDone }: { onDone: () => void }) {
if (done) {
return (
<div className="max-w-xl mx-auto text-center py-16">
<CheckCircle2 className="w-14 h-14 text-emerald-500 mx-auto" />
<div className="max-w-xl mx-auto text-center py-16 animate-fade-up">
<span className="mx-auto grid h-16 w-16 place-items-center rounded-2xl bg-brand-50">
<CheckCircle2 className="w-9 h-9 text-brand-600" />
</span>
<h1 className="mt-4 text-xl font-semibold text-gray-800"></h1>
<p className="mt-2 text-gray-500">
</p>
<button
onClick={onDone}
className="mt-6 px-5 py-2 rounded-lg bg-emerald-600 text-white hover:bg-emerald-700"
>
<button onClick={onDone} className="btn-primary mt-6">
</button>
</div>
);
}
const input =
"w-full border rounded-md px-3 py-2 text-sm focus:outline-none focus:ring-2 focus:ring-emerald-400";
const label = "block text-xs text-gray-500 mb-1";
const input = "input";
const label = "block text-xs font-medium text-gray-500 mb-1";
// Group the kind fields by their group label, preserving template order.
const groups: { label: string; fields: KindField[] }[] = [];
for (const f of kindFields) {
let g = groups.find((x) => x.label === f.group_label);
if (!g) {
g = { label: f.group_label, fields: [] };
groups.push(g);
}
g.fields.push(f);
}
const KIND_LABEL: Record<string, string> = {
drug: "药品",
electronics: "数码 3C",
food: "食品",
generic: "通用",
};
function renderField(f: KindField) {
const val = attrs[f.field_key] || "";
const set = (v: string) => setAttrs((p) => ({ ...p, [f.field_key]: v }));
const lbl = f.unit ? `${f.label_zh} (${f.unit})` : f.label_zh;
return (
<div key={f.field_key} className={f.field_type === "textarea" || f.field_type === "list" ? "sm:col-span-2" : ""}>
<label className={label}>
{lbl}
{f.qualified && <span className="text-brand-500"> *</span>}
</label>
{f.field_type === "select" ? (
<select className={input} value={val} onChange={(e) => set(e.target.value)}>
<option value=""></option>
{(f.options || []).map((o) => (
<option key={o} value={o}>
{o}
</option>
))}
</select>
) : f.field_type === "textarea" ? (
<textarea className={input} rows={3} value={val} placeholder={f.placeholder || ""} onChange={(e) => set(e.target.value)} />
) : f.field_type === "list" ? (
<textarea className={input} rows={2} value={val} placeholder={f.placeholder || "每行一项,或用、逗号分隔"} onChange={(e) => set(e.target.value)} />
) : (
<input
type={f.field_type === "number" ? "number" : "text"}
step={f.field_type === "number" ? "any" : undefined}
className={input}
value={val}
placeholder={f.placeholder || ""}
onChange={(e) => set(e.target.value)}
/>
)}
</div>
);
}
return (
<form onSubmit={submit} className="max-w-3xl mx-auto">
<h1 className="text-xl font-semibold text-gray-800"></h1>
<form onSubmit={submit} className="max-w-3xl mx-auto animate-fade-up">
<h1 className="text-2xl font-semibold tracking-tight text-gray-900"></h1>
<p className="mt-1 text-sm text-gray-500">
<b></b> *
<b></b>
</p>
{error && (
<div className="mt-4 bg-red-50 text-red-700 text-sm rounded-md px-4 py-2">{error}</div>
<div className="mt-4 rounded-xl bg-red-50 px-4 py-2.5 text-sm text-red-700">{error}</div>
)}
<div className="bg-white border rounded-lg p-5 mt-4">
<div className="card p-5 mt-4">
<h2 className="font-medium text-gray-700 mb-3"></h2>
<div className="grid grid-cols-1 sm:grid-cols-2 gap-4">
<div className="sm:col-span-2">
@@ -145,6 +250,11 @@ export default function Contribute({ onDone }: { onDone: () => void }) {
</option>
))}
</select>
{categoryID && (
<p className="mt-1 text-xs text-gray-400">
{KIND_LABEL[kind] || kind}
</p>
)}
</div>
<div>
<label className={label}></label>
@@ -167,41 +277,58 @@ export default function Contribute({ onDone }: { onDone: () => void }) {
</div>
</div>
<div className="bg-white border rounded-lg p-5 mt-4">
<h2 className="font-medium text-gray-700 mb-3"></h2>
<label className={label}></label>
<textarea
className={input}
rows={3}
value={ingredients}
onChange={(e) => setIngredients(e.target.value)}
/>
<div className="mt-3">
<label className={label}></label>
<select className={input} value={basis} onChange={(e) => setBasis(e.target.value)}>
<option value=""></option>
<option value="per_100g"> 100g</option>
<option value="per_100ml"> 100ml</option>
<option value="per_serving"></option>
</select>
{kind === "food" && (
<div className="card p-5 mt-4">
<h2 className="font-medium text-gray-700 mb-3"></h2>
<label className={label}></label>
<textarea
className={input}
rows={3}
value={ingredients}
onChange={(e) => setIngredients(e.target.value)}
/>
<div className="mt-3">
<label className={label}></label>
<select className={input} value={basis} onChange={(e) => setBasis(e.target.value)}>
<option value=""></option>
<option value="per_100g"> 100g</option>
<option value="per_100ml"> 100ml</option>
<option value="per_serving"></option>
</select>
</div>
<div className="grid grid-cols-2 sm:grid-cols-4 gap-3 mt-3">
{NUTRI_FIELDS.map((f) => (
<div key={f.key}>
<label className={label}>{f.label}</label>
<input
type="number"
step="any"
className={input}
value={nutri[f.key] || ""}
onChange={(e) => setNutri({ ...nutri, [f.key]: e.target.value })}
/>
</div>
))}
</div>
</div>
<div className="grid grid-cols-2 sm:grid-cols-4 gap-3 mt-3">
{NUTRI_FIELDS.map((f) => (
<div key={f.key}>
<label className={label}>{f.label}</label>
<input
type="number"
step="any"
className={input}
value={nutri[f.key] || ""}
onChange={(e) => setNutri({ ...nutri, [f.key]: e.target.value })}
/>
)}
{kind !== "food" && kind !== "generic" && kindFields.length > 0 && (
<div className="card p-5 mt-4">
<h2 className="font-medium text-gray-700 mb-1">{KIND_LABEL[kind] || kind}</h2>
<p className="text-xs text-gray-400 mb-3"> * </p>
{groups.map((g) => (
<div key={g.label} className="mt-3 first:mt-0">
{g.label && <h3 className="text-xs font-semibold text-gray-400 mb-2">{g.label}</h3>}
<div className="grid grid-cols-1 sm:grid-cols-2 gap-4">
{g.fields.map(renderField)}
</div>
</div>
))}
</div>
</div>
)}
<div className="bg-white border rounded-lg p-5 mt-4">
<div className="card p-5 mt-4">
<h2 className="font-medium text-gray-700 mb-3"> URL</h2>
{images.length > 0 && (
<ul className="mb-3 space-y-1">
@@ -234,14 +361,14 @@ export default function Contribute({ onDone }: { onDone: () => void }) {
setImageURL("");
}
}}
className="shrink-0 px-3 rounded-md border text-sm flex items-center gap-1 hover:bg-gray-50"
className="shrink-0 px-3 rounded-xl border border-gray-200 text-sm flex items-center gap-1 hover:bg-gray-50"
>
<PlusCircle className="w-4 h-4" />
</button>
</div>
</div>
<div className="bg-white border rounded-lg p-5 mt-4">
<div className="card p-5 mt-4">
<h2 className="font-medium text-gray-700 mb-3"></h2>
<div className="grid grid-cols-1 sm:grid-cols-2 gap-4">
<div>
@@ -260,11 +387,7 @@ export default function Contribute({ onDone }: { onDone: () => void }) {
</div>
<div className="mt-5 flex items-center gap-3">
<button
type="submit"
disabled={submitting}
className="px-6 py-2.5 rounded-lg bg-emerald-600 text-white font-medium hover:bg-emerald-700 disabled:opacity-60"
>
<button type="submit" disabled={submitting} className="btn-primary px-6">
{submitting ? "提交中…" : "提交审核"}
</button>
<button type="button" onClick={onDone} className="text-sm text-gray-500 hover:underline">
+64 -47
View File
@@ -1,16 +1,14 @@
import { useState } from "react";
import { Search, PlusCircle, Code2 } from "lucide-react";
import { Search, PlusCircle, ArrowRight } from "lucide-react";
import { api } from "../api";
import type { ProductSummary } from "../types";
export default function Home({
onOpen,
onContribute,
onApi,
}: {
onOpen: (id: string) => void;
onContribute: () => void;
onApi: () => void;
}) {
const [q, setQ] = useState("");
const [items, setItems] = useState<ProductSummary[]>([]);
@@ -19,12 +17,14 @@ export default function Home({
const [loading, setLoading] = useState(false);
const [error, setError] = useState("");
async function run(e?: React.FormEvent) {
async function run(term?: string, e?: React.FormEvent) {
e?.preventDefault();
const keyword = (term ?? q).trim();
if (term !== undefined) setQ(term);
setLoading(true);
setError("");
try {
const res = await api.search(q.trim(), 1, 30);
const res = await api.search(keyword, 1, 30);
setItems(res.items);
setTotal(res.total);
setSearched(true);
@@ -36,81 +36,98 @@ export default function Home({
}
return (
<div>
<div className="text-center py-10">
<h1 className="text-3xl font-bold text-gray-800"></h1>
<p className="mt-2 text-gray-500">
</p>
<form onSubmit={run} className="mt-6 max-w-2xl mx-auto flex gap-2">
<div className="flex-1 flex items-center gap-2 bg-white border rounded-lg px-3 shadow-sm focus-within:ring-2 focus-within:ring-emerald-400">
<Search className="w-5 h-5 text-gray-400" />
<div className="animate-fade-up">
<section
className={`flex flex-col items-center px-4 text-center ${
searched
? "pt-2 pb-6"
: "min-h-[calc(100vh-12rem)] justify-center"
}`}
>
<h1
className={`font-medium tracking-tight text-gray-800 ${
searched ? "text-3xl" : "text-6xl sm:text-7xl"
}`}
>
<span className="text-brand-600"></span>
</h1>
<form
onSubmit={(e) => run(undefined, e)}
className="mx-auto mt-8 w-full max-w-xl"
>
<div className="flex items-center gap-3 rounded-full border border-gray-200 bg-white px-5 py-3 shadow-sm transition hover:shadow-md focus-within:shadow-md">
<Search className="h-5 w-5 shrink-0 text-gray-400" />
<input
autoFocus
value={q}
onChange={(e) => setQ(e.target.value)}
placeholder="例如:可乐、Nutella、5449000000996"
className="flex-1 py-3 outline-none bg-transparent"
placeholder="搜索商品名称或条码"
className="flex-1 bg-transparent outline-none"
/>
</div>
<button
type="submit"
disabled={loading}
className="px-6 rounded-lg bg-emerald-600 text-white font-medium hover:bg-emerald-700 disabled:opacity-60"
>
{loading ? "检索中…" : "检索"}
</button>
{!searched && (
<div className="mt-8">
<button
type="submit"
disabled={loading}
className="rounded-md border border-gray-200 bg-gray-50 px-6 py-2.5 text-sm text-gray-700 transition hover:border-gray-300 hover:shadow-sm"
>
{loading ? "检索中…" : "检索"}
</button>
</div>
)}
</form>
<button
onClick={onApi}
className="mt-4 inline-flex items-center gap-1.5 text-sm text-emerald-700 hover:underline"
>
<Code2 className="w-4 h-4" /> API
</button>
</div>
</section>
{error && (
<div className="max-w-2xl mx-auto bg-red-50 text-red-700 text-sm rounded-md px-4 py-2">
<div className="mx-auto mt-6 max-w-2xl rounded-xl bg-red-50 px-4 py-2.5 text-sm text-red-700">
{error}
</div>
)}
{searched && (
<div className="mt-2">
<div className="text-sm text-gray-500 mb-2">
{total} {q ? `(关键词:${q}` : ""}
<div className="mt-8">
<div className="mb-3 text-sm text-gray-500">
<span className="font-semibold text-gray-700">{total}</span>
{q ? `(关键词:${q}` : ""}
</div>
{items.length === 0 ? (
<div className="bg-white border rounded-lg p-8 text-center text-gray-500">
<div className="card p-10 text-center text-gray-500">
<p></p>
<button
onClick={onContribute}
className="mt-3 inline-flex items-center gap-1.5 text-emerald-700 hover:underline"
className="mt-3 inline-flex items-center gap-1.5 font-medium text-brand-700 hover:underline"
>
<PlusCircle className="w-4 h-4" />
</button>
</div>
) : (
<ul className="bg-white border rounded-lg divide-y">
<ul className="card divide-y divide-gray-100 overflow-hidden">
{items.map((p) => (
<li key={p.id}>
<button
onClick={() => onOpen(p.id)}
className="w-full text-left px-4 py-3 hover:bg-gray-50 flex items-center justify-between gap-4"
className="group flex w-full items-center justify-between gap-4 px-4 py-3.5 text-left transition hover:bg-brand-50/50"
>
<div>
<div className="font-medium text-gray-800">{p.name}</div>
<div className="text-xs text-gray-500 mt-0.5">
<div className="min-w-0">
<div className="truncate font-medium text-gray-800 group-hover:text-brand-700">
{p.name}
</div>
<div className="mt-0.5 text-xs text-gray-500">
{p.brand || "未知品牌"}
{p.gtin ? ` · ${p.gtin}` : ""}
</div>
</div>
<span className="text-xs text-gray-400 text-right">
<span className="block">{p.category_path || ""}</span>
{p.country_of_origin ? (
<span className="block text-gray-400">{p.country_of_origin}</span>
) : null}
</span>
<div className="flex shrink-0 items-center gap-3">
<span className="text-right text-xs text-gray-400">
<span className="block">{p.category_path || ""}</span>
{p.country_of_origin ? (
<span className="block">{p.country_of_origin}</span>
) : null}
</span>
<ArrowRight className="h-4 w-4 text-gray-300 transition group-hover:translate-x-0.5 group-hover:text-brand-500" />
</div>
</button>
</li>
))}
+25 -8
View File
@@ -28,11 +28,11 @@ export default function ProductView({ id, onBack }: { id: string; onBack: () =>
<button onClick={onBack} className="text-sm text-gray-500 flex items-center gap-1 mb-4">
<ArrowLeft className="w-4 h-4" />
</button>
<div className="bg-red-50 text-red-700 text-sm rounded-md px-4 py-3">{error}</div>
<div className="rounded-xl bg-red-50 px-4 py-3 text-sm text-red-700">{error}</div>
</div>
);
}
if (!p) return <div className="text-gray-400"></div>;
if (!p) return <div className="text-gray-400 animate-pulse"></div>;
const basisLabel: Record<string, string> = {
per_100g: "每 100g",
@@ -42,17 +42,23 @@ export default function ProductView({ id, onBack }: { id: string; onBack: () =>
const nutriEntries = Object.entries(p.nutriments || {}).filter(
([, v]) => v !== null && v !== undefined,
);
const latestMsrp = (p.msrp || [])[0];
const msrpText = latestMsrp
? latestMsrp.currency === "CNY"
? `¥${latestMsrp.amount.toFixed(2)}`
: `${latestMsrp.amount.toFixed(2)} ${latestMsrp.currency}`
: null;
return (
<div>
<button onClick={onBack} className="text-sm text-gray-500 flex items-center gap-1 mb-4">
<button onClick={onBack} className="mb-4 inline-flex items-center gap-1 text-sm text-gray-500 hover:text-gray-700">
<ArrowLeft className="w-4 h-4" />
</button>
<div className="bg-white border rounded-lg p-5">
<div className="card p-6 animate-fade-up">
<div className="flex items-start justify-between gap-4">
<h1 className="text-xl font-semibold text-gray-800">{p.name}</h1>
<span className="shrink-0 text-xs bg-emerald-50 text-emerald-700 rounded px-2 py-1">
<h1 className="text-2xl font-semibold tracking-tight text-gray-900">{p.name}</h1>
<span className="shrink-0 rounded-full border border-brand-100 bg-brand-50 px-2.5 py-1 text-xs font-medium text-brand-700">
{Math.round(p.quality_score * 100)}
</span>
</div>
@@ -93,6 +99,17 @@ export default function ProductView({ id, onBack }: { id: string; onBack: () =>
}
/>
<Row label="产地" value={p.country_of_origin} />
<Row
label="建议零售价"
value={
msrpText ? (
<span className="inline-flex items-baseline gap-2">
<span className="font-medium text-gray-900">{msrpText}</span>
<span className="text-xs text-gray-400"></span>
</span>
) : null
}
/>
<Row label="Nutri-Score" value={p.nutri_score} />
<Row label="配料" value={p.ingredients_text} />
<Row
@@ -107,7 +124,7 @@ export default function ProductView({ id, onBack }: { id: string; onBack: () =>
</div>
{p.specs && p.specs.length > 0 && (
<div className="bg-white border rounded-lg p-5 mt-4">
<div className="card p-6 mt-4">
<h2 className="font-medium text-gray-700 mb-2"></h2>
<div className="grid grid-cols-1 sm:grid-cols-2 gap-2 text-sm">
{p.specs.map((s) => (
@@ -124,7 +141,7 @@ export default function ProductView({ id, onBack }: { id: string; onBack: () =>
)}
{nutriEntries.length > 0 && (
<div className="bg-white border rounded-lg p-5 mt-4">
<div className="card p-6 mt-4">
<h2 className="font-medium text-gray-700 mb-2">
{p.nutrition_basis ? `${basisLabel[p.nutrition_basis] || p.nutrition_basis}` : ""}
+32 -3
View File
@@ -10,7 +10,36 @@ body,
body {
margin: 0;
background: #f3f4f6;
font-family: system-ui, -apple-system, "Segoe UI", Roboto, "Helvetica Neue",
Arial, "PingFang SC", "Microsoft YaHei", sans-serif;
color: #1f2937;
font-family: theme("fontFamily.sans");
-webkit-font-smoothing: antialiased;
text-rendering: optimizeLegibility;
background-color: #f6f8f7;
background-image:
radial-gradient(60rem 60rem at 100% -10%, rgba(16, 185, 129, 0.10), transparent 60%),
radial-gradient(50rem 50rem at -10% 0%, rgba(45, 212, 191, 0.10), transparent 55%);
background-attachment: fixed;
}
@layer components {
.card {
@apply bg-white rounded-2xl border border-gray-100;
box-shadow: theme("boxShadow.card");
}
.input {
@apply w-full rounded-xl border border-gray-200 bg-white px-3.5 py-2.5 text-sm text-gray-800 placeholder:text-gray-400 transition focus:outline-none focus:border-brand-400 focus:ring-4 focus:ring-brand-500/10;
}
.btn-primary {
@apply inline-flex items-center justify-center gap-1.5 rounded-xl bg-brand-600 px-5 py-2.5 text-sm font-semibold text-white shadow-sm transition hover:bg-brand-700 active:bg-brand-800 disabled:opacity-60;
}
.btn-ghost {
@apply inline-flex items-center justify-center gap-1.5 rounded-xl px-3.5 py-2 text-sm font-medium text-gray-600 transition hover:bg-gray-100;
}
.chip {
@apply inline-flex items-center gap-1 rounded-full border border-gray-200 bg-white/70 px-3 py-1 text-xs font-medium text-gray-600 transition hover:border-brand-300 hover:text-brand-700 hover:bg-brand-50;
}
}
+25
View File
@@ -24,6 +24,15 @@ export interface ProductSpec {
unit?: string;
}
export interface Msrp {
amount: number;
currency: string;
region: string;
effective_date?: string | null;
source_url?: string | null;
note?: string | null;
}
export interface Product {
id: string;
gtin: string | null;
@@ -43,6 +52,7 @@ export interface Product {
ingredients_text?: string | null;
allergens?: string[] | null;
additives?: string[] | null;
msrp?: Msrp[] | null;
}
export interface Category {
@@ -51,6 +61,20 @@ export interface Category {
name_en: string | null;
path: string;
level: number;
archive_kind: string;
}
export interface KindField {
kind: string;
field_key: string;
group_label: string;
label_zh: string;
field_type: "text" | "number" | "textarea" | "select" | "list";
unit: string | null;
options: string[] | null;
placeholder: string | null;
sort_order: number;
qualified: boolean;
}
export interface SubmissionImage {
@@ -68,6 +92,7 @@ export interface SubmissionInput {
country_of_origin?: string | null;
ingredients_text?: string | null;
nutriments?: Record<string, number> | null;
attributes?: Record<string, unknown> | null;
nutrition_basis?: string | null;
serving_size?: string | null;
nutri_score?: string | null;
+50 -1
View File
@@ -1,6 +1,55 @@
/** @type {import('tailwindcss').Config} */
export default {
content: ["./index.html", "./src/**/*.{ts,tsx}"],
theme: { extend: {} },
theme: {
extend: {
colors: {
brand: {
50: "#ecfdf5",
100: "#d1fae5",
200: "#a7f3d0",
300: "#6ee7b7",
400: "#34d399",
500: "#10b981",
600: "#059669",
700: "#047857",
800: "#065f46",
900: "#064e3b",
},
},
fontFamily: {
sans: [
"Inter",
"system-ui",
"-apple-system",
"Segoe UI",
"Roboto",
"Helvetica Neue",
"Arial",
"PingFang SC",
"Microsoft YaHei",
"sans-serif",
],
},
boxShadow: {
card: "0 1px 2px rgba(16,24,40,0.04), 0 4px 16px -8px rgba(16,24,40,0.10)",
"card-hover": "0 10px 30px -10px rgba(16,24,40,0.18)",
glow: "0 12px 40px -12px rgba(5,150,105,0.45)",
},
borderRadius: {
"2xl": "1rem",
"3xl": "1.5rem",
},
keyframes: {
"fade-up": {
"0%": { opacity: "0", transform: "translateY(8px)" },
"100%": { opacity: "1", transform: "translateY(0)" },
},
},
animation: {
"fade-up": "fade-up 0.4s ease-out both",
},
},
},
plugins: [],
};
+9
View File
@@ -0,0 +1,9 @@
# build artifacts
bypos-collector.exe
bypos-collector
bypos-collector-linux
*.exe
# collected data / outputs
*.jsonl
*.csv
*.log
+82
View File
@@ -0,0 +1,82 @@
# bypos-collector
按条码批量采集商品档案的小工具(单文件 Windows/Linux 程序,自带本地 Web 控制台)。
数据来源是云店「新增商品」输入条码时所查的同一个中心商品库 `zc.bypos.net`
采集结果落地为 JSONL,供后续导入本项目(天工/goods)。
> 这是一个**独立模块**(有自己的 `go.mod`),与 `api/` 主服务互不影响,
> CI 不会编译它。放在 `tools/` 下仅作代码留存与后期迭代。
## 目录
| 文件 | 说明 |
| --- | --- |
| `main.go` | 入口:本地 HTTP 服务 + 启动浏览器 + API 路由(start/stop/stats/download/export.csv) |
| `collect.go` | 核心:签名、EAN-13 校验位、范围/清单枚举、并发+限速、JSONL 落库、断点续采 |
| `web/index.html` | 内嵌(`go:embed`)的控制台界面 |
| `build.sh` | 交叉编译出 `bypos-collector.exe`(windows/amd64)与 linux 测试二进制 |
| `使用说明.md` | 面向使用者的操作说明 |
## 构建
```bash
./build.sh
# 产物:bypos-collector.exe(发给 Windows 用户)/ bypos-collector-linux(本地测试)
```
二进制与采集产物(`*.jsonl`/`*.csv`)已在 `.gitignore` 中排除,不入库。
## 接口与签名(逆向所得,后期迭代参考)
云店新增商品页输入条码时,前端经服务端代理 `/prod-api/ZmSvr/httpUtil/getGet`
转发到中心库:
```
GET http://zc.bypos.net/byGoodsService/byMessage.asmx/GetGoodsinfo
?sdogid=<账号id>&regnum=1&barcode=<条码>
&sparm1=<md5(sdogid)> # 常量,随账号固定
&sparm2=<md5(barcode + tsMs)> # tsMs = 当前秒*1000(末尾恒为 000)
&sparm3=<tsMs 前 10 位 = 秒级时间戳>
&sparm4=&barcodetype=yunpos
```
返回 `<string>{...json...}</string>`,内层 JSON 字段:
| 上游字段 | 含义 | 归一化字段 |
| --- | --- | --- |
| item_name | 品名 | name |
| item_size | 规格 | spec |
| unit_no | 单位 | unit |
| item_area | 产地/地区 | area |
| birth_com | 生产企业(常空) | manufacturer |
| birth_doc | 生产许可(常空) | license |
| inprice | 建议进价 | in_price |
| sellprice | 建议零售价 | sell_price |
| retcode | 1=命中,0=失败 | status(hit/miss/invalid) |
`retmsg` 含「非国标条码 / 参数异常」=> invalid;含「条码不存在」=> miss。
## 配置
- `sdogid`:中心库账号 id(本项目所属云店账号的授权 id)。默认值见
`collect.go``defaultSdogID`,也可用 `-sdogid` 参数或控制台覆盖。
**这是账号级凭证**——若本仓库对外公开,建议改为从环境变量/外部配置读取。
## 导入 goods/天工库
采集得到的 JSONL 用 ingestion 里的导入任务入库(只导入 `status=hit` 的记录,
按 GTIN 去重 upsert,重复导入幂等):
```bash
cd ingestion
python -m opengoods.jobs.import_bypos --input products.jsonl
# 可选:--limit N 限制条数;--dsn 指定数据库
```
字段映射:`barcode→gtin``name→name``spec`(可解析的质量/体积如 500mL/5kg)
`→net_content`、其余规格/单位/产地/许可/进价留存到 `attributes``sell_price→`
CNY MSRP 快照。来源记为 `bypos中心库`,带字段级 provenance。
## 注意
批量自动查询比页面逐条更"重",上游可能对账号限频。请低速、分前缀/品类分批采集。
+11
View File
@@ -0,0 +1,11 @@
#!/usr/bin/env bash
# Build the bypos-collector for Windows (and a Linux binary for local testing).
set -euo pipefail
cd "$(dirname "$0")"
gofmt -w ./*.go
go vet ./...
echo "building windows/amd64 .exe ..."
GOOS=windows GOARCH=amd64 go build -ldflags "-s -w" -o bypos-collector.exe .
echo "building linux/amd64 (for testing) ..."
go build -o bypos-collector-linux .
ls -la bypos-collector.exe bypos-collector-linux
+481
View File
@@ -0,0 +1,481 @@
package main
import (
"context"
"crypto/md5"
"encoding/hex"
"encoding/json"
"fmt"
"io"
"net/http"
"net/url"
"os"
"path/filepath"
"regexp"
"strconv"
"strings"
"sync"
"sync/atomic"
"time"
)
// ---- upstream config ----
// sdogid is the 云店 account license id observed in the live request. It is
// configurable so the tool is not tied to a single account.
const defaultSdogID = "137966"
const endpoint = "http://zc.bypos.net/byGoodsService/byMessage.asmx/GetGoodsinfo"
var stringTagRe = regexp.MustCompile(`(?s)<string[^>]*>(.*)</string>`)
// Product is the normalized record we persist (one JSON object per line).
type Product struct {
Barcode string `json:"barcode"`
Name string `json:"name"` // item_name 品名
Spec string `json:"spec"` // item_size 规格
Unit string `json:"unit"` // unit_no 单位
Area string `json:"area"` // item_area 产地/地区
Manufacturer string `json:"manufacturer"` // birth_com 生产企业
License string `json:"license"` // birth_doc 生产许可
InPrice string `json:"in_price"` // 建议进价
SellPrice string `json:"sell_price"` // 建议零售价
Status string `json:"status"` // hit / miss / invalid / error
RetMsg string `json:"retmsg"` // 原始返回信息
FetchedAt string `json:"fetched_at"` // RFC3339
Source string `json:"source"` // zc.bypos.net
}
// upstream raw fields
type rawResp struct {
RetCode string `json:"retcode"`
RetMsg string `json:"retmsg"`
Barcode string `json:"barcode"`
ItemName string `json:"item_name"`
UnitNo string `json:"unit_no"`
ItemSize string `json:"item_size"`
ItemArea string `json:"item_area"`
BirthCom string `json:"birth_com"`
BirthDoc string `json:"birth_doc"`
InPrice string `json:"inprice"`
SellPrice string `json:"sellprice"`
}
func md5hex(s string) string {
h := md5.Sum([]byte(s))
return hex.EncodeToString(h[:])
}
// ean13Check computes the EAN-13 check digit for a 12-digit body.
func ean13Check(body string) (string, bool) {
if len(body) != 12 {
return "", false
}
sum := 0
for i := 0; i < 12; i++ {
c := body[i]
if c < '0' || c > '9' {
return "", false
}
d := int(c - '0')
if i%2 == 0 {
sum += d
} else {
sum += d * 3
}
}
chk := (10 - (sum % 10)) % 10
return body + strconv.Itoa(chk), true
}
// lookup queries the upstream central library for one barcode.
func (c *Collector) lookup(ctx context.Context, barcode string) (*Product, error) {
tsMs := strconv.FormatInt(time.Now().Unix()*1000, 10) // always ends in 000
sparm1 := md5hex(c.sdogID)
sparm2 := md5hex(barcode + tsMs)
sparm3 := tsMs[:10]
q := url.Values{}
q.Set("sdogid", c.sdogID)
q.Set("regnum", "1")
q.Set("barcode", barcode)
q.Set("sparm1", sparm1)
q.Set("sparm2", sparm2)
q.Set("sparm3", sparm3)
q.Set("sparm4", "")
q.Set("barcodetype", "yunpos")
reqURL := endpoint + "?" + q.Encode()
req, _ := http.NewRequestWithContext(ctx, "GET", reqURL, nil)
req.Header.Set("User-Agent", "Mozilla/5.0")
resp, err := c.client.Do(req)
if err != nil {
return nil, err
}
defer resp.Body.Close()
b, err := io.ReadAll(resp.Body)
if err != nil {
return nil, err
}
inner := b
if m := stringTagRe.FindSubmatch(b); m != nil {
inner = m[1]
}
var r rawResp
if err := json.Unmarshal(inner, &r); err != nil {
return nil, fmt.Errorf("parse: %v (body=%.120s)", err, string(b))
}
p := &Product{
Barcode: barcode,
RetMsg: r.RetMsg,
FetchedAt: time.Now().Format(time.RFC3339),
Source: "zc.bypos.net",
}
if r.RetCode == "1" {
p.Status = "hit"
p.Name = strings.TrimSpace(r.ItemName)
p.Spec = strings.TrimSpace(r.ItemSize)
p.Unit = strings.TrimSpace(r.UnitNo)
p.Area = strings.TrimSpace(r.ItemArea)
p.Manufacturer = strings.TrimSpace(r.BirthCom)
p.License = strings.TrimSpace(r.BirthDoc)
p.InPrice = strings.TrimSpace(r.InPrice)
p.SellPrice = strings.TrimSpace(r.SellPrice)
} else if strings.Contains(r.RetMsg, "非国标") || strings.Contains(r.RetMsg, "参数异常") {
p.Status = "invalid"
} else {
p.Status = "miss"
}
return p, nil
}
// ---- job / collector state ----
type Stats struct {
Running bool `json:"running"`
Total int64 `json:"total"`
Done int64 `json:"done"`
Hits int64 `json:"hits"`
Miss int64 `json:"miss"`
Invalid int64 `json:"invalid"`
Errors int64 `json:"errors"`
Skipped int64 `json:"skipped"`
Current string `json:"current"`
OutFile string `json:"out_file"`
StartedAt string `json:"started_at"`
Message string `json:"message"`
}
type Collector struct {
mu sync.Mutex
client *http.Client
sdogID string
outPath string
outFile *os.File
cancel context.CancelFunc
wg sync.WaitGroup
// atomic counters
total, done, hits, miss, invalid, errors, skipped int64
running int32
current atomic.Value // string
startedAt string
message string
seen map[string]struct{} // barcodes already in output (dedupe / resume)
recent []Product // ring of last results for UI
}
func NewCollector(sdogID string) *Collector {
if sdogID == "" {
sdogID = defaultSdogID
}
c := &Collector{
client: &http.Client{Timeout: 25 * time.Second},
sdogID: sdogID,
seen: map[string]struct{}{},
}
c.current.Store("")
return c
}
func (c *Collector) isRunning() bool { return atomic.LoadInt32(&c.running) == 1 }
func (c *Collector) snapshot() Stats {
cur, _ := c.current.Load().(string)
c.mu.Lock()
msg := c.message
out := c.outPath
started := c.startedAt
c.mu.Unlock()
return Stats{
Running: c.isRunning(),
Total: atomic.LoadInt64(&c.total),
Done: atomic.LoadInt64(&c.done),
Hits: atomic.LoadInt64(&c.hits),
Miss: atomic.LoadInt64(&c.miss),
Invalid: atomic.LoadInt64(&c.invalid),
Errors: atomic.LoadInt64(&c.errors),
Skipped: atomic.LoadInt64(&c.skipped),
Current: cur,
OutFile: out,
StartedAt: started,
Message: msg,
}
}
func (c *Collector) recentResults() []Product {
c.mu.Lock()
defer c.mu.Unlock()
out := make([]Product, len(c.recent))
copy(out, c.recent)
return out
}
func (c *Collector) pushRecent(p Product) {
c.mu.Lock()
c.recent = append(c.recent, p)
if len(c.recent) > 60 {
c.recent = c.recent[len(c.recent)-60:]
}
c.mu.Unlock()
}
// loadSeen reads an existing output file to build the dedupe set (for resume).
func (c *Collector) loadSeen(path string) error {
c.seen = map[string]struct{}{}
f, err := os.Open(path)
if err != nil {
if os.IsNotExist(err) {
return nil
}
return err
}
defer f.Close()
dec := json.NewDecoder(f)
for {
var p Product
if err := dec.Decode(&p); err != nil {
break
}
if p.Barcode != "" {
c.seen[p.Barcode] = struct{}{}
}
}
return nil
}
type JobReq struct {
Mode string `json:"mode"` // "range" | "list"
StartBody string `json:"start_body"` // 12-digit body (range mode)
EndBody string `json:"end_body"` // 12-digit body (range mode)
List string `json:"list"` // newline/space separated barcodes (list mode)
Concurrency int `json:"concurrency"` // parallel requests
DelayMs int `json:"delay_ms"` // min interval between request starts
LogMiss bool `json:"log_miss"` // also write miss/invalid lines
OutFile string `json:"out_file"`
SdogID string `json:"sdog_id"`
}
func sanitizeBarcodes(s string) []string {
fields := regexp.MustCompile(`[^0-9]+`).Split(s, -1)
var out []string
for _, f := range fields {
if f != "" {
out = append(out, f)
}
}
return out
}
// Start launches a collection job. Returns error if validation fails or busy.
func (c *Collector) Start(req JobReq) error {
if c.isRunning() {
return fmt.Errorf("已有任务在运行")
}
if req.Concurrency <= 0 {
req.Concurrency = 3
}
if req.Concurrency > 20 {
req.Concurrency = 20
}
if req.DelayMs < 0 {
req.DelayMs = 0
}
if req.OutFile == "" {
req.OutFile = "products.jsonl"
}
if req.SdogID != "" {
c.sdogID = req.SdogID
}
// Build the list of barcodes to query.
var barcodes []string
switch req.Mode {
case "list":
barcodes = sanitizeBarcodes(req.List)
if len(barcodes) == 0 {
return fmt.Errorf("条码清单为空")
}
case "range":
start, err := strconv.ParseInt(req.StartBody, 10, 64)
if err != nil || len(req.StartBody) != 12 {
return fmt.Errorf("起始码必须是 12 位数字(不含校验位)")
}
end, err := strconv.ParseInt(req.EndBody, 10, 64)
if err != nil || len(req.EndBody) != 12 {
return fmt.Errorf("结束码必须是 12 位数字(不含校验位)")
}
if end < start {
return fmt.Errorf("结束码不能小于起始码")
}
if end-start+1 > 5_000_000 {
return fmt.Errorf("单次范围过大(>500万),请缩小区间分批采集")
}
for v := start; v <= end; v++ {
body := fmt.Sprintf("%012d", v)
full, ok := ean13Check(body)
if ok {
barcodes = append(barcodes, full)
}
}
default:
return fmt.Errorf("未知模式: %s", req.Mode)
}
abs, _ := filepath.Abs(req.OutFile)
if err := c.loadSeen(abs); err != nil {
return fmt.Errorf("读取已有文件失败: %v", err)
}
f, err := os.OpenFile(abs, os.O_CREATE|os.O_WRONLY|os.O_APPEND, 0644)
if err != nil {
return fmt.Errorf("打开输出文件失败: %v", err)
}
c.outFile = f
c.outPath = abs
// reset counters
atomic.StoreInt64(&c.total, int64(len(barcodes)))
atomic.StoreInt64(&c.done, 0)
atomic.StoreInt64(&c.hits, 0)
atomic.StoreInt64(&c.miss, 0)
atomic.StoreInt64(&c.invalid, 0)
atomic.StoreInt64(&c.errors, 0)
atomic.StoreInt64(&c.skipped, 0)
c.mu.Lock()
c.recent = nil
c.startedAt = time.Now().Format(time.RFC3339)
c.message = ""
c.mu.Unlock()
atomic.StoreInt32(&c.running, 1)
ctx, cancel := context.WithCancel(context.Background())
c.cancel = cancel
go c.run(ctx, barcodes, req)
return nil
}
func (c *Collector) Stop() {
if c.cancel != nil {
c.cancel()
}
}
func (c *Collector) run(ctx context.Context, barcodes []string, req JobReq) {
defer func() {
atomic.StoreInt32(&c.running, 0)
if c.outFile != nil {
c.outFile.Sync()
c.outFile.Close()
c.outFile = nil
}
c.current.Store("")
}()
jobs := make(chan string, req.Concurrency*2)
var writeMu sync.Mutex
// global rate limiter: one token every DelayMs
var ticker *time.Ticker
if req.DelayMs > 0 {
ticker = time.NewTicker(time.Duration(req.DelayMs) * time.Millisecond)
defer ticker.Stop()
}
worker := func() {
defer c.wg.Done()
for bc := range jobs {
if ctx.Err() != nil {
return
}
if ticker != nil {
select {
case <-ticker.C:
case <-ctx.Done():
return
}
}
c.current.Store(bc)
p, err := c.lookup(ctx, bc)
if err != nil {
if ctx.Err() != nil {
return
}
atomic.AddInt64(&c.errors, 1)
atomic.AddInt64(&c.done, 1)
ep := Product{Barcode: bc, Status: "error", RetMsg: err.Error(), FetchedAt: time.Now().Format(time.RFC3339), Source: "zc.bypos.net"}
c.pushRecent(ep)
continue
}
switch p.Status {
case "hit":
atomic.AddInt64(&c.hits, 1)
case "miss":
atomic.AddInt64(&c.miss, 1)
case "invalid":
atomic.AddInt64(&c.invalid, 1)
}
atomic.AddInt64(&c.done, 1)
c.pushRecent(*p)
if p.Status == "hit" || req.LogMiss {
line, _ := json.Marshal(p)
writeMu.Lock()
c.outFile.Write(line)
c.outFile.Write([]byte("\n"))
c.seen[bc] = struct{}{}
writeMu.Unlock()
}
}
}
for i := 0; i < req.Concurrency; i++ {
c.wg.Add(1)
go worker()
}
for _, bc := range barcodes {
if ctx.Err() != nil {
break
}
if _, ok := c.seen[bc]; ok {
atomic.AddInt64(&c.skipped, 1)
atomic.AddInt64(&c.done, 1)
continue
}
select {
case jobs <- bc:
case <-ctx.Done():
}
}
close(jobs)
c.wg.Wait()
c.mu.Lock()
if ctx.Err() != nil {
c.message = "已停止"
} else {
c.message = "采集完成"
}
c.mu.Unlock()
}
+3
View File
@@ -0,0 +1,3 @@
module byposcollector
go 1.23.4
+156
View File
@@ -0,0 +1,156 @@
package main
import (
"embed"
"encoding/csv"
"encoding/json"
"flag"
"fmt"
"io"
"io/fs"
"log"
"net"
"net/http"
"os"
"os/exec"
"runtime"
"time"
)
//go:embed web/*
var webFS embed.FS
var collector = NewCollector("")
func main() {
addr := flag.String("addr", "127.0.0.1:8765", "本地监听地址")
noOpen := flag.Bool("no-open", false, "不自动打开浏览器")
sdog := flag.String("sdogid", "", "中心库账号 id(默认使用内置值)")
flag.Parse()
if *sdog != "" {
collector.sdogID = *sdog
}
sub, _ := fs.Sub(webFS, "web")
mux := http.NewServeMux()
mux.Handle("/", http.FileServer(http.FS(sub)))
mux.HandleFunc("/api/start", handleStart)
mux.HandleFunc("/api/stop", handleStop)
mux.HandleFunc("/api/stats", handleStats)
mux.HandleFunc("/api/download", handleDownload)
mux.HandleFunc("/api/export.csv", handleExportCSV)
ln, err := net.Listen("tcp", *addr)
if err != nil {
log.Fatalf("无法监听 %s: %v", *addr, err)
}
realAddr := ln.Addr().String()
urlStr := "http://" + realAddr + "/"
fmt.Println("==============================================")
fmt.Println(" 中心库商品采集器 bypos-collector")
fmt.Println(" 控制台: " + urlStr)
fmt.Println(" 关闭本窗口即停止程序")
fmt.Println("==============================================")
if !*noOpen {
go openBrowser(urlStr)
}
log.Fatal(http.Serve(ln, mux))
}
func openBrowser(url string) {
time.Sleep(600 * time.Millisecond)
var cmd string
var args []string
switch runtime.GOOS {
case "windows":
cmd = "rundll32"
args = []string{"url.dll,FileProtocolHandler", url}
case "darwin":
cmd = "open"
args = []string{url}
default:
cmd = "xdg-open"
args = []string{url}
}
_ = exec.Command(cmd, args...).Start()
}
func writeJSON(w http.ResponseWriter, code int, v interface{}) {
w.Header().Set("Content-Type", "application/json; charset=utf-8")
w.WriteHeader(code)
json.NewEncoder(w).Encode(v)
}
func handleStart(w http.ResponseWriter, r *http.Request) {
if r.Method != "POST" {
http.Error(w, "method", 405)
return
}
var req JobReq
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
writeJSON(w, 400, map[string]string{"error": "请求格式错误"})
return
}
if err := collector.Start(req); err != nil {
writeJSON(w, 400, map[string]string{"error": err.Error()})
return
}
writeJSON(w, 200, map[string]string{"ok": "started"})
}
func handleStop(w http.ResponseWriter, r *http.Request) {
collector.Stop()
writeJSON(w, 200, map[string]string{"ok": "stopping"})
}
func handleStats(w http.ResponseWriter, r *http.Request) {
writeJSON(w, 200, map[string]interface{}{
"stats": collector.snapshot(),
"recent": collector.recentResults(),
})
}
func handleDownload(w http.ResponseWriter, r *http.Request) {
s := collector.snapshot()
if s.OutFile == "" {
http.Error(w, "no output yet", 404)
return
}
f, err := os.Open(s.OutFile)
if err != nil {
http.Error(w, err.Error(), 404)
return
}
defer f.Close()
w.Header().Set("Content-Type", "application/x-ndjson; charset=utf-8")
w.Header().Set("Content-Disposition", "attachment; filename=products.jsonl")
io.Copy(w, f)
}
func handleExportCSV(w http.ResponseWriter, r *http.Request) {
s := collector.snapshot()
if s.OutFile == "" {
http.Error(w, "no output yet", 404)
return
}
f, err := os.Open(s.OutFile)
if err != nil {
http.Error(w, err.Error(), 404)
return
}
defer f.Close()
w.Header().Set("Content-Type", "text/csv; charset=utf-8")
w.Header().Set("Content-Disposition", "attachment; filename=products.csv")
w.Write([]byte{0xEF, 0xBB, 0xBF}) // UTF-8 BOM so Excel reads Chinese correctly
cw := csv.NewWriter(w)
cw.Write([]string{"barcode", "name", "spec", "unit", "area", "manufacturer", "license", "in_price", "sell_price", "status", "fetched_at"})
dec := json.NewDecoder(f)
for {
var p Product
if err := dec.Decode(&p); err != nil {
break
}
cw.Write([]string{p.Barcode, p.Name, p.Spec, p.Unit, p.Area, p.Manufacturer, p.License, p.InPrice, p.SellPrice, p.Status, p.FetchedAt})
}
cw.Flush()
}
+206
View File
@@ -0,0 +1,206 @@
<!DOCTYPE html>
<html lang="zh-CN">
<head>
<meta charset="utf-8"/>
<meta name="viewport" content="width=device-width, initial-scale=1"/>
<title>中心库商品采集器</title>
<style>
* { box-sizing: border-box; }
body { font-family: -apple-system, "Microsoft YaHei", Arial, sans-serif; margin: 0; background:#f4f6f9; color:#222; }
header { background:#1f6feb; color:#fff; padding:14px 22px; font-size:18px; font-weight:600; }
.wrap { max-width:1080px; margin:18px auto; padding:0 16px; }
.card { background:#fff; border:1px solid #e3e8ef; border-radius:10px; padding:18px 20px; margin-bottom:16px; }
.card h3 { margin:0 0 12px; font-size:15px; color:#1f6feb; }
label { display:block; font-size:13px; color:#555; margin:8px 0 4px; }
input[type=text], input[type=number], textarea, select {
width:100%; padding:8px 10px; border:1px solid #cdd5e0; border-radius:6px; font-size:14px;
}
textarea { height:90px; font-family:monospace; }
.row { display:flex; gap:14px; flex-wrap:wrap; }
.row > div { flex:1; min-width:160px; }
.tabs { display:flex; gap:8px; margin-bottom:12px; }
.tab { padding:7px 16px; border:1px solid #cdd5e0; border-radius:20px; cursor:pointer; font-size:13px; background:#fff; }
.tab.active { background:#1f6feb; color:#fff; border-color:#1f6feb; }
button.primary { background:#1f6feb; color:#fff; border:none; padding:10px 22px; border-radius:6px; font-size:14px; cursor:pointer; }
button.danger { background:#d1242f; color:#fff; border:none; padding:10px 22px; border-radius:6px; font-size:14px; cursor:pointer; }
button.ghost { background:#fff; color:#1f6feb; border:1px solid #1f6feb; padding:8px 16px; border-radius:6px; cursor:pointer; font-size:13px; }
button:disabled { opacity:.5; cursor:not-allowed; }
.stats { display:flex; gap:10px; flex-wrap:wrap; }
.stat { flex:1; min-width:90px; background:#f7f9fc; border:1px solid #e3e8ef; border-radius:8px; padding:10px; text-align:center; }
.stat .n { font-size:22px; font-weight:700; }
.stat .l { font-size:12px; color:#777; margin-top:2px; }
.bar { height:10px; background:#e3e8ef; border-radius:6px; overflow:hidden; margin:10px 0; }
.bar > div { height:100%; background:#2da44e; width:0%; transition:width .4s; }
table { width:100%; border-collapse:collapse; font-size:13px; }
th, td { text-align:left; padding:6px 8px; border-bottom:1px solid #eef1f5; white-space:nowrap; overflow:hidden; text-overflow:ellipsis; max-width:180px; }
th { color:#888; font-weight:600; }
.hit { color:#2da44e; } .miss { color:#999; } .invalid { color:#d1242f; } .error { color:#bf8700; }
.hint { font-size:12px; color:#888; margin-top:6px; line-height:1.5; }
.est { font-size:13px; color:#1f6feb; margin-top:6px; }
</style>
</head>
<body>
<header>中心库商品采集器 · bypos-collector</header>
<div class="wrap">
<div class="card">
<h3>① 规划采集范围</h3>
<div class="tabs">
<div class="tab active" data-mode="range" onclick="setMode('range')">按条码范围</div>
<div class="tab" data-mode="list" onclick="setMode('list')">按条码清单</div>
</div>
<div id="pane-range">
<div class="hint">EAN-13 国标条码共 13 位,最后一位是校验位由程序自动计算。下面填<b>前 12 位</b>(本体),程序逐个枚举并补校验位查询。常见前缀:69 开头为中国大陆。</div>
<label>快捷填充前缀(可选)</label>
<div class="row">
<div><input type="text" id="prefix" placeholder="如 690100,点下方按钮自动算区间"/></div>
<div style="flex:0"><button class="ghost" onclick="fillFromPrefix()">用前缀填充区间</button></div>
</div>
<div class="row">
<div>
<label>起始本体(12 位)</label>
<input type="text" id="start" value="690100000000" maxlength="12"/>
</div>
<div>
<label>结束本体(12 位)</label>
<input type="text" id="end" value="690100000999" maxlength="12"/>
</div>
</div>
<div class="est" id="est"></div>
</div>
<div id="pane-list" style="display:none">
<label>粘贴条码清单(每行一个,或用空格/逗号分隔)</label>
<textarea id="list" placeholder="6901028941068&#10;6920202888883"></textarea>
</div>
</div>
<div class="card">
<h3>② 采集参数</h3>
<div class="row">
<div>
<label>并发数</label>
<input type="number" id="concurrency" value="3" min="1" max="20"/>
</div>
<div>
<label>每次请求间隔(毫秒)</label>
<input type="number" id="delay" value="300" min="0"/>
</div>
<div>
<label>输出文件名</label>
<input type="text" id="outfile" value="products.jsonl"/>
</div>
</div>
<label style="margin-top:12px"><input type="checkbox" id="logmiss"/> 同时记录未命中/无效条码(默认只存命中)</label>
<div class="hint">速度越快越容易触发上游频控。建议并发 3、间隔 300ms 起步,稳定后再调。已采过的条码会自动跳过(断点续采)。</div>
</div>
<div class="card">
<h3>③ 运行</h3>
<div style="margin-bottom:12px">
<button class="primary" id="btnStart" onclick="start()">开始采集</button>
<button class="danger" id="btnStop" onclick="stop()" disabled>停止</button>
<button class="ghost" onclick="window.open('/api/download')">下载 JSONL</button>
<button class="ghost" onclick="window.open('/api/export.csv')">导出 CSV(Excel)</button>
</div>
<div class="bar"><div id="prog"></div></div>
<div class="stats">
<div class="stat"><div class="n" id="s-done">0</div><div class="l">已处理</div></div>
<div class="stat"><div class="n" id="s-total">0</div><div class="l">总计</div></div>
<div class="stat"><div class="n hit" id="s-hits">0</div><div class="l">命中</div></div>
<div class="stat"><div class="n miss" id="s-miss">0</div><div class="l">未命中</div></div>
<div class="stat"><div class="n invalid" id="s-invalid">0</div><div class="l">无效</div></div>
<div class="stat"><div class="n error" id="s-errors">0</div><div class="l">错误</div></div>
<div class="stat"><div class="n" id="s-skipped">0</div><div class="l">跳过</div></div>
</div>
<div class="hint" id="msg"></div>
</div>
<div class="card">
<h3>④ 实时结果(最近 60 条)</h3>
<div style="max-height:340px; overflow:auto">
<table>
<thead><tr><th>条码</th><th>品名</th><th>规格</th><th>单位</th><th>产地</th><th>进价</th><th>零售价</th><th>状态</th></tr></thead>
<tbody id="rows"></tbody>
</table>
</div>
</div>
</div>
<script>
let mode = 'range';
function setMode(m){
mode = m;
document.querySelectorAll('.tab').forEach(t=>t.classList.toggle('active', t.dataset.mode===m));
document.getElementById('pane-range').style.display = m==='range'?'block':'none';
document.getElementById('pane-list').style.display = m==='list'?'block':'none';
}
function fillFromPrefix(){
let p = document.getElementById('prefix').value.replace(/[^0-9]/g,'');
if(!p){ alert('请先填前缀'); return; }
if(p.length>=12){ alert('前缀太长,应少于 12 位'); return; }
let pad = 12 - p.length;
document.getElementById('start').value = p + '0'.repeat(pad);
document.getElementById('end').value = p + '9'.repeat(pad);
updateEst();
}
function updateEst(){
let s = document.getElementById('start').value.replace(/[^0-9]/g,'');
let e = document.getElementById('end').value.replace(/[^0-9]/g,'');
if(s.length===12 && e.length===12){
let n = (BigInt(e) - BigInt(s)) + 1n;
document.getElementById('est').textContent = '本次将查询约 ' + n.toString() + ' 个条码';
} else {
document.getElementById('est').textContent = '';
}
}
document.getElementById('start').addEventListener('input', updateEst);
document.getElementById('end').addEventListener('input', updateEst);
updateEst();
async function start(){
let body = {
mode: mode,
start_body: document.getElementById('start').value.trim(),
end_body: document.getElementById('end').value.trim(),
list: document.getElementById('list').value,
concurrency: parseInt(document.getElementById('concurrency').value)||3,
delay_ms: parseInt(document.getElementById('delay').value)||0,
log_miss: document.getElementById('logmiss').checked,
out_file: document.getElementById('outfile').value.trim()
};
let r = await fetch('/api/start', {method:'POST', headers:{'Content-Type':'application/json'}, body:JSON.stringify(body)});
let j = await r.json();
if(j.error){ alert('启动失败: ' + j.error); return; }
}
async function stop(){ await fetch('/api/stop', {method:'POST'}); }
function esc(s){ return (s||'').replace(/[&<>]/g, c=>({'&':'&amp;','<':'&lt;','>':'&gt;'}[c])); }
async function poll(){
try{
let r = await fetch('/api/stats'); let j = await r.json();
let s = j.stats;
document.getElementById('s-done').textContent = s.done;
document.getElementById('s-total').textContent = s.total;
document.getElementById('s-hits').textContent = s.hits;
document.getElementById('s-miss').textContent = s.miss;
document.getElementById('s-invalid').textContent = s.invalid;
document.getElementById('s-errors').textContent = s.errors;
document.getElementById('s-skipped').textContent = s.skipped;
let pct = s.total>0 ? Math.floor(s.done*100/s.total) : 0;
document.getElementById('prog').style.width = pct + '%';
document.getElementById('msg').textContent = (s.running? ('采集中… 当前 '+s.current) : (s.message||'空闲'));
document.getElementById('btnStart').disabled = s.running;
document.getElementById('btnStop').disabled = !s.running;
let rows = (j.recent||[]).slice().reverse().map(p=>
'<tr><td>'+esc(p.barcode)+'</td><td>'+esc(p.name)+'</td><td>'+esc(p.spec)+'</td><td>'+esc(p.unit)+'</td><td>'+esc(p.area)+'</td><td>'+esc(p.in_price)+'</td><td>'+esc(p.sell_price)+'</td><td class="'+p.status+'">'+esc(p.status)+'</td></tr>'
).join('');
document.getElementById('rows').innerHTML = rows;
}catch(e){}
}
setInterval(poll, 1000); poll();
</script>
</body>
</html>
+68
View File
@@ -0,0 +1,68 @@
# 中心库商品采集器 bypos-collector 使用说明
一个单文件 Windows 小程序,通过云店「新增商品」用到的同一个中心商品库
(`zc.bypos.net`)按条码批量采集商品档案(品名/规格/单位/产地/厂商/建议进价/建议零售价),
存到本地,供后续导入天工(goods)系统。
## 一、运行
1.`bypos-collector.exe` 放到任意空文件夹(采集结果会生成在同一文件夹)。
2. 双击运行。会弹出一个黑色命令行窗口(不要关它),并自动打开浏览器控制台
`http://127.0.0.1:8765/`
- 若没自动打开,手动在浏览器输入上面这个地址。
3. 用完直接关掉那个命令行窗口即可退出。
## 二、采集
控制台分四步:
**① 规划采集范围** —— 两种方式二选一:
- **按条码范围**:EAN-13 国标条码共 13 位,最后一位是校验位,程序自动算。
你只填**前 12 位**的起止区间。可在「快捷填充前缀」里填如 `690100`,
点按钮自动生成区间(`690100000000` ~ `690100999999`)。
- **按条码清单**:直接粘贴一批条码(每行一个,或空格/逗号分隔)。
**② 采集参数**:
- 并发数(默认 3)、请求间隔(默认 300ms):**越慢越安全**,上游可能对账号限频。
- 输出文件名(默认 `products.jsonl`)。
- 「同时记录未命中/无效条码」:默认只存命中的;勾上会把未命中也记下来。
**③ 运行**:点「开始采集」。进度、命中/未命中/错误实时显示。
已经采过的条码会自动跳过(可随时停了再开,断点续采)。
**④ 实时结果**:最近 60 条滚动显示。
## 三、导出
- 「下载 JSONL」:原始数据(每行一个 JSON),用于导入天工系统。
- 「导出 CSV」:Excel 可直接打开查看。
## 四、字段说明(JSONL 每行)
| 字段 | 含义 |
| --- | --- |
| barcode | 条码(GTIN/EAN-13) |
| name | 品名 |
| spec | 规格 |
| unit | 单位 |
| area | 产地/地区 |
| manufacturer | 生产企业(常为空) |
| license | 生产许可(常为空) |
| in_price | 建议进价 |
| sell_price | 建议零售价 |
| status | hit=命中 / miss=不存在 / invalid=非国标条码 / error=请求出错 |
| fetched_at | 采集时间 |
## 五、注意
- 这是用云店账号授权去查上游中心库,**批量自动**比页面里一条条查更"重",
上游厂商可能对账号做频控/限额。请低速、分批("一点点采"),发现大量报错就降速。
- 全量 69 段是个天文数字,不要无脑全跑;建议按你关心的品牌/品类前缀分批。
## 六、命令行参数(可选)
```
bypos-collector.exe -addr 127.0.0.1:8765 # 改监听端口
bypos-collector.exe -no-open # 不自动开浏览器
bypos-collector.exe -sdogid 137966 # 指定中心库账号 id
```