Compare commits

..

26 Commits

Author SHA1 Message Date
lixu 0da57a2fb3 Merge pull request 'feat(admin): 新增 /api/import/bypos 导入接口(修复采集器导入失败)' (#33) from devin/1782307670-bypos-import-endpoint into main
CI / Go (api) (push) Successful in 53s
CI / Python (ingestion) (push) Successful in 16s
CI / Migrations (postgres) (push) Successful in 28s
2026-06-24 21:33:02 +08:00
rosemariejebbjtxbfp 85c58fbd52 feat(admin): add /api/import/bypos endpoint for bypos-collector import
CI / Go (api) (pull_request) Successful in 58s
CI / Python (ingestion) (pull_request) Successful in 16s
CI / Migrations (postgres) (pull_request) Successful in 27s
Brings the bypos JSONL import backend (handler + store) into main so the
bypos-collector tool's import feature works end-to-end. Records are upserted
on GTIN with manufacturer/MSRP/barcode/source provenance, quality recomputed
per product. Only status=hit records with a name are imported.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-24 13:30:27 +00:00
lixu 3aec0b84fe Merge pull request 'feat(web+api): 贡献表单按 archive_kind 动态生成字段' (#32) from devin/1782295354-contribution-form-dynamic into main
CI / Go (api) (push) Successful in 55s
CI / Python (ingestion) (push) Successful in 32s
CI / Migrations (postgres) (push) Successful in 26s
2026-06-24 18:07:05 +08:00
rosemariejebbjtxbfp bd1d3bde7c feat(web+api): 贡献表单按 archive_kind 动态生成字段
CI / Go (api) (pull_request) Successful in 1m3s
CI / Python (ingestion) (pull_request) Successful in 23s
CI / Migrations (postgres) (pull_request) Successful in 26s
- categories 接口返回 archive_kind
- 新增公开只读接口 /api/v1/kind-fields?kind=X 返回字段模板
- 投稿 payload 支持通用 attributes,审核通过写入 product.attributes (JSONB 合并)
- food_detail 仅在含食品数据时才 upsert (药品/3C/通用不再产生空行)
- 前端 Contribute 按所选品类 archive_kind 动态渲染字段
  (食品营养 / 药品 18 字段 / 3C / 通用),除商品名外均选填

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-24 10:02:46 +00:00
lixu 998471f6df Merge pull request 'feat: 新增药品(drug)商品档案格式' (#31) from devin/1782286467-drug-archive-kind into main
CI / Go (api) (push) Successful in 47s
CI / Python (ingestion) (push) Successful in 18s
CI / Migrations (postgres) (push) Successful in 27s
2026-06-24 15:39:07 +08:00
rosemariejebbjtxbfp 1f9b1ecd30 feat(migrations): add drug (药品) archive kind template + category subtree
CI / Go (api) (pull_request) Successful in 55s
CI / Python (ingestion) (pull_request) Successful in 23s
CI / Migrations (postgres) (pull_request) Successful in 27s
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-24 07:36:43 +00:00
lixu 1d71617f71 Merge pull request 'chore(web): 添加微信域名校验文件' (#30) from devin/1782285002-wechat-domain-verify into main
CI / Go (api) (push) Successful in 51s
CI / Python (ingestion) (push) Successful in 30s
CI / Migrations (postgres) (push) Successful in 26s
2026-06-24 15:11:58 +08:00
rosemariejebbjtxbfp d18fc2470b chore(web): add WeChat domain verification file
CI / Go (api) (pull_request) Successful in 54s
CI / Python (ingestion) (pull_request) Successful in 26s
CI / Migrations (postgres) (pull_request) Successful in 27s
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-24 07:10:07 +00:00
lixu f29696607a Merge pull request 'feat(api): Redis read cache for product details and search (stage 0)' (#29) from devin/1782283345-redis-read-cache into main
CI / Go (api) (push) Successful in 50s
CI / Python (ingestion) (push) Successful in 37s
CI / Migrations (postgres) (push) Successful in 27s
2026-06-24 14:54:59 +08:00
rosemariejebbjtxbfp a75318b811 feat(api): Redis read cache for product details and search
CI / Go (api) (pull_request) Successful in 54s
CI / Python (ingestion) (pull_request) Successful in 19s
CI / Migrations (postgres) (pull_request) Successful in 27s
Stage-0 caching from docs/scalability.md. The Go API now caches hot
product-detail and search-result reads in Redis with a fail-open,
epoch-versioned scheme; Python ingestion bumps the epoch after a write
run to invalidate the cache globally in O(1).

- api/internal/cache: fail-open Cache (GetJSON/SetJSON) namespaced by an
  epoch counter (og:cache:epoch); disabled when Redis is unconfigured.
- store: ProductByID/ProductByGTIN (24h TTL) and SearchProducts (1h TTL)
  read-through the cache via WithCache.
- ingestion: bump_cache_epoch() called after update_off/seed_off/
  import_bypos/dedup commits when rows changed; best-effort, never fails
  a run. Adds redis dependency.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-24 06:52:56 +00:00
lixu 20b9eb0fc9 Merge pull request 'ci: 修复 main 上 Python CI 历史失败(在 python3.12+node 容器内运行)' (#28) from devin/1782280936-fix-python-ci into main
CI / Go (api) (push) Successful in 54s
CI / Python (ingestion) (push) Successful in 23s
CI / Migrations (postgres) (push) Successful in 27s
2026-06-24 14:32:51 +08:00
rosemariejebbjtxbfp 17bc0ed680 ci: remove GitHub-hosted actions, manual checkout + Go from CN mirror
CI / Go (api) (pull_request) Successful in 51s
CI / Python (ingestion) (pull_request) Successful in 21s
CI / Migrations (postgres) (pull_request) Successful in 26s
Self-hosted Gitea runner has flaky/blocked access to github.com, causing
actions/checkout and actions/setup-go to time out intermittently across all
jobs. Replace them with:
- manual git checkout against $GITHUB_SERVER_URL (the Gitea host, reachable
  from job containers)
- Go installed from mirrors.aliyun.com

Python job stays on the python3.12-nodejs20 container (fixes the original
PEP 660 editable-install failure). No more github.com network dependency.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-24 06:30:03 +00:00
rosemariejebbjtxbfp 5020fdcc19 ci: re-trigger with public gitea actions url
CI / Go (api) (pull_request) Failing after 11s
CI / Python (ingestion) (pull_request) Failing after 31s
CI / Migrations (postgres) (pull_request) Failing after 11s
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-24 06:24:52 +00:00
rosemariejebbjtxbfp 885f6c2b01 ci: re-trigger with self-hosted actions
CI / Go (api) (pull_request) Failing after 12s
CI / Python (ingestion) (pull_request) Failing after 1m31s
CI / Migrations (postgres) (pull_request) Failing after 32s
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-24 06:21:01 +00:00
rosemariejebbjtxbfp 2aff6287d2 ci: re-trigger after configuring action proxy
CI / Go (api) (pull_request) Failing after 13s
CI / Python (ingestion) (pull_request) Failing after 31s
CI / Migrations (postgres) (pull_request) Failing after 1m32s
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-24 06:10:22 +00:00
rosemariejebbjtxbfp 3364a50c68 ci: run Python job in python3.12+node container
CI / Go (api) (pull_request) Failing after 1m32s
CI / Python (ingestion) (pull_request) Failing after 31s
CI / Migrations (postgres) (pull_request) Failing after 19m16s
setup-python@v5 can't fetch CPython 3.12 on the self-hosted Gitea runner
(it queries the Gitea API for actions/python-versions, which 404s), so the
job silently fell back to the image's system Python 3.10 + pip 22.0.2. That
old pip lacks PEP 660 editable support, so 'pip install -e' failed with
'build backend is missing the build_editable hook', and 3.10 is below the
project's requires-python>=3.11 (code uses datetime.UTC).

Run the job inside nikolaik/python-nodejs:python3.12-nodejs20 which bundles
Python 3.12, Node 20 (for actions/checkout) and modern pip, removing the
GitHub download dependency entirely.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-24 06:02:28 +00:00
lixu 936828abae Merge pull request 'perf(search): 浏览路径部分复合索引 (0012)' (#27) from devin/1782279211-search-indexes into main
CI / Go (api) (push) Successful in 49s
CI / Python (ingestion) (push) Failing after 15s
CI / Migrations (postgres) (push) Successful in 31s
2026-06-24 13:35:50 +08:00
rosemariejebbjtxbfp ab7a964934 perf(search): 为浏览路径补部分复合索引 (0012)
CI / Go (api) (pull_request) Successful in 58s
CI / Python (ingestion) (pull_request) Failing after 21s
CI / Migrations (postgres) (pull_request) Successful in 32s
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-24 05:33:31 +00:00
lixu 649fcc711c Merge pull request 'docs: 可扩展性设计与路线图 (scalability roadmap)' (#26) from devin/1782278890-scalability-doc into main
CI / Go (api) (push) Successful in 52s
CI / Python (ingestion) (push) Failing after 16s
CI / Migrations (postgres) (push) Successful in 26s
2026-06-24 13:31:21 +08:00
rosemariejebbjtxbfp 3b62f61288 docs: 新增可扩展性设计与路线图 (scalability roadmap)
CI / Go (api) (pull_request) Successful in 59s
CI / Python (ingestion) (pull_request) Failing after 20s
CI / Migrations (postgres) (pull_request) Successful in 34s
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-24 05:29:08 +00:00
lixu ba36e3ff4c Merge pull request '新增联系我们页面' (#25) from devin/1782277231-contact-us into main
CI / Go (api) (push) Failing after 32s
CI / Python (ingestion) (push) Failing after 31s
CI / Migrations (postgres) (push) Successful in 35s
2026-06-24 13:02:07 +08:00
rosemariejebbjtxbfp d837dd38bf feat(public-frontend): 新增联系我们页面
CI / Go (api) (pull_request) Successful in 56s
CI / Python (ingestion) (pull_request) Failing after 1m31s
CI / Migrations (postgres) (pull_request) Failing after 32s
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-24 05:00:31 +00:00
lixu 53ce705572 Merge pull request '移除首页 API 调用说明入口' (#23) from devin/1782276245-home-remove-api-cta into main
CI / Go (api) (push) Failing after 16m4s
CI / Python (ingestion) (push) Failing after 17s
CI / Migrations (postgres) (push) Successful in 29s
2026-06-24 12:46:03 +08:00
rosemariejebbjtxbfp 836ee73d73 feat(public-frontend): 移除首页 API 调用说明入口
CI / Go (api) (pull_request) Failing after 1m31s
CI / Python (ingestion) (pull_request) Failing after 16s
CI / Migrations (postgres) (pull_request) Successful in 25s
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-24 04:44:05 +00:00
lixu dbbad274b6 Merge pull request 'feat(public-frontend): 公开前端 UI 视觉升级' (#22) from devin/1782270985-public-ui-redesign into main
CI / Go (api) (push) Successful in 55s
CI / Python (ingestion) (push) Failing after 14s
CI / Migrations (postgres) (push) Successful in 23s
2026-06-24 12:36:17 +08:00
rosemariejebbjtxbfp d58f46bc80 feat(public-frontend): 公开前端 UI 视觉升级
CI / Go (api) (pull_request) Failing after 32s
CI / Python (ingestion) (pull_request) Failing after 31s
CI / Migrations (postgres) (pull_request) Failing after 32s
引入统一品牌主题(色阶/字体/阴影/动效),重做首页 hero、卡片、导航与页脚,统一各页面的卡片/输入框/按钮样式。

- tailwind: 新增 brand 色阶、Inter 字体、card/glow 阴影与 fade-up 动效
- index.html: 引入 Inter 字体与 theme-color/description meta
- index.css: 双径向渐变背景 + @layer 组件类(.card/.input/.btn-primary 等)
- App/Home/ProductView/Contribute/ApiDocs/Account: 套用新设计系统

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-06-24 03:16:41 +00:00
40 changed files with 1593 additions and 439 deletions
+45 -13
View File
@@ -5,6 +5,10 @@ on:
branches: [main]
pull_request:
env:
GO_VERSION: "1.23.12"
GOPROXY: "https://goproxy.cn,direct"
jobs:
go:
name: Go (api)
@@ -25,11 +29,22 @@ jobs:
env:
OPENGOODS_DATABASE_URL: postgres://opengoods:opengoods@postgres:5432/opengoods?sslmode=disable
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
with:
go-version: "1.23"
cache-dependency-path: api/go.sum
- name: Checkout
working-directory: ${{ github.workspace }}
run: |
git config --global --add safe.directory '*'
git init -q .
git remote add origin "${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}.git"
git -c protocol.version=2 fetch -q --no-tags --depth 1 origin "${GITHUB_REF}"
git checkout -q --force FETCH_HEAD
- name: Setup Go
working-directory: ${{ github.workspace }}
run: |
curl -fsSL -o /tmp/go.tgz "https://mirrors.aliyun.com/golang/go${GO_VERSION}.linux-amd64.tar.gz"
rm -rf /usr/local/go
tar -C /usr/local -xzf /tmp/go.tgz
echo "/usr/local/go/bin" >> "$GITHUB_PATH"
echo "$HOME/go/bin" >> "$GITHUB_PATH"
- name: Apply migrations
working-directory: .
run: |
@@ -44,14 +59,20 @@ jobs:
python:
name: Python (ingestion)
runs-on: ubuntu-latest
container:
image: nikolaik/python-nodejs:python3.12-nodejs20
defaults:
run:
working-directory: ingestion
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.12"
- name: Checkout
working-directory: ${{ github.workspace }}
run: |
git config --global --add safe.directory '*'
git init -q .
git remote add origin "${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}.git"
git -c protocol.version=2 fetch -q --no-tags --depth 1 origin "${GITHUB_REF}"
git checkout -q --force FETCH_HEAD
- name: Install
run: pip install -e ".[dev]"
- name: Ruff lint
@@ -77,10 +98,21 @@ jobs:
env:
DBURL: postgres://opengoods:opengoods@postgres:5432/opengoods?sslmode=disable
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
with:
go-version: "1.23"
- name: Checkout
working-directory: ${{ github.workspace }}
run: |
git config --global --add safe.directory '*'
git init -q .
git remote add origin "${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}.git"
git -c protocol.version=2 fetch -q --no-tags --depth 1 origin "${GITHUB_REF}"
git checkout -q --force FETCH_HEAD
- name: Setup Go
run: |
curl -fsSL -o /tmp/go.tgz "https://mirrors.aliyun.com/golang/go${GO_VERSION}.linux-amd64.tar.gz"
rm -rf /usr/local/go
tar -C /usr/local -xzf /tmp/go.tgz
echo "/usr/local/go/bin" >> "$GITHUB_PATH"
echo "$HOME/go/bin" >> "$GITHUB_PATH"
- name: Install golang-migrate
run: go install -tags 'postgres' github.com/golang-migrate/migrate/v4/cmd/migrate@v4.18.1
- name: Migrate up
+7 -1
View File
@@ -9,6 +9,7 @@ import (
"github.com/jackc/pgx/v5/pgxpool"
"github.com/baicai2026-baicai/goods/api/internal/cache"
"github.com/baicai2026-baicai/goods/api/internal/config"
"github.com/baicai2026-baicai/goods/api/internal/handler"
"github.com/baicai2026-baicai/goods/api/internal/publicweb"
@@ -36,7 +37,12 @@ func main() {
if !limiter.Enabled() {
log.Print("warning: Redis not configured; public API rate limiting disabled")
}
h := handler.New(store.New(pool), publicweb.Dist()).
readCache := cache.New(cfg.RedisURL)
if !readCache.Enabled() {
log.Print("warning: Redis not configured; public API read cache disabled")
}
h := handler.New(store.New(pool).WithCache(readCache), publicweb.Dist()).
WithRateLimit(limiter, cfg.AnonRateLimitPerMin).
WithQuotas(cfg.AnonTotalQuota, cfg.RegisteredRateLimitPerMin, cfg.RegisteredQuotaTotal)
+2
View File
@@ -90,6 +90,8 @@ func (h *Handler) Router() http.Handler {
r.Put("/api/categories/{id}", h.UpdateCategory)
r.Delete("/api/categories/{id}", h.DeleteCategory)
r.Post("/api/import/bypos", h.ImportBypos)
r.Get("/api/submissions", h.ListSubmissions)
r.Get("/api/submissions/{id}", h.GetSubmission)
r.Post("/api/submissions/{id}/approve", h.ApproveSubmission)
+30
View File
@@ -0,0 +1,30 @@
package adminhandler
import (
"encoding/json"
"net/http"
"github.com/baicai2026-baicai/goods/api/internal/adminstore"
)
func (h *Handler) ImportBypos(w http.ResponseWriter, r *http.Request) {
var records []adminstore.ByposRecord
dec := json.NewDecoder(http.MaxBytesReader(w, r.Body, 50<<20))
for dec.More() {
var rec adminstore.ByposRecord
if err := dec.Decode(&rec); err != nil {
continue
}
records = append(records, rec)
}
if len(records) == 0 {
writeError(w, http.StatusBadRequest, "empty", "\u6ca1\u6709\u53ef\u5bfc\u5165\u7684\u8bb0\u5f55")
return
}
result, err := h.store.ImportByposRecords(r.Context(), records)
if err != nil {
writeError(w, http.StatusInternalServerError, "import_error", err.Error())
return
}
writeJSON(w, http.StatusOK, result)
}
+217
View File
@@ -0,0 +1,217 @@
package adminstore
import (
"context"
"encoding/json"
"strings"
"github.com/jackc/pgx/v5"
)
type ByposRecord struct {
Barcode string `json:"barcode"`
Name string `json:"name"`
Spec string `json:"spec"`
Unit string `json:"unit"`
Area string `json:"area"`
Manufacturer string `json:"manufacturer"`
License string `json:"license"`
InPrice string `json:"in_price"`
SellPrice string `json:"sell_price"`
Status string `json:"status"`
RetMsg string `json:"retmsg"`
FetchedAt string `json:"fetched_at"`
Source string `json:"source"`
}
type ImportByposResult struct {
Loaded int `json:"loaded"`
Skipped int `json:"skipped"`
Errored int `json:"errored"`
}
const byposSourceName = "bypos\u4e2d\u5fc3\u5e93"
const byposSourceURL = "https://zc.bypos.net"
func (s *Store) ensureByposSource(ctx context.Context, tx pgx.Tx) (string, error) {
var id string
err := tx.QueryRow(ctx, `
INSERT INTO source (name, homepage, license, trust_weight)
VALUES ($1, $2, 'proprietary', 0.6)
ON CONFLICT (name) DO UPDATE SET homepage = EXCLUDED.homepage
RETURNING id`, byposSourceName, byposSourceURL).Scan(&id)
return id, err
}
func (s *Store) ensureManufacturer(ctx context.Context, tx pgx.Tx, name string, country *string) (string, error) {
norm := strings.Join(strings.Fields(strings.ToLower(name)), " ")
var id string
err := tx.QueryRow(ctx, `
INSERT INTO manufacturer (name, normalized_name, country)
VALUES ($1, $2, $3)
ON CONFLICT (normalized_name) DO UPDATE SET name = manufacturer.name
RETURNING id`, name, norm, country).Scan(&id)
return id, err
}
func (s *Store) importByposRecord(ctx context.Context, tx pgx.Tx, rec ByposRecord, sourceID string) error {
if rec.Status != "hit" || strings.TrimSpace(rec.Name) == "" {
return nil
}
barcode := strings.TrimSpace(rec.Barcode)
name := strings.TrimSpace(rec.Name)
var country *string
if strings.HasPrefix(barcode, "69") {
c := "\u4e2d\u56fd"
country = &c
}
var manufacturerID *string
if mfr := strings.TrimSpace(rec.Manufacturer); mfr != "" {
mid, err := s.ensureManufacturer(ctx, tx, mfr, country)
if err != nil {
return err
}
manufacturerID = &mid
}
attrs := map[string]interface{}{}
if v := strings.TrimSpace(rec.Spec); v != "" {
attrs["spec"] = v
}
if v := strings.TrimSpace(rec.Unit); v != "" {
attrs["pack_unit"] = v
}
if v := strings.TrimSpace(rec.Area); v != "" {
attrs["origin_area"] = v
}
if v := strings.TrimSpace(rec.License); v != "" {
attrs["production_license"] = v
}
attrsJSON, _ := json.Marshal(attrs)
var productID string
if barcode != "" {
err := tx.QueryRow(ctx, `
INSERT INTO product (gtin, name, manufacturer_id, country_of_origin, attributes, status)
VALUES ($1, $2, $3, $4, $5, 'active')
ON CONFLICT (gtin) WHERE gtin IS NOT NULL DO UPDATE SET
name = EXCLUDED.name,
manufacturer_id = COALESCE(EXCLUDED.manufacturer_id, product.manufacturer_id),
country_of_origin = COALESCE(EXCLUDED.country_of_origin, product.country_of_origin),
attributes = product.attributes || EXCLUDED.attributes
RETURNING id`, barcode, name, manufacturerID, country, attrsJSON).Scan(&productID)
if err != nil {
return err
}
} else {
err := tx.QueryRow(ctx, `
INSERT INTO product (name, manufacturer_id, country_of_origin, attributes, status)
VALUES ($1, $2, $3, $4, 'active')
RETURNING id`, name, manufacturerID, country, attrsJSON).Scan(&productID)
if err != nil {
return err
}
}
_, _ = tx.Exec(ctx, "DELETE FROM product_msrp WHERE product_id = $1 AND source_id = $2", productID, sourceID)
if sp := strings.TrimSpace(rec.SellPrice); sp != "" && sp != "0" {
_, _ = tx.Exec(ctx, `
INSERT INTO product_msrp (product_id, amount, currency, region, source_id, source_url)
VALUES ($1, $2::numeric, 'CNY', 'CN', $3, $4)`, productID, sp, sourceID, byposSourceURL)
}
_, _ = tx.Exec(ctx, "DELETE FROM product_source WHERE product_id = $1 AND source_id = $2", productID, sourceID)
fields := []string{"name", "country_of_origin"}
if manufacturerID != nil {
fields = append(fields, "manufacturer")
}
if len(attrs) > 0 {
fields = append(fields, "attributes")
}
if barcode != "" {
fields = append(fields, "gtin")
}
rawJSON, _ := json.Marshal(rec)
fetchedAt := strings.TrimSpace(rec.FetchedAt)
if fetchedAt == "" {
fetchedAt = ""
}
if fetchedAt != "" {
_, err := tx.Exec(ctx, `
INSERT INTO product_source (product_id, source_id, url, fields, fetched_at, raw)
VALUES ($1, $2, $3, $4, $5::timestamptz, $6)`,
productID, sourceID, byposSourceURL, fields, fetchedAt, rawJSON)
if err != nil {
return err
}
} else {
_, err := tx.Exec(ctx, `
INSERT INTO product_source (product_id, source_id, url, fields, fetched_at, raw)
VALUES ($1, $2, $3, $4, now(), $5)`,
productID, sourceID, byposSourceURL, fields, rawJSON)
if err != nil {
return err
}
}
if barcode != "" {
gtinType := "EAN13"
switch len(barcode) {
case 8:
gtinType = "EAN8"
case 12:
gtinType = "UPC"
case 14:
gtinType = "GTIN14"
}
_, _ = tx.Exec(ctx, `
INSERT INTO product_barcode (product_id, gtin, gtin_type, pack_level, is_primary, source_id)
VALUES ($1, $2, $3, 'each', true, $4)
ON CONFLICT (gtin) DO NOTHING`, productID, barcode, gtinType, sourceID)
}
_, err := s.recomputeQualityTx(ctx, tx, productID)
return err
}
func (s *Store) ImportByposRecords(ctx context.Context, records []ByposRecord) (*ImportByposResult, error) {
result := &ImportByposResult{}
tx, err := s.pool.Begin(ctx)
if err != nil {
return nil, err
}
defer tx.Rollback(ctx)
sourceID, err := s.ensureByposSource(ctx, tx)
if err != nil {
return nil, err
}
for _, rec := range records {
if rec.Status != "hit" || strings.TrimSpace(rec.Name) == "" {
result.Skipped++
continue
}
sp, spErr := tx.Begin(ctx)
if spErr != nil {
result.Errored++
continue
}
if err := s.importByposRecord(ctx, sp, rec, sourceID); err != nil {
sp.Rollback(ctx)
result.Errored++
} else {
sp.Commit(ctx)
result.Loaded++
}
}
if err := tx.Commit(ctx); err != nil {
return nil, err
}
return result, nil
}
+34 -10
View File
@@ -31,6 +31,7 @@ type SubmissionInput struct {
CountryOfOrigin *string `json:"country_of_origin"`
IngredientsText *string `json:"ingredients_text"`
Nutriments map[string]any `json:"nutriments"`
Attributes map[string]any `json:"attributes"`
NutritionBasis *string `json:"nutrition_basis"`
ServingSize *string `json:"serving_size"`
NutriScore *string `json:"nutri_score"`
@@ -278,14 +279,19 @@ func (s *Store) ApproveSubmission(ctx context.Context, id, actor string) (*Produ
fields := submissionFields(in)
var attrJSON []byte
if len(in.Attributes) > 0 {
attrJSON, _ = json.Marshal(in.Attributes)
}
if productID == "" {
// Create a new product from the contribution.
err = tx.QueryRow(ctx, `
INSERT INTO product (gtin, name, brand_id, category_id, gpc_brick_code,
net_content_value, net_content_unit, net_content_canonical, country_of_origin, status)
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,'active') RETURNING id`,
net_content_value, net_content_unit, net_content_canonical, country_of_origin, attributes, status)
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,COALESCE($10::jsonb,'{}'::jsonb),'active') RETURNING id`,
in.GTIN, in.Name, brandID, in.CategoryID, gpc,
in.NetContentValue, in.NetContentUnit, canonical, in.CountryOfOrigin).Scan(&productID)
in.NetContentValue, in.NetContentUnit, canonical, in.CountryOfOrigin, attrJSON).Scan(&productID)
if err != nil {
return nil, err
}
@@ -302,21 +308,29 @@ UPDATE product SET
net_content_unit=COALESCE($7, net_content_unit),
net_content_canonical=COALESCE($8, net_content_canonical),
country_of_origin=COALESCE($9, country_of_origin),
gtin=COALESCE($10, gtin)
gtin=COALESCE($10, gtin),
attributes=product.attributes || COALESCE($11::jsonb,'{}'::jsonb)
WHERE id=$1`,
productID, in.Name, brandID, in.CategoryID, gpc,
in.NetContentValue, in.NetContentUnit, canonical, in.CountryOfOrigin, in.GTIN)
in.NetContentValue, in.NetContentUnit, canonical, in.CountryOfOrigin, in.GTIN, attrJSON)
if err != nil {
return nil, err
}
}
// food_detail: upsert, preserving existing values where not provided.
// food_detail: upsert only when the contribution carries food data, so a
// non-food submission (drug/3C/generic) doesn't create an empty row.
var nutriJSON []byte
if len(in.Nutriments) > 0 {
nutriJSON, _ = json.Marshal(in.Nutriments)
}
_, err = tx.Exec(ctx, `
foodPresent := len(in.Nutriments) > 0 ||
(in.IngredientsText != nil && *in.IngredientsText != "") ||
(in.NutritionBasis != nil && *in.NutritionBasis != "") ||
(in.ServingSize != nil && *in.ServingSize != "") ||
(in.NutriScore != nil && *in.NutriScore != "")
if foodPresent {
_, err = tx.Exec(ctx, `
INSERT INTO food_detail (product_id, ingredients_text, nutriments, nutrition_basis, serving_size, nutri_score)
VALUES ($1,$2,$3,$4,$5,$6)
ON CONFLICT (product_id) DO UPDATE SET
@@ -325,9 +339,10 @@ ON CONFLICT (product_id) DO UPDATE SET
nutrition_basis=COALESCE(EXCLUDED.nutrition_basis, food_detail.nutrition_basis),
serving_size=COALESCE(EXCLUDED.serving_size, food_detail.serving_size),
nutri_score=COALESCE(EXCLUDED.nutri_score, food_detail.nutri_score)`,
productID, in.IngredientsText, nutriJSON, in.NutritionBasis, in.ServingSize, in.NutriScore)
if err != nil {
return nil, err
productID, in.IngredientsText, nutriJSON, in.NutritionBasis, in.ServingSize, in.NutriScore)
if err != nil {
return nil, err
}
}
for _, im := range in.Images {
@@ -417,5 +432,14 @@ func submissionFields(in SubmissionInput) []string {
add("ingredients", in.IngredientsText != nil && *in.IngredientsText != "")
add("nutriments", len(in.Nutriments) > 0)
add("image", len(in.Images) > 0)
for k, v := range in.Attributes {
if v == nil {
continue
}
if s, ok := v.(string); ok && s == "" {
continue
}
fields = append(fields, k)
}
return fields
}
+126
View File
@@ -0,0 +1,126 @@
// Package cache is a Redis-backed, fail-open read cache for the public API.
//
// It caches hot product details and search results so repeated reads avoid
// PostgreSQL. Like the ratelimit package, every operation fails open: if Redis
// is unavailable or misconfigured the caller simply falls back to the database,
// so the cache can never take the API down or serve stale data after Redis loss.
//
// Invalidation is global and O(1): keys are namespaced by an epoch counter
// stored in Redis (og:cache:epoch). The Python ingestion bumps that counter
// after a write run, which logically invalidates every cached entry at once
// while old keys age out via their TTL. The epoch is read at most once per
// refresh interval per process, so it adds no per-request round trip.
package cache
import (
"context"
"encoding/json"
"errors"
"log"
"strconv"
"sync"
"time"
"github.com/redis/go-redis/v9"
)
// epochKey is the Redis key holding the global cache generation counter.
const epochKey = "og:cache:epoch"
// epochRefresh bounds how often a process re-reads the epoch from Redis.
const epochRefresh = 10 * time.Second
// opTimeout caps any single Redis operation so a slow backend never blocks a
// request beyond this; on timeout the cache fails open.
const opTimeout = 150 * time.Millisecond
// Cache wraps a Redis client. A nil-backed Cache (Redis unconfigured) disables
// caching: every Get misses and every Set is a no-op.
type Cache struct {
rdb *redis.Client
mu sync.RWMutex
epoch int64
epochSetAt time.Time
epochOK bool
}
// New builds a Cache from a redis:// URL. On a parse error it logs and returns a
// disabled (fail-open) cache so the server still boots.
func New(redisURL string) *Cache {
opt, err := redis.ParseURL(redisURL)
if err != nil {
log.Printf("cache: invalid redis url %q: %v (caching disabled)", redisURL, err)
return &Cache{}
}
return &Cache{rdb: redis.NewClient(opt)}
}
// Enabled reports whether a Redis backend is configured.
func (c *Cache) Enabled() bool { return c != nil && c.rdb != nil }
// epochNow returns the current cache generation, reading it from Redis at most
// once per epochRefresh. On any Redis error it keeps the last known value and
// throttles re-reads so a down backend cannot slow the hot path.
func (c *Cache) epochNow(ctx context.Context) int64 {
c.mu.RLock()
if c.epochOK && time.Since(c.epochSetAt) < epochRefresh {
e := c.epoch
c.mu.RUnlock()
return e
}
c.mu.RUnlock()
cctx, cancel := context.WithTimeout(ctx, opTimeout)
defer cancel()
n, err := c.rdb.Get(cctx, epochKey).Int64()
c.mu.Lock()
defer c.mu.Unlock()
switch {
case err == nil:
c.epoch = n
case errors.Is(err, redis.Nil):
c.epoch = 0
}
c.epochSetAt = time.Now()
c.epochOK = true
return c.epoch
}
// key namespaces a logical suffix under the current epoch.
func (c *Cache) key(ctx context.Context, suffix string) string {
return "og:v" + strconv.FormatInt(c.epochNow(ctx), 10) + ":" + suffix
}
// GetJSON unmarshals the cached value for suffix into dest and reports a hit.
// Any miss, decode error, or Redis error returns false (fail-open).
func (c *Cache) GetJSON(ctx context.Context, suffix string, dest any) bool {
if !c.Enabled() {
return false
}
k := c.key(ctx, suffix)
cctx, cancel := context.WithTimeout(ctx, opTimeout)
defer cancel()
b, err := c.rdb.Get(cctx, k).Bytes()
if err != nil {
return false
}
return json.Unmarshal(b, dest) == nil
}
// SetJSON stores val (JSON-encoded) for suffix with the given TTL. Best effort:
// marshal or Redis errors are ignored.
func (c *Cache) SetJSON(ctx context.Context, suffix string, val any, ttl time.Duration) {
if !c.Enabled() {
return
}
b, err := json.Marshal(val)
if err != nil {
return
}
k := c.key(ctx, suffix)
cctx, cancel := context.WithTimeout(ctx, opTimeout)
defer cancel()
_ = c.rdb.Set(cctx, k, b, ttl).Err()
}
+84
View File
@@ -0,0 +1,84 @@
package cache
import (
"context"
"fmt"
"os"
"testing"
"time"
)
// TestDisabledFailsOpen verifies a Cache without a Redis backend never panics,
// always misses, and silently drops writes.
func TestDisabledFailsOpen(t *testing.T) {
c := New("not-a-valid-url") // parse error => disabled
if c.Enabled() {
t.Fatal("expected cache to be disabled for invalid url")
}
c.SetJSON(context.Background(), "k", map[string]int{"a": 1}, time.Minute)
var dst map[string]int
if c.GetJSON(context.Background(), "k", &dst) {
t.Fatalf("disabled cache must always miss, got %+v", dst)
}
}
func testCache(t *testing.T) *Cache {
t.Helper()
url := os.Getenv("OPENGOODS_REDIS_URL")
if url == "" {
url = "redis://localhost:6379/0"
}
c := New(url)
if !c.Enabled() {
t.Skip("redis not configured")
}
ctx, cancel := context.WithTimeout(context.Background(), time.Second)
defer cancel()
if err := c.rdb.Ping(ctx).Err(); err != nil {
t.Skipf("redis not reachable: %v", err)
}
return c
}
// TestRoundTrip stores then reads a value back.
func TestRoundTrip(t *testing.T) {
c := testCache(t)
ctx := context.Background()
suffix := fmt.Sprintf("test:rt:%d", time.Now().UnixNano())
c.SetJSON(ctx, suffix, map[string]any{"name": "foo", "n": float64(3)}, time.Minute)
got := map[string]any{}
if !c.GetJSON(ctx, suffix, &got) {
t.Fatal("expected cache hit after set")
}
if got["name"] != "foo" || got["n"] != float64(3) {
t.Fatalf("unexpected payload: %+v", got)
}
}
// TestEpochInvalidation verifies that bumping the epoch counter logically drops
// every previously cached entry.
func TestEpochInvalidation(t *testing.T) {
c := testCache(t)
ctx := context.Background()
suffix := fmt.Sprintf("test:epoch:%d", time.Now().UnixNano())
c.SetJSON(ctx, suffix, map[string]int{"v": 1}, time.Minute)
var dst map[string]int
if !c.GetJSON(ctx, suffix, &dst) {
t.Fatal("expected hit before epoch bump")
}
// Simulate an ingestion write bumping the global epoch.
if err := c.rdb.Incr(ctx, epochKey).Err(); err != nil {
t.Fatalf("incr epoch: %v", err)
}
// Force the process to re-read the epoch rather than use its cached value.
c.mu.Lock()
c.epochOK = false
c.mu.Unlock()
if c.GetJSON(ctx, suffix, &dst) {
t.Fatal("entry should be invisible after epoch bump")
}
}
+16
View File
@@ -127,6 +127,7 @@ func (h *Handler) Router() http.Handler {
})
r.Get("/brands", h.ListBrands)
r.Get("/categories", h.ListCategories)
r.Get("/kind-fields", h.ListKindFields)
r.Get("/sources/{id}", h.SourceByID)
r.Get("/stats", h.Stats)
})
@@ -275,6 +276,21 @@ func (h *Handler) ListCategories(w http.ResponseWriter, r *http.Request) {
writeJSON(w, http.StatusOK, map[string]any{"items": items})
}
// ListKindFields returns the read-only spec field template for an archive kind,
// used by the contribution form to render kind-specific inputs.
func (h *Handler) ListKindFields(w http.ResponseWriter, r *http.Request) {
kind := strings.TrimSpace(r.URL.Query().Get("kind"))
if kind == "" {
writeError(w, r, http.StatusBadRequest, "bad_request", "missing kind")
return
}
items, err := h.store.ListKindFields(r.Context(), kind)
if h.handleErr(w, r, err) {
return
}
writeJSON(w, http.StatusOK, map[string]any{"items": items, "kind": kind})
}
// SourceByID returns a single data source.
func (h *Handler) SourceByID(w http.ResponseWriter, r *http.Request) {
src, err := h.store.SourceByID(r.Context(), chi.URLParam(r, "id"))
+119 -5
View File
@@ -4,21 +4,35 @@ package store
import (
"context"
"crypto/sha1"
"encoding/hex"
"encoding/json"
"errors"
"strconv"
"strings"
"time"
"github.com/jackc/pgx/v5"
"github.com/jackc/pgx/v5/pgxpool"
"github.com/baicai2026-baicai/goods/api/internal/cache"
)
// Cache TTLs for the public read cache. Product details are far less volatile
// than search result sets, so they live longer; both are also invalidated
// wholesale whenever ingestion bumps the cache epoch.
const (
productCacheTTL = 24 * time.Hour
searchCacheTTL = time.Hour
)
// ErrNotFound is returned when a requested row does not exist.
var ErrNotFound = errors.New("not found")
// Store wraps a PostgreSQL connection pool.
// Store wraps a PostgreSQL connection pool and an optional read cache.
type Store struct {
pool *pgxpool.Pool
pool *pgxpool.Pool
cache *cache.Cache
}
// New constructs a Store from an existing pgx pool.
@@ -26,6 +40,30 @@ func New(pool *pgxpool.Pool) *Store {
return &Store{pool: pool}
}
// WithCache attaches a Redis-backed read cache. A nil or disabled cache leaves
// the Store reading straight from PostgreSQL.
func (s *Store) WithCache(c *cache.Cache) *Store {
s.cache = c
return s
}
// cacheGet reads a cached JSON value into dest, reporting a hit. It is a no-op
// miss when no cache is attached.
func (s *Store) cacheGet(ctx context.Context, suffix string, dest any) bool {
if s.cache == nil {
return false
}
return s.cache.GetJSON(ctx, suffix, dest)
}
// cacheSet stores a JSON value when a cache is attached.
func (s *Store) cacheSet(ctx context.Context, suffix string, val any, ttl time.Duration) {
if s.cache == nil {
return
}
s.cache.SetJSON(ctx, suffix, val, ttl)
}
// Ping verifies database connectivity.
func (s *Store) Ping(ctx context.Context) error {
return s.pool.Ping(ctx)
@@ -222,6 +260,10 @@ func stringifyAttr(v any) string {
// ProductByGTIN looks up an active product by any of its barcodes.
func (s *Store) ProductByGTIN(ctx context.Context, gtin string) (*Product, error) {
const suffix = "prod:gtin:"
if cached := new(Product); s.cacheGet(ctx, suffix+gtin, cached) {
return cached, nil
}
row := s.pool.QueryRow(ctx, productSelect+`
WHERE p.status = 'active'
AND (p.gtin = $1 OR EXISTS (
@@ -238,11 +280,16 @@ func (s *Store) ProductByGTIN(ctx context.Context, gtin string) (*Product, error
if p.Specs, err = s.buildSpecs(ctx, p.ArchiveKind, attrs); err != nil {
return nil, err
}
s.cacheSet(ctx, suffix+gtin, p, productCacheTTL)
return p, nil
}
// ProductByID looks up a product by its UUID.
func (s *Store) ProductByID(ctx context.Context, id string) (*Product, error) {
const suffix = "prod:id:"
if cached := new(Product); s.cacheGet(ctx, suffix+id, cached) {
return cached, nil
}
row := s.pool.QueryRow(ctx, productSelect+" WHERE p.id = $1", id)
p, attrs, err := scanProduct(row)
if err != nil {
@@ -254,6 +301,7 @@ func (s *Store) ProductByID(ctx context.Context, id string) (*Product, error) {
if p.Specs, err = s.buildSpecs(ctx, p.ArchiveKind, attrs); err != nil {
return nil, err
}
s.cacheSet(ctx, suffix+id, p, productCacheTTL)
return p, nil
}
@@ -268,6 +316,11 @@ const fuzzyThreshold = "0.42"
// similarity blended with quality_score so the best, most-complete records
// surface first. Without a query, results are ordered by quality_score.
func (s *Store) SearchProducts(ctx context.Context, f SearchFilters, limit, offset int) ([]ProductSummary, int, error) {
suffix := searchCacheSuffix(f, limit, offset)
if entry := new(searchCacheEntry); s.cacheGet(ctx, suffix, entry) {
return entry.Items, entry.Total, nil
}
args := []any{}
where := "WHERE p.status = 'active'"
@@ -334,7 +387,28 @@ LEFT JOIN category c ON c.id = p.category_id `
}
out = append(out, ps)
}
return out, total, rows.Err()
if err := rows.Err(); err != nil {
return nil, 0, err
}
s.cacheSet(ctx, suffix, searchCacheEntry{Items: out, Total: total}, searchCacheTTL)
return out, total, nil
}
// searchCacheEntry is the cached payload for a SearchProducts call.
type searchCacheEntry struct {
Items []ProductSummary `json:"items"`
Total int `json:"total"`
}
// searchCacheSuffix derives a stable cache key from the full filter set and
// paging window so distinct queries never collide.
func searchCacheSuffix(f SearchFilters, limit, offset int) string {
raw := strings.Join([]string{
f.Query, f.Category, f.Brand, f.Country,
strconv.Itoa(limit), strconv.Itoa(offset),
}, "\x1f")
sum := sha1.Sum([]byte(raw))
return "search:" + hex.EncodeToString(sum[:])
}
// Nutriments returns just the nutrition payload for a product.
@@ -427,12 +501,13 @@ type Category struct {
Path string `json:"path"`
GPCBrickCode *string `json:"gpc_brick_code"`
Level int `json:"level"`
ArchiveKind string `json:"archive_kind"`
}
// ListCategories returns the full category tree ordered by path.
func (s *Store) ListCategories(ctx context.Context) ([]Category, error) {
rows, err := s.pool.Query(ctx,
"SELECT id, name_zh, name_en, path::text, gpc_brick_code, level FROM category ORDER BY path")
"SELECT id, name_zh, name_en, path::text, gpc_brick_code, level, COALESCE(archive_kind, 'generic') FROM category ORDER BY path")
if err != nil {
return nil, err
}
@@ -440,7 +515,7 @@ func (s *Store) ListCategories(ctx context.Context) ([]Category, error) {
out := []Category{}
for rows.Next() {
var c Category
if err := rows.Scan(&c.ID, &c.NameZH, &c.NameEN, &c.Path, &c.GPCBrickCode, &c.Level); err != nil {
if err := rows.Scan(&c.ID, &c.NameZH, &c.NameEN, &c.Path, &c.GPCBrickCode, &c.Level, &c.ArchiveKind); err != nil {
return nil, err
}
out = append(out, c)
@@ -448,6 +523,45 @@ func (s *Store) ListCategories(ctx context.Context) ([]Category, error) {
return out, rows.Err()
}
// KindField describes one editable spec field for an archive kind. It drives
// the dynamic contribution form (public, read-only view of the template).
type KindField struct {
Kind string `json:"kind"`
FieldKey string `json:"field_key"`
GroupLabel string `json:"group_label"`
LabelZH string `json:"label_zh"`
FieldType string `json:"field_type"`
Unit *string `json:"unit"`
Options []string `json:"options"`
Placeholder *string `json:"placeholder"`
SortOrder int `json:"sort_order"`
Qualified bool `json:"qualified"`
}
// ListKindFields returns the ordered field template for one archive kind so the
// public contribution form can render kind-specific inputs.
func (s *Store) ListKindFields(ctx context.Context, kind string) ([]KindField, error) {
rows, err := s.pool.Query(ctx, `
SELECT kind, field_key, group_label, label_zh, field_type, unit, options,
placeholder, sort_order, qualified
FROM kind_field WHERE kind = $1 ORDER BY sort_order, field_key`, kind)
if err != nil {
return nil, err
}
defer rows.Close()
out := []KindField{}
for rows.Next() {
var f KindField
if err := rows.Scan(&f.Kind, &f.FieldKey, &f.GroupLabel, &f.LabelZH,
&f.FieldType, &f.Unit, &f.Options, &f.Placeholder, &f.SortOrder,
&f.Qualified); err != nil {
return nil, err
}
out = append(out, f)
}
return out, rows.Err()
}
// APIKey is the minimal metadata the public API needs to authorize a caller.
type APIKey struct {
ID string
+112
View File
@@ -0,0 +1,112 @@
# 可扩展性设计与路线图 (Scalability Roadmap)
本文档回答一个长期问题:随着商品越来越多、品类越来越杂(食品 / 电子 3C / 药品 /
……),**检索与新增会不会压垮数据库?要不要按品类「分表」?**
> 结论先行:**现阶段不要按品类手动分表。** 现有「单表 + JSONB + archive_kind 框架」
> 的设计方向是对的。扩展应当靠 **分区(非分表) + 读副本 + 缓存 + 专用搜索引擎**,
> 按数据量分阶段推进,避免提前过度设计。
---
## 1. 现状盘点
### 1.1 数据模型
- 所有商品落在**一张 `product` 表**;非食品的领域字段存 `product.attributes`(JSONB)。
- 食品有独立明细表 `food_detail`(配料 / 营养 / 过敏原等结构化字段)。
- `0010_archive_kinds` 引入 **archive_kind 框架**:每个品类带一个 `archive_kind`
(`food` / `electronics` / `generic`),`kind_field` 表按 kind 定义字段模板,
驱动后台动态表单与合格度评分。
- **加新品类(如药品)不需要新建表**:只要新增一组 `kind_field` 行 + 一棵品类子树;
仅当某品类有大量需被独立筛选/排序的结构化字段时,才考虑像 `food_detail` 那样补一张
明细表。
### 1.2 已有索引(检索性能的基础)
| 对象 | 索引 | 用途 |
|------|------|------|
| `product.gtin` | 唯一索引 | 条码精确查 |
| `product.name` | trigram GIN (`pg_trgm`) | 名称模糊/相似匹配 |
| `product.search_tsv` | 全文 GIN (`tsvector`) | 全文检索 |
| `product.attributes` | JSONB GIN | 属性过滤 |
| `brand.name` | trigram GIN | 品牌模糊匹配 |
| `product.country_of_origin` | btree | 产地精确/前缀过滤 |
| category / brand / updated_at | btree | 关联与增量 |
### 1.3 检索方式
- 有关键词时:`name ILIKE` `word_similarity ≥ 阈值(~0.42)` 条码匹配,
排序按 `相似度 × (0.5 + quality_score)`
- 无关键词时:按 `quality_score` 排序。
- 翻页:`LIMIT / OFFSET`
### 1.4 写入特征
- 写库**只有** Python ingestion 一条路径(批量 ETL),不是高并发 OLTP。
- 插入瓶颈极低;`search_tsv` 由触发器逐行重算,正常增量下开销可忽略。
---
## 2. 为什么不建议按品类「分表」
1. **核心场景是全局检索**:用户通常不知道商品属于哪个品类,搜索要跨所有品类。
按品类拆成多表后,一次搜索得 `UNION ALL` 所有表,**更慢、代码更复杂、排序更难统一**。
2. **单表足够能打**:Postgres 单表配好索引,**几千万行**量级的检索完全可承载。
"表大"很少是真正瓶颈,"搜索方式"和"读并发"才是。
3. **分表会侵蚀框架优势**:archive_kind 框架的价值就是"加品类零建表";手动分表等于
把这套通用能力又拆碎。
> 区分两个概念:**分表(sharding,应用层拆多张表)** ≠ **分区(Postgres 原生
> declarative partitioning,对上层透明的一张逻辑表)**。后者在超大规模时才有意义,
> 见第 3 阶段。
---
## 3. 分阶段路线图(按数据量触发,不提前做)
### 阶段 0 — 现在 ~ 数百万条:维持现状 + 低成本优化
触发:当前规模。改动小、收益稳,建议尽早做:
- **深翻页改 keyset 分页**:`OFFSET` 越翻越慢(需扫描并丢弃前 N 行);改用
`WHERE (score, id) < (:last_score, :last_id)` 形式的游标分页。
- **部分索引**:绝大多数查询限定 `status='active'`,可建
`CREATE INDEX ... WHERE status='active'` 缩小索引、提速。
- **常用筛选复合索引**:如 `(category_id, quality_score DESC)`
`(archive_kind, quality_score DESC)` 配合域内列表。
- **Redis 缓存**(已在技术栈内):缓存热门搜索结果与商品详情,挡住重复读。
### 阶段 1 — 千万级以上:读扩展 + 调优
触发:单库读 QPS 升高、P99 变慢。
- **只读副本(read replica)**:本服务是**只读公益 API**,天然适合一主多从,
把检索/详情读流量分到副本,主库只承接 ingestion 写入。
- **索引与查询调优**:按慢查询日志补/删索引,`EXPLAIN ANALYZE` 校核计划。
- **(可选)Postgres 原生分区**:若多数检索"限定单一域"(只搜药品 / 只搜食品),
可按 `archive_kind`**LIST 分区**(对上层透明,仍是一张逻辑表)。主要利好
**维护**(分区级 vacuum / 归档)与**域内查询裁剪**;对真正的全局搜索帮助有限。
### 阶段 2 — 搜索相关性/规模成为痛点:引入专用搜索引擎
触发:`pg_trgm`/`tsvector` 在相关性排序、跨字段检索、规模上吃力。
- 把**检索**迁到专用倒排引擎:**OpenSearch / Meilisearch / Typesense**,
或 Postgres 内的 **ParadeDB(`pg_search`,BM25)**
- **Postgres 仍是唯一事实来源**;搜索引擎只做索引,由 ingestion 在写库后同步。
- 这才是"搜索量大"的正解,**比分表有效得多**。
---
## 4. 大批量导入的建议
- 海量初始化/回填用 `COPY` 而非逐行 `INSERT`
- 超大批量时可"先停建二级索引 → COPY → 重建索引",比边插边维护索引快得多。
- ETL 控制并发与批大小,避免与在线读争抢。
---
## 5. 药品档案怎么落地(回到最初的问题)
在上述设计下,加"药品"属于**阶段 0 的常规扩展**,不触动架构:
1. 新增 `drug``kind_field` 模板(批准文号 / 通用名 / 商品名 / 剂型 / 规格 /
生产企业 / OTC 分类 / 适应症 / 用法用量 / 不良反应 / 禁忌 / 注意事项 / 贮藏 /
有效期 等),`qualified` 标记关键字段参与合格度评分。
2. 加一棵药品品类子树,并把这些品类的 `archive_kind` 置为 `drug`
3. 仅当药品需要**被独立筛选/排序的强结构化字段**(如按批准文号精确查、按 OTC 分类
过滤)时,才考虑补一张 `drug_detail` 明细表;否则继续走 `attributes` JSONB。
---
## 6. 版本
- 本路线图随规模演进更新;任何落地改动需同步:迁移(SQL) + 本文档 +(涉及对外字段时)
`docs/data-contract.md` / `docs/openapi.yaml`
+52
View File
@@ -0,0 +1,52 @@
"""Read-cache invalidation signal for the public API.
The Go API caches hot product details and search results in Redis, namespacing
every key by a global generation counter (``og:cache:epoch``). Bumping that
counter logically invalidates the entire cache in one O(1) operation while the
old keys age out via their TTL.
Ingestion is the only writer to the database, so after a run that changed data
it calls :func:`bump_cache_epoch` to make those changes visible immediately
instead of waiting for per-key TTLs to expire.
Like the API's cache, this is strictly best effort and fails open: if Redis is
unconfigured or unreachable the ingestion run still succeeds, and stale entries
simply expire on their own.
"""
from __future__ import annotations
import logging
import os
logger = logging.getLogger(__name__)
EPOCH_KEY = "og:cache:epoch"
def default_redis_url() -> str | None:
"""Return the configured Redis URL, or ``None`` when caching is disabled."""
return os.environ.get("OPENGOODS_REDIS_URL") or None
def bump_cache_epoch(url: str | None = None) -> bool:
"""Increment the API cache generation counter.
Returns ``True`` if the counter was bumped, ``False`` if caching is disabled
or Redis was unreachable. Never raises: invalidation failures must not fail
an ingestion run.
"""
url = url or default_redis_url()
if not url:
return False
try:
import redis # imported lazily so the dependency is optional at runtime
client = redis.Redis.from_url(url, socket_timeout=2, socket_connect_timeout=2)
new_epoch = client.incr(EPOCH_KEY)
client.close()
logger.info("bumped API cache epoch to %s", new_epoch)
return True
except Exception as exc: # noqa: BLE001 - invalidation is best effort
logger.warning("cache epoch bump skipped: %s", exc)
return False
+3
View File
@@ -12,6 +12,7 @@ import sys
import psycopg
from opengoods.cache import bump_cache_epoch
from opengoods.etl.dedup import dedup_all
from opengoods.etl.load import default_dsn
@@ -23,6 +24,8 @@ def run(args: argparse.Namespace) -> int:
conn.rollback()
else:
conn.commit()
if not args.dry_run and summary["merged"]:
bump_cache_epoch()
mode = "dry-run" if args.dry_run else "applied"
print(f"{mode} groups={summary['groups']} merged={summary['merged']}")
return 0
+3
View File
@@ -25,6 +25,7 @@ from collections.abc import Iterator
import psycopg
from opengoods.adapters.bypos import transform_bypos
from opengoods.cache import bump_cache_epoch
from opengoods.etl.load import default_dsn, ensure_bypos_source, load_bypos_record_safe
@@ -59,6 +60,8 @@ def run(args: argparse.Namespace) -> int:
else:
errored += 1
conn.commit()
if loaded:
bump_cache_epoch()
print(f"loaded={loaded} skipped={skipped} errored={errored}")
return 0
+3
View File
@@ -28,6 +28,7 @@ from opengoods.adapters.openfoodfacts import (
is_cn_gs1,
read_dump,
)
from opengoods.cache import bump_cache_epoch
from opengoods.etl.load import default_dsn, ensure_source, load_record_safe
from opengoods.etl.transform import transform
@@ -67,6 +68,8 @@ def run(args: argparse.Namespace) -> int:
else:
errored += 1
conn.commit()
if loaded:
bump_cache_epoch()
print(f"loaded={loaded} skipped={skipped} errored={errored}")
return 0
+3
View File
@@ -17,6 +17,7 @@ import sys
import psycopg
from opengoods.adapters.openfoodfacts import SOURCE_NAME, OpenFoodFactsAdapter
from opengoods.cache import bump_cache_epoch
from opengoods.etl.load import default_dsn, ensure_source, load_record_safe
from opengoods.etl.state import get_watermark, set_watermark
from opengoods.etl.transform import transform
@@ -49,6 +50,8 @@ def run(args: argparse.Namespace) -> int:
stats={"loaded": loaded, "skipped": skipped, "errored": errored, "since": since},
)
conn.commit()
if loaded:
bump_cache_epoch()
print(
f"since={since} loaded={loaded} skipped={skipped} "
f"errored={errored} watermark={high_watermark}"
+1
View File
@@ -6,6 +6,7 @@ requires-python = ">=3.11"
dependencies = [
"httpx>=0.27",
"psycopg[binary]>=3.2",
"redis>=5.0,<6",
]
[project.optional-dependencies]
+44
View File
@@ -0,0 +1,44 @@
"""Tests for the best-effort API cache invalidation signal."""
from __future__ import annotations
import sys
import opengoods.cache as cache
def test_disabled_when_no_url(monkeypatch):
monkeypatch.delenv("OPENGOODS_REDIS_URL", raising=False)
assert cache.default_redis_url() is None
assert cache.bump_cache_epoch() is False
def test_bump_fails_open_when_unreachable(monkeypatch):
# An unroutable URL must not raise; the run still succeeds.
monkeypatch.setenv("OPENGOODS_REDIS_URL", "redis://127.0.0.1:1/0")
assert cache.bump_cache_epoch() is False
def test_bump_increments_epoch(monkeypatch):
calls = {}
class FakeClient:
def incr(self, key):
calls["key"] = key
return 7
def close(self):
calls["closed"] = True
class FakeRedis:
@staticmethod
def from_url(url, **kwargs):
calls["url"] = url
return FakeClient()
monkeypatch.setitem(sys.modules, "redis", type("M", (), {"Redis": FakeRedis}))
assert cache.bump_cache_epoch("redis://example:6379/0") is True
assert calls["key"] == "og:cache:epoch"
assert calls["url"] == "redis://example:6379/0"
assert calls["closed"] is True
+2
View File
@@ -0,0 +1,2 @@
DROP INDEX IF EXISTS idx_product_active_category;
DROP INDEX IF EXISTS idx_product_active_quality;
+15
View File
@@ -0,0 +1,15 @@
-- 检索优化(可扩展性路线图 阶段0):为最常见的"浏览"路径补部分/复合索引。
-- 搜索查询恒带 WHERE status = 'active';无关键词时按 quality_score DESC, name 排序。
-- 现有索引无法同时满足"过滤 active + 按 quality_score/name 排序",深翻页时需要对全部
-- active 行排序。下面的部分复合索引让规划器直接走索引顺序扫描,省掉排序、加速深翻页与
-- count(*)。索引只覆盖 active 行,体积更小。
-- 默认浏览(无关键词、无品类):ORDER BY quality_score DESC, name
CREATE INDEX IF NOT EXISTS idx_product_active_quality
ON product (quality_score DESC, name)
WHERE status = 'active';
-- 品类内浏览:先按 category_id 收敛,再按 quality_score 排序
CREATE INDEX IF NOT EXISTS idx_product_active_category
ON product (category_id, quality_score DESC)
WHERE status = 'active';
@@ -0,0 +1,8 @@
-- Reverse 0013: detach products from the seeded drug categories, drop the drug
-- category subtree, and remove the drug field template.
UPDATE product SET category_id = NULL
WHERE category_id IN (SELECT id FROM category WHERE path <@ 'drug');
DELETE FROM category WHERE path <@ 'drug';
DELETE FROM kind_field WHERE kind = 'drug';
+43
View File
@@ -0,0 +1,43 @@
-- Drug (药品) archive kind: a new domain on the generic archive_kind framework
-- (see 0010). No new table — drug spec values live in product.attributes JSONB,
-- driven by the kind_field template below, plus a 'drug' category subtree.
-- Field template for the drug domain. field_type is constrained to
-- text/number/textarea/select/list by kind_field_type_chk. The `qualified`
-- fields are the identifying / regulatory essentials that count toward the
-- kind-aware completeness score.
INSERT INTO kind_field (kind, field_key, group_label, label_zh, field_type, unit, options, sort_order, qualified) VALUES
('drug','approval_number', '基础信息','批准文号', 'text', NULL, '{}', 10, true),
('drug','generic_name', '基础信息','通用名', 'text', NULL, '{}', 20, true),
('drug','trade_name', '基础信息','商品名', 'text', NULL, '{}', 30, false),
('drug','dosage_form', '基础信息','剂型', 'select', NULL, '{片剂,胶囊剂,颗粒剂,散剂,丸剂,注射剂,口服液,糖浆剂,软膏剂,乳膏剂,凝胶剂,滴剂,喷雾剂,气雾剂,栓剂,贴剂,其他}', 40, true),
('drug','specification', '基础信息','规格', 'text', NULL, '{}', 50, true),
('drug','pack_spec', '基础信息','包装规格', 'text', NULL, '{}', 60, false),
('drug','rx_class', '分类与企业','处方分类', 'select', NULL, '{处方药,甲类OTC,乙类OTC}', 70, true),
('drug','manufacturer', '分类与企业','生产企业', 'text', NULL, '{}', 80, true),
('drug','mah', '分类与企业','上市许可持有人','text', NULL, '{}', 90, false),
('drug','indications', '说明书','适应症/功能主治', 'textarea', NULL, '{}', 100, true),
('drug','dosage_usage', '说明书','用法用量', 'textarea', NULL, '{}', 110, false),
('drug','adverse_reactions', '说明书','不良反应', 'textarea', NULL, '{}', 120, false),
('drug','contraindications', '说明书','禁忌', 'textarea', NULL, '{}', 130, false),
('drug','precautions', '说明书','注意事项', 'textarea', NULL, '{}', 140, false),
('drug','interactions', '说明书','药物相互作用', 'textarea', NULL, '{}', 150, false),
('drug','storage', '贮藏与有效期','贮藏', 'text', NULL, '{}', 160, false),
('drug','shelf_life', '贮藏与有效期','有效期', 'text', NULL, '{}', 170, false),
('drug','spec_source', '贮藏与有效期','说明书来源', 'text', NULL, '{}', 180, false);
-- Drug category tree. ltree labels are English slugs (no spaces/Chinese);
-- Chinese names live in category.name_zh.
INSERT INTO category (name_zh, name_en, parent_id, path, level, archive_kind)
VALUES ('药品', 'Drug', NULL, 'drug', 0, 'drug');
INSERT INTO category (name_zh, name_en, parent_id, path, level, archive_kind)
SELECT v.name_zh, v.name_en, c.id, v.path::ltree, 1, 'drug'
FROM (VALUES
('化学药品', 'Chemical drug', 'drug.chemical'),
('中成药', 'Chinese patent medicine', 'drug.tcm_patent'),
('中药饮片', 'Chinese herbal pieces', 'drug.tcm_herb'),
('生物制品', 'Biological product', 'drug.biological'),
('疫苗', 'Vaccine', 'drug.vaccine')
) AS v(name_zh, name_en, path)
JOIN category c ON c.path = 'drug';
+12 -1
View File
@@ -3,7 +3,18 @@
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>天工商品档案公共仓</title>
<meta name="theme-color" content="#059669" />
<meta
name="description"
content="天工商品档案公共仓——公益、开放、可溯源的商品事实查询与开放 API。"
/>
<title>天工 · 商品档案公共仓</title>
<link rel="preconnect" href="https://fonts.googleapis.com" />
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin />
<link
href="https://fonts.googleapis.com/css2?family=Inter:wght@400;500;600;700;800&display=swap"
rel="stylesheet"
/>
</head>
<body>
<div id="root"></div>
@@ -0,0 +1 @@
9ef4c7d0ea804c11bc637f4e82198e98af908894
+66 -52
View File
@@ -1,10 +1,11 @@
import { useEffect, useState } from "react";
import { Boxes, Search, PlusCircle, Code2, KeyRound } from "lucide-react";
import { Boxes, Search, PlusCircle, Code2, KeyRound, Headset } from "lucide-react";
import Home from "./components/Home";
import ProductView from "./components/ProductView";
import Contribute from "./components/Contribute";
import ApiDocs from "./components/ApiDocs";
import Account from "./components/Account";
import Contact from "./components/Contact";
import { api } from "./api";
type View =
@@ -12,7 +13,16 @@ type View =
| { name: "product"; id: string }
| { name: "contribute" }
| { name: "api" }
| { name: "account" };
| { name: "account" }
| { name: "contact" };
const NAV: { key: View["name"]; label: string; icon: typeof Search }[] = [
{ key: "home", label: "检索", icon: Search },
{ key: "contribute", label: "贡献档案", icon: PlusCircle },
{ key: "api", label: "API", icon: Code2 },
{ key: "account", label: "API 密钥", icon: KeyRound },
{ key: "contact", label: "联系我们", icon: Headset },
];
export default function App() {
const [view, setView] = useState<View>({ name: "home" });
@@ -27,58 +37,50 @@ export default function App() {
return (
<div className="min-h-full flex flex-col">
<header className="bg-white border-b">
<div className="max-w-5xl mx-auto px-4 h-14 flex items-center justify-between">
<header className="sticky top-0 z-30 border-b border-gray-200/70 bg-white/80 backdrop-blur-md">
<div className="max-w-5xl mx-auto px-4 h-16 flex items-center justify-between gap-4">
<button
className="flex items-center gap-2 font-semibold text-gray-800"
className="flex items-center gap-2.5 font-semibold text-gray-800 group"
onClick={() => setView({ name: "home" })}
>
<Boxes className="w-6 h-6 text-emerald-600" />
<span className="text-gray-400 font-normal text-sm"></span>
<span className="grid h-9 w-9 place-items-center rounded-xl bg-gradient-to-br from-brand-500 to-brand-700 text-white shadow-glow transition group-hover:scale-105">
<Boxes className="w-5 h-5" />
</span>
<span className="flex items-baseline gap-1.5">
<span className="text-lg tracking-tight"></span>
<span className="hidden sm:inline text-gray-400 font-normal text-xs">
</span>
</span>
</button>
<nav className="flex items-center gap-1 text-sm">
<button
className={`px-3 py-1.5 rounded-md flex items-center gap-1.5 ${
view.name === "home" ? "bg-emerald-50 text-emerald-700" : "text-gray-600 hover:bg-gray-100"
}`}
onClick={() => setView({ name: "home" })}
>
<Search className="w-4 h-4" />
</button>
<button
className={`px-3 py-1.5 rounded-md flex items-center gap-1.5 ${
view.name === "contribute" ? "bg-emerald-50 text-emerald-700" : "text-gray-600 hover:bg-gray-100"
}`}
onClick={() => setView({ name: "contribute" })}
>
<PlusCircle className="w-4 h-4" />
</button>
<button
className={`px-3 py-1.5 rounded-md flex items-center gap-1.5 ${
view.name === "api" ? "bg-emerald-50 text-emerald-700" : "text-gray-600 hover:bg-gray-100"
}`}
onClick={() => setView({ name: "api" })}
>
<Code2 className="w-4 h-4" /> API
</button>
<button
className={`px-3 py-1.5 rounded-md flex items-center gap-1.5 ${
view.name === "account" ? "bg-emerald-50 text-emerald-700" : "text-gray-600 hover:bg-gray-100"
}`}
onClick={() => setView({ name: "account" })}
>
<KeyRound className="w-4 h-4" /> API
</button>
<nav className="flex items-center gap-0.5 text-sm">
{NAV.map(({ key, label, icon: Icon }) => {
const active =
view.name === key || (key === "home" && view.name === "product");
return (
<button
key={key}
className={`px-3 py-1.5 rounded-lg flex items-center gap-1.5 font-medium transition ${
active
? "bg-brand-50 text-brand-700"
: "text-gray-600 hover:bg-gray-100"
}`}
onClick={() => setView({ name: key } as View)}
>
<Icon className="w-4 h-4" />
<span className="hidden sm:inline">{label}</span>
</button>
);
})}
</nav>
</div>
</header>
<main className="flex-1 max-w-5xl w-full mx-auto px-4 py-6">
<main className="flex-1 max-w-5xl w-full mx-auto px-4 py-8">
{view.name === "home" && (
<Home
onOpen={(id) => setView({ name: "product", id })}
onContribute={() => setView({ name: "contribute" })}
onApi={() => setView({ name: "api" })}
/>
)}
{view.name === "product" && (
@@ -89,24 +91,36 @@ export default function App() {
)}
{view.name === "api" && <ApiDocs onRegister={() => setView({ name: "account" })} />}
{view.name === "account" && <Account />}
{view.name === "contact" && <Contact />}
</main>
<footer className="border-t bg-white">
<div className="max-w-5xl mx-auto px-4 py-4 text-xs text-gray-400 leading-relaxed text-center">
<footer className="border-t border-gray-200/70 bg-white/60">
<div className="max-w-5xl mx-auto px-4 py-6 text-xs text-gray-400 leading-relaxed text-center">
{qualified != null && (
<div className="mb-2 text-gray-500">
<span className="mx-1 font-semibold text-emerald-600">
<div className="mb-3 inline-flex items-center gap-1.5 rounded-full border border-brand-100 bg-brand-50 px-3 py-1 text-gray-500">
<span className="font-semibold text-brand-600">
{qualified.toLocaleString()}
</span>
</div>
)}
<button onClick={() => setView({ name: "api" })} className="ml-1 text-emerald-600 hover:underline">
API
</button>
<div className="mt-2">
<p className="max-w-2xl mx-auto">
<button
onClick={() => setView({ name: "api" })}
className="ml-1 text-brand-600 font-medium hover:underline"
>
API
</button>
<button
onClick={() => setView({ name: "contact" })}
className="ml-1 text-brand-600 font-medium hover:underline"
>
</button>
</p>
<div className="mt-3">
<a
href="https://beian.miit.gov.cn/"
target="_blank"
+5 -1
View File
@@ -1,4 +1,4 @@
import type { Category, Product, ProductSummary, SubmissionInput } from "./types";
import type { Category, KindField, Product, ProductSummary, SubmissionInput } from "./types";
async function req<T>(path: string, init?: RequestInit): Promise<T> {
const res = await fetch(path, {
@@ -67,6 +67,10 @@ export const api = {
},
product: (id: string) => req<Product>(`/api/v1/products/${id}`),
categories: () => req<{ items: Category[] }>(`/api/v1/categories`),
kindFields: (kind: string) =>
req<{ items: KindField[]; kind: string }>(
`/api/v1/kind-fields?kind=${encodeURIComponent(kind)}`,
),
submit: (input: SubmissionInput) =>
req<{ id: string; status: string }>(`/api/public/submissions`, {
method: "POST",
+11 -11
View File
@@ -5,7 +5,7 @@ import { api, type AccountInfo, type KeyResponse } from "../api";
function KeyReveal({ data }: { data: KeyResponse }) {
const [copied, setCopied] = useState(false);
return (
<div className="mt-4 rounded-lg border border-emerald-200 bg-emerald-50 p-4">
<div className="mt-4 rounded-lg border border-brand-100 bg-brand-50 p-4">
<div className="flex items-start gap-2 text-amber-700 text-sm">
<AlertTriangle className="w-4 h-4 mt-0.5 shrink-0" />
<span></span>
@@ -27,7 +27,7 @@ function KeyReveal({ data }: { data: KeyResponse }) {
className="text-gray-400 hover:text-gray-600 shrink-0"
title="复制"
>
{copied ? <Check className="w-5 h-5 text-emerald-600" /> : <Copy className="w-5 h-5" />}
{copied ? <Check className="w-5 h-5 text-brand-600" /> : <Copy className="w-5 h-5" />}
</button>
</div>
<div className="mt-3 text-sm text-gray-600">
@@ -92,9 +92,9 @@ export default function Account() {
return (
<div className="max-w-xl mx-auto space-y-5">
<div className="bg-white border rounded-lg p-5">
<div className="card p-5">
<h1 className="flex items-center gap-2 text-2xl font-bold text-gray-800">
<KeyRound className="w-6 h-6 text-emerald-600" /> API
<KeyRound className="w-6 h-6 text-brand-600" /> API
</h1>
<p className="mt-2 text-sm text-gray-600 leading-relaxed">
IP <strong>1000</strong> API
@@ -104,7 +104,7 @@ export default function Account() {
<div className="mt-4 inline-flex rounded-md border bg-gray-50 p-0.5 text-sm">
<button
className={`px-4 py-1.5 rounded ${
mode === "register" ? "bg-white shadow-sm text-emerald-700" : "text-gray-500"
mode === "register" ? "bg-white shadow-sm text-brand-700" : "text-gray-500"
}`}
onClick={() => {
setMode("register");
@@ -115,7 +115,7 @@ export default function Account() {
</button>
<button
className={`px-4 py-1.5 rounded ${
mode === "manage" ? "bg-white shadow-sm text-emerald-700" : "text-gray-500"
mode === "manage" ? "bg-white shadow-sm text-brand-700" : "text-gray-500"
}`}
onClick={() => {
setMode("manage");
@@ -135,7 +135,7 @@ export default function Account() {
value={email}
onChange={(e) => setEmail(e.target.value)}
placeholder="you@example.com"
className="w-full border rounded-md px-3 py-2 text-sm focus:outline-none focus:ring-2 focus:ring-emerald-500"
className="w-full border rounded-md px-3 py-2 text-sm focus:outline-none focus:ring-2 focus:ring-brand-500"
/>
</div>
<div>
@@ -146,14 +146,14 @@ export default function Account() {
value={password}
onChange={(e) => setPassword(e.target.value)}
placeholder="••••••••"
className="w-full border rounded-md px-3 py-2 text-sm focus:outline-none focus:ring-2 focus:ring-emerald-500"
className="w-full border rounded-md px-3 py-2 text-sm focus:outline-none focus:ring-2 focus:ring-brand-500"
/>
</div>
{error && <div className="text-sm text-red-600">{error}</div>}
<button
type="submit"
disabled={loading}
className="w-full bg-emerald-600 text-white rounded-md py-2 text-sm font-medium hover:bg-emerald-700 disabled:opacity-50"
className="w-full bg-brand-600 text-white rounded-md py-2 text-sm font-medium hover:bg-brand-700 disabled:opacity-50"
>
{loading ? "处理中…" : mode === "register" ? "注册并领取密钥" : "查询账号"}
</button>
@@ -175,12 +175,12 @@ export default function Account() {
<div>
<strong>{info.quota_used.toLocaleString()}</strong> /{" "}
{info.quota_total.toLocaleString()} {" "}
<strong className="text-emerald-600">{info.quota_remaining.toLocaleString()}</strong>
<strong className="text-brand-600">{info.quota_remaining.toLocaleString()}</strong>
</div>
<button
onClick={regenerate}
disabled={loading}
className="mt-2 text-emerald-600 hover:underline disabled:opacity-50"
className="mt-2 text-brand-600 hover:underline disabled:opacity-50"
>
</button>
+7 -7
View File
@@ -20,7 +20,7 @@ function CopyBtn({ text }: { text: string }) {
className="text-gray-400 hover:text-gray-600"
title="复制"
>
{done ? <Check className="w-4 h-4 text-emerald-600" /> : <Copy className="w-4 h-4" />}
{done ? <Check className="w-4 h-4 text-brand-600" /> : <Copy className="w-4 h-4" />}
</button>
);
}
@@ -39,7 +39,7 @@ function Code({ children }: { children: string }) {
}
function Method({ m }: { m: string }) {
const color = m === "GET" ? "bg-sky-100 text-sky-700" : "bg-emerald-100 text-emerald-700";
const color = m === "GET" ? "bg-sky-100 text-sky-700" : "bg-brand-100 text-brand-700";
return <span className={`text-xs font-mono font-semibold rounded px-1.5 py-0.5 ${color}`}>{m}</span>;
}
@@ -63,7 +63,7 @@ function Endpoint({
response: string;
}) {
return (
<div className="bg-white border rounded-lg p-5">
<div className="card p-5">
<div className="flex items-center gap-2 flex-wrap">
<Method m={method} />
<code className="text-sm text-gray-800 font-mono break-all">{path}</code>
@@ -105,7 +105,7 @@ function Endpoint({
export default function ApiDocs({ onRegister }: { onRegister?: () => void }) {
return (
<div className="space-y-5">
<div className="bg-white border rounded-lg p-5">
<div className="card p-5">
<h1 className="text-2xl font-bold text-gray-800">API </h1>
<p className="mt-2 text-gray-600 text-sm leading-relaxed">
<strong></strong> REST API
@@ -119,7 +119,7 @@ export default function ApiDocs({ onRegister }: { onRegister?: () => void }) {
<ul className="mt-2 list-disc pl-5 text-gray-600 space-y-1">
<li>
API Key / Token GET IP <strong>1000</strong>
<button onClick={onRegister} className="text-emerald-600 hover:underline"></button>
<button onClick={onRegister} className="text-brand-600 hover:underline"></button>
</li>
<li>
@@ -135,13 +135,13 @@ export default function ApiDocs({ onRegister }: { onRegister?: () => void }) {
</div>
</div>
<div className="bg-white border rounded-lg p-5">
<div className="card p-5">
<h2 className="text-lg font-semibold text-gray-800"></h2>
<p className="mt-2 text-gray-600 text-sm leading-relaxed">
API <strong></strong> IP
<strong> 1000 </strong>
<code className="font-mono">403</code> <code className="font-mono">quota_exhausted</code>
<button onClick={onRegister} className="text-emerald-600 hover:underline"></button>
<button onClick={onRegister} className="text-brand-600 hover:underline"></button>
API Key
</p>
<div className="mt-3">
+113
View File
@@ -0,0 +1,113 @@
import { useState } from "react";
import { Phone, Mail, Globe, MessageSquare, Copy, Check, Headset } from "lucide-react";
type Channel = {
key: string;
icon: typeof Phone;
label: string;
value: string;
href?: string;
copy: string;
};
const CHANNELS: Channel[] = [
{
key: "phone",
icon: Phone,
label: "电话",
value: "188 6595 7520",
href: "tel:18865957520",
copy: "18865957520",
},
{
key: "email",
icon: Mail,
label: "邮箱",
value: "1115084741@qq.com",
href: "mailto:1115084741@qq.com",
copy: "1115084741@qq.com",
},
{
key: "site",
icon: Globe,
label: "官网",
value: "www.wenyaoyu.com",
href: "https://www.wenyaoyu.com",
copy: "https://www.wenyaoyu.com",
},
{
key: "wechat",
icon: MessageSquare,
label: "微信",
value: "s-b-m-y",
copy: "s-b-m-y",
},
];
export default function Contact() {
const [copied, setCopied] = useState<string | null>(null);
async function copy(channel: Channel) {
try {
await navigator.clipboard.writeText(channel.copy);
setCopied(channel.key);
setTimeout(() => setCopied((k) => (k === channel.key ? null : k)), 1500);
} catch {
/* clipboard unavailable */
}
}
return (
<div className="max-w-3xl mx-auto animate-fade-up">
<div className="text-center">
<span className="mx-auto grid h-14 w-14 place-items-center rounded-2xl bg-gradient-to-br from-brand-500 to-brand-700 text-white shadow-glow">
<Headset className="w-7 h-7" />
</span>
<h1 className="mt-4 text-2xl font-semibold tracking-tight text-gray-900"></h1>
<p className="mx-auto mt-2 max-w-xl text-sm text-gray-500">
</p>
</div>
<div className="mt-8 grid grid-cols-1 sm:grid-cols-2 gap-4">
{CHANNELS.map((c) => {
const Icon = c.icon;
return (
<div key={c.key} className="card p-5 flex items-center gap-4">
<span className="grid h-11 w-11 shrink-0 place-items-center rounded-xl bg-brand-50 text-brand-600">
<Icon className="w-5 h-5" />
</span>
<div className="min-w-0 flex-1">
<div className="text-xs font-medium text-gray-400">{c.label}</div>
{c.href ? (
<a
href={c.href}
target={c.key === "site" ? "_blank" : undefined}
rel={c.key === "site" ? "noreferrer" : undefined}
className="block truncate font-medium text-gray-800 hover:text-brand-600 hover:underline"
>
{c.value}
</a>
) : (
<div className="truncate font-medium text-gray-800">{c.value}</div>
)}
</div>
<button
type="button"
onClick={() => copy(c)}
title="复制"
className="shrink-0 grid h-9 w-9 place-items-center rounded-lg border border-gray-200 text-gray-400 transition hover:bg-gray-50 hover:text-brand-600"
>
{copied === c.key ? (
<Check className="w-4 h-4 text-brand-600" />
) : (
<Copy className="w-4 h-4" />
)}
</button>
</div>
);
})}
</div>
</div>
);
}
+181 -58
View File
@@ -1,7 +1,7 @@
import { useEffect, useState } from "react";
import { useEffect, useMemo, useState } from "react";
import { CheckCircle2, PlusCircle, Trash2 } from "lucide-react";
import { api } from "../api";
import type { Category, SubmissionImage, SubmissionInput } from "../types";
import type { Category, KindField, SubmissionImage, SubmissionInput } from "../types";
const NUTRI_FIELDS: { key: string; label: string }[] = [
{ key: "energy_kcal", label: "能量 (kcal)" },
@@ -35,6 +35,8 @@ export default function Contribute({ onDone }: { onDone: () => void }) {
const [ingredients, setIngredients] = useState("");
const [basis, setBasis] = useState("");
const [nutri, setNutri] = useState<Record<string, string>>({});
const [kindFields, setKindFields] = useState<KindField[]>([]);
const [attrs, setAttrs] = useState<Record<string, string>>({});
const [images, setImages] = useState<SubmissionImage[]>([]);
const [imageURL, setImageURL] = useState("");
const [submitter, setSubmitter] = useState("");
@@ -45,6 +47,33 @@ export default function Contribute({ onDone }: { onDone: () => void }) {
api.categories().then((r) => setCategories(r.items)).catch(() => undefined);
}, []);
// Derive the archive kind from the selected category. No category => generic.
const kind = useMemo(() => {
const c = categories.find((x) => x.id === categoryID);
return c?.archive_kind || "generic";
}, [categories, categoryID]);
// Fetch the kind-specific field template for non-food kinds.
useEffect(() => {
setAttrs({});
if (kind === "food" || kind === "generic") {
setKindFields([]);
return;
}
let alive = true;
api
.kindFields(kind)
.then((r) => {
if (alive) setKindFields(r.items);
})
.catch(() => {
if (alive) setKindFields([]);
});
return () => {
alive = false;
};
}, [kind]);
async function submit(e: React.FormEvent) {
e.preventDefault();
if (name.trim() === "") {
@@ -55,9 +84,31 @@ export default function Contribute({ onDone }: { onDone: () => void }) {
setError("");
const nutriments: Record<string, number> = {};
for (const [k, v] of Object.entries(nutri)) {
const n = parseFloat(v);
if (!Number.isNaN(n)) nutriments[k] = n;
if (kind === "food") {
for (const [k, v] of Object.entries(nutri)) {
const n = parseFloat(v);
if (!Number.isNaN(n)) nutriments[k] = n;
}
}
const attributes: Record<string, unknown> = {};
if (kind !== "food" && kind !== "generic") {
for (const f of kindFields) {
const raw = (attrs[f.field_key] || "").trim();
if (raw === "") continue;
if (f.field_type === "number") {
const n = parseFloat(raw);
if (!Number.isNaN(n)) attributes[f.field_key] = n;
} else if (f.field_type === "list") {
const parts = raw
.split(/[\n,,、]/)
.map((s) => s.trim())
.filter((s) => s !== "");
if (parts.length) attributes[f.field_key] = parts;
} else {
attributes[f.field_key] = raw;
}
}
}
const input: SubmissionInput = {
@@ -68,9 +119,10 @@ export default function Contribute({ onDone }: { onDone: () => void }) {
net_content_value: field(netValue) ? parseFloat(netValue) : null,
net_content_unit: field(netUnit),
country_of_origin: field(country),
ingredients_text: field(ingredients),
ingredients_text: kind === "food" ? field(ingredients) : null,
nutriments: Object.keys(nutriments).length ? nutriments : null,
nutrition_basis: basis || null,
attributes: Object.keys(attributes).length ? attributes : null,
nutrition_basis: kind === "food" && basis ? basis : null,
images: images.length ? images : undefined,
submitter_name: field(submitter),
submitter_contact: field(contact),
@@ -89,38 +141,91 @@ export default function Contribute({ onDone }: { onDone: () => void }) {
if (done) {
return (
<div className="max-w-xl mx-auto text-center py-16">
<CheckCircle2 className="w-14 h-14 text-emerald-500 mx-auto" />
<div className="max-w-xl mx-auto text-center py-16 animate-fade-up">
<span className="mx-auto grid h-16 w-16 place-items-center rounded-2xl bg-brand-50">
<CheckCircle2 className="w-9 h-9 text-brand-600" />
</span>
<h1 className="mt-4 text-xl font-semibold text-gray-800"></h1>
<p className="mt-2 text-gray-500">
</p>
<button
onClick={onDone}
className="mt-6 px-5 py-2 rounded-lg bg-emerald-600 text-white hover:bg-emerald-700"
>
<button onClick={onDone} className="btn-primary mt-6">
</button>
</div>
);
}
const input =
"w-full border rounded-md px-3 py-2 text-sm focus:outline-none focus:ring-2 focus:ring-emerald-400";
const label = "block text-xs text-gray-500 mb-1";
const input = "input";
const label = "block text-xs font-medium text-gray-500 mb-1";
// Group the kind fields by their group label, preserving template order.
const groups: { label: string; fields: KindField[] }[] = [];
for (const f of kindFields) {
let g = groups.find((x) => x.label === f.group_label);
if (!g) {
g = { label: f.group_label, fields: [] };
groups.push(g);
}
g.fields.push(f);
}
const KIND_LABEL: Record<string, string> = {
drug: "药品",
electronics: "数码 3C",
food: "食品",
generic: "通用",
};
function renderField(f: KindField) {
const val = attrs[f.field_key] || "";
const set = (v: string) => setAttrs((p) => ({ ...p, [f.field_key]: v }));
const lbl = f.unit ? `${f.label_zh} (${f.unit})` : f.label_zh;
return (
<div key={f.field_key} className={f.field_type === "textarea" || f.field_type === "list" ? "sm:col-span-2" : ""}>
<label className={label}>
{lbl}
{f.qualified && <span className="text-brand-500"> *</span>}
</label>
{f.field_type === "select" ? (
<select className={input} value={val} onChange={(e) => set(e.target.value)}>
<option value=""></option>
{(f.options || []).map((o) => (
<option key={o} value={o}>
{o}
</option>
))}
</select>
) : f.field_type === "textarea" ? (
<textarea className={input} rows={3} value={val} placeholder={f.placeholder || ""} onChange={(e) => set(e.target.value)} />
) : f.field_type === "list" ? (
<textarea className={input} rows={2} value={val} placeholder={f.placeholder || "每行一项,或用、逗号分隔"} onChange={(e) => set(e.target.value)} />
) : (
<input
type={f.field_type === "number" ? "number" : "text"}
step={f.field_type === "number" ? "any" : undefined}
className={input}
value={val}
placeholder={f.placeholder || ""}
onChange={(e) => set(e.target.value)}
/>
)}
</div>
);
}
return (
<form onSubmit={submit} className="max-w-3xl mx-auto">
<h1 className="text-xl font-semibold text-gray-800"></h1>
<form onSubmit={submit} className="max-w-3xl mx-auto animate-fade-up">
<h1 className="text-2xl font-semibold tracking-tight text-gray-900"></h1>
<p className="mt-1 text-sm text-gray-500">
<b></b> *
<b></b>
</p>
{error && (
<div className="mt-4 bg-red-50 text-red-700 text-sm rounded-md px-4 py-2">{error}</div>
<div className="mt-4 rounded-xl bg-red-50 px-4 py-2.5 text-sm text-red-700">{error}</div>
)}
<div className="bg-white border rounded-lg p-5 mt-4">
<div className="card p-5 mt-4">
<h2 className="font-medium text-gray-700 mb-3"></h2>
<div className="grid grid-cols-1 sm:grid-cols-2 gap-4">
<div className="sm:col-span-2">
@@ -145,6 +250,11 @@ export default function Contribute({ onDone }: { onDone: () => void }) {
</option>
))}
</select>
{categoryID && (
<p className="mt-1 text-xs text-gray-400">
{KIND_LABEL[kind] || kind}
</p>
)}
</div>
<div>
<label className={label}></label>
@@ -167,41 +277,58 @@ export default function Contribute({ onDone }: { onDone: () => void }) {
</div>
</div>
<div className="bg-white border rounded-lg p-5 mt-4">
<h2 className="font-medium text-gray-700 mb-3"></h2>
<label className={label}></label>
<textarea
className={input}
rows={3}
value={ingredients}
onChange={(e) => setIngredients(e.target.value)}
/>
<div className="mt-3">
<label className={label}></label>
<select className={input} value={basis} onChange={(e) => setBasis(e.target.value)}>
<option value=""></option>
<option value="per_100g"> 100g</option>
<option value="per_100ml"> 100ml</option>
<option value="per_serving"></option>
</select>
{kind === "food" && (
<div className="card p-5 mt-4">
<h2 className="font-medium text-gray-700 mb-3"></h2>
<label className={label}></label>
<textarea
className={input}
rows={3}
value={ingredients}
onChange={(e) => setIngredients(e.target.value)}
/>
<div className="mt-3">
<label className={label}></label>
<select className={input} value={basis} onChange={(e) => setBasis(e.target.value)}>
<option value=""></option>
<option value="per_100g"> 100g</option>
<option value="per_100ml"> 100ml</option>
<option value="per_serving"></option>
</select>
</div>
<div className="grid grid-cols-2 sm:grid-cols-4 gap-3 mt-3">
{NUTRI_FIELDS.map((f) => (
<div key={f.key}>
<label className={label}>{f.label}</label>
<input
type="number"
step="any"
className={input}
value={nutri[f.key] || ""}
onChange={(e) => setNutri({ ...nutri, [f.key]: e.target.value })}
/>
</div>
))}
</div>
</div>
<div className="grid grid-cols-2 sm:grid-cols-4 gap-3 mt-3">
{NUTRI_FIELDS.map((f) => (
<div key={f.key}>
<label className={label}>{f.label}</label>
<input
type="number"
step="any"
className={input}
value={nutri[f.key] || ""}
onChange={(e) => setNutri({ ...nutri, [f.key]: e.target.value })}
/>
)}
{kind !== "food" && kind !== "generic" && kindFields.length > 0 && (
<div className="card p-5 mt-4">
<h2 className="font-medium text-gray-700 mb-1">{KIND_LABEL[kind] || kind}</h2>
<p className="text-xs text-gray-400 mb-3"> * </p>
{groups.map((g) => (
<div key={g.label} className="mt-3 first:mt-0">
{g.label && <h3 className="text-xs font-semibold text-gray-400 mb-2">{g.label}</h3>}
<div className="grid grid-cols-1 sm:grid-cols-2 gap-4">
{g.fields.map(renderField)}
</div>
</div>
))}
</div>
</div>
)}
<div className="bg-white border rounded-lg p-5 mt-4">
<div className="card p-5 mt-4">
<h2 className="font-medium text-gray-700 mb-3"> URL</h2>
{images.length > 0 && (
<ul className="mb-3 space-y-1">
@@ -234,14 +361,14 @@ export default function Contribute({ onDone }: { onDone: () => void }) {
setImageURL("");
}
}}
className="shrink-0 px-3 rounded-md border text-sm flex items-center gap-1 hover:bg-gray-50"
className="shrink-0 px-3 rounded-xl border border-gray-200 text-sm flex items-center gap-1 hover:bg-gray-50"
>
<PlusCircle className="w-4 h-4" />
</button>
</div>
</div>
<div className="bg-white border rounded-lg p-5 mt-4">
<div className="card p-5 mt-4">
<h2 className="font-medium text-gray-700 mb-3"></h2>
<div className="grid grid-cols-1 sm:grid-cols-2 gap-4">
<div>
@@ -260,11 +387,7 @@ export default function Contribute({ onDone }: { onDone: () => void }) {
</div>
<div className="mt-5 flex items-center gap-3">
<button
type="submit"
disabled={submitting}
className="px-6 py-2.5 rounded-lg bg-emerald-600 text-white font-medium hover:bg-emerald-700 disabled:opacity-60"
>
<button type="submit" disabled={submitting} className="btn-primary px-6">
{submitting ? "提交中…" : "提交审核"}
</button>
<button type="button" onClick={onDone} className="text-sm text-gray-500 hover:underline">
+103 -54
View File
@@ -1,16 +1,34 @@
import { useState } from "react";
import { Search, PlusCircle, Code2 } from "lucide-react";
import { Search, PlusCircle, Code2, ScanBarcode, ShieldCheck, ArrowRight } from "lucide-react";
import { api } from "../api";
import type { ProductSummary } from "../types";
const EXAMPLES = ["可乐", "Nutella", "牛奶", "5449000000996"];
const FEATURES = [
{
icon: Search,
title: "客观可查",
desc: "按名称或条码检索商品的成分、营养、规格等官方事实。",
},
{
icon: ShieldCheck,
title: "可溯源",
desc: "每条资料标注数据来源与质量分,公开透明、人人可核。",
},
{
icon: Code2,
title: "开放 API",
desc: "免鉴权只读 REST 接口,开发者可直接接入商品档案。",
},
];
export default function Home({
onOpen,
onContribute,
onApi,
}: {
onOpen: (id: string) => void;
onContribute: () => void;
onApi: () => void;
}) {
const [q, setQ] = useState("");
const [items, setItems] = useState<ProductSummary[]>([]);
@@ -19,12 +37,14 @@ export default function Home({
const [loading, setLoading] = useState(false);
const [error, setError] = useState("");
async function run(e?: React.FormEvent) {
async function run(term?: string, e?: React.FormEvent) {
e?.preventDefault();
const keyword = (term ?? q).trim();
if (term !== undefined) setQ(term);
setLoading(true);
setError("");
try {
const res = await api.search(q.trim(), 1, 30);
const res = await api.search(keyword, 1, 30);
setItems(res.items);
setTotal(res.total);
setSearched(true);
@@ -36,87 +56,116 @@ export default function Home({
}
return (
<div>
<div className="text-center py-10">
<h1 className="text-3xl font-bold text-gray-800"></h1>
<p className="mt-2 text-gray-500">
</p>
<form onSubmit={run} className="mt-6 max-w-2xl mx-auto flex gap-2">
<div className="flex-1 flex items-center gap-2 bg-white border rounded-lg px-3 shadow-sm focus-within:ring-2 focus-within:ring-emerald-400">
<Search className="w-5 h-5 text-gray-400" />
<input
autoFocus
value={q}
onChange={(e) => setQ(e.target.value)}
placeholder="例如:可乐、Nutella、5449000000996"
className="flex-1 py-3 outline-none bg-transparent"
/>
<div className="animate-fade-up">
<section className="relative overflow-hidden rounded-3xl border border-gray-100 bg-gradient-to-b from-white to-brand-50/40 px-6 py-14 text-center shadow-card">
<div
aria-hidden
className="pointer-events-none absolute -top-24 left-1/2 h-72 w-[42rem] -translate-x-1/2 rounded-full bg-brand-200/30 blur-3xl"
/>
<div className="relative">
<span className="inline-flex items-center gap-1.5 rounded-full border border-brand-200 bg-white/70 px-3 py-1 text-xs font-medium text-brand-700">
<ShieldCheck className="w-3.5 h-3.5" /> · ·
</span>
<h1 className="mt-5 text-4xl sm:text-5xl font-extrabold tracking-tight text-gray-900">
<span className="bg-gradient-to-r from-brand-600 to-emerald-500 bg-clip-text text-transparent"></span>
</h1>
<p className="mx-auto mt-3 max-w-xl text-gray-500">
</p>
<form onSubmit={(e) => run(undefined, e)} className="mx-auto mt-7 flex max-w-2xl gap-2">
<div className="flex flex-1 items-center gap-2 rounded-xl border border-gray-200 bg-white px-3.5 shadow-sm transition focus-within:border-brand-400 focus-within:ring-4 focus-within:ring-brand-500/10">
<Search className="h-5 w-5 shrink-0 text-gray-400" />
<input
autoFocus
value={q}
onChange={(e) => setQ(e.target.value)}
placeholder="例如:可乐、Nutella、5449000000996"
className="flex-1 bg-transparent py-3.5 outline-none"
/>
</div>
<button type="submit" disabled={loading} className="btn-primary px-7">
{loading ? "检索中…" : "检索"}
</button>
</form>
<div className="mt-4 flex flex-wrap items-center justify-center gap-2">
<span className="text-xs text-gray-400"></span>
{EXAMPLES.map((ex) => (
<button key={ex} type="button" className="chip" onClick={() => run(ex)}>
<ScanBarcode className="w-3 h-3" /> {ex}
</button>
))}
</div>
<button
type="submit"
disabled={loading}
className="px-6 rounded-lg bg-emerald-600 text-white font-medium hover:bg-emerald-700 disabled:opacity-60"
>
{loading ? "检索中…" : "检索"}
</button>
</form>
<button
onClick={onApi}
className="mt-4 inline-flex items-center gap-1.5 text-sm text-emerald-700 hover:underline"
>
<Code2 className="w-4 h-4" /> API
</button>
</div>
</div>
</section>
{error && (
<div className="max-w-2xl mx-auto bg-red-50 text-red-700 text-sm rounded-md px-4 py-2">
<div className="mx-auto mt-6 max-w-2xl rounded-xl bg-red-50 px-4 py-2.5 text-sm text-red-700">
{error}
</div>
)}
{searched && (
<div className="mt-2">
<div className="text-sm text-gray-500 mb-2">
{total} {q ? `(关键词:${q}` : ""}
{searched ? (
<div className="mt-8">
<div className="mb-3 text-sm text-gray-500">
<span className="font-semibold text-gray-700">{total}</span>
{q ? `(关键词:${q}` : ""}
</div>
{items.length === 0 ? (
<div className="bg-white border rounded-lg p-8 text-center text-gray-500">
<div className="card p-10 text-center text-gray-500">
<p></p>
<button
onClick={onContribute}
className="mt-3 inline-flex items-center gap-1.5 text-emerald-700 hover:underline"
className="mt-3 inline-flex items-center gap-1.5 font-medium text-brand-700 hover:underline"
>
<PlusCircle className="w-4 h-4" />
</button>
</div>
) : (
<ul className="bg-white border rounded-lg divide-y">
<ul className="card divide-y divide-gray-100 overflow-hidden">
{items.map((p) => (
<li key={p.id}>
<button
onClick={() => onOpen(p.id)}
className="w-full text-left px-4 py-3 hover:bg-gray-50 flex items-center justify-between gap-4"
className="group flex w-full items-center justify-between gap-4 px-4 py-3.5 text-left transition hover:bg-brand-50/50"
>
<div>
<div className="font-medium text-gray-800">{p.name}</div>
<div className="text-xs text-gray-500 mt-0.5">
<div className="min-w-0">
<div className="truncate font-medium text-gray-800 group-hover:text-brand-700">
{p.name}
</div>
<div className="mt-0.5 text-xs text-gray-500">
{p.brand || "未知品牌"}
{p.gtin ? ` · ${p.gtin}` : ""}
</div>
</div>
<span className="text-xs text-gray-400 text-right">
<span className="block">{p.category_path || ""}</span>
{p.country_of_origin ? (
<span className="block text-gray-400">{p.country_of_origin}</span>
) : null}
</span>
<div className="flex shrink-0 items-center gap-3">
<span className="text-right text-xs text-gray-400">
<span className="block">{p.category_path || ""}</span>
{p.country_of_origin ? (
<span className="block">{p.country_of_origin}</span>
) : null}
</span>
<ArrowRight className="h-4 w-4 text-gray-300 transition group-hover:translate-x-0.5 group-hover:text-brand-500" />
</div>
</button>
</li>
))}
</ul>
)}
</div>
) : (
<div className="mt-8 grid gap-4 sm:grid-cols-3">
{FEATURES.map(({ icon: Icon, title, desc }) => (
<div key={title} className="card p-5 transition hover:shadow-card-hover">
<span className="grid h-10 w-10 place-items-center rounded-xl bg-brand-50 text-brand-600">
<Icon className="h-5 w-5" />
</span>
<h3 className="mt-3 font-semibold text-gray-800">{title}</h3>
<p className="mt-1 text-sm leading-relaxed text-gray-500">{desc}</p>
</div>
))}
</div>
)}
</div>
);
@@ -28,11 +28,11 @@ export default function ProductView({ id, onBack }: { id: string; onBack: () =>
<button onClick={onBack} className="text-sm text-gray-500 flex items-center gap-1 mb-4">
<ArrowLeft className="w-4 h-4" />
</button>
<div className="bg-red-50 text-red-700 text-sm rounded-md px-4 py-3">{error}</div>
<div className="rounded-xl bg-red-50 px-4 py-3 text-sm text-red-700">{error}</div>
</div>
);
}
if (!p) return <div className="text-gray-400"></div>;
if (!p) return <div className="text-gray-400 animate-pulse"></div>;
const basisLabel: Record<string, string> = {
per_100g: "每 100g",
@@ -45,14 +45,14 @@ export default function ProductView({ id, onBack }: { id: string; onBack: () =>
return (
<div>
<button onClick={onBack} className="text-sm text-gray-500 flex items-center gap-1 mb-4">
<button onClick={onBack} className="mb-4 inline-flex items-center gap-1 text-sm text-gray-500 hover:text-gray-700">
<ArrowLeft className="w-4 h-4" />
</button>
<div className="bg-white border rounded-lg p-5">
<div className="card p-6 animate-fade-up">
<div className="flex items-start justify-between gap-4">
<h1 className="text-xl font-semibold text-gray-800">{p.name}</h1>
<span className="shrink-0 text-xs bg-emerald-50 text-emerald-700 rounded px-2 py-1">
<h1 className="text-2xl font-semibold tracking-tight text-gray-900">{p.name}</h1>
<span className="shrink-0 rounded-full border border-brand-100 bg-brand-50 px-2.5 py-1 text-xs font-medium text-brand-700">
{Math.round(p.quality_score * 100)}
</span>
</div>
@@ -107,7 +107,7 @@ export default function ProductView({ id, onBack }: { id: string; onBack: () =>
</div>
{p.specs && p.specs.length > 0 && (
<div className="bg-white border rounded-lg p-5 mt-4">
<div className="card p-6 mt-4">
<h2 className="font-medium text-gray-700 mb-2"></h2>
<div className="grid grid-cols-1 sm:grid-cols-2 gap-2 text-sm">
{p.specs.map((s) => (
@@ -124,7 +124,7 @@ export default function ProductView({ id, onBack }: { id: string; onBack: () =>
)}
{nutriEntries.length > 0 && (
<div className="bg-white border rounded-lg p-5 mt-4">
<div className="card p-6 mt-4">
<h2 className="font-medium text-gray-700 mb-2">
{p.nutrition_basis ? `${basisLabel[p.nutrition_basis] || p.nutrition_basis}` : ""}
+32 -3
View File
@@ -10,7 +10,36 @@ body,
body {
margin: 0;
background: #f3f4f6;
font-family: system-ui, -apple-system, "Segoe UI", Roboto, "Helvetica Neue",
Arial, "PingFang SC", "Microsoft YaHei", sans-serif;
color: #1f2937;
font-family: theme("fontFamily.sans");
-webkit-font-smoothing: antialiased;
text-rendering: optimizeLegibility;
background-color: #f6f8f7;
background-image:
radial-gradient(60rem 60rem at 100% -10%, rgba(16, 185, 129, 0.10), transparent 60%),
radial-gradient(50rem 50rem at -10% 0%, rgba(45, 212, 191, 0.10), transparent 55%);
background-attachment: fixed;
}
@layer components {
.card {
@apply bg-white rounded-2xl border border-gray-100;
box-shadow: theme("boxShadow.card");
}
.input {
@apply w-full rounded-xl border border-gray-200 bg-white px-3.5 py-2.5 text-sm text-gray-800 placeholder:text-gray-400 transition focus:outline-none focus:border-brand-400 focus:ring-4 focus:ring-brand-500/10;
}
.btn-primary {
@apply inline-flex items-center justify-center gap-1.5 rounded-xl bg-brand-600 px-5 py-2.5 text-sm font-semibold text-white shadow-sm transition hover:bg-brand-700 active:bg-brand-800 disabled:opacity-60;
}
.btn-ghost {
@apply inline-flex items-center justify-center gap-1.5 rounded-xl px-3.5 py-2 text-sm font-medium text-gray-600 transition hover:bg-gray-100;
}
.chip {
@apply inline-flex items-center gap-1 rounded-full border border-gray-200 bg-white/70 px-3 py-1 text-xs font-medium text-gray-600 transition hover:border-brand-300 hover:text-brand-700 hover:bg-brand-50;
}
}
+15
View File
@@ -51,6 +51,20 @@ export interface Category {
name_en: string | null;
path: string;
level: number;
archive_kind: string;
}
export interface KindField {
kind: string;
field_key: string;
group_label: string;
label_zh: string;
field_type: "text" | "number" | "textarea" | "select" | "list";
unit: string | null;
options: string[] | null;
placeholder: string | null;
sort_order: number;
qualified: boolean;
}
export interface SubmissionImage {
@@ -68,6 +82,7 @@ export interface SubmissionInput {
country_of_origin?: string | null;
ingredients_text?: string | null;
nutriments?: Record<string, number> | null;
attributes?: Record<string, unknown> | null;
nutrition_basis?: string | null;
serving_size?: string | null;
nutri_score?: string | null;
+50 -1
View File
@@ -1,6 +1,55 @@
/** @type {import('tailwindcss').Config} */
export default {
content: ["./index.html", "./src/**/*.{ts,tsx}"],
theme: { extend: {} },
theme: {
extend: {
colors: {
brand: {
50: "#ecfdf5",
100: "#d1fae5",
200: "#a7f3d0",
300: "#6ee7b7",
400: "#34d399",
500: "#10b981",
600: "#059669",
700: "#047857",
800: "#065f46",
900: "#064e3b",
},
},
fontFamily: {
sans: [
"Inter",
"system-ui",
"-apple-system",
"Segoe UI",
"Roboto",
"Helvetica Neue",
"Arial",
"PingFang SC",
"Microsoft YaHei",
"sans-serif",
],
},
boxShadow: {
card: "0 1px 2px rgba(16,24,40,0.04), 0 4px 16px -8px rgba(16,24,40,0.10)",
"card-hover": "0 10px 30px -10px rgba(16,24,40,0.18)",
glow: "0 12px 40px -12px rgba(5,150,105,0.45)",
},
borderRadius: {
"2xl": "1rem",
"3xl": "1.5rem",
},
keyframes: {
"fade-up": {
"0%": { opacity: "0", transform: "translateY(8px)" },
"100%": { opacity: "1", transform: "translateY(0)" },
},
},
animation: {
"fade-up": "fade-up 0.4s ease-out both",
},
},
},
plugins: [],
};
+3 -4
View File
@@ -58,10 +58,9 @@ GET http://zc.bypos.net/byGoodsService/byMessage.asmx/GetGoodsinfo
## 配置
- `sdogid`中心库账号 id本项目所属云店账号的授权 id)。
优先从环境变量 `BYPOS_SDOGID` 读取,其次是命令行 `-sdogid` 参数
也可以在 Web 控制台的输入框中填写。**三者都未设时启动会警告、开始采集时会报错。**
- 网络错误自动重试(最多 3 次,指数退避 500ms/1s/2s)。
- `sdogid`:中心库账号 id(本项目所属云店账号的授权 id)。默认值见
`collect.go``defaultSdogID`,也可用 `-sdogid` 参数或控制台覆盖。
**这是账号级凭证**——若本仓库对外公开,建议改为从环境变量/外部配置读取。
## 导入 goods/天工库
+8 -42
View File
@@ -20,21 +20,12 @@ import (
)
// ---- upstream config ----
// sdogid is the 云店 account license id observed in the live request. It is
// configurable so the tool is not tied to a single account.
const defaultSdogID = "137966"
const endpoint = "http://zc.bypos.net/byGoodsService/byMessage.asmx/GetGoodsinfo"
// maxRetries is the number of retry attempts for transient network errors.
const maxRetries = 3
// resolveSdogID reads the account id from $BYPOS_SDOGID, falling back to the
// explicit argument (CLI flag or Web UI input). Returns empty if neither set.
func resolveSdogID(explicit string) string {
if v := os.Getenv("BYPOS_SDOGID"); v != "" {
return v
}
return explicit
}
var stringTagRe = regexp.MustCompile(`(?s)<string[^>]*>(.*)</string>`)
// Product is the normalized record we persist (one JSON object per line).
@@ -96,8 +87,8 @@ func ean13Check(body string) (string, bool) {
return body + strconv.Itoa(chk), true
}
// lookupOnce performs a single HTTP request to the upstream central library.
func (c *Collector) lookupOnce(ctx context.Context, barcode string) (*Product, error) {
// lookup queries the upstream central library for one barcode.
func (c *Collector) lookup(ctx context.Context, barcode string) (*Product, error) {
tsMs := strconv.FormatInt(time.Now().Unix()*1000, 10) // always ends in 000
sparm1 := md5hex(c.sdogID)
sparm2 := md5hex(barcode + tsMs)
@@ -156,30 +147,6 @@ func (c *Collector) lookupOnce(ctx context.Context, barcode string) (*Product, e
return p, nil
}
// lookup queries the upstream with up to maxRetries retries on transient errors.
func (c *Collector) lookup(ctx context.Context, barcode string) (*Product, error) {
var lastErr error
for attempt := 0; attempt <= maxRetries; attempt++ {
if ctx.Err() != nil {
return nil, ctx.Err()
}
p, err := c.lookupOnce(ctx, barcode)
if err == nil {
return p, nil
}
lastErr = err
if attempt < maxRetries {
backoff := time.Duration(1<<uint(attempt)) * 500 * time.Millisecond
select {
case <-time.After(backoff):
case <-ctx.Done():
return nil, ctx.Err()
}
}
}
return nil, lastErr
}
// ---- job / collector state ----
type Stats struct {
@@ -220,7 +187,9 @@ type Collector struct {
}
func NewCollector(sdogID string) *Collector {
sdogID = resolveSdogID(sdogID)
if sdogID == "" {
sdogID = defaultSdogID
}
c := &Collector{
client: &http.Client{Timeout: 25 * time.Second},
sdogID: sdogID,
@@ -339,9 +308,6 @@ func (c *Collector) Start(req JobReq) error {
if req.SdogID != "" {
c.sdogID = req.SdogID
}
if c.sdogID == "" {
return fmt.Errorf("未指定 sdogid,请通过控制台输入框、环境变量 $BYPOS_SDOGID 或 -sdogid 参数配置")
}
// Build the list of barcodes to query.
var barcodes []string
-113
View File
@@ -1,113 +0,0 @@
package main
import (
"os"
"testing"
)
func TestEan13Check_valid(t *testing.T) {
tests := []struct {
body string
want string
}{
{"692045990501", "6920459905012"},
{"690100000000", "6901000000004"},
{"690100000099", "6901000000998"},
}
for _, tt := range tests {
got, ok := ean13Check(tt.body)
if !ok {
t.Errorf("ean13Check(%q) returned not ok", tt.body)
continue
}
if got != tt.want {
t.Errorf("ean13Check(%q) = %q, want %q", tt.body, got, tt.want)
}
}
}
func TestEan13Check_invalid(t *testing.T) {
cases := []string{"", "12345", "12345678901", "1234567890123", "69010000a000"}
for _, body := range cases {
_, ok := ean13Check(body)
if ok {
t.Errorf("ean13Check(%q) should return not ok", body)
}
}
}
func TestMd5hex(t *testing.T) {
got := md5hex("137966")
if len(got) != 32 {
t.Errorf("md5hex should return 32-char hex, got len=%d", len(got))
}
if got != md5hex("137966") {
t.Error("md5hex should be deterministic")
}
if got == md5hex("other") {
t.Error("md5hex should differ for different inputs")
}
}
func TestSanitizeBarcodes(t *testing.T) {
got := sanitizeBarcodes("6920459905012\n6901028941068 123")
want := []string{"6920459905012", "6901028941068", "123"}
if len(got) != len(want) {
t.Fatalf("len = %d, want %d", len(got), len(want))
}
for i := range want {
if got[i] != want[i] {
t.Errorf("got[%d] = %q, want %q", i, got[i], want[i])
}
}
}
func TestSanitizeBarcodes_empty(t *testing.T) {
got := sanitizeBarcodes(" \n\t ")
if len(got) != 0 {
t.Errorf("expected empty, got %v", got)
}
}
func TestResolveSdogID_envOverride(t *testing.T) {
os.Setenv("BYPOS_SDOGID", "999999")
defer os.Unsetenv("BYPOS_SDOGID")
got := resolveSdogID("111111")
if got != "999999" {
t.Errorf("expected env var override, got %q", got)
}
}
func TestResolveSdogID_fallback(t *testing.T) {
os.Unsetenv("BYPOS_SDOGID")
got := resolveSdogID("111111")
if got != "111111" {
t.Errorf("expected fallback to explicit, got %q", got)
}
}
func TestResolveSdogID_empty(t *testing.T) {
os.Unsetenv("BYPOS_SDOGID")
got := resolveSdogID("")
if got != "" {
t.Errorf("expected empty, got %q", got)
}
}
func TestNewCollector_sdogFromEnv(t *testing.T) {
os.Setenv("BYPOS_SDOGID", "888888")
defer os.Unsetenv("BYPOS_SDOGID")
c := NewCollector("")
if c.sdogID != "888888" {
t.Errorf("expected sdogID from env, got %q", c.sdogID)
}
}
func TestStartRejectsEmptySdogID(t *testing.T) {
os.Unsetenv("BYPOS_SDOGID")
c := NewCollector("")
err := c.Start(JobReq{Mode: "list", List: "6920459905012"})
if err == nil {
t.Fatal("expected error for empty sdogid")
}
}
+9 -50
View File
@@ -1,7 +1,6 @@
package main
import (
"bufio"
"embed"
"encoding/csv"
"encoding/json"
@@ -21,48 +20,18 @@ import (
//go:embed web/*
var webFS embed.FS
var collector *Collector
// waitExit keeps the console window open on Windows so the user can read errors.
func waitExit() {
if runtime.GOOS == "windows" {
fmt.Println("\n按回车键退出...")
bufio.NewReader(os.Stdin).ReadBytes('\n')
}
}
var collector = NewCollector("")
func main() {
// Log to file so crashes are diagnosable even if console closes.
lf, lfErr := os.OpenFile("bypos-collector.log", os.O_CREATE|os.O_WRONLY|os.O_APPEND, 0644)
if lfErr == nil {
log.SetOutput(io.MultiWriter(os.Stderr, lf))
defer lf.Close()
}
defer func() {
if r := recover(); r != nil {
log.Printf("程序崩溃: %v", r)
waitExit()
}
}()
log.Println("bypos-collector 启动中...")
addr := flag.String("addr", "127.0.0.1:8765", "本地监听地址")
noOpen := flag.Bool("no-open", false, "不自动打开浏览器")
sdog := flag.String("sdogid", "", "中心库账号 id(优先读 $BYPOS_SDOGID 环境变量)")
sdog := flag.String("sdogid", "", "中心库账号 id(默认使用内置值)")
flag.Parse()
collector = NewCollector(*sdog)
if collector.sdogID == "" {
log.Println("警告: 未配置 sdogid,请通过 $BYPOS_SDOGID 环境变量、-sdogid 参数或控制台输入框指定")
if *sdog != "" {
collector.sdogID = *sdog
}
sub, err := fs.Sub(webFS, "web")
if err != nil {
log.Printf("错误: 无法加载内嵌 web 资源: %v", err)
waitExit()
return
}
sub, _ := fs.Sub(webFS, "web")
mux := http.NewServeMux()
mux.Handle("/", http.FileServer(http.FS(sub)))
mux.HandleFunc("/api/start", handleStart)
@@ -73,13 +42,7 @@ func main() {
ln, err := net.Listen("tcp", *addr)
if err != nil {
log.Printf("端口 %s 被占用,自动选择可用端口...", *addr)
ln, err = net.Listen("tcp", "127.0.0.1:0")
if err != nil {
log.Printf("错误: 无法监听: %v", err)
waitExit()
return
}
log.Fatalf("无法监听 %s: %v", *addr, err)
}
realAddr := ln.Addr().String()
urlStr := "http://" + realAddr + "/"
@@ -88,14 +51,10 @@ func main() {
fmt.Println(" 控制台: " + urlStr)
fmt.Println(" 关闭本窗口即停止程序")
fmt.Println("==============================================")
log.Printf("监听地址: %s", realAddr)
if !*noOpen {
go openBrowser(urlStr)
}
if err := http.Serve(ln, mux); err != nil {
log.Printf("HTTP 服务异常退出: %v", err)
waitExit()
}
log.Fatal(http.Serve(ln, mux))
}
func openBrowser(url string) {
@@ -184,14 +143,14 @@ func handleExportCSV(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Disposition", "attachment; filename=products.csv")
w.Write([]byte{0xEF, 0xBB, 0xBF}) // UTF-8 BOM so Excel reads Chinese correctly
cw := csv.NewWriter(w)
cw.Write([]string{"barcode", "name", "spec", "unit", "area", "manufacturer", "license", "in_price", "sell_price", "status", "retmsg", "fetched_at", "source"})
cw.Write([]string{"barcode", "name", "spec", "unit", "area", "manufacturer", "license", "in_price", "sell_price", "status", "fetched_at"})
dec := json.NewDecoder(f)
for {
var p Product
if err := dec.Decode(&p); err != nil {
break
}
cw.Write([]string{p.Barcode, p.Name, p.Spec, p.Unit, p.Area, p.Manufacturer, p.License, p.InPrice, p.SellPrice, p.Status, p.RetMsg, p.FetchedAt, p.Source})
cw.Write([]string{p.Barcode, p.Name, p.Spec, p.Unit, p.Area, p.Manufacturer, p.License, p.InPrice, p.SellPrice, p.Status, p.FetchedAt})
}
cw.Flush()
}
-5
View File
@@ -79,10 +79,6 @@
<div class="card">
<h3>② 采集参数</h3>
<div class="row">
<div>
<label>中心库账号 sdogid</label>
<input type="text" id="sdogid" placeholder="留空则使用环境变量或启动参数"/>
</div>
<div>
<label>并发数</label>
<input type="number" id="concurrency" value="3" min="1" max="20"/>
@@ -169,7 +165,6 @@ async function start(){
start_body: document.getElementById('start').value.trim(),
end_body: document.getElementById('end').value.trim(),
list: document.getElementById('list').value,
sdog_id: document.getElementById('sdogid').value.trim(),
concurrency: parseInt(document.getElementById('concurrency').value)||3,
delay_ms: parseInt(document.getElementById('delay').value)||0,
log_miss: document.getElementById('logmiss').checked,