Stage-0 caching from docs/scalability.md: the read-only API now caches hot product-detail and search-result reads in Redis so repeated requests skip PostgreSQL. It mirrors the existing ratelimit package — every cache operation fails open, so a missing/unreachable Redis never takes the API down and never serves stale data after a Redis loss.
Invalidation is global and O(1) via an epoch counter. Keys are namespaced og:v<epoch>:...; ingestion bumps og:cache:epoch after a write run, which logically drops the entire cache at once while old keys age out by TTL. The epoch is read from Redis at most once per 10s per process, so caching adds no extra per-request round trip beyond the value lookup.
API (Go)
New api/internal/cache: Cache.GetJSON/SetJSON, namespaced by epochNow(). Disabled (always-miss / no-op) when Redis is unconfigured; all ops capped at 150ms and fail open.
store.Store gains WithCache(c); reads become read-through:
Same for ProductByGTIN (24h). SearchProducts caches {items,total} keyed by a sha1 of the full filter set + paging window (1h TTL).
cmd/server: wires store.New(pool).WithCache(cache.New(cfg.RedisURL)) (same URL as the limiter).
Ingestion (Python)
New opengoods/cache.py → bump_cache_epoch(): best-effort INCR og:cache:epoch, never raises (an invalidation failure must not fail a run).
Called after a successful commit, only when rows actually changed, in update_off, seed_off, import_bypos (loaded), and dedup (merged). schedule.py inherits it through those jobs.
Adds redis>=5.0,<6 dependency.
Testing
Go: go build/vet, gofmt, go test ./... clean. New cache_test.go (disabled-fails-open + round-trip + epoch-invalidation) verified green against a real Redis.
Python: ruff check, ruff format --check, pytest → 45 passed / 7 skipped; new test_cache.py covers disabled, unreachable-fail-open, and epoch increment.
## Summary
Stage-0 caching from `docs/scalability.md`: the read-only API now caches hot **product-detail** and **search-result** reads in Redis so repeated requests skip PostgreSQL. It mirrors the existing `ratelimit` package — every cache operation **fails open**, so a missing/unreachable Redis never takes the API down and never serves stale data after a Redis loss.
Invalidation is **global and O(1)** via an epoch counter. Keys are namespaced `og:v<epoch>:...`; ingestion bumps `og:cache:epoch` after a write run, which logically drops the entire cache at once while old keys age out by TTL. The epoch is read from Redis at most once per 10s per process, so caching adds no extra per-request round trip beyond the value lookup.
### API (Go)
- New `api/internal/cache`: `Cache.GetJSON/SetJSON`, namespaced by `epochNow()`. Disabled (always-miss / no-op) when Redis is unconfigured; all ops capped at 150ms and fail open.
- `store.Store` gains `WithCache(c)`; reads become read-through:
```go
func (s *Store) ProductByID(ctx, id) (*Product, error) {
if hit := cacheGet("prod:id:"+id); hit { return cached }
p := /* query postgres */
cacheSet("prod:id:"+id, p, productCacheTTL) // 24h
return p
}
```
Same for `ProductByGTIN` (24h). `SearchProducts` caches `{items,total}` keyed by a sha1 of the full filter set + paging window (1h TTL).
- `cmd/server`: wires `store.New(pool).WithCache(cache.New(cfg.RedisURL))` (same URL as the limiter).
### Ingestion (Python)
- New `opengoods/cache.py` → `bump_cache_epoch()`: best-effort `INCR og:cache:epoch`, never raises (an invalidation failure must not fail a run).
- Called after a successful commit, only when rows actually changed, in `update_off`, `seed_off`, `import_bypos` (`loaded`), and `dedup` (`merged`). `schedule.py` inherits it through those jobs.
- Adds `redis>=5.0,<6` dependency.
## Testing
- Go: `go build/vet`, `gofmt`, `go test ./...` clean. New `cache_test.go` (disabled-fails-open + round-trip + epoch-invalidation) verified green against a real Redis.
- Python: `ruff check`, `ruff format --check`, `pytest` → 45 passed / 7 skipped; new `test_cache.py` covers disabled, unreachable-fail-open, and epoch increment.
Stage-0 caching from docs/scalability.md. The Go API now caches hot
product-detail and search-result reads in Redis with a fail-open,
epoch-versioned scheme; Python ingestion bumps the epoch after a write
run to invalidate the cache globally in O(1).
- api/internal/cache: fail-open Cache (GetJSON/SetJSON) namespaced by an
epoch counter (og:cache:epoch); disabled when Redis is unconfigured.
- store: ProductByID/ProductByGTIN (24h TTL) and SearchProducts (1h TTL)
read-through the cache via WithCache.
- ingestion: bump_cache_epoch() called after update_off/seed_off/
import_bypos/dedup commits when rows changed; best-effort, never fails
a run. Adds redis dependency.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
lixu
merged commit f29696607a into main2026-06-24 14:54:59 +08:00
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Summary
Stage-0 caching from
docs/scalability.md: the read-only API now caches hot product-detail and search-result reads in Redis so repeated requests skip PostgreSQL. It mirrors the existingratelimitpackage — every cache operation fails open, so a missing/unreachable Redis never takes the API down and never serves stale data after a Redis loss.Invalidation is global and O(1) via an epoch counter. Keys are namespaced
og:v<epoch>:...; ingestion bumpsog:cache:epochafter a write run, which logically drops the entire cache at once while old keys age out by TTL. The epoch is read from Redis at most once per 10s per process, so caching adds no extra per-request round trip beyond the value lookup.API (Go)
api/internal/cache:Cache.GetJSON/SetJSON, namespaced byepochNow(). Disabled (always-miss / no-op) when Redis is unconfigured; all ops capped at 150ms and fail open.store.StoregainsWithCache(c); reads become read-through:ProductByGTIN(24h).SearchProductscaches{items,total}keyed by a sha1 of the full filter set + paging window (1h TTL).cmd/server: wiresstore.New(pool).WithCache(cache.New(cfg.RedisURL))(same URL as the limiter).Ingestion (Python)
opengoods/cache.py→bump_cache_epoch(): best-effortINCR og:cache:epoch, never raises (an invalidation failure must not fail a run).update_off,seed_off,import_bypos(loaded), anddedup(merged).schedule.pyinherits it through those jobs.redis>=5.0,<6dependency.Testing
go build/vet,gofmt,go test ./...clean. Newcache_test.go(disabled-fails-open + round-trip + epoch-invalidation) verified green against a real Redis.ruff check,ruff format --check,pytest→ 45 passed / 7 skipped; newtest_cache.pycovers disabled, unreachable-fail-open, and epoch increment.