feat(api): Redis read cache for product details and search
Stage-0 caching from docs/scalability.md. The Go API now caches hot product-detail and search-result reads in Redis with a fail-open, epoch-versioned scheme; Python ingestion bumps the epoch after a write run to invalidate the cache globally in O(1). - api/internal/cache: fail-open Cache (GetJSON/SetJSON) namespaced by an epoch counter (og:cache:epoch); disabled when Redis is unconfigured. - store: ProductByID/ProductByGTIN (24h TTL) and SearchProducts (1h TTL) read-through the cache via WithCache. - ingestion: bump_cache_epoch() called after update_off/seed_off/ import_bypos/dedup commits when rows changed; best-effort, never fails a run. Adds redis dependency. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
This commit is contained in:
@@ -17,6 +17,7 @@ import sys
|
||||
import psycopg
|
||||
|
||||
from opengoods.adapters.openfoodfacts import SOURCE_NAME, OpenFoodFactsAdapter
|
||||
from opengoods.cache import bump_cache_epoch
|
||||
from opengoods.etl.load import default_dsn, ensure_source, load_record_safe
|
||||
from opengoods.etl.state import get_watermark, set_watermark
|
||||
from opengoods.etl.transform import transform
|
||||
@@ -49,6 +50,8 @@ def run(args: argparse.Namespace) -> int:
|
||||
stats={"loaded": loaded, "skipped": skipped, "errored": errored, "since": since},
|
||||
)
|
||||
conn.commit()
|
||||
if loaded:
|
||||
bump_cache_epoch()
|
||||
print(
|
||||
f"since={since} loaded={loaded} skipped={skipped} "
|
||||
f"errored={errored} watermark={high_watermark}"
|
||||
|
||||
Reference in New Issue
Block a user