14.1 Serving Features Online with Feature Store
Key Takeaways
- A feature view must be synced at least once before Feature Store can serve its feature values online.
- Bigtable online serving handles terabyte-scale feature data, supports scheduled and continuous sync, and doesn't manage embeddings, which belong in Vector Search.
- Feature Store fetches online values with fetchFeatureValues by entity ID, returning KEY_VALUE JSON or PROTO_STRUCT formats.
- Direct write (preview) updates feature values in a Bigtable-served feature view with about 100 ms freshness, without changing the BigQuery source.
- Bigtable online stores autoscale between a minimum and maximum node count around a CPU utilization target.
The exam guide's scaling section begins with managing and serving features using Agent Platform Feature Store. Section 5.3 covered creating features in BigQuery and the Feature Registry. This section is about getting those features to a model within milliseconds.
Online Serving Architecture
| Resource | Role |
|---|---|
| FeatureOnlineStore | The serving cluster, with its serving type, node scaling, and encryption |
| FeatureView | A logical set of features inside an online store, sourced from feature groups and features or directly from a BigQuery table or view |
| Data sync | Copies the latest values from BigQuery into the feature view |
| fetchFeatureValues | Reads a feature record for an entity ID at request time |
A feature view must be synced at least once before it can serve.
Online Serving Types
| Type | Status | Characteristics |
|---|---|---|
| Bigtable online serving | Current, recommended | Large data volumes (terabytes) with high durability. Scheduled and continuous sync. Supports CMEK. No embedding management, so use Vector Search for embeddings |
| Optimized online serving (public or Private Service Connect endpoint) | Deprecated February 17, 2026. Shutdown February 17, 2027 | Ultra-low latency plus embedding management, scheduled sync only. Migrate to Bigtable online serving |
Bigtable online store capacity
Create the store with autoscaling settings: minimum node count, maximum node count, and a CPU utilization target. For example, 1-3 nodes at 50% CPU. Size minimum nodes for baseline traffic so lookups don't wait for scale-up.
Keeping Online Features Fresh
| Mechanism | How it works | Freshness | Notes |
|---|---|---|---|
| Scheduled sync | A cron schedule re-syncs from BigQuery. You can also trigger a sync manually | Minutes to hours, based on schedule | Works for any serving type. BigQuery reads cost money, so don't over-sync |
| Continuous sync | Syncs as new rows land in BigQuery | Near real time for new records | Bigtable serving plus registered feature groups, with the source in us, eu, or us-central1. Updates and deletes in BigQuery aren't synced. A feature view with continuous sync can't be updated |
| Direct write (preview) | Write values straight into a Bigtable-served feature view | About 100 ms | Doesn't update the BigQuery source. The next sync overwrites with any newer BigQuery value. Each feature view needs its own writes |
Null handling: by default the store serves the latest non-null value, falling back to the most recent non-null historical value. To serve latest values including nulls, register the feature group with dense = true, use Bigtable online serving, and use scheduled sync.
Fetching Features in the Serving Path
A typical online inference request flow:
- The client calls the app with
customer_id=C123and the transaction details. - The app, or a CPR
preprocess(), callsfetchFeatureValueson the feature view forC123. Results come back asKEY_VALUEJSON orPROTO_STRUCT. - The app merges the stored features (such as
spend_30dandchargebacks_90d) with request-time features (transaction amount, merchant). - It sends the combined instance to the model endpoint.
- It logs the exact feature values used, for monitoring and future training.
aiplatform.init(project="my-project", location="us-central1")
fos = FeatureOnlineStore("fraud_online_store")
fv = FeatureView("customer_features", feature_online_store_id=fos.name)
record = fv.read("C123") # latest values for this entity ID
Latency budgeting
If an end-to-end SLA is 80 ms, measure feature lookup p99 separately from model inference. Put the application, the online store, and the endpoint in the same region, and use private connectivity where possible.
Training-Serving Consistency
- Online and offline features come from the same BigQuery-defined feature, which avoids re-implementing aggregations in application code.
- Build training data with point-in-time lookups (offline serving or
ML.FEATURES_AT_TIME), so training values match what online serving would have returned then. - Log served feature values. Comparing them with training distributions is how training-serving skew monitoring works (Chapter 19).
Online Store vs. Other Low-Latency Options
| Need | Option |
|---|---|
| Per-entity feature values defined in BigQuery, served to models | Feature Store online store (Bigtable serving) |
| Nearest-neighbor search over embeddings | Vector Search |
| Application state unrelated to ML features | The application's own database |
| Precomputed predictions for lookup | A key-value store or a table read by the app (Chapter 12 hybrid patterns) |
Security and Operations
- IAM on online stores and feature views controls who can read features.
- CMEK is supported for Bigtable online stores.
- Monitoring: track fetch latency, error rates, sync job status, and Bigtable CPU.
- Cost: online stores bill for provisioned nodes. Syncs incur BigQuery query costs, so match the sync frequency to how often the source changes.
Worked Scenario
A card issuer scores transactions in under 100 ms. Its features are 30-day spend aggregates (updated hourly in BigQuery) and a "transactions in last 10 minutes" counter.
- Serve the hourly aggregates from a Bigtable online store with an hourly scheduled sync.
- Write the 10-minute counter from the streaming Dataflow pipeline with direct write for about 100 ms freshness. Also write it to BigQuery so training data and the next sync stay consistent.
- Fetch both in the scoring service, send to a Private Service Connect endpoint, and log the served values to BigQuery.
A team creates a new feature view on a Bigtable online store, but fetchFeatureValues returns nothing for known entity IDs. What is the most likely cause?
A recommendation system needs to store and retrieve item embeddings for similarity search next to regular user features. Using the current Feature Store, where should the embeddings go?
A fraud model needs a transaction-velocity feature available within about 100 ms of events, while hourly aggregates come from BigQuery. Which Feature Store capability addresses the velocity feature?