PracticeVideosBlogFlashcards

8.1 Cloud Storage Lifecycle and Signed Access

Key Takeaways

  • Bucket Lock cannot be undone or shortened after locking.

  • A signed PUT URL and a resumable-upload session are distinct bearer-access mechanisms.

  • Cloud Storage FUSE offers a filesystem interface with object-store constraints, not a general POSIX database volume.

Last updated: October 2026

Cloud Storage (GCS): Classes, Lifecycle & Governance

Google Cloud Storage delivers exabyte-scale object storage with 99.999999999% (11 nines) annual durability.

Storage Classes & Autoclass

Storage ClassMinimum Storage DurationAvailability SLATypical Access Pattern / Use Case
StandardNone99.95% SLA (multi-region)Actively accessed data, static websites, dynamic media serving.
Nearline30 Days99.9% SLA (multi-region)Read less than once per month; scheduled monthly reports, backups.
Coldline90 Days99.9% SLA (multi-region)Read less than once per quarter; disaster recovery, raw historical data.
Archive365 Days99.9% SLA (multi-region)Read less than once per year; regulatory compliance, long-term legal tape.
AutoclassFeature-dependent billingUnderlying storage-class/location rulesAutomates supported transitions; review management and enablement charges.
RapidNone99.9% SLA (zone)Specialized high-performance zonal buckets; different geography/recovery trade-offs.

Object Versioning & Lifecycle Management

  • Object Versioning: When enabled, overwriting or deleting the live object without a generation can preserve a noncurrent generation. Explicit generation deletion, lifecycle rules and soft-delete settings have separate behavior; versioning is not a promise that nothing can be permanently removed.
  • Object Lifecycle Management Rules: Configured via declarative JSON policies to automate object retention and cost optimization:
    • Condition: Object age > 30 days, current storage class is Standard -> Action: Set storage class to Nearline.
    • Condition: Noncurrent versions older than 14 days, numNewerVersions > 3 -> Action: Delete permanently.
    • Condition: abortIncompleteMultipartUpload after 7 days -> Cleans up abandoned chunked uploads to prevent hidden storage costs.

Retention Policies & Bucket Lock (WORM Compliance)

  • Retention Policy: Enforces a minimum retention period (e.g., 7 years) during which objects in the bucket cannot be deleted or overwritten by anyone, supporting write-once retention designs; regulatory compliance also depends on the full system and applicable requirements.
  • Bucket Lock: Permanently locks the retention policy. Critical Safety Rule: Once a retention policy is locked, it cannot be deleted, disabled, or shortened, even by Project Owners or Google Cloud Support. The retention period can only be extended.

Direct Client Operations: Signed URLs & Cloud Storage FUSE

Cloud Storage Signed URLs

Routing multi-gigabyte media uploads or downloads through an intermediate application server (such as Cloud Run or GKE) consumes excessive CPU, memory, and double network bandwidth. Developers resolve this by generating Signed URLs:

+--------------------------------------------------------------------------+
| Cloud Storage Signed URL Workflow (Direct Client Upload)                 |
+--------------------------------------------------------------------------+
|                                                                          |
| [ Client Browser ]                                                       |
|        |                                                                 |
|        | 1. Request Upload Permission (POST /api/upload-ticket)          |
|        v                                                                 |
| [ Cloud Run API ] ---> 2. Calls IAM SignBlob API with GSA Credentials    |
|        |                  (Generates V4 Signed PUT URL, Valid for 15m)   |
|        | 3. Returns Signed URL to Client                                 |
|        v                                                                 |
| [ Client Browser ]                                                       |
|        |                                                                 |
|        | 4. Direct upload using signed PUT or authorized resumable session            |
|        v                                                                 |
| [ Cloud Storage Bucket ] (Bypasses Cloud Run Compute & Egress Completely)|
+--------------------------------------------------------------------------+
  • Mechanics: The signer needs the IAM signBlob permission and storage permission for the chosen operation (for example Object Creator for a new upload or Object Viewer for a download). It can generate a cryptographic V4 Signed URL with a defined HTTP method (PUT, GET), expiration window (e.g., 15 minutes), and optional Content-MD5 checksum.
  • Security: The client uploads the file directly to the Cloud Storage bucket without needing Google Cloud credentials or public bucket access permissions.

Cloud Storage FUSE

Cloud Storage FUSE is an open-source FUSE adapter that allows Linux instances, GKE Pods, and Cloud Run containers to mount Cloud Storage buckets as local filesystems.

  • Operational Constraints: Cloud Storage is an object store, not a POSIX filesystem. GCS FUSE does not support file locking, symlinks, or in-place byte modifications ; supported streaming or append behavior varies by FUSE version, but it does not provide general database-grade POSIX semantics.
  • Optimal Use Cases: Machine learning model training data ingestion, read-heavy media assets, and legacy file-processing containers requiring a directory-like interface.

NoSQL and Unstructured Storage Decision Matrix

Evaluation CriterionCloud FirestoreCloud BigtableCloud Storage
Data StructureHierarchical JSON documentsWide-column sorted tablesUnstructured binary objects (blobs)
Query CapabilitiesRich filters, ranges, composite indexes, full document scansRow key prefix scans, regex filters (Core row-key access model; newer SQL/materialized-view features require explicit evaluation)Key-value retrieval via object name/prefix
Maximum Item Size1 MiB per document (subject to documented overhead)Use current cell/row limits and size recommendationsUp to 5 TiB per individual object
Latency ProfileWorkload-dependent latencyLow latency with suitable keys and capacityTens of milliseconds to first byte
Scale ThresholdGigabytes to TerabytesTerabytes to Petabytes (> 1 TB baseline)Terabytes to Exabytes
Client ModelDirect Web, Mobile, Server SDKsServer Client Libraries (gRPC / HBase API)REST API, Signed URLs, SDKs, GCS FUSE

Storage Consistency and Replication Boundaries

Cloud Storage object read-after-write, metadata reads, deletion and object listing are strongly consistent after a successful operation, including dual/multi-region buckets. This does not mean every administrative change propagates immediately: granting/revoking access and recreating deleted buckets have eventual behavior. Public caches can also serve an older object until the configured cache lifetime ends. Use generation preconditions for operations that must target a specific version instead of assuming successive ranged reads always see the same bytes.

Strong API visibility is different from replication recovery objectives. If an acknowledged object's initial location fails before cross-region replication completes, a retryable availability error can occur rather than returning stale object data. Choose the supported location/replication option and recovery commitments for the actual failure model. A regional application and multi-region object bucket do not share one universal RPO/RTO.

Official references checked October 18, 2026: Google Cloud 1, Google Cloud 2.

Loading diagram...
Firestore vs Bigtable vs Cloud Storage Direct I/O Architecture
Test Your Knowledge

A media sharing application running on Cloud Run allows users to upload raw 4K video files ranging from 2 GB to 10 GB each. Currently, the client application sends video bytes directly to an API endpoint on Cloud Run, which streams them to a Cloud Storage bucket. Users frequently report HTTP 504 Gateway Timeout errors, and Cloud Run network egress costs are surging. How should the cloud developer rearchitect the upload workflow?

A

Have the Cloud Run API generate a time-limited Cloud Storage signed PUT URL or an authorized resumable-upload session, allowing the client to upload the video file directly to the Cloud Storage bucket

B

Mount the Cloud Storage bucket as a local filesystem on Cloud Run using Cloud Storage FUSE and write files using standard POSIX file operations

C

Deploy a GKE cluster with local SSD storage to buffer incoming video files before transferring them to Cloud Storage via gsutil

D

Increase the Cloud Run request timeout to 3600 seconds and increase container instance memory to 32 GiB

Sections you finish are checked off in the contents.