feat(serverless): network-volume warm cache (VolumeCache)

README.md

@@ -149,6 +149,19 @@ runpod.serverless.start({"handler": handler})
 
 See [Worker Fitness Checks](https://github.com/runpod/runpod-python/blob/main/docs/serverless/worker_fitness_checks.md) documentation for more examples and best practices.
 
+### Network-Volume Warm Cache
+
+When a network volume is attached, `VolumeCache` warms local directories (such as a model cache) across cold starts — hydrating them on startup and syncing new files back on exit — so a repeated multi-GB model download becomes a one-time cost per endpoint. It is stdlib-only and best-effort.
+
+```python
+from runpod.serverless import VolumeCache
+
+with VolumeCache(dirs=["/root/.cache/huggingface"]):
+    model = load_model()
+```
+
+See [Network-Volume Warm Cache](https://github.com/runpod/runpod-python/blob/main/docs/serverless/volume_cache.md) documentation for configuration and details.
+
 ## 📚 | API Language Library (GraphQL Wrapper)
 
 When interacting with the Runpod API you can use this library to make requests to the API.

docs/serverless/volume_cache.md

@@ -0,0 +1,89 @@
+# Network-Volume Warm Cache (VolumeCache)
+
+`VolumeCache` warms local directories across serverless workers using a mounted
+network volume. It keeps a browsable mirror of your cache directories on the
+volume and reconciles it against the container on each use: on cold start it
+restores previously-cached files into place, and after use it copies newly
+written files back to the volume. This turns a repeated multi-GB model
+download on every cold start into a one-time cost per endpoint.
+
+It is stdlib-only and best-effort: any failure degrades to a cold worker and
+never raises into your handler or worker loop.
+
+## Requirements
+
+- A **network volume** attached to the endpoint (mounted at `/runpod-volume`).
+- Set on serverless automatically: `RUNPOD_ENDPOINT_ID` (used to scope the
+  mirror per endpoint/namespace).
+
+If no volume is mounted, or the namespace is empty, every operation is a safe
+no-op.
+
+## Usage
+
+`VolumeCache` is a context-manager closure — using it around a model load
+hydrates the cache before the block runs and syncs any changes back after:
+
+```python
+from runpod.serverless import VolumeCache
+
+with VolumeCache(dirs=["/root/.cache/huggingface"]):
+    model = load_model()   # downloads land in the cached directory
+```
+
+- **On enter**, `hydrate()` copies files that are missing or newer on the
+  volume mirror into the container.
+- **On exit**, `sync()` copies files that are missing or newer in the
+  container onto the volume mirror. By default this runs on a background
+  daemon thread and returns immediately, so the `with` block doesn't block on
+  the sync; a process-exit hook joins any outstanding syncs so short-lived
+  processes (including local test runs) still complete the sync before
+  exiting.
+
+You can also call the phases directly when they happen at different points in
+your worker's lifecycle:
+
+```python
+vc = VolumeCache(dirs=["/data/models"], namespace="my-model-cache")
+
+vc.hydrate()                        # restore cached files (e.g. at startup)
+model = load_model()                # populate the cache
+vc.sync(background=False)           # persist new files back to the volume, inline
+```
+
+## Constructor
+
+| Argument | Default | Purpose |
+| --- | --- | --- |
+| `dirs` | required | Local directories to cache. |
+| `namespace` | `RUNPOD_ENDPOINT_ID` | Isolation key for the on-volume mirror. Must be a single safe path component. |
+| `volume_path` | `/runpod-volume` | Network-volume mount point. |
+| `best_effort` | `True` | Swallow and log errors instead of raising. Set `False` while debugging. |
+
+## How it works
+
+- **Directory mirror.** Cached files live at `{volume_path}/.cache/{namespace}`,
+  laid out at the same absolute path they occupy in the container (so the
+  mirror is directly browsable — no archive format to unpack).
+- **Per-file, atomic reconcile.** Each direction (`hydrate`/`sync`) walks the
+  source tree and copies any file whose size differs or whose mtime is newer
+  than the destination's (beyond a small tolerance for coarse network-filesystem
+  mtimes). Copies are written to a temp file and atomically renamed into place
+  (`os.replace`), so a crash mid-copy never leaves a half-written file visible.
+- **Idempotent.** Copies preserve mtime (`shutil.copy2`), so re-running
+  `hydrate`/`sync` after nothing has changed copies zero files.
+- **Last-writer-wins.** Under concurrent workers, the mirror simply reflects
+  whichever worker synced most recently — there's no locking or merge.
+- **Safety.** Symlinked sources are never followed/copied. Hydration destinations
+  are checked to resolve inside one of the configured `dirs` before any write,
+  so a mirror entry can't be used to write outside the cached directories.
+
+## Limitations
+
+- **Cold-scale write amplification.** If N workers cold-start at the same time,
+  each may miss the still-empty mirror, download the model, and sync a full
+  copy back. There's no coordination between concurrent syncs.
+- **Background sync on short-lived processes.** `sync()` schedules the copy on
+  a daemon thread; if the process exits without going through normal
+  interpreter shutdown (e.g. `os._exit`, `SIGKILL`), the atexit hook never
+  runs and the sync may not complete.

examples/serverless/volume_cache_handler.py

@@ -0,0 +1,48 @@
+"""Warm-cache model weights across cold starts with VolumeCache.
+
+`VolumeCache` keeps a browsable mirror of local cache directories on a mounted
+network volume and reconciles the two on each use: it restores cached files on
+cold start (`hydrate`) and syncs new downloads back afterward (`sync`). Wrapping
+a model load in `with VolumeCache(...)` turns a repeated multi-GB download into a
+one-time cost per endpoint.
+
+Why not just point HF_HOME at the volume? Because then every read hits the
+network mount. Here the Hugging Face cache stays on fast local disk and
+VolumeCache mirrors it to the volume, so inference reads are local while cold
+starts stay warm.
+
+Requirements to run:
+- Attach a network volume to the endpoint (mounted at /runpod-volume).
+- RUNPOD_ENDPOINT_ID is set automatically on Runpod serverless (it scopes the
+  cache per endpoint). Without a mounted volume or endpoint id, VolumeCache is a
+  safe no-op and the handler still works.
+- pip install "transformers" "torch" (the model library used below).
+
+Local test:
+    python volume_cache_handler.py --rp_serve_api
+"""
+
+import os
+
+import runpod
+from runpod.serverless import VolumeCache
+
+# Hugging Face caches models here by default. Keep it on local disk and let
+# VolumeCache mirror it to the network volume.
+HF_CACHE = os.environ.get("HF_HOME", os.path.expanduser("~/.cache/huggingface"))
+
+# Hydrate the cache from the volume before loading, then sync new files back on
+# exit. On a warm endpoint the model is already local and nothing is downloaded.
+with VolumeCache(dirs=[HF_CACHE]):
+    from transformers import pipeline
+
+    classifier = pipeline("sentiment-analysis")
+
+
+def handler(job):
+    """Classify the sentiment of an input string."""
+    text = job["input"].get("text", "")
+    return {"input": text, "predictions": classifier(text)}
+
+
+runpod.serverless.start({"handler": handler})

runpod/serverless/__init__.py

@@ -17,12 +17,14 @@
 from .modules.rp_logger import RunPodLogger
 from .modules.rp_progress import progress_update
 from .modules.rp_fitness import register_fitness_check
+from .utils.rp_volume_cache import VolumeCache
 
 __all__ = [
     "start",
     "progress_update",
     "register_fitness_check",
-    "runpod_version"
+    "runpod_version",
+    "VolumeCache",
 ]
 
 log = RunPodLogger()

runpod/serverless/utils/__init__.py

@@ -2,9 +2,11 @@
 
 from .rp_download import download_files_from_urls
 from .rp_upload import upload_file_to_bucket, upload_in_memory_object
+from .rp_volume_cache import VolumeCache
 
 __all__ = [
     "download_files_from_urls",
     "upload_file_to_bucket",
-    "upload_in_memory_object"
+    "upload_in_memory_object",
+    "VolumeCache",
 ]