Modal offers instant GPU access, sub-second container starts, and native storage, so it's simple to serve low-latency inference, fine-tune models, and access production-ready sandboxes at scale.

Key Capabilities

Functions as the scale unit. @app.function and @app.cls define independently autoscaling workers. Default behavior is scale-to-zero. @modal.concurrent allows multiple in-flight inputs per container. GPU count is gpu="H100:8" (or a fallback list such as gpu=["H100", "A100-40GB:2"]).
Web and event triggers. @modal.fastapi_endpoint, @modal.asgi_app (FastAPI, Starlette, FastHTML), @modal.wsgi_app (Flask, Django), and @modal.web_server(port) expose Functions on the public internet. Cron schedules, Python .remote() / .map(), and proxy-token auth sit on the same Functions. Request bodies up to 4 GiB; WebSocket per RFC 6455.
GPU and CPU pool. Documented GPU strings: T4, L4, A10, L40S, A100 / A100-40GB / A100-80GB, RTX-PRO-6000, H100 / H100!, H200, B200 / B200+, B300. H100/H200 on Modal are SXM. Single-node Functions attach up to eight GPUs on B300, B200, H100, A100, L4, T4, and L40S (up to four on A10). Multi-node training with InfiniBand is a private-beta path, not general availability in the public GPU guide.
Images, storage, and platform primitives. Images are built in Python (debian_slim(), uv_pip_install, snapshotting). modal.Volume, cloud-bucket mounts, Queues, Dicts, Secrets, tunnels, and static-IP proxies are the bundled data/network layer. Memory snapshotting and a lazy filesystem are how Modal keeps container start times short when models and engines are large.
Inference product surface (serving secondary). /products/inference and modal endpoint create --model … deploy open-weight or custom checkpoints (Hugging Face or a Volume) onto Modal’s runtime, with token streaming and dynamic/offline batching. The catalog includes hosted variants of Qwen, Gemma, DeepSeek, GLM, Kimi, GPT-OSS, and Nemotron. This is BYO-engine serving on FaaS (vLLM and similar run in the user’s image), not a proprietary inference server.
Training, batch, sandboxes, notebooks. Training is the same Function decorator with longer timeouts and more GPUs: SFT, LoRA, full fine-tunes, RL rollouts, hyperparameter sweeps; user-chosen stacks (PyTorch, Axolotl, Unsloth, TRL, torchtune) and user-chosen experiment trackers (W&B, TensorBoard). Sandboxes are isolated containers for untrusted or model-generated code, coding agents, and RL environments, with snapshotting, Volume mounts, and readiness probes. Notebooks are GPU-backed collaborative notebooks on the same runtime.
Operations and tenancy. Dashboard logs/metrics per Function, sandbox, and container. Starter / Team / Enterprise plans (free tier includes a modest monthly compute credit; Team is a few hundred dollars per month plus usage; Enterprise is custom). SOC 2, HIPAA on Enterprise, data-residency controls, Okta SSO, audit logs, environment budgets. AWS and GCP marketplace for committed-spend pass-through.

Audience & Use Cases

Python ML/platform engineers who want GPU and CPU capacity without owning Kubernetes or a cloud account. Typical jobs: low-latency LLM or multimodal endpoints; bursty batch embeddings, evals, and transcription; LoRA/SFT jobs that should die when the run ends; isolated execution for coding agents and RL; one-off GPU notebooks. JS/TypeScript and Go clients can invoke deployed Functions and Sandboxes; Function bodies themselves are Python.

Modal

What is Modal?

Categories & Use Cases