Serverless GPU
Modal
Serverless containers and GPUs billed for what you run — strong fit for Python inference jobs and bursty workloads without managing VMs.
Product facts checked October 2, 2026Editorial scores last reviewed September 3, 2026
Some outbound links use a first-party redirect hop for click counting. Commission is only claimed when a partner programme is active for that specific link — most vendor hops here are not paid placements. Affiliate disclosure.
Marketplace referrals (for example RunPod or Vast) may return credits or kickbacks when a programme is active — that is a material connection even when it is not a cash CPA.
Best for
Developers who want GPU code as functions with autoscaling
Watch out
You can choose a specific supported GPU type, but containers and scaling remain managed. Warm idle containers consume billable resources. Starter compute credits do not cover Shared Endpoints tokens.
| Billing | Functions: per-second CPU, memory and GPU resources; Shared Endpoints: per-token. Storage and network egress have separate allowances and rates. |
|---|---|
| GPU choice | Choose supported GPU types through the function GPU setting, including T4, L4, A10, L40S, A100, RTX PRO 6000, H100, H200, B200 and B300; allocation and multi-GPU limits vary. |
| Cold start | Can be noticeable on cold containers; benchmark the real image and model. |
| Model access | Package models in images or pull at runtime |
Sources for the dated product and announcement claims
- Supported GPU types and allocation limits — source checked October 2, 2026
- modal official product and billing documentation — source checked October 2, 2026
- modal official product and billing documentation — source checked October 2, 2026
- modal official product and billing documentation — source checked October 2, 2026
Fit detail
When Modal is the right shortlist — and when it is not
Use these lists to kill bad comparisons early, before you compare logos.
Ideal for
- Python inference jobs that spike and then go quiet
- Teams that want functions + GPUs without VM babysitting
- Pipelines that already look like jobs, not long-lived pods
Usually not ideal for
- Requiring exclusive ownership or SSH administration of a particular physical host
- First requests that cannot tolerate container/model loading without paid warm capacity
- Container workloads that do not fit Modal's Python deployment interface
Spend & ops
How cost behaves — and what breaks first
No invented $/hour quotes. These notes explain the failure modes that show up on the first real invoice.
Cost mental model
Function compute is billed by second during startup/load, processing, and the configured idle window (60 seconds by default after last input); it stops billing after scale-to-zero. Volumes/storage are separate. Shared Endpoints use per-token pricing and are a different product.
Ops notes
- Measure first-request latency using the actual image and weights; the default maximum idle window is 60 seconds and is configurable.
- Minimum or buffer containers and allocated idle GPUs/memory remain billable. Set warm capacity from the latency target.
- Starter includes $30/month of compute; Shared Endpoints tokens have been billed from the first request since September 1, 2026. Volumes include 1 TiB/month free; egress allowances depend on plan.
Fit scores
Modal on the decision axes
Same editorial scale as the hub chart — useful for shortlists, not as a price quote.
Modal
- GPU choice6/10
- Time-to-serving (editorial)7/10
- Price clarity7/10
- Production ops8/10
- Open-model breadth7/10
Among all 16 providers
GPU choice
Pick exact GPUs
Time-to-serving (editorial)
Warm-path fit
Price clarity
Easy to forecast
Production ops
Less DIY ops
Open-model breadth
Catalog depth
Scores are editorial planning ratings (1–10) for product shape — not published $/hour quotes or vendor SLAs. Verify current pricing on each provider site.
Compare
Modal vs alternatives
Open a head-to-head when you are deciding between product shapes.
Related
Keep the decision attached to the rest of the stack
Cloud rental is one path. Local VRAM and open-weight fit still matter.