Home/Online Tools/GPU Matcher

GPU Matcher

Recommend suitable GPU configurations based on model size, quantization, and deployment constraints.

Model-GPU Matcher

Recommend more suitable GPU options based on model size, quantization, context, and concurrency, and determine whether single-GPU deployment is feasible.

ModelDeepSeek-R1DeepSeek-R1-0528DeepSeek-R1-0528-Qwen3-8BDeepSeek-R1-Distill-Llama-70BDeepSeek-R1-Distill-Qwen-14BDeepSeek-R1-Distill-Qwen-32BDeepSeek-V3DeepSeek-V3-0324DeepSeek-V3.1DeepSeek-V3.2DeepSeek-V3.2-ExpDeepSeek-V3.2-SpecialeDeepSeek-V4-FlashDeepSeek-V4-ProMiniMax-M1-80k-hfMiniMax-M2MiniMax-M2.1MiniMax-M2.5MiniMax-M2.7MiniMax-Text-01MiniMax-Text-01-hfKimi-K2-InstructKimi-K2-Instruct-0905Kimi-K2-ThinkingKimi-Linear-48B-A3B-InstructMoonlight-16B-A3BMoonlight-16B-A3B-InstructMiniCPM-SALAQwen2.5-7B-InstructQwen3-0.6BQwen3-1.7BQwen3-14BQwen3-235B-A22BQwen3-235B-A22B-Instruct-2507Qwen3-235B-A22B-Thinking-2507Qwen3-30B-A3BQwen3-30B-A3B-Instruct-2507Qwen3-30B-A3B-Thinking-2507Qwen3-32BQwen3-4BQwen3-4B-Instruct-2507Qwen3-4B-Thinking-2507Qwen3-8BQwen3-Coder-30B-A3B-InstructQwen3-Coder-480B-A35B-InstructQwen3-Coder-NextQwen3-Next-80B-A3B-InstructQwen3-Next-80B-A3B-ThinkingQwen3.5-0.8BQwen3.5-122B-A10BQwen3.5-27BQwen3.5-2BQwen3.5-35B-A3BQwen3.5-397B-A17BQwen3.5-4BQwen3.5-9BQwen3.6-27BQwen3.6-35B-A3BWebWorld-14BWebWorld-32BWebWorld-8BStep-3.5-FlashMiMo-7B-RL-0530MiMo-V2-FlashMiMo-V2.5-ProGLM-4-32B-0414GLM-4-9B-0414GLM-4.5GLM-4.5-AirGLM-4.6GLM-4.7GLM-4.7-FlashGLM-5GLM-5.1GLM-Z1-32B-0414GLM-Z1-9B-0414GLM-Z1-Rumination-32B-0414QuantizationINT4INT8FP8FP16BF16Prompt tokensMax output tokensBatch sizeConcurrencySafety marginGPU categoryAllConsumerWorkstationDatacenter

Enable lower runtime / KV management overhead for mainstream inference runtimes

Recommended total VRAM456.22 GiB

Per-GPU VRAM456.22 GiB

Model size685B / A37B

Priority recommendations

Sorted by quantization support, required GPU count, deployment tier, and recommendation priority.

NVIDIA H200 141GB

Datacenter

Production · 141 GB

4 GPUs

Estimated minimum requirement: 4 identical GPUs for tensor parallel deployment.

NVIDIA A100 80GB

Datacenter

Production · 80 GB

7 GPUs

Estimated minimum requirement: 7 identical GPUs for tensor parallel deployment.

NVIDIA H100 80GB

Datacenter

Production · 80 GB

7 GPUs

Estimated minimum requirement: 7 identical GPUs for tensor parallel deployment.

NVIDIA A40 48GB

Datacenter

Production · 48 GB

11 GPUs

Estimated minimum requirement: 11 identical GPUs for tensor parallel deployment.

NVIDIA L40 48GB

Datacenter

Production · 48 GB

11 GPUs

Estimated minimum requirement: 11 identical GPUs for tensor parallel deployment.

NVIDIA L40S 48GB

Datacenter

Production · 48 GB

11 GPUs

Estimated minimum requirement: 11 identical GPUs for tensor parallel deployment.

					Deployment notes
NVIDIA H200 141GBDatacenter · Production	141 GB	INT4	4 GPUs	Multi GPU	Use 4 GPUs with tensor parallelism and validate further with your target context and concurrency.
NVIDIA A100 80GBDatacenter · Production	80 GB	INT4	7 GPUs	Multi GPU	Use 7 GPUs with tensor parallelism and validate further with your target context and concurrency.
NVIDIA H100 80GBDatacenter · Production	80 GB	INT4	7 GPUs	Multi GPU	Use 7 GPUs with tensor parallelism and validate further with your target context and concurrency.
NVIDIA A40 48GBDatacenter · Production	48 GB	INT4	11 GPUs	Multi GPU	Use 11 GPUs with tensor parallelism and validate further with your target context and concurrency.
NVIDIA L40 48GBDatacenter · Production	48 GB	INT4	11 GPUs	Multi GPU	Use 11 GPUs with tensor parallelism and validate further with your target context and concurrency.
NVIDIA L40S 48GBDatacenter · Production	48 GB	INT4	11 GPUs	Multi GPU	Use 11 GPUs with tensor parallelism and validate further with your target context and concurrency.
NVIDIA RTX A6000 48GBWorkstation · Department	48 GB	INT4	11 GPUs	Multi GPU	Use 11 GPUs with tensor parallelism and validate further with your target context and concurrency.
NVIDIA L20 48GBDatacenter · Department	48 GB	INT4	11 GPUs	Multi GPU	Use 11 GPUs with tensor parallelism and validate further with your target context and concurrency.
NVIDIA RTX 6000 Ada 48GBWorkstation · Department	48 GB	INT4	11 GPUs	Multi GPU	Use 11 GPUs with tensor parallelism and validate further with your target context and concurrency.
NVIDIA A100 40GBDatacenter · Production	40 GB	INT4	13 GPUs	Multi GPU	Use 13 GPUs with tensor parallelism and validate further with your target context and concurrency.
GeForce RTX 5090 32GBConsumer · Lab	32 GB	INT4	17 GPUs	Multi GPU	Use 17 GPUs with tensor parallelism and validate further with your target context and concurrency.
NVIDIA A10 24GBDatacenter · Production	24 GB	INT4	24 GPUs	Multi GPU	Use 24 GPUs with tensor parallelism and validate further with your target context and concurrency.
NVIDIA A30 24GBDatacenter · Production	24 GB	INT4	24 GPUs	Multi GPU	Use 24 GPUs with tensor parallelism and validate further with your target context and concurrency.
NVIDIA L4 24GBDatacenter · Production	24 GB	INT4	24 GPUs	Multi GPU	Use 24 GPUs with tensor parallelism and validate further with your target context and concurrency.
NVIDIA TITAN RTX 24GBConsumer · Lab	24 GB	INT4	24 GPUs	Multi GPU	Use 24 GPUs with tensor parallelism and validate further with your target context and concurrency.
GeForce RTX 3090 24GBConsumer · Lab	24 GB	INT4	24 GPUs	Multi GPU	Use 24 GPUs with tensor parallelism and validate further with your target context and concurrency.
GeForce RTX 4090 24GBConsumer · Lab	24 GB	INT4	24 GPUs	Multi GPU	Use 24 GPUs with tensor parallelism and validate further with your target context and concurrency.
NVIDIA T4 16GBDatacenter · Production	16 GB	INT4	-	Multi GPU	Not fit within 32-way tensor parallelism; validate model sharding and offload manually.
NVIDIA A2 16GBDatacenter · Production	16 GB	INT4	-	Multi GPU	Not fit within 32-way tensor parallelism; validate model sharding and offload manually.
GeForce RTX 2060 6GBConsumer · Lab	6 GB	INT4	-	Multi GPU	Not fit within 32-way tensor parallelism; validate model sharding and offload manually.
GeForce RTX 2060 SUPER 8GBConsumer · Lab	8 GB	INT4	-	Multi GPU	Not fit within 32-way tensor parallelism; validate model sharding and offload manually.
GeForce RTX 2070 8GBConsumer · Lab	8 GB	INT4	-	Multi GPU	Not fit within 32-way tensor parallelism; validate model sharding and offload manually.
GeForce RTX 2070 SUPER 8GBConsumer · Lab	8 GB	INT4	-	Multi GPU	Not fit within 32-way tensor parallelism; validate model sharding and offload manually.
GeForce RTX 2080 8GBConsumer · Lab	8 GB	INT4	-	Multi GPU	Not fit within 32-way tensor parallelism; validate model sharding and offload manually.
GeForce RTX 2080 SUPER 8GBConsumer · Lab	8 GB	INT4	-	Multi GPU	Not fit within 32-way tensor parallelism; validate model sharding and offload manually.
GeForce RTX 5050 8GBConsumer · Lab	8 GB	INT4	-	Multi GPU	Not fit within 32-way tensor parallelism; validate model sharding and offload manually.
GeForce RTX 5060 8GBConsumer · Lab	8 GB	INT4	-	Multi GPU	Not fit within 32-way tensor parallelism; validate model sharding and offload manually.
GeForce RTX 2080 Ti 11GBConsumer · Lab	11 GB	INT4	-	Multi GPU	Not fit within 32-way tensor parallelism; validate model sharding and offload manually.
GeForce RTX 3060 12GBConsumer · Lab	12 GB	INT4	-	Multi GPU	Not fit within 32-way tensor parallelism; validate model sharding and offload manually.
GeForce RTX 5070 12GBConsumer · Lab	12 GB	INT4	-	Multi GPU	Not fit within 32-way tensor parallelism; validate model sharding and offload manually.
GeForce RTX 5060 Ti 16GBConsumer · Lab	16 GB	INT4	-	Multi GPU	Not fit within 32-way tensor parallelism; validate model sharding and offload manually.
GeForce RTX 5070 Ti 16GBConsumer · Lab	16 GB	INT4	-	Multi GPU	Not fit within 32-way tensor parallelism; validate model sharding and offload manually.
GeForce RTX 5080 16GBConsumer · Lab	16 GB	INT4	-	Multi GPU	Not fit within 32-way tensor parallelism; validate model sharding and offload manually.
GeForce RTX 4060 Ti 16GBConsumer · Lab	16 GB	INT4	-	Multi GPU	Not fit within 32-way tensor parallelism; validate model sharding and offload manually.
GeForce RTX 4080 16GBConsumer · Lab	16 GB	INT4	-	Multi GPU	Not fit within 32-way tensor parallelism; validate model sharding and offload manually.