English

LLM Benchmark

model	BPU core num	qtype	max context	TTFT (ms)	Decode (TPS)	memory (GB)
DeepSeek-R1-Distill-Qwen-1.5B	prefill 2 decode 2	prefill w4 decode w4	4096	68.9	92.4	2.2
Qwen3-0.6B	prefill 4 decode 4	prefill w8 decode w8	4096	75.4	92.9	3.0
Qwen3-1.7B	prefill 4 decode 4	prefill w4 decode w4	4096	91.2	75.0	3.7
Qwen3-4B	prefill 4 decode 4	prefill w4 decode w4	4096	232.1	45.8	6.6
Qwen3-4B	prefill 4 decode 4	prefill w8 decode w8	4096	235.3	32.3	8.3
Qwen3-8B	prefill 4 decode 4	prefill w4 decode w4	4096	283.6	31.4	9.1

LLM Development Guide