#Benchmark Specifications

#Test Conditions

• Development Board: RDK S600

• Operating Environment: Linux

#Metrics and Statistics

• model: Name of the model.

• BPU core num: Number of BPU (Brain Processing Unit) cores utilized during model execution.

• qtype: Quantization precision of the model.

• max context: Maximum cumulative token sequence length the model can handle.

• TTFT(ms): Time To First Token; the latency for generating the first token.

• Decode(TPS): Tokens Per Second; the generation rate during the decoding stage.

• memory(GB): Memory usage of the model.