Performance Metrics

All speed test were run on AMD Ryzen Threadripper 3960X using only 1 thread and batch size equal to 1, 16000 sampling rate:

torch.set_num_threads(1) # JIT
session.intra_op_num_threads = 1 # ONNX
session.inter_op_num_threads = 1 # ONNX

RTS - seconds of audio processed per second, real time speed, or inverse RTF

This represents the most common use case, but with batch size > 1 metrics can be even more impressive.