Server Inference

Triton-inference-server

ONNX Runtime

https://github.com/microsoft/onnxruntime-inference-examples

https://github.com/microsoft/DeepSpeed-MII

https://github.com/microsoft/onnx-script