NVIDIA Triton Inference Server

An open-source inference serving software that lets teams deploy trained AI models from any framework on any GPU or CPU-based infrastructure.