Seldon Core

Seldon Core is a Kubernetes-native AI Model Serving & Inference runtime. It loads trained machine learning (ML) and large language model (LLM) weights onto inference servers, exposes REST and gRPC endpoints, and executes those models in production so applications can obtain predictions or generated tokens at scale. The current architecture (Core 2) is bring-your-own-weights serving software that the operator runs on a cluster; it is not a hosted model-lab API and not a training platform.