kfserving

Serving 고려사항

1. Cost
   • Resource, Scale
2. Monitoring
   • health, performance, trace, log, prediction, anomaly dectection
3. Rollouts
   • Rollout, Rollback safe
4. Protocal
   • HTTP, GRPC, Kafka
5. Prediction
   • Realtime, Batch
6. Frameworks
   • Tensorflow, Pytorch, XGBoost

Inference Service 기능 (predictor, explainer, outlier detector)

Inference Service 구조 (kfserving 0.5)

Multi-model serving (v2)