Article Source

• Exploring ML Model Serving with KServe

Exploring ML Model Serving with KServe

• Exploring ML Model Serving with KServe (with fun drawings) - Alexa Nicole Griffith, Bloomberg

Abstract

KServe (formerly known as KFServing) provides an easy-to-use platform for deploying machine learning (ML) models. KServe is built on top of Kubernetes and provides performant, high abstraction interfaces that allow data scientists to spend more time focusing on building new models, and less time worrying about the underlying infrastructure. This open source project provides a simple, pluggable solution for common infrastructure issues with inference models, like GPU scaling and ModelMesh serving for high volume/density use cases. From the perspective of an eager engineer new to the KServe community, we will explore the KServe features that solve common issues for engineers and data scientists who are interested in or responsible for machine learning model deployment. Expect to learn about KServe’s fundamental offerings, like out-of-the box model serving and monitoring, and its exciting new, advanced functionalities, such as its inference graph capabilities and ModelMesh features. We will discuss the host of new features added to the project since its publication in 2019 and also outline KServe’s roadmap as it moves forward towards its v1.0 release.