Climate change is a pressing reality, and every step taken to combat it counts as a success. As more data centres are built, their CO₂ emissions continue to rise every year. Smart use of resources is therefore essential to slow this growth.
When a data scientist creates a new machine learning model, the Machine Learning Operations (or MLOps for short) engineer must manage the rest of the model lifecycle. The model needs to be correctly deployed to generate predictions, and they may later be replaced by a newer version or removed entirely if it is no longer needed.
This is where the FAME Smart Deployment module comes in. It is designed to help MLOps engineers manage models efficiently while optimizing data centre resource usage.
FAME Smart Deployment is built on Kubernetes, an open-source platform that ensures applications run reliably across multiple servers or nodes. Kubernetes keeps apps running even if a node fails and can scale resources automatically when more processing power is needed.
The module is structured into four submodules:
- API Server: Prepares everything required to deploy the model.
- Node Resolver: Determines the best node for deployment.
- Model Server: Hosts the model itself and uses MLFlow to store it.
- WebAPP: Provides an intuitive graphical interface to interact with the API Server.
FAME Smart Deployment also integrates seamlessly with Analytics CO₂ Monitoring, another module that tracks CO₂ emissions across countries. Together, these modules simplify model deployment, optimize distributed resources, and reduce the CO₂ emissions generated during model inference.
When an MLOps engineer starts deploying a new model, the API Server handles all steps automatically:
- Checks that the model is ready for deployment.
- Consults the Node Resolver, which collaborates with Analytics CO₂ Monitoring to choose a node in a country with lower emissions and sufficient resources.
- Builds the required Kubernetes resources and verifies their correctness.
- Deploys the model.
All of this happens with a single command from the MLOps engineer. The system then provides feedback on whether the deployment was successful or if any issues occurred during the process. In other words, the Smart Deployment is a transversal technology that optimizes the deployment of any containerized analytic purchased by in the FAME marketplace, thereby closing the MLOps lifecycle with a module that can be purchased and used by any MLOps engineer.
Author: Sergio Martín Hernández (ATOS)
—
Subscribe to our newsletter for the latest updates, and follow FAME on LinkedIn and X to be part of the journey.
More info:
ATOS