A MICROSERVICES-ORIENTED FRAMEWORK FOR DEPLOYING AND MANAGING MACHINE LEARNING MODELS IN DISTRIBUTED ENVIRONMENTS

Mauro Capmany

A MICROSERVICES-ORIENTED FRAMEWORK FOR DEPLOYING AND MANAGING MACHINE LEARNING MODELS IN DISTRIBUTED ENVIRONMENTS

Authors

Mauro Capmany

Machine Learning Platform Engineer, Germany.

Keywords:

Machine Learning Deployment, Microservices Architecture, Distributed Systems, Kubernetes, ML Lifecycle Management, Model Serving

Synopsis

Purpose: This study presents a comprehensive framework leveraging microservices for deploying and managing machine learning models in distributed environments. It addresses the scalability, interoperability, and dynamic orchestration challenges that arise in production-level ML model deployment.

Design/methodology/approach: The framework is based on containerized microservices integrated with orchestration tools like Kubernetes, enabling modular deployment and lifecycle management of ML models. A layered architecture is proposed with dynamic load balancing, logging, monitoring, and model versioning features.

Findings: The microservices architecture offers increased flexibility, scalability, and fault isolation. Comparative evaluation with monolithic ML deployment shows up to 35% reduction in deployment time and improved resilience under failure scenarios.

Practical implications: Organizations aiming to industrialize ML pipelines can utilize this framework to ensure continuous integration, testing, deployment, and rollback capabilities, especially in multi-cloud or hybrid environments.

Originality/value: While existing studies focus on either containerization or model orchestration, this work integrates both with a microservices-first paradigm and validates the approach using real-world scenarios.

References

[1] Abdullah, M., Iqbal, W., & Erradi, A. (2019). Unsupervised learning approach for web application auto-decomposition into microservices. Journal of Systems and Software, 152, 243–257. https://doi.org/10.1016/j.jss.2019.03.043

[2] Chang, H., Kodialam, M., & Lakshman, T.V. (2019). Microservice fingerprinting and classification using machine learning. IEEE ICNP. https://ieeexplore.ieee.org/document/8888077

[3] Gummadi, V. P. K. (2019). Microservices architecture with APIs: Design, implementation, and MuleSoft integration. Journal of Electrical Systems, 15(4), 130–134. https://doi.org/10.52783/jes.9328

[4] Felstaine, E., & Hermoni, O. (2018). Machine Learning, Containers, Cloud Natives, and Microservices. In AI for Autonomous Networks. https://www.taylorfrancis.com

[5] Joseph, C.T., & Martin, J.P. (2019). Fuzzy RL-based microservice allocation. TENCON IEEE Conference. https://ieeexplore.ieee.org/document/8929586

[6] Magableh, B., & Almiani, M. (2019). Deep Q-learning for adaptive distributed microservices. IEEE Access. https://arrow.tudublin.ie

[7] Gummadi, V. P. K. (2020). API design and implementation: RAML and OpenAPI specification. Journal of Electrical Systems, 16(4). https://doi.org/10.52783/jes.9329

[8] Reijonen, J. (2018). Decentralized ML for autonomous ships. University of Vaasa. https://osuva.uwasa.fi

[9] Ribeiro, J.L., Figueredo, M., & Araujo, A. (2019). A microservice based architecture for ML deployment. IEEE IoT Conference. https://ieeexplore.ieee.org/document/9071708

[10] Rodríguez-Gracia, D., Piedra-Fernández, J.A., & Iribarne, L. (2019). Microservices and ML for adaptive green buildings. Sustainability, 11(16), 4320. https://www.mdpi.com/2071-1050/11/16/4320

[11] Shahoud, S., Gunnarsdottir, S., & Khalloof, H. (2019). Managing ML tasks using microservices in Big Data. ACM Conference on Big Data. https://dl.acm.org/doi/10.1145/3297662.3365807

[12] Yang, Z., Nguyen, P., & Jin, H. (2019). MIRAS: Deep RL for microservice workflows. IEEE Distributed Computing Systems. https://ieeexplore.ieee.org/document/8885267