Challenges and Opportunities of Applying Microservices Architecture in Artificial Intelligence and Machine Learning Systems

Karl Marx Celan

Challenges and Opportunities of Applying Microservices Architecture in Artificial Intelligence and Machine Learning Systems

Authors

Karl Marx Celan

AI/ML Systems Architect – Microservices & Scalable Infrastructure, Russia

Keywords:

Microservices Architecture, Artificial Intelligence, Machine Learning, Scalability, Orchestration, System Design

Synopsis

Purpose: This paper explores the application of microservices architecture (MSA) in artificial intelligence (AI) and machine learning (ML) systems, aiming to understand its benefits and limitations in real-world deployments.

Design/methodology/approach: A literature-based approach was used to analyze existing studies up to 2019, focusing on integrating MSA with AI/ML systems. We examined practical implementations and proposed architecture patterns.

Findings: While MSA brings scalability, modularity, and maintainability, challenges like data consistency, orchestration, and increased complexity remain. AI/ML pipelines require orchestration that often conflicts with decentralized microservices logic.

Practical implications: Adopting MSA in AI/ML enables faster development, isolated scaling of components, and team autonomy but demands new tools for monitoring, deployment, and model lifecycle management.

Originality/value: This paper contributes a consolidated view of early academic and industrial attempts to integrate AI/ML workflows with MSA, and presents comparative visual models and tables to summarize findings.

References

[1] Rodríguez-Gracia, D., Piedra-Fernández, J. A., & Iribarne, L. (2019). Microservices and machine learning algorithms for adaptive green buildings. Sustainability, 11(16), 4320. https://www.mdpi.com/2071-1050/11/16/4320

[2] Ribeiro, J. L., Figueredo, M., & Araujo, A. (2019). A microservice based architecture topology for machine learning deployment. IEEE, https://ieeexplore.ieee.org/document/9071708

[3] Gummadi, V. P. K. (2019). Microservices architecture with APIs: Design, implementation, and MuleSoft integration. Journal of Electrical Systems, 15(4), 130–134. https://doi.org/10.52783/jes.9328

[4] Felstaine, E., & Hermoni, O. (2018). Machine Learning, Containers, Cloud Natives, and Microservices. AI for Autonomous Networks, Taylor & Francis.

[5] Pahl, M. O., & Loipfinger, M. (2018). Machine learning as a reusable microservice. IEEE/IFIP NOMS, https://ieeexplore.ieee.org/document/8406165

[6] Bierzynski, K., & Lutskov, P. (2019). Supporting the self-learning of systems at the network edge with microservices. IEEE, https://ieeexplore.ieee.org/document/8727784

[7] Fernández-Garcıa, A. J. (2019). A Recommender System for Smart User Interfaces using ML and Microservices. Torrossa.

[8] Magableh, B., & Almiani, M. (2019). Deep Q Learning for Self Adaptive Distributed Microservices. IEEE Access.

[9] Thompson, A. (2019). Integrating Automation and AI in Microservices Architectures. IJAIML, https://itaimle.com/index.php/ijaiml/article/download/80/203

[10] Lv, J., Wei, M., & Yu, Y. (2019). A container scheduling strategy based on ML in microservices. IEEE, https://ieeexplore.ieee.org/document/8813951

[11] Lee, G. M., Um, T. W., & Choi, J. K. (2018). AI as a Microservice (AIMS) over 5G networks. IEEE, https://ieeexplore.ieee.org/document/8597704