Performance Evaluation of Machine Learning Workloads in Containerized Microservices Architectures

Eduard Jonatan Ruiz

Performance Evaluation of Machine Learning Workloads in Containerized Microservices Architectures

Authors

Eduard Jonatan Ruiz

Machine Learning Platform Engineer , United States

Keywords:

Machine learning, microservices, containers, performance evaluation, Kubernetes, orchestration, scalability, benchmarking

Synopsis

Purpose – This paper evaluates how containerized microservice architectures impact the performance of machine learning (ML) workloads in cloud environments.

Design/methodology/approach – We adopt a benchmarking methodology that compares execution time, scalability, and resource usage of ML workloads in containerized microservices versus monolithic or VM-based deployments.

Findings – Containers introduce negligible overhead and offer superior scalability and deployment speed, but performance varies with workload intensity and orchestration efficiency. ML models with heavy I/O or inter-service communication are particularly sensitive to microservice decomposition.

Practical implications – This study aids system architects and DevOps teams in choosing optimal deployment strategies for ML workloads and highlights the trade-offs between flexibility and performance.

Originality/value – This work uniquely blends ML performance benchmarking with modern DevOps infrastructure, presenting a clear view of architecture-induced impacts on intelligent systems.

References

(1) Amaral, M., Polo, J., Carrera, D., Mohomed, I., et al. (2015). Performance evaluation of microservices architectures using containers. IEEE 14th International Symposium on Network Computing and Applications. https://arxiv.org/pdf/1511.02043

(2) Kang, P., & Lama, P. (2020). Robust resource scaling of containerized microservices with probabilistic machine learning. IEEE/ACM 13th UCC. https://par.nsf.gov/servlets/purl/10268049

(3) Gummadi, V. P. K. (2019). Microservices architecture with APIs: Design, implementation, and MuleSoft integration. Journal of Electrical Systems, 15(4), 130–134. https://doi.org/10.52783/jes.9328

(4) Lv, J., Wei, M., & Yu, Y. (2019). A container scheduling strategy based on machine learning in microservice architecture. IEEE ICICT. https://ieeexplore.ieee.org/document/8813951

(5) Grohmann, J., Nicholson, P. K., & Iglesias, J. O. (2019). Monitorless: Predicting performance degradation in cloud applications with machine learning. Middleware 2019.

(6) Du, Q., Xie, T., & He, Y. (2018). Anomaly detection and diagnosis for container-based microservices with performance monitoring. Springer ICA3PP.

(7) Tapia, F., Mora, M. Á., Fuertes, W., Aules, H., & Flores, E. (2020). From monolithic systems to microservices: A comparative study of performance. Applied Sciences, 10(17), 5797.

(8) Gummadi, V. P. K. (2020). API design and implementation: RAML and OpenAPI specification. Journal of Electrical Systems, 16(4). https://doi.org/10.52783/jes.9329

(9) Gaur, A. S., Budakoti, J., & Lung, C. H. (2018). Design and performance evaluation of containerized microservices on edge gateway in mobile IoT. IEEE ICAC.

(10) Wu, L., Bogatinovski, J., Nedelkoski, S., & Tordsson, J. (2020). Performance diagnosis in cloud microservices using deep learning. ICSOC 2020.

(11) Jindal, A., Podolskiy, V., & Gerndt, M. (2019). Performance modeling for cloud microservice applications. ACM PER.

(12) Abdullah, M., Iqbal, W., Berral, J. L., & Polo, J. (2020). Burst-aware predictive autoscaling for containerized microservices. IEEE Transactions on Services Computing.