Automated Machine Learning (AutoML) Pipelines Enabled by Microservices Architecture
Keywords:
AutoML, microservices, pipelines, MLOps, orchestration, serverless, automationSynopsis
Purpose – This paper explores the integration of Automated Machine Learning (AutoML) pipelines with microservices architecture to enhance flexibility, scalability, and automation in machine learning operations.
Design/methodology/approach – A conceptual synthesis of key research from 10 papers is presented, examining modular design, orchestration mechanisms, and serverless deployment strategies.
Findings – Microservices-based AutoML pipelines offer improved component reuse, scalability, and continuous deployment potential. They reduce coupling between data preprocessing, model training, and inference stages.
Practical implications – Enterprises can integrate AutoML capabilities within production systems more efficiently by leveraging containerized services and orchestration tools.
Originality/value – This work synthesizes early architectural patterns and implementation cases that demonstrate how microservices enhance AutoML effectiveness, paving the way for robust MLOps infrastructures.
References
(1) Kamath, P., Singh, A., & Dutta, D. (2018). AMLA: an AutoML frAmework for Neural Network Design. Automatic Machine Learning Workshop at ICML. http://pkamath.com/publications/papers/amla_automl18.pdf
(2) Gummadi, V. P. K. (2019). Microservices architecture with APIs: Design, implementation, and MuleSoft integration. Journal of Electrical Systems, 15(4), 130–134. https://doi.org/10.52783/jes.9328
(3) Jonnakuti, S. (2019). Serverless AutoML: Automated Model Selection with Cloud Functions. Academia.edu. https://www.academia.edu/download/123132455/Serverless_AutoML_April_2019.pdf
(4) Sinha, R. (2017). Automation of Data Pipelines in Machine Learning Workflows. Int. J. of AI and Machine Learning, 2(4), 33–44. https://itaimle.com/index.php/ijaiml/article/download/104/177
(5) Aderaldo, C. M., Mendonça, N. C., & Pahl, C. (2017). Benchmark Requirements for Microservices Architecture Research. IEEE Software, 34(3), 78–86. https://ieeexplore.ieee.org/document/7968049
(6) Sriraman, A., & Wenisch, T. F. (2018). µTune: Auto-Tuned Threading for OLDI Microservices. OSDI’18. https://www.usenix.org/conference/osdi18/presentation/sriraman
(7) Felstaine, E., & Hermoni, O. (2018). Machine Learning, Containers, Cloud Natives, and Microservices. In AI for Autonomous Networks (pp. 115-130). CRC Press. https://www.taylorfrancis.com
(8) Pham, A. (2018). Building Continuous Delivery Pipeline for Microservices. Theseus.fi. https://www.theseus.fi/handle/10024/145611
(9) Mendonça, N. C., Jamshidi, P., Garlan, D., & Pahl, C. (2019). Developing Self-Adaptive Microservice Systems. IEEE Software, 36(2), 70–76. https://ieeexplore.ieee.org/document/8913688
(10) Micheal, L. (2019). Hybrid Cloud Architectures for Scalable Serverless AutoML Workflows. ResearchGate. https://www.researchgate.net/publication/394276648
(11) El-Assady, M., Jentner, W., Sperrle, F., et al. (2019). lingvis.io – A Linguistic Visual Analytics Framework. ACL 2019. https://aclanthology.org/P19-3003/
Published
Series
Categories
License
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
