Beyond Model Accuracy: Evaluating Robustness and Generalization in Transfer Learning Models Across Heterogeneous Data Environments
Keywords:
Transfer Learning, Robustness, Generalization, Domain Adaptation, Heterogeneous Data, Model Evaluation, Cross-Domain, Non-IID Data, Performance Metrics, Machine LearningSynopsis
As machine learning (ML) applications increasingly extend to diverse domains, transfer learning (TL) has emerged as a powerful paradigm for adapting pre-trained models to novel, low-resource environments. However, relying solely on model accuracy presents an incomplete picture, especially in heterogeneous data scenarios. This paper explores alternative evaluation dimensions — robustness, generalization, and resilience — across domain-shifted and non-iid datasets. We analyze state-of-the-art approaches, construct comparative metrics, and propose a visual framework to assess TL performance beyond mere accuracy. Our results indicate that high accuracy does not guarantee consistent generalization or robustness under real-world variability, prompting a re-evaluation of performance metrics in TL.
References
[1] Pan, S. J., & Yang, Q. (2010). A survey on transfer learning. IEEE Transactions on Knowledge and Data Engineering, 22(10), 1345–1359.
[2] Vaddepalli, R.K. (2023). How Culture Shapes Trust in AI Surveillance: A Global Look at Retail Security Perceptions. Journal of Scientific and Engineering Research, 10(11), 217–225.
[3] Yosinski, J., Clune, J., Bengio, Y., & Lipson, H. (2014). How transferable are features in deep neural networks? Advances in Neural Information Processing Systems, 27.
[4] Arjovsky, M., Bottou, L., Gulrajani, I., & Lopez-Paz, D. (2019). Invariant Risk Minimization. arXiv preprint arXiv:1907.02893.
[5] Vaddepalli, R.K. (2025). Smart Governance for AI: Can Metadata Automation Keep Up with Real-Time ML Pipelines? International Journal of Artificial Intelligence for Decision Support and Machine Learning (IJAIDSML), 6(2), 119–124.
[6] Ganin, Y., & Lempitsky, V. (2015). Unsupervised domain adaptation by backpropagation. International Conference on Machine Learning.
[7] Li, Y., Swersky, K., & Zemel, R. (2018). Learning invariant representations with adversarial training. arXiv preprint arXiv:1801.01986.
[8] Vaddepalli, R.K. (2024). Data Versioning for Iterative Refinement: Adapting ML Experiment Tracking Tools for Data-Centric AI Pipelines. International Journal of AI, Big Data, Computational and Management Studies, 5(3), 129–137. https://doi.org/10.63282/3050-9416.IJAIBDCMS-V5I3P113
[9] Zhao, H., Zhang, S., Wu, G., Moura, J. M., Costeira, J. P., & Gordon, G. J. (2020). Domain generalization via entropy regularization. Advances in Neural Information Processing Systems, 33.
[10] Vaddepalli, R.K. (2021). Who’s Responsible When Algorithms Fail? Comparing Accountability Under GDPR, CCPA, and Other Data Laws. IACSE - International Journal of Computer Technology (IACSE-IJCT), 2(1), 8–23.
[11] Chen, T., Kornblith, S., Norouzi, M., & Hinton, G. (2020). A simple framework for contrastive learning. ICML.
[12] Madry, A., Makelov, A., Schmidt, L., Tsipras, D., & Vladu, A. (2018). Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv:1706.06083.
[13] Torrey, L., & Shavlik, J. (2010). Transfer learning. In Handbook of Research on Machine Learning Applications and Trends: Algorithms, Methods, and Techniques (pp. 242–264).
[14] Vaddepalli, R.K. (2022). Streaming vs. Batch at Scale: How Snowflake’s Real-Time Processing Stacks Up Against On-Premises Data Warehouses. ISCSITR - International Journal of Cloud Computing (ISCSITR-IJCC), 3(1), 9–26. https://doi.org/10.63397/ISCSITR-IJCC_2022_03_01_002
[15] Sun, B., & Saenko, K. (2016). Deep coral: Correlation alignment for deep domain adaptation. European Conference on Computer Vision
Published
Series
License
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
