Physically interpretable multimodal prototypical networks for seismo-volcanic signal classification via few-shot learning
Introduction: This article stems from the doctoral thesis “Characterization and Classification of Seismo-Volcanic Signals Using Artificial Intelligence Techniques and Data Analytics” at the University of Cauca (2022–2026).
Problem: Real-time seismic monitoring requires a rapid and accurate classification process for various types of events to assess volcanic activity and thus activate possible early warnings. However, this field faces a fundamental operational bottleneck: in a newly instrumented volcano, monitoring networks record different types of earthquakes with a lower volume of data than required by standard deep learning classifiers, which need hundreds or thousands of labeled samples to achieve adequate performance.
Objective: To overcome the data scarcity problem, we developed and evaluated a Multimodal Prototypical Network (MPN) designed to maintain good classification performance even when labeled data are extremely limited.
Methodology: Our approach merges CLIP ViT-B/32 visual embeddings, extracted directly from seismic spectrograms, with a 19-dimensional tabular descriptor vector based on the physical characteristics of the signals under study. This multimodal input is processed using a 5-fold ensemble strategy driven by focal loss. The model was rigorously evaluated using the Chiles-Cerro Negro Volcanic Complex (CCNVC) dataset, categorizing three distinct signal classes: Volcanic-Tectonic (VT), Long-Period (LP), and Very-Long-Period (VLP) events.
Results: Under full supervision (6,366 labeled samples), the MPN achieved exceptional performance with a 98.73% balanced accuracy and a 99.89% ROC AUC. Crucially, in a few-shot scenario with only 33 labeled waveforms (14 VT, 14 LP, and 5 VLP), the model retained a highly robust 85.27% balanced accuracy and a 95.36% ROC AUC. Furthermore, physical consistency analysis confirmed that the network’s decisions are driven by spectral morphology (amplitude–geometry correlation, r = 0.248) rather than simple amplitude geometry, aligning with the expected physics of volcanic source mechanisms.
Conclusion: These findings establish intermodal transfer from large-scale vision-and-language pretraining as a highly viable and physically consistent strategy for volcanic monitoring environments with a scarcity of labeled data.
Originality: Instead of relying solely on raw waveform statistics or standard deep learning, this work uniquely combines foundation models (CLIP) with strict tabular constraints based on physical features to solve the specific few-shot labeling problem inherent in operational volcanology.
Limitations: While performance with limited labeled data is noteworthy, the absolute lower bound for labeling requirements (e.g., fewer than 5 VLP samples) was not tested. Furthermore, the model’s physical consistency is tied to the specific origin mechanisms of VT, LP, and VLP events in the CCNVC; applying this exact multimodal framework to signals with greater overlap (such as hybrid events or tremors) might require recalibrating the tabular vector based on the physics of those signals.
How to Cite
License
Copyright (c) 2026 Ingeniería Solidaria
This work is licensed under a Creative Commons Attribution 4.0 International License.
Cession of rights and ethical commitment
As the author of the article, I declare that is an original unpublished work exclusively created by me, that it has not been submitted for simultaneous evaluation by another publication and that there is no impediment of any kind for concession of the rights provided for in this contract.
In this sense, I am committed to await the result of the evaluation by the journal Ingeniería Solidaría before considering its submission to another medium; in case the response by that publication is positive, additionally, I am committed to respond for any action involving claims, plagiarism or any other kind of claim that could be made by third parties.
At the same time, as the author or co-author, I declare that I am completely in agreement with the conditions presented in this work and that I cede all patrimonial rights, in other words, regarding reproduction, public communication, distribution, dissemination, transformation, making it available and all forms of exploitation of the work using any medium or procedure, during the term of the legal protection of the work and in every country in the world, to the Universidad Cooperativa de Colombia Press.
[1] J. Martí, “From rest to eruption: How we should anticipate volcanic eruptions,” npj Natural Hazards, vol. 1, Art. no. 31, 2024, doi: https://doi.org/10.1038/s44304-024-00033-8. [Online]. Available: https://www.nature.com/articles/s44304-024-00033-8 DOI: https://doi.org/10.1038/s44304-024-00033-8
[2] K. I. Konstantinou, “A review of the source characteristics and physical mechanisms of very long period (VLP) seismic signals at active volcanoes,” Surveys in Geophysics, vol. 45, no. 1, pp. 117–149, 2024, doi: https://doi.org/10.1007/s10712-023-09800-0. [Online]. Available: https://link.springer.com/article/10.1007/s10712-023-09800-0 DOI: https://doi.org/10.1007/s10712-023-09800-0
[3] B. A. Chouet and R. S. Matoza, “A multi-decadal view of seismic methods for detecting precursors of magma movement and eruption,” Journal of Volcanology and Geothermal Research, vol. 252, pp. 108–175, 2013, doi: https://doi.org/10.1016/j.jvolgeores.2012.11.013. [Online]. Available: https://doi.org/10.1016/j.jvolgeores.2012.11.013 DOI: https://doi.org/10.1016/j.jvolgeores.2012.11.013
[4] I. W. McBrearty and P. Segall, “Deep learning forecasts caldera collapse events at Kīlauea Volcano,” arXiv, Art. no. 2404.19351, 2024, doi: https://doi.org/10.48550/arXiv.2404.19351. [Online]. Available: https://arxiv.org/abs/2404.19351
[5] G. F. Manley, T. A. Mather, D. M. Pyle, D. A. Clifton, M. Rodgers, G. Thompson, and J. M. Londoño, “A deep active learning approach to the automatic classification of volcano-seismic events,” Frontiers in Earth Science, vol. 10, Art. no. 807926, 2022, doi: https://doi.org/10.3389/feart.2022.807926. DOI: https://doi.org/10.3389/feart.2022.807926
[6] J. P. Canario, M. Curilem, G. Fuentealba, G. Acuña, M. Chacón, M. S. Khan, and N. Becerra Yoma, “Adaptive classification using incremental learning for seismic-volcanic signals with concept drift,” Journal of Volcanology and Geothermal Research, vol. 413, Art. no. 107211, 2021, doi: https://doi.org/10.1016/j.jvolgeores.2021.107211. [Online]. Available: https://doi.org/10.1016/j.jvolgeores.2021.107211 DOI: https://doi.org/10.1016/j.jvolgeores.2021.107211
[7] P. Rey-Devesa, J. Carthy, M. Titos, J. M. Ibáñez, and C. Benítez, “Universal machine learning approach to volcanic eruption forecasting using seismic features,” Frontiers in Earth Science, vol. 12, Art. no. 1342468, 2024, doi: https://doi.org/10.3389/feart.2024.1342468. [Online]. Available: https://www.frontiersin.org/articles/10.3389/feart.2024.1342468 DOI: https://doi.org/10.3389/feart.2024.1342468
[8] M. Lara, M. Londoño, G. P. Cortés, and J. F. Pacheco, “The Chiles-Cerro Negro volcanic complex seismic crisis (2013–2015): Evolution and implications for volcanic hazard assessment,” Boletín de Geología, vol. 38, no. 1, pp. 67–84, 2016, doi: https://doi.org/10.18273/re
[9] Servicio Geológico Colombiano, “Boletines mensuales de actividad del Complejo Volcánico Chiles-Cerro Negro,” Bogotá, Colombia: Servicio Geológico Colombiano, 2024. [Online]. Available: https://www2.sgc.gov.co/Publicaciones/Informes%20tcnicos/Forms/AllItems.aspx
[10] J. Carthy, M. Titos, F. Cannavó, L. Zuccarello, and C. Benítez, “Learning feature representations from unlabeled data for volcano-seismic event classification,” Journal of Volcanology and Geothermal Research, vol. 445, Art. no. 107970, 2024, doi: https://doi.org/10.1016/j.jvolgeores.2023.107970. [Online]. Available: https://doi.org/10.1016/j.jvolgeores.2023.107970 DOI: https://doi.org/10.1016/j.jvolgeores.2023.107970
[11] A. Radford et al., “Learning transferable visual models from natural language supervision,” in Proc. 38th International Conference on Machine Learning (ICML), vol. 139, 2021, pp. 8748–8763. [Online]. Available: https://proceedings.mlr.press/v139/radford21a.html
[12] K. He, X. Chen, S. Xie, Y. Li, P. Dollár, and R. Girshick, “Masked autoencoders are scalable vision learners,” in Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022, pp. 16000–16009, doi: https://doi.org/10.1109/CVPR52688.2022.01553. [Online]. Available: https://doi.org/10.1109/CVPR52688.2022.01553 DOI: https://doi.org/10.1109/CVPR52688.2022.01553
[13] Y. Zhao, B. Chai, L. Shuo, Z. Li, H. Wu, and T. Wang, “Few-shot learning for seismic facies segmentation via prototype learning,” Geophysics, vol. 88, no. 3, pp. IM41–IM49, 2023, doi: https://doi.org/10.1190/geo2022-0281.1. [Online]. Available: https://doi.org/10.1190/geo2022-0281.1 DOI: https://doi.org/10.1190/geo2022-0281.1
[14] F. Münchmeyer, A. Bizzarri, J. P. Ampuero, and U. Leser, “Which picker fits my data? A quantitative evaluation of deep learning-based seismic pickers,” Journal of Geophysical Research: Solid Earth, vol. 127, no. 1, Art. no. e2021JB023499, 2022, doi: https://doi.org/10.1029/2021JB023499. DOI: https://doi.org/10.1029/2021JB023499
[15] M. Titos, A. Bueno, L. García, C. Benítez, and J. M. Ibáñez, “A deep neural networks approach to automatic recognition systems for volcano-seismic events,” IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 11, no. 5, pp. 1533–1544, 2018, doi: https://doi.org/10.1109/JSTARS.2018.2803198. DOI: https://doi.org/10.1109/JSTARS.2018.2803198
[16] K. I. Konstantinou and V. Schlindwein, “Nature, wavefield properties and source mechanism of volcanic tremor: A review,” Journal of Volcanology and Geothermal Research, vol. 119, no. 1–4, pp. 161–188, 2003, doi: https://doi.org/10.1016/S0377-0273(02)00311-6. DOI: https://doi.org/10.1016/S0377-0273(02)00311-6
[17] M. Curilem, F. Huenupan, D. Beltrán, C. San Martín, G. Fuentealba, L. Franco, C. Cardona, G. Acuña, M. Chacón, M. S. Khan, and N. Becerra Yoma, “Pattern recognition applied to seismic signals of Llaima Volcano (Chile): An evaluation of station-dependent classifiers,” Journal of Volcanology and Geothermal Research, vol. 315, pp. 15–27, 2016, doi: https://doi.org/10.1016/j.jvolgeores.2016.02.006. [Online]. Available: https://doi.org/10.1016/j.jvolgeores.2016.02.006 DOI: https://doi.org/10.1016/j.jvolgeores.2016.02.006
[18] M. Mousavi and G. C. Beroza, “Deep-learning seismology,” Science, vol. 377, no. 6607, pp. 384–388, 2022, doi: https://doi.org/10.1126/science.abm4470. [Online]. Available: https://doi.org/10.1126/science.abm4470 DOI: https://doi.org/10.1126/science.abm4470
[19] S. M. Mousavi, Y. Sheng, W. Zhu, and G. C. Beroza, “STanford earthquake dataset (STEAD): A global data set of seismic signals for AI,” IEEE Access, vol. 7, pp. 179464–179476, 2019, doi: https://doi.org/10.1109/ACCESS.2019.2947848. [Online]. Available: https://doi.org/10.1109/ACCESS.2019.2947848 DOI: https://doi.org/10.1109/ACCESS.2019.2947848
[20] R. B. Aster, B. Borchers, and C. H. Thurber, Parameter Estimation and Inverse Problems, 3rd ed. Amsterdam, The Netherlands: Elsevier, 2018, doi: https://doi.org/10.1016/C2009-0-61134-X. [Online]. Available: https://www.sciencedirect.com/book/9780123850485/parameter-estimation -and-inverse-problems
[21] P. M. Shearer, Introduction to Seismology, 3rd ed. Cambridge, U.K.: Cambridge University Press, 2019, doi: https://doi.org/10.1017/9781316877111. [Online]. Available: https://www.cambridge.org/highereducation/books/introduction-to-seismology/C1471C1B553C05997E2BC7EB26D4C26D
[22] C. M. Bishop, Pattern Recognition and Machine Learning. New York, NY, USA: Springer, 2006. [Online]. Available: https://link.springer.com/book/9780387310732
[23] D. P. Kingma and M. Welling, “Auto-encoding variational Bayes,” in Proc. International Conference on Learning Representations (ICLR), 2014, doi: https://doi.org/10.48550/arXiv.1312.6114. [Online]. Available: https://arxiv.org/abs/1312.6114
[24] G. Carleo et al., “Machine learning and the physical sciences,” Reviews of Modern Physics, vol. 91, no. 4, Art. no. 045002, 2019, doi: https://doi.org/10.1103/RevModPhys.91.045002. [Online]. Available: https://doi.org/10.1103/RevModPhys.91.045002 DOI: https://doi.org/10.1103/RevModPhys.91.045002
[25] J. Snell, K. Swersky, and R. Zemel, “Prototypical networks for few-shot learning,” in Advances in Neural Information Processing Systems, vol. 30, 2017, pp. 4077–4087, doi: https://doi.org/10.48550/arXiv.1703.05175. [Online]. Available: https://arxiv.org/abs/1703.05175
[26] T.-Y. Lin, P. Goyal, R. Girshick, K. He, and P. Dollár, “Focal loss for dense object detection,” in Proc. IEEE International Conference on Computer Vision (ICCV), Venice, Italy, 2017, pp. 2980–2988, doi: https://doi.org/10.1109/ICCV.2017.324. [Online]. Available: https://doi.org/10.1109/ICCV.2017.324 DOI: https://doi.org/10.1109/ICCV.2017.324
[27] O. Vinyals, C. Blundell, T. Lillicrap, K. Kavukcuoglu, and D. Wierstra, “Matching networks for one-shot learning,” in Advances in Neural Information Processing Systems, vol. 29, 2016, pp. 3630–3638, doi: https://doi.org/10.48550/arXiv.1606.04080. [Online]. Available: https://arxiv.org/abs/1606.04080
[28] A. Dosovitskiy et al., “An image is worth 16 × 16 words: Transformers for image recognition at scale,” in Proc. International Conference on Learning Representations (ICLR), 2021, doi: https://doi.org/10.48550/arXiv.2010.11929. [Online]. Available: https://arxiv.org/abs/2010.11929
[29] C. Finn, P. Abbeel, and S. Levine, “Model-agnostic meta-learning for fast adaptation of deep networks,” in Proc. 34th International Conference on Machine Learning (ICML), vol. 70, 2017, pp. 1126–1135, doi: https://doi.org/10.48550/arXiv.1703.03400. [Online]. Available: https://arxiv.org/abs/1703.03400
[30] M. Long, Y. Cao, J. Wang, and M. I. Jordan, “Learning transferable features with deep adaptation networks,” in Proc. 32nd International Conference on Machine Learning (ICML), vol. 37, 2015, pp. 97–105, doi: https://doi.org/10.48550/arXiv.1502.02791. [Online]. Available: https://arxiv.org/abs/1502.02791
