Sistem Otentikasi Biometrik Multimodal Berbasis Face Recognition Dan Voiceprint Pada Aplikasi Website
DOI:
https://doi.org/10.69693/ijim.v4i3.2147Keywords:
Otentikasi Biometrik, Multimodal, Voiceprint, Face Recognition, Automatic Speech RecognitionAbstract
Keamanan sistem otentikasi berbasis kata sandi semakin rentan terhadap ancaman pencurian identitas, phishing, dan serangan modern lainnya. Penelitian ini merancang dan mengimplementasikan sistem otentikasi biometrik multimodal berbasis website yang mengintegrasikan pengenalan wajah, verifikasi voiceprint, dan verifikasi passphrase sebagai pendekatan autentikasi passwordless. Sistem dikembangkan dengan arsitektur client-server menggunakan Flask pada sisi backend serta WebRTC dan MediaRecorder API pada sisi frontend. Pengenalan wajah dilakukan menggunakan DeepFace dengan model FaceNet, sedangkan voiceprint diperoleh dari fusi berbobot tiga model deep learning, yaitu ECAPA-TDNN, WavLM, dan Wav2Vec 2.0. Passphrase yang diucapkan pengguna ditranskripsikan melalui Whisper ASR dan dicocokkan dengan teks referensi untuk memperkuat deteksi keaktifan (liveness detection). Keputusan akhir otentikasi menggunakan mekanisme cascading dengan ambang suara 0,70, wajah 0,75, passphrase 0,80, dan skor gabungan 0,85. Sistem diuji melalui skenario identifikasi 1:N, meliputi black box testing, pengujian genuine, pengujian impostor, pengujian performa, dan simulasi replay attack. Hasil pengujian menunjukkan Rank-1 Accuracy sebesar 72% dengan tingkat penolakan impostor 100% dari 250 percobaan penyusupan, serta rata-rata waktu pemrosesan 3,42 detik per autentikasi. Kegagalan pengguna sah pada pengujian genuine disebabkan oleh kesalahan pengucapan passphrase, bukan oleh kegagalan model biometrik. Penelitian ini membuktikan bahwa autentikasi biometrik multimodal passwordless dapat diimplementasikan secara praktis pada infrastruktur web standar tanpa perangkat keras tambahan, serta memberikan lapisan keamanan berlapis yang lebih tahan terhadap serangan dibandingkan autentikasi konvensional berbasis kata sandi.
Downloads
References
Alharbi, B., & Alshanbari, H. S. (2023). Face-Voice Based Multimodal Biometric Authentication System Via Facenet And Gmm. Https://Doi.Org/10.7717/Peerj-Cs.1468
Atenco-Vázquez, J. C., Moreno-Rodríguez, J. C., & Ramírez-Cortés, J. M. (2024). Bimodal Biometric Recognition System Using Convolutional Neural Networks And Fusion Of Deep Audiovisual Feature Vectors Biometric Verification And Identification Multimodal Biometric Systems. 15(3), 4–14.
Chen, S., Wang, C., Chen, Z., Wu, Y., Liu, S., & Chen, Z. (2022). Wavlm : Large-Scale Self-Supervised Pre-Training For Full Stack Speech Processing. 1–14.
Choi, H., Kim, J., Woo, S. S., & Kim, H. (2024). Blind-Match : Efficient Homomorphic Encryption-Based 1 : N Matching For Privacy-Preserving Biometric Identification. In Proceedings Of The 33rd Acm International Conference On Information And Knowledge Management (Cikm ’24), October 21â•fi25, 2024, Boise, Id, Usa (Vol. 1, Issue 1). Association For Computing Machinery. Https://Doi.Org/10.1145/3627673.3680017
Dawalatabad, N., & Ravanelli, M. (2021). Ecapa-Tdnn Embeddings For Speaker Diarization.
Dobrowolski, A. P., Piotrowski, Z., & Adam, K. (2023). Enhancing Web Application Security : Advanced Biometric Voice Verification For Two-Factor Authentication. 1–19.
Fan, Z., Li, M., Zhou, S., & Xu, B. (2021). Exploring Wav2vec 2.0 On Speaker Verification And Language Identification. Lid, 1–5.
Farhadipour, A., Chapariniya, M., Vukovic, T., & Dellwo, V. (2024). Comparative Analysis Of Modality Fusion Approaches For Audio-Visual Person Identification And Verification.
Mugalu, B. W., Wamala, R. C., Serugunda, J., & Katumba, A. (2021). Face Recognition As A Method Of Authentication In A Web-Based System.
Rachapalli, D. R., Dondeti, V., & Kalluri, H. K. (2024). Multimodal Cancellable Biometric Template Protection And Person Verification In Transformed Domain. Ieee Access, 12(November), 173557–173582. Https://Doi.Org/10.1109/Access.2024.3501368
Radford, A., Wook, J., Tao, K., Greg, X., Christine, B., & Ilya, M. (2021). Robust Speech Recognition Via Large-Scale Weak Supervision.
Radytia, A. A. (2026). Keamanan Informasi Di Era Digital: Tanggung Jawab Kita Bersama. Https://Www.Djkn.Kemenkeu.Go.Id/Kpknl-Biak/Baca-Artikel/18443/Keamanan-Informasi-Di-Era-Digital-Tanggung-Jawab-Kita-Bersama.Html
Rahman, S. A. El, Saleh, A., & Id, A. (2024). Enhanced Multimodal Biometric Recognition Systems Based On Deep Learning And Traditional Methods In Smart Environments. 1–24. Https://Doi.Org/10.1371/Journal.Pone.0291084
Unnithan, M. R., Sabir, M., Sherief, S., Afiya, S., & Aziz, A. A. (2026). Fusion Strategies And Open-Set Recognition In Multimodal Biometric Authentication Systems : A Comprehensive Review. 04(April), 2350–2361.
Utami, S. N. (2023). 8 Macam Ancaman Pada Keamanan Informasi Digital. Https://Www.Kompas.Com/Skola/Read/2023/01/28/150000069/8-Macam-Ancaman-Pada-Keamanan-Informasi-Digital
Weng, S., Yang, L., & Mao, J. (2025). Effective Modeling Of Critical Contextual Information For Tdnn-Based Speaker Verification. 1–5.
Baevski, A., Zhou, Y., Mohamed, A., & Auli, M. (2020). Wav2vec 2.0: A Framework For Self-Supervised Learning Of Speech Representations. Advances In Neural Information Processing Systems, 33, 12449–12460.
Desplanques, B., Thienpondt, J., & Demuynck, K. (2020). Ecapa-Tdnn: Emphasized Channel Attention, Propagation And Aggregation In Tdnn Based Speaker Verification. Interspeech 2020, 3830–3834. Https://Doi.Org/10.21437/Interspeech.2020-2650
Schroff, F., Kalenichenko, D., & Philbin, J. (2015). Facenet: A Unified Embedding For Face Recognition And Clustering. Proceedings Of The Ieee Conference On Computer Vision And Pattern Recognition (Cvpr), 815–823. Https://Doi.Org/10.1109/Cvpr.2015.7298682
Snyder, D., Garcia-Romero, D., Povey, D., & Khudanpur, S. (2017). Deep Neural Network Embeddings For Text-Independent Speaker Verification. Interspeech 2017, 999–1003. Https://Doi.Org/10.21437/Interspeech.2017-620
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Indonesian Journal of Innovation Multidisipliner Research

This work is licensed under a Creative Commons Attribution 4.0 International License.


















