Comparative Analysis of CNN-RNN Models for Hatespeech Detection Incorporating L2 regularization

Tri Pratiwi Handayani, Wahyudin Hasyim

Abstract


This study aims to address the challenge of detecting hate speech in text data by comparing two experimental CNN-RNN models. The primary issue is achieving a balance between precision and recall in hate speech detection while preventing overfitting and ensuring good generalization. Two different approaches were applied: the first model used standard training techniques, while the second model incorporated L2 regularization and early stopping. The research involved using Keras Tokenizer for text tokenization, layering with CNN and LSTM for feature extraction and temporal context capturing, and applying dropout to prevent overfitting. L2 regularization and early stopping were added to the second model to enhance generalization. The findings reveal that the first model, although exhibiting some overfitting, attained a higher overall accuracy of 78% and more balanced F1-scores for both the "Not Hate Speech" and "Hate Speech" categories. The second model, although achieving higher precision for hate speech (0.81), had lower recall (0.58), resulting in an overall accuracy of 75%. This suggests that regularization and early stopping need careful tuning to avoid reducing sensitivity to hate speech detection.


Keywords


CNN-RNN, Hate Speech, Regularization, LSTM.

Full Text:

PDF

References


J. Howarth, “17 Cyberbullying Facts & Statistics (2024),” Exploding Topics. Accessed: Jun. 06, 2024. [Online]. Available: https://explodingtopics.com/blog/cyberbullying-stats.

G. Gambino and R. Pirrone, “CHILab @ HaSpeeDe 2: Enhancing hate speech detection with part-of-speech tagging,” in CEUR Workshop Proceedings, 2020. doi: 10.4000/books.aaccademia.7057.

M. Sajjad, F. Zulifqar, M. U. G. Khan, and M. Azeem, “Hate Speech Detection using Fusion Approach,” in 2019 International Conference on Applied and Engineering Mathematics, ICAEM 2019 - Proceedings, 2019. doi: 10.1109/ICAEM.2019.8853762.

T. Parcollet et al., “Quaternion recurrent neural networks,” in 7th International Conference on Learning Representations, ICLR 2019, 2019.

S. Riyadi, A. D. Andriyani, A. M. Masyhur, C. Damarjati, and M. I. Solihin, “Detection of Indonesian Hate Speech on Twitter Using Hybrid CNN-RNN,” in Proceeding - International Conference on Information Technology and Computing 2023, ICITCOM 2023, 2023, pp. 352–356. doi: 10.1109/ICITCOM60176.2023.10442041.

Y. Yu, X. Si, C. Hu, and J. Zhang, “A review of recurrent neural networks: Lstm cells and network architectures,” Neural Computation, vol. 31, no. 7. 2019. doi: 10.1162/neco_a_01199.

M. Buda, A. Maki, and M. A. Mazurowski, “A systematic study of the class imbalance problem in convolutional neural networks,” Neural Networks, vol. 106, 2018, doi: 10.1016/j.neunet.2018.07.011.

M. Kamyab, G. Liu, and M. Adjeisah, “Attention-Based CNN and Bi-LSTM Model Based on TF-IDF and GloVe Word Embedding for Sentiment Analysis,” Appl. Sci., vol. 11, no. 23, 2021, doi: 10.3390/app112311255.

X. Xie, M. Xie, A. J. Moshayedi, and M. H. Noori Skandari, “A Hybrid Improved Neural Networks Algorithm Based on L2 and Dropout Regularization,” Math. Probl. Eng., vol. 2022, 2022, doi: 10.1155/2022/8220453.

C. Yu, P. H. Hung, J. H. Hong, and H. Y. Chiang, “Efficient Max Pooling Architecture with Zero-Padding for Convolutional Neural Networks,” in GCCE 2023 - 2023 IEEE 12th Global Conference on Consumer Electronics, 2023. doi: 10.1109/GCCE59613.2023.10315268.

M. O. Ibrohim and I. Budi, “Multi-label Hate Speech and Abusive Language Detection in Indonesian Twitter,” ALW3 3rd Work. Abus. Lang. Online, pp. 46–57, 2019, [Online]. Available: https://www.aclweb.org/anthology/W19-3506.pdf

Y. Setiawan, D. Gunawan, and R. Efendi, “Feature Extraction TF-IDF to Perform Cyberbullying Text Classification: A Literature Review and Future Research Direction,” in 2022 International Conference on Information Technology Systems and Innovation, ICITSI 2022 - Proceedings, 2022, pp. 283–288. doi: 10.1109/ICITSI56531.2022.9970942.

S. D. A. Putri, M. O. Ibrohim, and I. Budi, Abusive Language and Hate Speech Detection for Indonesian-Local Language in Social Media Text, vol. 251. 2021. doi: 10.1007/978-3-030-79757-7_9.

R. Hendrawan, Adiwijaya, and S. Al Faraby, “Multilabel Classification of Hate Speech and Abusive Words on Indonesian Twitter Social Media,” in 2020 International Conference on Data Science and Its Applications, ICoDSA 2020, 2020. doi: 10.1109/ICoDSA50139.2020.9212962.

A. B. Syahputri and Y. Sibaroni, “Comparative Analysis of CNN and LSTM Performance for Hate Speech Detection on Twitter,” in International Conference on ICT Convergence, 2023, pp. 190–195. doi: 10.1109/ICoICT58202.2023.10262656.

R. Larracy, A. Phinyomark, and E. Scheme, “Machine Learning Model Validation for Early Stage Studies with Small Sample Sizes,” in Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS, 2021. doi: 10.1109/EMBC46164.2021.9629697.

A. Adli and P. Tyrrell, “Impact of Training Sample Size on the Effects of Regularization in a Convolutional Neural Network-based Dental X-ray Artifact Prediction Model,” J. Undergrad. Life Sci., vol. 14, no. 1, 2020, doi: 10.33137/juls.v14i1.35883.

F. Chollet and others, “Keras Tokenizer.” 2023.

K. Mukai and T. Yamanaka, “Improving Translation Invariance in Convolutional Neural Networks with Peripheral Prediction Padding,” in Proceedings - International Conference on Image Processing, ICIP, 2023. doi: 10.1109/ICIP49359.2023.10223102.

A. D. Nguyen, S. Choi, W. Kim, S. Ahn, J. Kim, and S. Lee, “Distribution Padding in Convolutional Neural Networks,” in Proceedings - International Conference on Image Processing, ICIP, 2019. doi: 10.1109/ICIP.2019.8803537.

M. Sun, X. Yang, and Y. Xie, “Deep learning in aquaculture: A review,” J. Comput. csroc.org.tw, 2020. [Online]. Available: http://www.csroc.org.tw/journal/JOC31-1/JOC3101-28.pdf

A. (2022). . Khan, “Balanced Split: A new train-test data splitting strategy for imbalanced datasets. ArXiv,” arxiv.org, 2022, [Online]. Available: https://arxiv.org/abs/2212.11116

M. Zhu et al., “Class weights random forest algorithm for processing class imbalanced medical data,” IEEE Access, vol. 6, 2018, doi: 10.1109/ACCESS.2018.2789428.

P. Wibowo and C. Fatichah, “An in-depth performance analysis of the oversampling techniques for high-class imbalanced dataset,” Regist. J. Ilm. Teknol. Sist. Inf., vol. 7, no. 1, 2021, doi: 10.26594/register.v7i1.2206.

V. M. González-Barcenas, E. Rendón, R. Alejo, E. E. Granda-Gutiérrez, and R. M. Valdovinos, “Addressing the Big Data Multi-class Imbalance Problem with Oversampling and Deep Learning Neural Networks,” in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2019. doi: 10.1007/978-3-030-31332-6_19.

O. Hrinchuk, V. Khrulkov, L. Mirvakhabova, E. Orlova, and I. Oseledets, “Tensorized embedding layers,” in Findings of the Association for Computational Linguistics Findings of ACL: EMNLP 2020, 2020. doi: 10.18653/v1/2020.findings-emnlp.436.

F. Heimerl and M. Gleicher, “Interactive Analysis of Word Vector Embeddings,” Comput. Graph. Forum, vol. 37, no. 3, 2018, doi: 10.1111/cgf.13417.

A. Templeton, “Word Equations: Inherently Interpretable Sparse Word Embeddings through Sparse Coding,” in BlackboxNLP 2021 - Proceedings of the 4th BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP, 2021. doi: 10.18653/v1/2021.blackboxnlp-1.12.

S. Ariwibowo, A. S. Girsang, and Diana, “Hate Speech Text Classification Using Long Short-Term Memory (LSTM),” in ICOSNIKOM 2022 - 2022 IEEE International Conference of Computer Science and Information Technology: Boundary Free: Preparing Indonesia for Metaverse Society, 2022. doi: 10.1109/ICOSNIKOM56551.2022.10034908.

K. Sanjar, A. Rehman, A. Paul, and K. Jeonghong, “Weight dropout for preventing neural networks from overfitting,” in 2020 8th International Conference on Orange Technology, ICOT 2020, 2020. doi: 10.1109/ICOT51877.2020.9468799.

J. Tong, Z. Wang, and X. Rui, “A Multimodel-Based Deep Learning Framework for Short Text Multiclass Classification with the Imbalanced and Extremely Small Data Set,” Comput. Intell. Neurosci., vol. 2022, 2022, doi: 10.1155/2022/7183207.

S. Afaq and S. Rao, “Significance Of Epochs On Training A Neural Network,” Int. J. Sci. Technol. Res., vol. 9, no. 06, 2020.




DOI: https://doi.org/10.52088/ijesty.v4i1.491

Article Metrics

Abstract view : 0 times
PDF - 0 times

Refbacks

  • There are currently no refbacks.


Copyright (c) 2024 Tri Pratiwi Handayani, Wahyudin Hasyim

International Journal of Engineering, Science and Information Technology (IJESTY) eISSN 2775-2674