Cross Modal-FT Net: A Multimodal Fake News Detection Framework using Text, Images, and User Behavior

Abstract

An unprecedented proliferation of fake news across digital platforms is a major hurdle for reliable information, people trust, and social stability. Current fake news detection techniques, primarily based on text analysis, frequently overlook the multimodal and behavioral indicators associated with contemporary misinformation. Multimodal approaches are rarer and typically classify news as either genuine or fraudulent. To address this problem, this paper proposes a CrossModal-FTNet (Fake News Transformer Network), a transformer-centric multimodal system that identifies fake news by analyzing text, associated images, and user actions such as likes, shares, and the reliability of sources. The suggested model includes three dedicated encoders: a BERT-inspired text encoder for contextual interpretation, a ResNet-50-inspired image encoder for visual cues, and a lightweight behavioral feature encoder for examining user interaction information. These varied representations are subsequently merged through a cross-modal fusion transformer, which synchronizes and enhances data from various sources into a single united feature space. Experiments on benchmark datasets such as Fakeddit, Weibo, MM-COVID, and Twitter15 indicate that the suggested model excels, attaining 94.3% accuracy and a 92.8% F1-score, outpacing multiple unimodal and early fusion baselines. The findings confirm that using cross-modal data greatly boosts the ability to detect fake news. Thus, CrossModal-FTNet offers a scalable, real-time, and precise solution for combating misinformation in the ever-changing online environment.