Processing Multimodal Information

Challenges and Solutions for Multimodal Sentiment Analysis and Hate Speech Detection

authored by

Sherzod Hakimov, Gullal S. Cheema, Ralph Ewerth

Abstract

This chapter explores the challenges and solutions for processing multimodal information, specifically in the context of multimodal sentiment analysis and hate speech detection. The increasing amount of multimodal data, such as text, images and videos, presents unique challenges for machine learning algorithms. These challenges include the integration and fusion of information from multiple modalities to acquire the overall context. In this chapter, first, we present an overview of recent developments on multimodal learning techniques in the context of sentiment and hate speech detection; second, we present a multimodal model that combines different visual aspects and features for multimodal sentiment detection; and third, we present a multi-task multimodal model for misogyny detection in multimodal memes.

Organisation(s)

L3S Research Centre

External Organisation(s)

University of Potsdam
German National Library of Science and Technology (TIB)

Type

Contribution to book/anthology

Pages

71-94

No. of pages

24

Publication date

2025

Publication status

Published

Peer reviewed

Yes

ASJC Scopus subject areas

General Computer Science, General Social Sciences

Sustainable Development Goals

SDG 5 - Gender Equality

Electronic version(s)

https://doi.org/10.1007/978-3-031-64451-1_4 (Access: Open)