An Ensemble of Classifiers for Automatic Annotation of Toxic Language in Portuguese under Data Scarcity

Abstract

Messages containing toxic language are a recurring problem on social media, highlighting the urgent need for effective automatic methods to mitigate their impact. Most existing approaches rely on large volumes of annotated data, which are costly, time-consuming, and highly labor-intensive. To address this challenge, this work proposes an ensemble of classifiers for the automatic annotation of toxic language in Portuguese, designed to operate under limited labeled data. The ensemble integrates three complementary strategies: a semi-supervised method based on heterogeneous graphs, a few-shot learning approach, and a Retrieval-Augmented Generation method, both grounded in large language models. The proposal is evaluated across multiple corpora, considering both their original versions and subsets filtered by total inter-annotator agreement. The results indicate that the ensemble exhibits competitive performance, surpassing the best individual method by up to 2% in scenarios of greater balance among the constituent classifiers and maintaining comparable performance in the remaining ones, while preserving moderate to substantial agreement with the original labels, demonstrating its potential for constructing annotated linguistic resources under data scarcity.