Robust Methods for Classifying Argument Components in Public Participation Processes for Mobility Planning

In this publication in the Workshop on Argument Mining, Julia Romberg and Stefan Conrad address the robustness of classification algorithms for argument mining to build reliable models that generalize across datasets.

Abstract

Public participation processes allow citizens to engage in municipal decision-making processes by expressing their opinions on specific issues. Municipalities often only have limited resources to analyze a possibly large amount of textual contributions that need to be evaluated in a timely and detailed manner. Automated support for the evaluation is therefore essential, e.g. to analyze arguments.

In this paper, we address (A) the identification of argumentative discourse units and (B) their classification as major position or premise in German public participation processes. The objective of our work is to make argument mining viable for use in municipalities. We compare different argument mining approaches and develop a generic model that can successfully detect argument structures in different datasets of mobility-related urban planning. We introduce a new data corpus comprising five public participation processes. In our evaluation, we achieve high macro F1 scores (0.76 – 0.80 for the identification of argumentative units; 0.86 – 0.93 for their classification) on all datasets. Additionally, we improve previous results for the classification of argumentative units on a similar German online participation dataset.

Key findings

  • We conducted a comprehensive evaluation of machine learning methods across five public participation process in German municipalities that differ in format (online participation platforms and questionnaires) and process subject.
  • BERT surpasses previously published argument mining approaches for public participation processes on German data for both tasks, reaching macro F1 scores of 0.76 – 0.80 for the identification of argumentative units and macro F1 scores of 0.86 – 0.93 for their classification.
  • In a cross-dataset evaluation, BERT models trained on one dataset can recognize argument structures in other public participation datasets (which were not part of the training) with comparable goodness of fit.
  • Such model robustness across datasets is an important step towards the practical application of argument mining in municipalities.

Publication

Romberg, Julia; Conrad, Stefan (2021, November). Citizen Involvement in Urban Planning – How Can Municipalities Be Supported in Evaluating Public Participation Processes for Mobility Transitions?. In Proceedings of the 8th Workshop on Argument Mining (pp. 89-99), Punta Cana, Dominican Republic. Association for Computational Linguistics. https://aclanthology.org/2021.argmining-1.9

(Opportunities to) Mobilise for local political online participation

In this article in the Zeitschrift für Politikwissenschaft, Bastian Rottinghaus and Tobias Escher explore the question of who does (not) participate in digital participation formats for local mobility-related planning and to what extent personalised invitations can contribute to the mobilisation of a larger and more diverse group of participants.

Summary

Consistent findings of unequal political participation have been motivating different democratic innovations, including those that utilize the opportunities of information and communication technologies for political online participation. While previous research has established only a limited mobilizing potential of digital media, we still lack a good understanding of the mechanisms leading citizens to decide for or against engagement online. Therefore, we investigate who participates in opportunities for political online participation, what explains (non)-engagement and how effective personalized invitations are to increase and diversify participation. To address these questions, we conducted a comparative study of three almost identical instances of local online participation, relying on evidence from surveys of registered users and random samples of the local population.

Our results show that engagement in online participation is indeed significantly biased from the population towards resource-rich individuals who also differ in their assessment of the participation process and its results. This is despite the fact that knowledge of these participation opportunities is equally distributed among all social groups. While online aversion is a barrier for some, distrust in the participation process and lack of interest are more powerful reasons to refrain from engagement. Using a randomized-controlled field experiment we can confirm that personalized invitations are an effective instrument for mobilization that increased participation by a factor of four to seven and that can to a limited degree elicit participation from under-represented groups. These findings have a number of important implications for researchers and practitioners who aim to increase equality in political participation.

Key Findings

  • In 2017, largely identical online participation processes were carried out and analysed in three cities in North Rhine-Westphalia. In Bonn, Cologne-Ehrenfeld and Moers, the population was invited to submit suggestions for improvements to cycling on an online platform.
  • Initially, the usual participation patterns emerged, with above-average participation by highly educated and middle-aged men. One of the main reasons for participation was dissatisfaction with the cycling infrastructure.
  • A lack of knowledge about the participation process is the main reason for not participating. Our findings could show that all population groups were equally-well informed about the process, but in the end resource-rich groups were significantly more likely to decide to participate. Furthermore, there are some specific reservations about the online format, which represent an obstacle to online participation.
  • As part of a controlled field experiment, personalised letters were sent to a random selection of citizens with an invitation to the participation process. It was found that this increased participation by a factor of four to seven.
  • This invitation also mobilises additional groups that are otherwise under-represented in the consultation process. This applies, for example, to women and people with a lower level of formal education. They also have slightly different attitudes to ‘the usual suspects’ in that they are somewhat less critical of the existing transport infrastructure, but are also less positive about the results of the participation process.
  • Overall, this shows that personal invitations are an important means of mobilising a larger and more diverse group of citizens to participate in consultation processes, but they cannot eliminate the fundamental under-representation of people with fewer resources and political interests.

Publication

Rottinghaus, B., & Escher, T. (2020). Mechanisms for inclusion and exclusion through digital political participation: Evidence from a comparative study of online consultations in three German cities. Zeitschrift für Politikwissenschaft, 30(2), 261–298. https://doi.org/10.1007/s41358-020-00222-7