Category: results

Mobility Transition through Participation? Policy impact of discursive, consultative public participation on urban transport projects for sustainability

Dissertation Projekt, Laura Mark

In my dissertation project at the Faculty of Architecture at RWTH Aachen University, I am using two case studies to investigate the substantive impact of consultative public participation on political decisions and the implications for sustainable development. My object of investigation is planning for the sustainable mobility transition, since on the one hand it is important and urgent for sustainable development and on the other hand it directly affects people’s everyday lives and thus often leads to resistance.

Abstract

A socio-ecological shift in transport requires profound changes in public space that affect the daily lives of users. This redistribution of road space and change in conditions of use is primarily carried out through spatial planning on the part of the public sector, in which the public is also increasingly involved. This is usually associated (implicitly or explicitly) with the public having an influence on the content of the planning; however, the actual effect has hardly been researched.

I am investigating the mechanisms through which the substantive impact of public participation comes about or is prevented, and which factors influence these mechanisms. I am interested in the conditions under which these substantive effects contribute to integrated transport planning, measured both in terms of democratic theory and substantive criteria.

Two municipal transport transition projects in Hamburg serve as case studies, in which the public can participate or has participated through consultation offers and other forms of participation: the redesign of the Elbchaussee in Hamburg and the low-car design of the Ottensen neighbourhood in Hamburg. The processes differ, among other things, in their framework conditions, spatial scale, tasks and participation offerings. For the detailed reconstruction and analysis of these processes, I mainly rely on data from qualitative interviews, document and media analyses, supplemented by results of quantitative population and participant surveys.

Expected Results

Expected results are theses on public participation in the context of the mobility transition. These deal with the mechanisms and factors that influence policy impact and come about through a detailed analysis of the individual case studies, a targeted comparison of the two case studies with each other and the embedding of the empirical results in the state of research as well as other results from the project. These theses are intended to contribute to the discussion on the role of the public in the context of a socio-ecological transformation.

Supporting the Manual Evaluation Process of Citizen’s Contributions Through Natural Language Processing

Doctoral thesis (full text) of Julia Romberg

Engaging citizens in decision-making processes is a widely implemented instrument in democracies. On the one hand, such public participation processes serve the goal of achieving a more informed process and thus to potentially improve the process outcome, i.e. resulting policies, through the ideas and suggestions of the citizens. On the other hand, involving the citizenry is an attempt to increase the public acceptance of decisions made.

As public officials try to evaluate the often large quantities of citizen input, they regularly face challenges due to restricted resources (e.g. lack of personnel,time limitations). When it comes to textual contributions, natural language processing (NLP) offers the opportunity to provide automated support for the evaluation, which to date is still carried out mainly manually. Although some research has already been conducted in this area, important questions have so far been insufficiently addressed or have remained completely unanswered.

My dissertation, which I successfully completed in 2023, therefore focused on how existing research gaps can be overcome with the help of text classification methods. A particular emphasis was placed on the sub-tasks of thematic structuring and argument analysis of public participation data.

The thesis begins with a systematic literature review of previous approaches to the machine-assisted evaluation of textual contributions (for more insights, please refer to this article). Given the identified shortage of language resources, subsequently the newly created multidimensionally annotated CIMT corpus to facilitate the development of text classification models for German-language public participation is presented (for more insights, please refer to this article).

The first focus is on the thematic structuring of public input, particularly considering the uniqueness of many public participation processes in
terms of content and context. To make customized models for automation worthwhile, we leverage the concept of active learning to reduce manual workload by optimizing training data selection. In a comparison across three participation processes, we show that transformer-based active learning can significantly reduce manual classification efforts for process sizes starting at a few hundred contributions while maintaining high accuracy and affordable runtimes (for more insights, please refer to this article). We then turn to the criteria of practical applicability that conventional evaluation does not encompass. By proposing measures that reflect class-related demands users place on data acquisition, we provide insights into the behavior of different active learning strategies on class-imbalanced datasets, which is a common characteristic in collections of public input.

Afterward, we shift the focus to the analysis of citizens’ reasoning. Our first contribution lies in the development of a robust model for the detection of argumentative structures across different processes of public participation. Our approach improves upon previous techniques in the application domain for the recognition of argumentative sentences and, in particular, their classification as argument components (for more insights, please refer to this article). Following that, we explore the machine prediction of argument concreteness. In this context, the subjective nature of argumentation was accounted for by presenting a first approach to model different perspectives in the input representation of machine learning in argumentation mining (for more insights, please refer to this article).

Expert evidence: State of research on opportunities, challenges and limitations of digital participation

As set out in the German Site Selection Act (StandAG), the Federal Office for the Safety of Nuclear Waste Management (BASE) is charged with the comprehensive information and participation of the public in regards procedure for the search and selection of a repository site for the final disposal of high-level radioactive waste. In this context, in February 2022 BASE commissioned an expert report on the “Possibilities and limits of digital participation tools for public participation in the repository site selection procedure (DigiBeSt)” from the Düsseldorf Institute for Internet and Democracy (DIID) at Heinrich Heine University Düsseldorf in cooperation with the nexus Institute Berlin. For this purpose, lead by Tobias Escher a review of the state of research and current developments (work package 2) was prepared has been summarised in a detailed report (in German).

Selected findings from the report are:

  • Social inequalities in digital participation are mainly based on the second-level digital divide, i.e. differences in the media- and content-related skills required for independent and constructive use of the internet for political participation.
  • Knowledge about the effectiveness of activation factors is still often incomplete and anecdotal, making it difficult for initiators to estimate the costs and benefits of individual measures.
  • Personal invitations have been proven to be suitable for (target group-specific) mobilisation, but the established mass media also continue to play an important role.
  • Broad and inclusive participation requires a combination of different digital and analogue participation formats.
  • Participation formats at the national level face particular challenges due to the complexity of the issues at stake and the size of the target group. Therefore, these require the implementation of cascaded procedures (interlocking formats of participation at different political levels) as well as the creation of new institutions.

Publication

Lütters, Stefanie; Escher, Tobias; Soßdorf, Anna; Gerl, Katharina; Haas, Claudia; Bosch, Claudia (2024): Möglichkeiten und Grenzen digitaler Beteiligungsinstrumente für die Beteiligung der Öffentlichkeit im Standortauswahlverfahren (DigiBeSt). Hg. v. Düsseldorfer Institut für Internet und Demokratie und nexus Institut. Bundesamt für die Sicherheit der nuklearen Entsorgung (BASE). Berlin (BASE-RESFOR 026/24). Available online https://www.base.bund.de/DE/themen/fa/sozio/projekte-ende/projekte-ende.html .

CAIS Working Group: AI in digital public participation

As participants in a workshop organised by the Center for Advanced Internet Studies (CAIS) in Bochum, Julia Romberg and Tobias Escher presented results of the CIMT research on AI-supported evaluation of participation contributions and discussed further possibilities for using artificial intelligence to support public participation with experts from research as well as participation practice. It became clear that the practitioners see potential not only in the evaluation (output), but also in the activation of participants (input) and in the support of interactions (throughput) in participation processes. Nevertheless, these potentials face challenges and risks, including the adequate technical implementation and ensuring data protection and non-discrimination.

The workshop was organised by Dr Dennis Frieß and Anke Stoll and took place from 8 to 10 February 2023 in Bochum. Further information can be found on the website of the Düsseldorf Institute for Internet and Democracy.

3rd workshop for practitioners on first results from surveys in case study municipalities

On 30 November we invited representatives of the municipalities with whom we cooperate in order to discuss the first results of the extensive surveys conducted by our research group. The focus was on the question of how the respective participation procedures are assessed by those participating and which aspects motivate or discourage such participation.

Despite the diversity of the five projects we examined (and the still small number of participants), the assessments of the people participating in such processes show a relatively high degree of agreement. Overall, the evaluations of the participation processes are rather positive with regard to the course of discussion and transparency. At the same time, however, there are also comparable challenges in all processes. For example, the representation of one’s own interests is rated as relatively good, but gaps in the representation of other opinions are perceived. Also, a balance of interests is not always achieved. Furthermore, the participants are rather sceptical about the actual impact of the participation results on the political process, even though they still deem such an impact possible.

There is more information available in German.

Enriching Machine Prediction with Subjectivity Using the Example of Argument Concreteness in Public Participation

In this publication in the Workshop on Argument Mining, Julia Romberg develops a method to incorporate human perspectivism in machine prediction. The method is tested on the task of argument concreteness in public participation contributions.

Abstract

Although argumentation can be highly subjective, the common practice with supervised machine learning is to construct and learn from an aggregated ground truth formed from individual judgments by majority voting, averaging, or adjudication. This approach leads to a neglect of individual, but potentially important perspectives and in many cases cannot do justice to the subjective character of the tasks. One solution to this shortcoming are multi-perspective approaches, which have received very little attention in the field of argument mining so far.

In this work we present PerspectifyMe, a method to incorporate perspectivism by enriching a task with subjectivity information from the data annotation process. We exemplify our approach with the use case of classifying argument concreteness, and provide first promising results for the recently published CIMT PartEval Argument Concreteness Corpus.

Key findings

  • Machine learning often assumes a single ground truth to learn from, but this does not hold for subjective tasks.
  • PerspectifyMe is a simple method to incorporate perspectivism in existing machine learning workflows by complementing an aggregated label with a subjectivity score.
  • An example of a subjective task is the classification of the concreteness of an argument (low, medium, high), a task whose solution can also benefit the machine-assisted evaluation of public participation processes.
  • First approaches to classifying the concreteness of arguments (aggregated label) show an accuracy of 0.80 and an F1 value of 0.67.
  • The subjectivity of concreteness perception (objective vs. subjective) can be predicted with an accuracy of 0.72 resp. an F1 value of 0.74.

Publication

Romberg, Julia (2022, October). Is Your Perspective Also My Perspective? Enriching Prediction with Subjectivity. In Proceedings of the 9th Workshop on Argument Mining (pp.115-125), Gyeongju, Republic of Korea. Association for Computational Linguistics. https://aclanthology.org/2022.argmining-1.11

Automated Topic Categorization of Citizens’ Contributions: Reducing Manual Labeling Efforts Through Active Learning

In this publication in Electronic Government, Julia Romberg and Tobias Escher investigate the potential of active learning for reducing the manual labeling efforts in categorizing public participation contributions thematically.

Abstract

Political authorities in democratic countries regularly consult the public on specific issues but subsequently evaluating the contributions requires substantial human resources, often leading to inefficiencies and delays in the decision-making process. Among the solutions proposed is to support human analysts by thematically grouping the contributions through automated means.

While supervised machine learning would naturally lend itself to the task of classifying citizens’ proposal according to certain predefined topics, the amount of training data required is often prohibitive given the idiosyncratic nature of most public participation processes. One potential solution to minimize the amount of training data is the use of active learning. While this semi-supervised procedure has proliferated in recent years, these promising approaches have never been applied to the evaluation of participation contributions.

Therefore we utilize data from online participation processes in three German cities, provide classification baselines and subsequently assess how different active learning strategies can reduce manual labeling efforts while maintaining a good model performance. Our results show not only that supervised machine learning models can reliably classify topic categories for public participation contributions, but that active learning significantly reduces the amount of training data required. This has important implications for the practice of public participation because it dramatically cuts the time required for evaluation from which in particular processes with a larger number of contributions benefit.

Key findings

  • We compare a variety of state-of-the-art approaches for text classification and active learning on a case study of three nearly identical participation processes for cycling infrastructure in the German municipalities of Bonn, Ehrenfeld (a district of Cologne) and Moers.
  • We find that BERT can predict the correct topic(s) for about 77% of the cases.
  • Active learning significantly reduces manual labeling efforts: it was sufficient to manually label 20% to 50% of the datasets to maintain the level of accuracy. Efficiency-improvements grow with the size of the dataset.
  • At the same time, the models operate within an efficient runtime.
  • We therefore hypothesize that active learning should significantly reduce human efforts in most use cases.

Publication

J. Romberg and T. Escher. Automated topic categorisation of citizens’ contributions: Reducing manual labelling efforts through active learning. In M. Janssen, C. Csáki,I. Lindgren, E. Loukis, U. Melin, G. Viale Pereira, M. P. Rodríguez Bolívar, and E. Tambouris, editors,Electronic Government, pages 369–385, Cham, 2022. SpringerInternational Publishing. ISBN 978-3-031-15086-9

Socio-spatial justice through public participation?

In this presentation at the AESOP (Assosiation of European Schools of Planning) annual Congress in 2022, Laura Mark, Katharina Huseljić and Tobias Escher introduced a framework of distributive socio-spatial justice and the way consultation procedures can contribute, before evaluating the case study Elbchaussee in Hamburg regarding socio-spatial justice, using qualitative and quantitative results. 

Abstract

Our current transport system exhibits significant socio-spatial injustices as it has both major negative environmental effects and structurally disadvantages certain socio-economic groups. Planning processes increasingly include elements of public participation, often linked to the hope of better understanding and integrating different mobility needs into the planning process. However, so far there is little knowledge on whether public participation results indeed in more socio-spatial justice.

To approach this question, we focus on socio-spatial justice as distributive justice and investigate how well consultative planning procedures do actually lead to measures that both contribute to sustainability (i.e. reduce or redistribute negative external effects) and cater for the needs of disadvantaged groups (e.g. those with low income or education, women and disabled people). To this end, we have investigated in detail the case study of the reconstruction of the Elbchaussee, a representative main road of citywide importance in the district of Altona in Hamburg, Germany. We are drawing on both qualitative and quantitative data including expert interviews and public surveys.  

We first show that the process did result in planning measures that contribute slightly to ecological sustainability. Second, in particular through improving the situation for pedestrians and cyclists as well as the quality of stay, the measures should contribute to more justice for some groups but this is recognized only by non-male groups. Beyond this there are no effects for people with low income, low education, those with mobility restrictions or with particular mobility needs often associated with these groups. Overall, we conclude that the consultative planning process provides only a small contribution to socio-spatial justice and we discuss potential explanations.

Key Findings

  • The consultative planning process as a whole resulted in measures that contribute slightly to socio-spatial justice, since they support the transition to more sustainable mobility and will benefit some disadvantages groups, though both to a limited degree.
  • We find that the consultation procedure had no significant influence on the policy. In terms of socio-spatial justice, no positive effects can be traced back to the consultation procedure. Notably, those that participated in the consultation did indeed report less satisfaction with the measures.
  • We trace those limited contributions back to some general features of consultation and the current planning system, but also find that in the case study the scope of possible influence was very limited due to external restrictions and power imbalances.

Publication

We are working on a publication for a peer-reviewed journal. The publication will be linked here as soon as it is published.

A Corpus of German Citizen Contributions in Mobility Planning: Supporting Evaluation Through Multidimensional Classification

In this publication in the Conference on Language Resources and Evaluation, Julia Romberg, Laura Mark and Tobias Escher introduce a collection of annotated datasets that promotes the development of machine learning approaches to support the evaluation of public participation contributions.

Abstract

Political authorities in democratic countries regularly consult the public in order to allow citizens to voice their ideas and concerns on specific issues. When trying to evaluate the (often large number of) contributions by the public in order to inform decision-making, authorities regularly face challenges due to restricted resources.

We identify several tasks whose automated support can help in the evaluation of public participation. These are i) the recognition of arguments, more precisely premises and their conclusions, ii) the assessment of the concreteness of arguments, iii) the detection of textual descriptions of locations in order to assign citizens’ ideas to a spatial location, and iv) the thematic categorization of contributions. To enable future research efforts to develop techniques addressing these four tasks, we introduce the CIMT PartEval Corpus, a new publicly-available German-language corpus that includes several thousand citizen contributions from six mobility-related planning processes in five German municipalities. The corpus provides annotations for each of these tasks which have not been available in German for the domain of public participation before either at all or in this scope and variety.

Key findings

  • The CIMT PartEval Argument Component Corpus comprises 17,852 sentences from German public participation processes annotated as non-argumentative, premise, or major position.
  • The CIMT PartEval Argument Concreteness Corpus consists of 1,127 argumentative text spans that are annotated according to three levels of concreteness: low, intermediate, and high.
  • Der CIMT PartEval Geographic Location Corpus consists of 4,830 locations and the GPS coordinates for 2,529 proposals from public consultations.
  • The CIMT PartEval Thematic Categorization Corpus relies on a new hierarchical categorization scheme for mobility that captures modes of transport (non-motorized transport: cycling, walking, scooters; motorized transport: local public transport, long-distance public transport, commercial transport) and a number of specifications, such as moving or stationary traffic, new services, and inter- and multimodality. In total, 697 documents have been annotated according to this scheme.

Publication

Romberg, Julia; Mark, Laura; Escher, Tobias (2022, June). A Corpus of German Citizen Contributions in Mobility Planning: Supporting Evaluation Through Multidimensional Classification. In Proceedings of the Language Resources and Evaluation Conference (pp. 2874–2883), Marseille, France. European Language Resources Association. https://aclanthology.org/2022.lrec-1.308

Corpus available under

https://github.com/juliaromberg/cimt-argument-mining-dataset

https://github.com/juliaromberg/cimt-argument-concreteness-dataset

https://github.com/juliaromberg/cimt-geographic-location-dataset

https://github.com/juliaromberg/cimt-thematic-categorization-dataset

2nd workshop for practitioners on automated text analysis for citizen contributions

Part of the efforts of the research group is to develop tools that support the evaluation of citizen contributions from participation processes. On 10 December 2021 the research group hosted a workshop with practitioners (including local planning officials, participation officers and planning experts) to discuss our recent developments, part of which have been published in the Proceedings of the 8th Workshop on Argument Mining.

More information on the insights from the workshop is available in German.