In my PhD thesis at the University of Düsseldorf I look at the effects of decriptive and substantive representation in consultative citizen participation on legitimacy beliefs of individuals.
Summary
Legitimacy – as a sum of individual beliefs about the appropriateness and acceptability of a political community, its regime and authorities – is the key element in stabilizing nowadays democratic systems. But, dissatisfaction with the performance of political systems is increasing and understandings of democracy can be divergent. Especially when political involvement is reduced to the possibility of choosing representatives legitimacy beliefs remain hard to rebuilt and understandings of democracy remain hard to align between different citizens. To solve this “legitimacy problem” plenty democratic theorists and researchers suggest more possibilities for political participation in the democratic process. Consultation is one mean often used by local municipalities to increase satisfaction and understanding of political processes. But consultative participation often promises too much. Like all political participation consultation is biased. Social inequality in society influences who participates. And who participates will ultimately influence a processes outcome. . The risk of losing marginalized voices in the process is high.
I want to enable a detailed understanding of the advantages of including these voices for local democratic legitimacy beliefs. Therefore, I follow Pitkin’s (1972) ideas on descriptive and substantive representation applying them to a consultative participation process. I ask
(a) Does descriptive representation in the input of a consultative participation process increase substantive representation in the throughput and outcome of a political process?
(b) How important are descriptive and substantive representation for increasing legitimacy beliefs after the political process?
I focus specifically on three levels of the policy making process (1) the input level, where I consider descriptive representation to be relevant, (2) the throughput level, where I consider substantive representation as ‘speaking for’ relevant and (3) the outcome level, where I consider substantive representation as ‘acting for’ by local municipalities relevant. While I consider (1) and (2) to be relevant criteria for increasing legitimacy beliefs by improving the political process, I consider (3) to be relevant for increasing legitimacy beliefs by improving real life living conditions.
In the meet the team series, we introduce a member of the research group every week to give an impression beyond the scientific work. For this purpose, our student assistant Philippe Sander asked us a few questions.
Today in the interview: Laura Mark. She is an urban planner and works on the effect of participation on political and planning decisions. More information on Laura’s research can be found here.
Foto: Tilman Schenk
What inspired you to pursue a career in your research field, and how did you get started in your field?
I first studied urban and regional planning. What I found exciting about it was that it unites different subject areas, such as ecology and social and cultural aspects. The task then is to take all these different aspects into account at the same time. After graduating, I worked in an engineering office in the field of transportation and realized that it was a little too technical and focused on numbers for me and that I was missing the social aspect. So now I’ve ended up in this intersection between planning science and social science.
Can you describe your current research project and what you hope to achieve with it? What do you personally find the most interesting about it?
I’m currently looking at the impact of public participation on planning projects for sustainable mobility. What interests me most about it is that it is a topic of current interest. Participation is widely discussed at the moment, and although research on it is still quite thin, the substantive effect on a decision is often taken for granted. Some parties also have little interest in taking a closer look.
How do you go about your research? What methods, theories or frameworks do you use?
I am doing qualitative research and look at two case studies in particular. My approach is modelled on process tracing. Unlike the various studies that have been done, it’s a very detailed approach and takes a lot of different influences on effect into focus. I work mainly with interviews and media analysis as well as participatory observation and partly with project survey data.
What are some of the biggest challenges you face in your work and how do you overcome them?
The biggest challenge is that the planning processes do not work in practice as they do in theory. For me, it’s important to do in-process research, because I mainly look at the substantive effect. However, some of the planning processes have been severely delayed, and not only because of Corona. Therefore, realistic time planning is definitely a challenge. Thus, schedules must always be flexibly adapted to the progress of the project and the situation in the case studies.
How do you stay on top of the latest trends and developments in your field?
I am connected with other researchers on Twitter, and I am also at conferences and in a working group of the Academy for Territorial Development in the Leibniz Association (ARL) with a focus on mobility and social participation.
As my field is not only about science, and since a lot is happening in practice right now – the topic of the transport transition is very much in discussion regarding climate change, etc. – I am of course also staying informed through newspapers and podcasts. There are also often free webinars or workshops from various (‘practitioner’) organisations where you can get further training.
How do you collaborate with other researchers or experts in your field to improve your projects?
The working group I mentioned before is definitely important because although it mainly discusses mobility as a topic, I can also contribute with the aspect of participation. I also participate in various interdisciplinary colloquia and regularly present my work. For example, there is one on socio-ecological transformations with different topics. In addition, we work together closely in the research group and, for example, present our results at conferences together.
What impact do you hope your research will have on society or the field?
I hope that my research will contribute to a better understanding of participation processes and perhaps to better planning for the public sector. I also hope that the research about substantive effects of participation will make public authorities consider this issue more closely. In the future I hope that civil society is taken more seriously and there will be more accountability on how the contributions are used.
What are some emerging trends or future directions you see in your research area?
The focus on participation of civil society is becoming stronger in practice and research. The important question is how this participation can be improved and how to deal with conflicts over urgently needed changes, for example in urban space. In addition, the question of how to reach certain groups that traditionally do not participate much.
Trends in the field of mobility are (among many others), on the one hand, technical aspects such as autonomous driving. In addition, a large part of the issue is how knowledge about the mobility transition can now be implemented in practice – the role of the municipalities is particularly relevant here, as they are trying harder to get more leeway to expand the 30 km/h speed limit or to make certain areas car-free.
Can you tell us about an interesting or meaningful experiences you had during your research?
For me, the most interesting thing was the experience that practice is completely different from theory. What I also found surprising is that the different interviewees contradicted each other very strongly, because their perceptions of the planning process were very different. In the minutes, for example from committee meetings, things looked completely different again; these are all truths and perspectives that I can then use to approach the process.
What advice do you have for students and aspiring scientists just starting out in their careers?
I think you shouldn’t plan too much but go by what interests you. You should have confidence that everything will come together in the end to form something coherent.
Lastly, can you tell us a little about yourself outside of your work? What hobbies or interests do you pursue in your spare time, and how do they complement your research?
I really enjoy being outside, I like hiking and biking; I do martial arts and yoga in my spare time. It definitely complements my research in the sense that I can switch off my head. It’s something completely different, not as intellectual. Sometimes it’s stressful, though, because every now and then when I’m riding my bike, I think to myself, “who approved that, that’s way too narrow, that doesn’t even comply with the standard width, who would do something like that!”.
In the meet the team series, we introduce a member of the research group every week to give an impression beyond the scientific work. For this purpose, our student assistant Philippe Sander asked us a few questions.
Today in the interview: Julia Romberg. As a computer scientist, she develops methods for the (partially) automated classification of contributions in participation processes. More info on Julia’s research can be found here.
Foto: Tilman Schenk
What inspired you to pursue a career in your research field, and how did you get started in your field?
I studied computer science because I enjoyed math at school and wanted to try something technical. In my master’s degree I started working with language data. Since I always found human language very interesting, I stuck with it.
Can you describe your current research project and what you hope to achieve with it? What do you personally find the most interesting about it?
The aim of my research project is to support the evaluation of textual contributions from citizen participation processes. One challenge is that often large amounts of data are generated, e.g., as emails or via online platforms. These are supposed to be evaluated within a certain time frame, but at the same time their evaluation must fulfill certain democratic norms (e.g., every voice must be heard). These requirements are difficult to comply by a pure manual evaluation, and that’s where computational approaches come in.
How do you go about your research? What methods, theories or frameworks do you use?
I use natural language processing tools to pre-structure public contributions thematically and to identify citizens’ arguments in order to highlight them fora subsequent manual analysis.
What are some of the biggest challenges you face in your work and how do you overcome them?
We are a transdisciplinary and interdisciplinary project. Communication across different disciplines is always a bit difficult, e.g., because of different terminologies and the strong focus on very specific research questions. That’s why we took our time at the beginning of the project to build a common ground to better understand each other and raising awareness for this particular challenge. The same applies for our communication with the practice, so that people without closer connection to the research area can understand our research. My recommendation for good science communication is “learning by doing”, e.g., by regularly preparing presentations for a non-specialist audience.
How do you stay on top of the latest trends and developments in your field?
Of course, it is an advantage that AI and natural language processing have aroused the broad interest of the media by now (keyword ChatGPT). Additionally, reading the current literature is a must. At the same time, the fast pace of the research field makes it difficult to maintain a comprehensive overview. For this purpose, the exchange with colleagues as well as the participation in tutorials and workshops is important in order to stay up to date with the latest research.
How do you collaborate with other researchers or experts in your field to improve your projects?
I participate in and organize colloquia and workshops in which people exchange ideas on various thematic focal points. Suggestions from such talks and discussions naturally flow back into my own work and sometimes even result in collaborations.
What impact do you hope your research will have on society or the field?
I hope that the methods developed can find application in practice.
What are some emerging trends or future directions you see in your research area?
A current trend are “prompt-based” approaches, where one consults large language models with different objectives.
Can you tell us about any interesting or meaningful experiences you had during your research?
Before I started working on the project, I worked at a chair in computer science, where rather advanced concepts were developed. However, I learned that practical use cases often need down-to-earth solutions first.
What advice do you have for students and aspiring scientists just starting out in their careers?
A good network and a clear research agenda are essential. It helps to set as narrow a scope as possible to develop a realistic project management. Even from a small delineated framework, quite a lot of research usually results. Especially in interdisciplinary and transdisciplinary projects, it should also be ensured that there is enough time for research that is relevant within one’s own discipline.
Lastly, can you tell us a little about yourself outside of your work? What hobbies or interests do you pursue in your spare time, and how do they complement your research?
I play bass guitar in a band and do Ashtanga yoga, where you only get ahead if you persevere. It’s the same as in science: you have to stick with something until it pays off.
In this publication in Digital Government: Research and Practice Julia Romberg and Tobias Escher offer a review of the computational techniques that have been used in order to support the evaluation of contributions in public participation processes. Based on a systematic literature review, they assess their performance and offer future research directions.
Abstract
Public sector institutions that consult citizens to inform decision-making face the challenge of evaluating the contributions made by citizens. This evaluation has important democratic implications but at the same time, consumes substantial human resources. However, until now the use of artificial intelligence such as computer-supported text analysis has remained an under-studied solution to this problem. We identify three generic tasks in the evaluation process that could benefit from natural language processing (NLP). Based on a systematic literature search in two databases on computational linguistics and digital government, we provide a detailed review of existing methods and their performance. While some promising approaches exist, for instance to group data thematically and to detect arguments and opinions, we show that there remain important challenges before these could offer any reliable support in practice. These include the quality of results, the applicability to non-English language corpora and making algorithmic models available to practitioners through software. We discuss a number of avenues that future research should pursue that can ultimately lead to solutions for practice. The most promising of these bring in the expertise of human evaluators, for example through active learning approaches or interactive topic modelling.
Key findings
There are a number of tasks in the evaluation processes that could be supported through Natural Language Processing (NLP). Broadly speaking, these are i) detecting (near) duplicates, ii) grouping of contributions by topic and iii) analyzing the individual contributions in depth. Most of the literature in this review focused on the automated recognition and analysis of arguments, one particular aspect of the task of in-depth analysis of contribution.
We provide a comprehensive overview of the datasets used as well as the algorithms employed and aim to assess their performance. Generally, despite promising results so far the significant advances of NLP techniques in recent years have barely been exploited in this domain.
A particular gap is that few applications exist that would enable practitioners to easily apply NLP to their data and reap the benefits of these methods.
The manual labelling efforts required for training machine learning models risk any efficiency gains from automation.
We suggest a number of fruitful future research avenues, many of which draw upon the expertise of humans, for example through active learning or interactive topic modelling.
Publication
Romberg, Julia; Escher, Tobias (2023): Making Sense of Citizens’ Input through Artificial Intelligence. In: Digital Government: Research and Practice, Artikel 3603254. DOI: 10.1145/3603254.
In my dissertation project at the Faculty of Architecture at RWTH Aachen University, I am using two case studies to investigate the substantive impact of consultative public participation on political decisions and the implications for sustainable development. My object of investigation is planning for the sustainable mobility transition, since on the one hand it is important and urgent for sustainable development and on the other hand it directly affects people’s everyday lives and thus often leads to resistance.
Abstract
A socio-ecological shift in transport requires profound changes in public space that affect the daily lives of users. This redistribution of road space and change in conditions of use is primarily carried out through spatial planning on the part of the public sector, in which the public is also increasingly involved. This is usually associated (implicitly or explicitly) with the public having an influence on the content of the planning; however, the actual effect has hardly been researched.
I am investigating the mechanisms through which the substantive impact of public participation comes about or is prevented, and which factors influence these mechanisms. I am interested in the conditions under which these substantive effects contribute to integrated transport planning, measured both in terms of democratic theory and substantive criteria.
Two municipal transport transition projects in Hamburg serve as case studies, in which the public can participate or has participated through consultation offers and other forms of participation: the redesign of the Elbchaussee in Hamburg and the low-car design of the Ottensen neighbourhood in Hamburg. The processes differ, among other things, in their framework conditions, spatial scale, tasks and participation offerings. For the detailed reconstruction and analysis of these processes, I mainly rely on data from qualitative interviews, document and media analyses, supplemented by results of quantitative population and participant surveys.
Expected Results
Expected results are theses on public participation in the context of the mobility transition. These deal with the mechanisms and factors that influence policy impact and come about through a detailed analysis of the individual case studies, a targeted comparison of the two case studies with each other and the embedding of the empirical results in the state of research as well as other results from the project. These theses are intended to contribute to the discussion on the role of the public in the context of a socio-ecological transformation.
Engaging citizens in decision-making processes is a widely implemented instrument in democracies. On the one hand, such public participation processes serve the goal of achieving a more informed process and thus to potentially improve the process outcome, i.e. resulting policies, through the ideas and suggestions of the citizens. On the other hand, involving the citizenry is an attempt to increase the public acceptance of decisions made.
As public officials try to evaluate the often large quantities of citizen input, they regularly face challenges due to restricted resources (e.g. lack of personnel,time limitations). When it comes to textual contributions, natural language processing (NLP) offers the opportunity to provide automated support for the evaluation, which to date is still carried out mainly manually. Although some research has already been conducted in this area, important questions have so far been insufficiently addressed or have remained completely unanswered.
My dissertation, which I successfully completed in 2023, therefore focused on how existing research gaps can be overcome with the help of text classification methods. A particular emphasis was placed on the sub-tasks of thematic structuring and argument analysis of public participation data.
The thesis begins with a systematic literature review of previous approaches to the machine-assisted evaluation of textual contributions (for more insights, please refer to this article). Given the identified shortage of language resources, subsequently the newly created multidimensionally annotated CIMT corpus to facilitate the development of text classification models for German-language public participation is presented (for more insights, please refer to this article).
The first focus is on the thematic structuring of public input, particularly considering the uniqueness of many public participation processes in terms of content and context. To make customized models for automation worthwhile, we leverage the concept of active learning to reduce manual workload by optimizing training data selection. In a comparison across three participation processes, we show that transformer-based active learning can significantly reduce manual classification efforts for process sizes starting at a few hundred contributions while maintaining high accuracy and affordable runtimes (for more insights, please refer to this article). We then turn to the criteria of practical applicability that conventional evaluation does not encompass. By proposing measures that reflect class-related demands users place on data acquisition, we provide insights into the behavior of different active learning strategies on class-imbalanced datasets, which is a common characteristic in collections of public input.
Afterward, we shift the focus to the analysis of citizens’ reasoning. Our first contribution lies in the development of a robust model for the detection of argumentative structures across different processes of public participation. Our approach improves upon previous techniques in the application domain for the recognition of argumentative sentences and, in particular, their classification as argument components (for more insights, please refer to this article). Following that, we explore the machine prediction of argument concreteness. In this context, the subjective nature of argumentation was accounted for by presenting a first approach to model different perspectives in the input representation of machine learning in argumentation mining (for more insights, please refer to this article).
As set out in the German Site Selection Act (StandAG), the Federal Office for the Safety of Nuclear Waste Management (BASE) is charged with the comprehensive information and participation of the public in regards procedure for the search and selection of a repository site for the final disposal of high-level radioactive waste. In this context, in February 2022 BASE commissioned an expert report on the “Possibilities and limits of digital participation tools for public participation in the repository site selection procedure (DigiBeSt)” from the Düsseldorf Institute for Internet and Democracy (DIID) at Heinrich Heine University Düsseldorf in cooperation with the nexus Institute Berlin. For this purpose, lead by Tobias Escher a review of the state of research and current developments (work package 2) was prepared has been summarised in a detailed report (in German).
Selected findings from the report are:
Social inequalities in digital participation are mainly based on the second-level digital divide, i.e. differences in the media- and content-related skills required for independent and constructive use of the internet for political participation.
Knowledge about the effectiveness of activation factors is still often incomplete and anecdotal, making it difficult for initiators to estimate the costs and benefits of individual measures.
Personal invitations have been proven to be suitable for (target group-specific) mobilisation, but the established mass media also continue to play an important role.
Broad and inclusive participation requires a combination of different digital and analogue participation formats.
Participation formats at the national level face particular challenges due to the complexity of the issues at stake and the size of the target group. Therefore, these require the implementation of cascaded procedures (interlocking formats of participation at different political levels) as well as the creation of new institutions.
Publication
Lütters, Stefanie; Escher, Tobias; Soßdorf, Anna; Gerl, Katharina; Haas, Claudia; Bosch, Claudia (2024): Möglichkeiten und Grenzen digitaler Beteiligungsinstrumente für die Beteiligung der Öffentlichkeit im Standortauswahlverfahren (DigiBeSt). Hg. v. Düsseldorfer Institut für Internet und Demokratie und nexus Institut. Bundesamt für die Sicherheit der nuklearen Entsorgung (BASE). Berlin (BASE-RESFOR 026/24). Available online https://www.base.bund.de/DE/themen/fa/sozio/projekte-ende/projekte-ende.html .
As participants in a workshop organised by the Center for Advanced Internet Studies (CAIS) in Bochum, Julia Romberg and Tobias Escher presented results of the CIMT research on AI-supported evaluation of participation contributions and discussed further possibilities for using artificial intelligence to support public participation with experts from research as well as participation practice. It became clear that the practitioners see potential not only in the evaluation (output), but also in the activation of participants (input) and in the support of interactions (throughput) in participation processes. Nevertheless, these potentials face challenges and risks, including the adequate technical implementation and ensuring data protection and non-discrimination.
On 30 November we invited representatives of the municipalities with whom we cooperate in order to discuss the first results of the extensive surveys conducted by our research group. The focus was on the question of how the respective participation procedures are assessed by those participating and which aspects motivate or discourage such participation.
Despite the diversity of the five projects we examined (and the still small number of participants), the assessments of the people participating in such processes show a relatively high degree of agreement. Overall, the evaluations of the participation processes are rather positive with regard to the course of discussion and transparency. At the same time, however, there are also comparable challenges in all processes. For example, the representation of one’s own interests is rated as relatively good, but gaps in the representation of other opinions are perceived. Also, a balance of interests is not always achieved. Furthermore, the participants are rather sceptical about the actual impact of the participation results on the political process, even though they still deem such an impact possible.
In this publication in the Workshop on Argument Mining, Julia Romberg develops a method to incorporate human perspectivism in machine prediction. The method is tested on the task of argument concreteness in public participation contributions.
Abstract
Although argumentation can be highly subjective, the common practice with supervised machine learning is to construct and learn from an aggregated ground truth formed from individual judgments by majority voting, averaging, or adjudication. This approach leads to a neglect of individual, but potentially important perspectives and in many cases cannot do justice to the subjective character of the tasks. One solution to this shortcoming are multi-perspective approaches, which have received very little attention in the field of argument mining so far.
In this work we present PerspectifyMe, a method to incorporate perspectivism by enriching a task with subjectivity information from the data annotation process. We exemplify our approach with the use case of classifying argument concreteness, and provide first promising results for the recently published CIMT PartEval Argument Concreteness Corpus.
Key findings
Machine learning often assumes a single ground truth to learn from, but this does not hold for subjective tasks.
PerspectifyMe is a simple method to incorporate perspectivism in existing machine learning workflows by complementing an aggregated label with a subjectivity score.
An example of a subjective task is the classification of the concreteness of an argument (low, medium, high), a task whose solution can also benefit the machine-assisted evaluation of public participation processes.
First approaches to classifying the concreteness of arguments (aggregated label) show an accuracy of 0.80 and an F1 value of 0.67.
The subjectivity of concreteness perception (objective vs. subjective) can be predicted with an accuracy of 0.72 resp. an F1 value of 0.74.
Publication
Romberg, Julia (2022, October). Is Your Perspective Also My Perspective? Enriching Prediction with Subjectivity. In Proceedings of the 9th Workshop on Argument Mining (pp.115-125), Gyeongju, Republic of Korea. Association for Computational Linguistics. https://aclanthology.org/2022.argmining-1.11
