Digital resources in the Social Sciences and Humanities OpenEdition Our platforms OpenEdition Books OpenEdition Journals Hypotheses Calenda Libraries OpenEdition Freemium Follow us

Research Scenario: RePublic III

A reputational perspective on structural reforms

The aim of the CLARIAH-VL Open Humanities Service Infrastructure is to advance digitally-enabled research in Humanities and the Arts by, among other disciplines, providing data-level access to digitized and born-digital resources. In this blogpost series, we will communicate on research scenarios leading to and building upon the software and datasets made available through CLARIAH-VL. In the previous blogposts in this series, the RePublic model was introduced, and it was explained how the model was used to show the interaction between parliamentary attention and media attention of public agencies. This third and final part discusses the research conducted in Boon et al. (2025), in which RePublic was utilized.

Introduction

The study described in this blogpost explores how media coverage and different sentiments in media coverage impact the likelihood of structural reforms in public agencies. 

Research questions

  1. Does the amount of media attention influence the likelihood of structural reforms in public agencies? 
  2. Does the tone of media coverage (positive or negative sentiment) impact the likelihood of reforms? 
  3. Are negative reputations more influential than positive reputations in triggering reforms?

Hypotheses

  1. Agencies with more media attention are less likely to face structural reforms. 
  2. Positive reputations reduce the likelihood of reforms, while negative reputations increase it. 
  3. Negative reputations have a stronger impact on reform likelihood than positive reputations. 

Methodology

The study used two main datasets to answer the research questions mentioned above. The first dataset was the same corpus that was mentioned in the previous blogpost in this series: it consists of Flemish news articles (published between 2000 and 2015) that discuss a public agency. The RePublic model was used to provide sentiment annotations (positive, negative, neutral) to these articles. The second corpus, on the other hand, consisted of  a dataset extracted from the Belgian State Administration Database (Kleizen, Verhoest, and Wynen 2018), which contains information regarding the structural reforms which the same agencies that occur in the first corpus underwent.

In order to detect any effects of sentiment in media attention on structural reform likelihood, both linear and non-linear statistical models were used. The fact whether an agency experienced a structural reform in a given year (binary: yes/no) was treated as the dependent variable. The independent variables, on the other hand, were media sentiment and total media attention. Political turnover and neutral sentiment were used as control variables. 

Results

The results indicate an inverted U-shaped relationship between negative media coverage and reform likelihood: negative media reputations initially increase the likelihood of reforms, but this effect diminishes when negativity becomes extreme. Agencies with consistently negative reputations are less likely to experience reforms, as negativity becomes normalized. Positive media reputations, on the other hand, do not significantly impact the likelihood of reforms. 

References

Jan Boon, Jan Wynen, Koen Verhoest, Walter Daelemans, Jens Lemmens. 2025. A Reputational Perspective on Structural Reforms: How Media Reputations are Related to the Structural Reform Likelihood of Public Agencies. In Journal of Public Administration Research and Theory, pp. 1-15. Oxford University Press.

Jan Boon, Jan Wynen, Walter Daelemans, Jens Lemmens, Koen Verhoest. 2023. Agencies on the Parliamentary Radar: Exploring the Relations between Media Attention and Parliamentary Attention for Public Agencies Using Machine Learning Methods. In Public Administration 102:3, pp. 1026-1044. Wiley Online Library.

Bjorn Kleizen, Koen Verhoest, and Jan Wynen. 2018. Structural Reform Histories and Perceptions of Organizational Autonomy: Do Senior Managers Perceive Less Strategic Policy Autonomy When Faced with Frequent and Intense Restructuring? Public Administration 96: pp. 349-67. https://doi.org/10.1111/padm.12399

Evelien Willems and Frederik Heylen. 2023. FlemPar: An interface to the API of the Flemish Parliament. https://github.com/PolscienceAntwerp/Flempar

Authors

Jens Lemmens*, Jan Boon**, Koen Verhoest*, and Walter Daelemans*

(*University of Antwerp, **University of Hasselt)

Research scenario: RePublic II

Introducing the case of reputation analysis of government organizations 

The aim of the CLARIAH-VL Open Humanities Service Infrastructure is to advance digitally-enabled research in Humanities and the Arts by, among other disciplines, providing data-level access to digitized and born-digital resources. In this blogpost series, we will communicate on research scenarios leading to and building upon the software and datasets made available through CLARIAH-VL. In the previous blogpost in this series, the RePublic model was introduced. This second part discusses the research conducted in Boon et al. (2023), in which RePublic was utilized.

Introduction

Public agencies operate with significant autonomy, often holding more information about their activities than legislators. This information imbalance makes it challenging for politicians to monitor the performance of said agencies. Previous research has shown that news media play an important role in shaping political debates by drawing attention to societal issues, which helps to fill this knowledge gap. While earlier studies have explored the media’s impact on politics broadly, it is still unclear how media sentiment affects political scrutiny of agencies (Boon et al., 2023). By using RePublic to analyze sentiment in news media and its influence on political debates we aimed to provide new insights into this matter.

Research question

How are media attention and parliamentary attention for public agencies related?

Hypotheses

  • Media attention in newspapers precedes parliamentary questions about public agencies. This effect is more pronounced for news with a negative tone, compared to news with a neutral or positive tone.
  • Negative media attention for public agencies in newspapers is more likely to precede negatively toned parliamentary questions than positive and neutral media attention.

Methodology

In order to investigate the effect of news media attention (main independent variable) on parliamentary attention (main dependent variable), news data and parliamentary data about 24 public agencies was collected and statistical regression tests were applied.

Attention and reputation analysis

The number of published news articles and  parliamentary questions about public agencies were used as a metric for attention. In order to provide reputation annotations, RePublic was used to predict a “neutral”, “positive”, or “negative” label for each document. To determine which documents talk about which organizations, regular expressions were used. These statistics, aggregated per month, were used as the unit of analysis. 

Data

More than 90.000 news articles about 24 government organizations were collected. These were all published in one of three popular Flemish newspapers (De Standaard, De Morgen, Het Laatste Nieuws) between 2000 and 2020. For the parliamentary data, written questions from commissions and plenary sessions that originate from the same time span and that mentioned the same organizations were scraped using the FlemPar package for R (Willems and Heylen, 2023).

Results

The study revealed that media coverage influences parliamentary attention to public agencies, with media attention often preceding parliamentary attention. It was shown that positive media prompts favorable questions within the same month, but that negative coverage has a larger impact and increases all types of questions. Surprisingly, majority legislators, not just the opposition, actively respond to negative news, likely to protect their reputation. Written questions, though symbolic, reflect how legislators rely on media to monitor agencies. While causality isn’t definitive, the media’s agenda-setting role is clear—negative coverage triggers scrutiny, while positive coverage results in more favorable treatment of agencies in parliament.

Next Steps

In the third post of this series, the research described in Boon et al. (2025) will be presented.

References

Jan Boon, Jan Wynen, Koen Verhoest, Walter Daelemans, Jens Lemmens. 2025. A Reputational Perspective on Structural Reforms: How Media Reputations are Related to the Structural Reform Likelihood of Public Agencies. In Journal of Public Administration Research and Theory, pp. 1-15. Oxford University Press.

Jan Boon, Jan Wynen, Walter Daelemans, Jens Lemmens, Koen Verhoest. 2023. Agencies on the Parliamentary Radar: Exploring the Relations between Media Attention and Parliamentary Attention for Public Agencies Using Machine Learning Methods. In Public Administration 102:3, pp. 1026-1044. Wiley Online Library.

Evelien Willems and Frederik Heylen. 2023. FlemPar: An interface to the API of the Flemish Parliament. https://github.com/PolscienceAntwerp/Flempar

Authors

Jens Lemmens*, Jan Boon**, Koen Verhoest*, and Walter Daelemans*

(*University of Antwerp, **University of Hasselt)

Research Scenario: RePublic I

Introducing the case of reputation analysis of government organizations

The aim of the CLARIAH-VL Open Humanities Service Infrastructure is to advance digitally-enabled research in Humanities and the Arts by, among other disciplines, providing data-level access to digitized and born-digital resources. In this blogpost series, we will communicate on research scenarios leading to and building upon the software and datasets made available through CLARIAH-VL. This blogpost introduces the research scenario of the RePublic NLP model.

Introduction

To evaluate government organizations, their (mal)performance is discussed both in news media and during parliamentary sessions. The relationship between news attention and parliamentary attention (and its contingent nature), however, is understudied. In this three-piece blogpost, we describe a tool developed during CLARIAH-VL that can be used to predict the reputation of public agencies from text data, and 2 political studies conducted with this tool, which have been published as peer reviewed journal articles. In this first part, the tool itself – RePublic – is described.

Research question

Both news media and parliamentary discussions play an important role in the evaluation of government organisations. Due to the large scale of the available data, however, it is  necessary to utilize automatic methods to estimate the reputation of these organizations and gain comprehensive, diachronic insights. Hence, we proposed the following research question: How can we leverage Natural Language Processing methods to automatically analyze the reputation of government organisations from text?

Method

Data

An annotation task was set up to collect 4404 sentences mentioning Flemish government organizations, of which 1257 sentences were positive, 1485 sentences were negative and 1662 sentences were neutral. The sentences were extracted from news articles published between 2000 and 2020 in “Het Laatste Nieuws”, “De Standaard” or “De Morgen”, and which contained at least one of 24 government organizations, such as De Lijn, NMBS, Agentschap Natuur en Bos, etc. The latter was determined by using regular expressions.

Model

We used BERTje, the Dutch version of BERT – a pre-trained transformer model – to build a tool for automatic reputation prediction (De Vries et al., 2019). Initially, we used a Masked Language Modeling task to allow the model to learn the text genre using a corpus of more than 90.000 unlabeled news articles that mentioned at least one of the 24 government organisations. Then, a fine-tuning task was conducted to predict whether a given text about a certain organization expresses a positive, negative, or neutral attitude towards its reputation. For this task, the labeled data mentioned above was used. Our final model, which we named ‘RePublic’ (reputation analyzer for public agencies), is publicly available on the HuggingFace/transformers hub: https://huggingface.co/clips/republic.

Evaluation

A 10-fold cross validation experiment was conducted on the labeled data to optimize the hyperparameters of the model and evaluate it. The results can be found below.

ClassPrecision (%)Recall (%)F1-score (%)
Positive87.388.688.0
Negative86.486.586.5
Neutral85.384.284.7
Macro-averaged86.386.486.4

Table 1. Results of the 10-fold cross-validation experiment with RePublic using optimal hyper- parameters.

Next Steps

Using RePublic, two reputation studies have been conducted. These are published in Boon et al. (2025) and Boon et al. (2024), and will be described in two separate blogposts.

References

Jan Boon, Jan Wynen, Koen Verhoest, Walter Daelemans, Jens Lemmens. 2025. A Reputational Perspective on Structural Reforms: How Media Reputations are Related to the Structural Reform Likelihood of Public Agencies. In Journal of Public Administration Research and Theory, pp. 1-15. Oxford University press.

Jan Boon, Jan Wynen, Walter Daelemans, Jens Lemmens, Koen Verhoest. 2023. Agencies on the Parliamentary Radar: Exploring the Relations between Media Attention and Parliamentary Attention for Public Agencies Using Machine Learning Methods. In Public Administration 102:3, pp. 1026-1044. Wiley Online Library.

Wietse de Vries, Andreas van Cranenburgh, Arianna Bisazza, Tommaso Caselli, Gertjan van Noord, Malvina Nissim. 2019. BERTje: A Dutch BERT Model. arXiv:1912.09582.

Authors

Jens Lemmens*, Jan Boon**, Koen Verhoest*, and Walter Daelemans*

(*University of Antwerp, **University of Hasselt)

Reading historical maps in a Digital Era

The promises of Artificial Intelligence technologies to extract historical information from maps are becoming more impressive each day. However, these tools have been developed not only on the basis of but also, for modern maps. Older ones, including hand-drawn maps (early nineteenth century and earlier) therefore remain, as often with AI, the poor siblings of AI revolution, because the models used to extract information from maps do not work on those older documents: older maps have other characteristics than modern ones, often do not display the same sort of information which also means that researchers that are working on this kind of documents do not always have the same research questions than their colleagues working on modern ones.

On the 17th of November, members of the Antwerp Group of CLARIAH-VL (Iason Jongepier and Léa Hermenault assisted by Lamyk Bekius and Rein Debrulle), organized a workshop in Antwerp, with the support of CLARIAH-VL, the University of Antwerp and Ghent University, that aimed to tackle this issue. The underlying idea of this workshop was to facilitate brainstorming around technical solutions that allow older maps to also benefit from AI technology and AI-related workflows.

The workshop gathered 25 participants and welcomed 13 speakers from various horizons. In the morning, researchers from the leading Alan Turing Institute gave presentations and demos regarding two applications from the “Living with machines” project (namely MapReader and Machines Reading Maps). In the afternoon, colleagues from the University of Antwerp, Ghent University and University of Amsterdam presented their own work on AI and AI-related workflows, among others.

Program of the ‘Reading Maps in a Digital Era’ workshop

MapReader, a computer vision pipelines for exploring and analysing images

We first had the great pleasure to hear Katherine McDonough (Lancaster University & Alan Turing Institute) and Daniel Wilson (Alan Turing Institute) who came to introduce the tool MapReader they developed with their team in the framework of the “Living with Machines” project. This tool is open source and can be installed and used by anyone thanks to the instructions available here. Originally developed to automatically browse railways components on the Ordnance survey of England, it can be used to help researchers finding on a raster document any elements they are looking for by automatically identifying the later in pre-defined patches.

MapReader can also simply be used as a way to annotate patches. The tool first needs to be trained on a sample of patches, whose number depend on the size of the patches and the number of the specific characteristics of the browsed element: if the latter is not easily distinguishable from other elements, then the tool will need to be trained on a very large number of patches, but if on the contrary the element has very clear and specific characteristics, then the model only needs to be trained on a relative small number of patches. It should be possible to use the tool to explore old and hand-written maps if elements are easily identifiable, given that enough maps with these elements exist. One of the main problems about older maps indeed remains that we usually do not have enough material to train the models: very few maps collections dated from before the nineteenth century constitute series.  

MapReader patches on the Ordnance Survey

“Machines Reading Maps”, a tool to automatically transcribe texts on maps

After a short break, Katherine McDonough and Valeria Vitale (Sheffield University & Alan Turing Institute) introduced the audience to another tool that has been developed by the University of Southern California Digital Library, the Computer Science & Engineering Department at the University of Minnesota and the Alan Turing Institute, called “Machines Reading Maps”. This tool is trained to identify printed text on maps and to transcribe it. It has been first tested on the Rumsey collection, and then added to the platform that allows to browse it. It enables everyone to look for a toponym not only in the metadata, but also on the map itself. Machines Reading Maps can, therefore, also be used to count the occurrences of a specific place name variant in one collection for instance, or to gather text written with a specific graphic style (Bold, italic, etc.). If the later possibility would only be of interest for maps produced in series, which therefore limits drastically its use for old maps, the tool still looks promising for pre-nineteenth century cartography since writings tend to be more quickly standardized than symbology. It should definitely be tested on maps with non-printed text. 

Example of the results given by the search “Antwerpen” in the Rumsey Collection

Applying computer vision on historical documents

The afternoon was organized in three different sessions with two short papers in each of them. The first session, entitled ‘Computer vision‘ aimed at broadening our scope to the application of computer vision on geospatial and related data in general. José Oramas (University of Antwerp, Imec/IDLab) gave a paper related to his research on computer vision models applied to pictures where he tries to understand how those models really work in order to eventually improve their results. Then Thomas Smits (University of Amsterdam) introduced the audience to a research that he did together with Mona Allaert, Loren Verreyen, Wouter Haverals and Mike Kestemont at the University of Antwerp and which consisted of the use of computer vision, HTR and Large Language Models to transcribe and geo-localize addresses found on 100,000 historical postcards. This research shows how promising HTR technics are but also reveals how important it is to have a solid addresses database that can be used to geo-localize information, which is certainly reachable for modern periods, but which is more challenging for older ones.  

The second session was dedicated to the specific challenges of historical maps. The later have specificities, advantages and inconveniences that we have to be aware of if we want to build efficient and relevant applications and workflows to facilitate their digital use. The aim of this session was to focus on two very different corpus of maps to broaden our knowledge of their specificities. Dieter De Witte (Ghent University and Imec/IDLab) and Iason Jongepier (University of Antwerp and State Archives) introduced us to the specificities of historical maps of Belgium but also to the first attempts that have been done to extract information from them. Then, Katherine McDonough and Daniel Wilson showed how they used MapReader to explore “railway spaces” on the Ordnance Survey and explained the advantages of reflecting on those spaces using patches instead of vector data. 

The audience,at the end of a long day of work

Pipelines and workflows to scale up the digitization of data

The third and last session aimed at pipelines and workflows that can be use for the handling data derived from maps, or historical data with a strong spatial component. Janna Aerts (University of Amsterdam) and Leon van Wissen (University of Amsterdam and UvACreate) presented different projects for which historical data have been gathered and are connected via the linked open data system AdamLink. Next, Vincent Ducatteeuw (University of Ghent) and Léa Hermenault (University of Antwerp) gave a paper related to an article they are currently writing and that aims to show that small-scale/local gazetteers can greatly contribute to the debate regarding the structure of gazetteers by helping to choose information that should be available in a gazetteer to secure its interest for research purposes but also to meet FAIR standards. 

This fruitful day helped each of the participants to get to know new tools and to reflect on new methodological issues. It will without a doubt lead to further explorations and discussions that will hopefully help to unlock the access that historical information that old maps are packed with.