Digital resources in the Social Sciences and Humanities OpenEdition Our platforms OpenEdition Books OpenEdition Journals Hypotheses Calenda Libraries OpenEdition Freemium Follow us

Multiscope: A User-Friendly Multi-Label Text Classification Dashboard

Natural Language Processing (NLP) has seen rapid developments in the past few years with the introduction of the transformers model architecture and Large Language Models (LLMs). These developments have led to improved results for a multitude of NLP tasks, including text classification. A complex subproblem in text classification is Multi-Label Text Classification (MLTC), which aims to assign multiple labels (e.g., topics or emotions) to a single text. In this post, we introduce Multiscope, a Python tool that aims to facilitate the process of preparing multi-label data, in addition to training and evaluating multi-label classification models. The code and detailed user guidelines can be found on GitHub.

Loading Data

The tool accepts JSON, CSV and Excel files. The user can also opt to load in an existing dataset from the HuggingFace hub. Then, the user can choose to create custom train, validation and test splits from the data. The tool then provides statistics and visualizations from the data.

Training 

After loading the data, the user can choose to train either a BERT-like transformer (Devlin et al. 2019) or a Support Vector Machine (SVM). For both options, the user can specify the models’ hyperparameters. For this, the tool leverages the Transformers library (Wolf et al. 2020) and the Sci-kit learn library (Pedregosa et al. 2011). The tool also integrates Weights & Biases, which tracks the training and evaluation process of the models.

Evaluation

After training a model, the tool provides multiple automatic evaluation methods. The tool calculates several classification metrics adapted to MLTC, provides a classification report and visualizes a multi-label confusion matrix (Heydarian et al. 2022).

How to Cite

Jens Van Nooten and Walter Daelemans. 2024. Multiscope: A User-Friendly Multi-Label Text Classification Dashboard. CLiPS Technical Report Series 11 (CTRS 11). ISSN 2033-3544. Computational Linguistics, Psycholinguistics, and Sociolinguistics Research Center.

References

Devlin, Jacob, Chang, Ming-Wei, Lee, Kenton, & Toutanova, Kristina. (2019). BERT: Pre-training of deep bidirectional transformers for language understanding. In Jill Burstein, Christy Doran, & Thamar Solorio (Eds.), Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) (pp. 4171–4186). Minneapolis, Minnesota: Association for Computational Linguistics. https://doi.org/10.18653/v1/N19-1423

Mohammadreza Heydarian, Thomas E. Doyle and Reza Samavi. 2022. MLCM: Multi-label confusion matrix. IEEE Access, 10, 19083–19095. https://doi.org/10.1109/ACCESS.2022.3151048

Pedregosa, Fabian, Varoquaux, Gaël, Gramfort, Alexandre, Michel, Vincent, Thirion, Bertrand, Grisel, Olivier, Blondel, Mathieu, Prettenhofer, Peter, Weiss, Ron, Dubourg, Vincent, Vanderplas, Jake, Passos, Alexandre, Cournapeau, David, Brucher, Matthieu, Perrot, Matthieu, & Duchesnay, Édouard. (2011). Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12(85), 2825–2830. Retrieved from http://jmlr.org/papers/v12/pedregosa11a.html

Wolf, Thomas, Debut, Lysandre, Sanh, Victor, Chaumond, Julien, Delangue, Clement, Moi, Anthony, Cistac, Pierric, Rault, Tim, Louf, Rémi, Funtowicz, Morgan, Davison, Joe, Shleifer, Sam, von Platen, Patrick, Ma, Clara, Jernite, Yacine, Plu, Julien, Xu, Canwen, Le Scao, Teven, Gugger, Sylvain, Drame, Mariama, Lhoest, Quentin, & Rush, Alexander M. (2020). HuggingFace’s Transformers: State-of-the-art natural language processing. arXivhttps://arxiv.org/abs/1910.03771

Research Scenario: Mapping flood damages in the past (part II)

Building the dataset and identifying locations

The aim of the CLARIAH-VL Open Humanities Service Infrastructure is to advance digitally-enabled research in Humanities and the Arts by, among other things, providing data-level access to digitized and born-digital resources. In this blogpost series, we will communicate on research scenarios leading to and building upon the datasets made available through CLARIAH-VL. The previous blogpost introduced the research scenario “Mapping flood damages”. This one presents the methodology that was used, while the last one will display the results obtained. 

Aim of the research scenario

Our societies are facing major meteorological disasters because climate changes greatly and because societies are no longer prepared to face them: many people live in risky areas when the risk itself is culturally no longer accepted. Recent research has shown that past societies used to be more resilient towards meteorological bad events (Soens 2018, De Keyzer et al. 2024), at least to the non-exceptional ones. Thinking about the daily resilience of societies to such events in the past means taking stock of past bad weather events: how frequent were they? What were the causes? Which places were the most impacted? This can be difficult because these events have whether not always left much trace in the documentation (because they were common), whether so much trace (because they were numerous) that they are difficult to manage. However, digital tools can help researchers in both cases. They can for instance greatly facilitate the visualization of specific natural disasters by automatizing the mapping of (forgotten) affected places. The aim of this research scenario is precisely to map semi-automatically the locations that have been mentioned in the Gazet van Antwerpen as having been under water because of a flood or a storm between 1911 and 1921, in order to try to answer the following question: where are the places that were impacted by storms and floods in provinces Antwerp and East-Flanders between 1911 and 1921? and To what extent were they affected ? 

Steps taken

Step 1: Building a the dataset

We looked for every mention of the words “onder water” from 1911 to 1921 (“under water” in English. Ex: “this hamlet was under water because of the storm”) in the newspaper Gazet van Antwerpen accessed on BelgicaPress, the online newspaper database of the Royal Library of Belgium (KBR).

Extract from the Gazet van Antwerpen, 2d of August, 1911 (https://uurl.kbr.be/1361477)

The words “onder water” appear on 317 pages between 1911 and 1921. Most of the time, this expression is used in the following contexts:

    1. consequently to a flood or a storm in Belgium 
    2. consequently to a flood or a storm abroad
    3. when someone died from drowning
    4. when a fire broke out (most of the time on a ship but not always) and had to be extinguished using massive quantities of water

Once set aside the newspaper articles published to report one of the last three events, we can count 58 newspapers where the expression “onder water” is used at least one time to report the consequences of a flood or a storm that occurred somewhere in Belgium. 

Unfortunately, BelgicaPress does not yet give access to the OCR version of the newspapers which means that, to be extracted, the results of a research has to be manually copied. We therefore copied manually the content of all the relevant newspapers articles containing the expression “onder water” in an Excel sheet. 

Step 2:  Matching the articles with the Belgian Historical Gazetteer

All these articles contain at least one place name. However localizing those places can prove difficult since names of town, villages or hamlets changed greatly in Belgium in the last two centuries, because of incorrect transfers or translations and the variability introduced by local dialects (Von Busekist 1998, Taeldeman 2001, Witte 2011). We therefore cannot use a contemporary gazetteer to locate places mentioned in those articles, and have to use an historical one, namely the Belgian Historical Gazetteer. This gazetteer – still under construction (CLARIAH-VL, SIC 4) – aims at providing researchers with a collection of data that 1) does not stop at Belgian provincial borders, which 2) goes beyond the level of municipalities and 3) that does not stop to the 19th century but go deeper in the past.

For the needs of this research scenario, we made a first matching test between the content of the articles and the Belgian Historical gazetteer (BHG). We did this using a formula that asks Excel to look for specific character strings (place names registered in the gazetteer) within a text (the newspaper articles). 

The BHG currently covers only provinces Antwerp and East-Flanders. A lot of the place names mentioned in the articles could therefore not be found in de gazetteer (36% of the total). For 42% of the remaining data, the matching worked perfectly giving the good result immediately. In 15% of the cases, we had to manually do a small intervention to get the good result (for instance a small correction in the spelling of the place name that had been incompletely or badly transcribed). In 38% of the cases, the formula did not succeed in finding the correct place name in the gazetteer, for various reasons:

  • the formula found the good toponym but not in the right province (disambiguation problem)
  • the formula could only find one toponym among the several that were mentioned in the article (the formula is not made to repeat the search once one toponym has been found)
  • the formula stopped the search once it has found a corresponding toponym in the gazetteer that was only part of the one mentioned in the article (example: “Gentbrugge” was matched with “Gent”)

In the previous cases we had to manually correct of supply the toponym found by the formula. Those problem will be solve in the future, in the framework of CLARIAH +, when a collaboration with experts in text extraction will help us to get a better matching query. 

In the next blogpost we will explain we get a visualization of the results of this matching test and how it offers research perspective.

Next step (and next blogpost)

Visualizing floods damages (blogpost 3/3)

References

Von Busekist, Astrid. La Belgique. Politique des langues et construction de l’État, de 1780 à nos jours, Paris-Bruxelles, Éditions Duculot, 1998.

Maïka De Keyzer, Tim Soens and Christophe Verbruggen, 2024. Mens en natuur : een geschiedenis, (Gent: Academia Press), 2014, 313p. 

Taeldeman, Johan. “De Regenboog van de Vlaamse Dialecten.” Het Taallandschap in Vlaanderen, Johan Taeldeman et al. (eds.), Academia Press, 2001, 49–58.

Witte, Els. “La question linguistique en Belgique dans une perspective historique », Pouvoirs 136(1), 37-50.

Tim Soens, 2018. “Resilient Societies, Vulnerable People: Coping with North Sea Floods Before 1800”, Past & Present, 241 (1), 2018 : 143–177, https://doi.org/10.1093/pastj/gty018

CLARIAH-VL+ receives funding: 4 more years!

CLARIAH-VL Presentation

Our project CLARIAH-VL+: Paving the Way for an SSH Open Science Cloud for Flanders has been funded by the Research Foundation Flanders (FWO) as an International Research Infrastructure (IRI) for an additional 4 years. A grant of €4,286,252 will support the continued collaboration between the universities of Ghent, Antwerp, Leuven, and Brussels and their partners. Building on the achievements of the CLARIAH-VL Open Humanities Service Infrastructure (2018–2020) and CLARIAH-VL Advancing Open Humanities Service Infrastructure (2021–2024), this new phase will consolidate previous efforts.

CLARIAH-VL+ partners
CLARIAH-VL+ partners

Tackling pressing societal challenges

To advance digitally-enabled research in the Social Sciences and Humanities (SSH), CLARIAH-VL+ encompasses disciplines such as literature, linguistics, history, sociology, political science, and economics. The initiative establishes a robust technical, knowledge, and social infrastructure to connect Flemish SSH scholars with to data-driven research communities in Europe and beyond. We will address unique infrastructural needs, such as the management of heritage and historical sources.

We want to bridge the Humanities and Social Sciences to tackle pressing societal challenges, including environmental change, social inequalities, and cultural diversity with CLARIAH-VL+. The project will promote interdisciplinary collaboration by integrating SSH with fields like ecology, climatology, and geography. The transformation of archival and historical materials, including maps, meteorological data, and censuses, into digital formats is of key importance to address critical societal questions.

High-quality user-friendly tools

Leveraging advancements in artificial intelligence (AI), Linked Open Data (LOD), and semantic web technologies, CLARIAH-VL+ will develop modular Service Infrastructure Components (SICs) to support researchers in analysing, sharing, and enriching diverse multilingual Humanities data. These SICs include high-quality, user-friendly tools and resources for analysing, sharing, connecting and enriching dense SSH research data such as the Belgian Historical Gazetteer, a crowdsourcing and annotation platform, and AI-driven automation for as many aspects of the workflows of researchers working with Humanities data as possible.

Public engagement

CLARIAH-VL+ will also foster public engagement by adhering to Open Science principles by operating as an Open Humanities Service infrastructure, and making research accessible and participatory. We will continue our collaboration with established entities such as CLARIN-ERIC and DARIAH-EU, and align with European Science Cluster developments, ensuring compatibility with existing resources and standards. The CLARIAH-VL+ Academy will serve as a central hub for training and educational resources, providing an entry point for DH researchers and practitioners by integrating with internal and external platforms such DARIAH-Campus, the CLARIN Learning Hub, the Digital Humanities Course Registry, and many more.

The roadmap of CLARIAH-VL+, paving the way for a SSH Open Science Cloud for Flanders


Within the CLARIAH-VL+ framework, we will also extend our partnerships with EU research infrastructures, including DARIAH and CLARIN, while reaching out to established ERICs with Flemish coordinators such as LifeWatch (coordinated by VLIZ) and DiSSCo (coordinated by Meise Botanic Garden). Furthermore, we will engage with emerging ESFRI research infrastructures, such as RESILIENCE, EHRI (the European Holocaust Research Infrastructure) via Kazerne Dossin, and CESSDA.

We would like to thank the FWO for the continued support and look forward to strengthening the SSH research infrastructure in Flanders and advancing our collaboration with existing and new partners over the next four years!

Bibundina: a Writer’s Library app

The Writer’s Library application “Bibundina”, developed at the University of Antwerp as part of eXtant, a toolkit for digital scholarly editing, aims to provide an environment both to create and to publish an edition of a collection of books and the reading traces in them. In origin, it was created to publish an edition of a writer’s library. It could for instance be used to make an edition of Virginia Woolf’s personal library, or Toni Morrison’s, or James Joyce’s—a digital reconstruction of the books that occupied their shelves. Such an edition can contain not just the bibliographic details of those books, but also focus on the marginalia left in them by the writers, as well as include information about the books’ provenance and when they were read.

Target audience

The project’s target audience consists of people who have an interest in creating such an edition of a library and are looking for a publication environment. The only technical skill users will need is an understanding of XML: to fill up an XML document following a well-documented schema. Bibundina was written as an application in eXist-db, an open-source NoSQL XML database and application platform. Bibundina can be easily installed via eXist’s “package manager” by dropping a compressed version of the app into the dropzone. Included with the eXist installation is eXide, eXist-db’s code editor. Users can edit the Writer’s Library app’s XML files in eXide and immediately see their work visualized in the publication environment. There are just two XML files to edit: a config file in which users can give their edition a name and select which sorting and browsing categories to use, and a file called “library.xml”, the main data file in which to encode the books.

Images

Images of the books can be added. The app accommodates three different ways of incorporating images, including IIIF, which will allow scholars to create editions based on their own research data or on collections of images that have been made available via IIIF at institutions all around the world. In the book view, users can browse through the images of the pages the editor wishes to include, cover and title page, for instance, and all pages with reading traces.

Reading traces

For pages with reading traces, zone numbers appear in the left margin of the image, at the same level as their corresponding trace. Clicking on a zone number will activate two pop-up windows on the right: a cropped picture of the reading trace in question and a text box with a transcription of the reading trace, the marked passage and an extract of the passage in context.

Some admin tools have been embedded into the interface to facilitate the data input. There is a tool to help editors create a new book entry, to extract image links from a IIIF manifest and formats them into the required XML schema, and a tool that assists admins with drawing rectangular zones around reading traces on IIIF images.

Available on GitHub: the source code, the current binary release, the documentation, and the user manual.

https://github.com/eXtant-CMG/writerslibrary

CLARIN Annual Conference 2024: report from the CLARIAH-VL representatives

The CLARIN Annual Conference 2024 was held in Barcelona from October 15 to 17 and brought together over 200 in-person attendees and nearly 100 virtual participants, establishing it as a central forum for advancing language technology and resource sharing across Europe. Chaired by Vincent Vandeghinste, coordinator of CLARIN-Belgium, the conference explored both theoretical and applied aspects of linguistic resource management and computational processing.

One of the keynote speakers was Maite Melero from the Barcelona Supercomputing Centre, whose talk, “The Future of Language (and Cultural) Diversity in the Age of AI,” addressed AI’s complex impact on linguistic diversity. Melero discussed AI’s dual potential: while it can support endangered languages, it also risks reinforcing the dominance of major languages and perpetuating biases. Her presentation highlighted the necessity for careful AI integration in multilingual contexts, with researchers and policymakers playing a crucial role in guiding AI toward linguistic inclusion.

Steven Bird from Charles Darwin University provided a complementary perspective in his address, “Making it Meaningful.” Drawing on his extensive work with under-resourced languages, Bird stressed the importance of community-driven approaches to linguistic preservation, especially through tools that enable local communities to document endangered languages. He argued that such technology should empower communities to preserve their linguistic heritage actively.

The conference program featured more than 20 oral presentations and multiple poster sessions, fostering lively discussions on the latest research and methodologies in the field.

The conference program included over 20 oral presentations and several poster sessions, encouraging engagement with recent research and methodologies in the field. The CLARIAH-VL delegation included Els Lefever, Belgium’s representative in CLARIN’s User Involvement Committee, Jonas Doumen, who gave a first demo of TextLens and Tess Dejaeghere, a PhD researcher at Ghent CDH. Tess contributed to both the main conference with an oral presentation and to the PhD session with a poster.

Further details on the conference program, presentations, and proceedings can be found on the official CLARIN Annual Conference 2024 page.

The Belgian Historical Gazetteer: an index of Belgian historical place names to link archival collections and explore landscapes histories

Legend: extracts from various sheets of the ‘reduced cadastre’ (©IGN/NGI)

Gazetteers can help historians with mapping toponyms that appear in the sources they analyse by providing them with lists of historical place names and extra information which can be used to disambiguate the latter, if it happens that several different places share a same name. This is the aim of initiatives such as the World Historical Gazetteer or Pleiades. However, despite their merits, the international scope of these tools impedes the use for regional and local historical research. Historians thus need more suitable gazetteers if they want to work at a more local scale.

CLARIAH-VL aims to fill in this gap for Belgium by launching the project “Belgian Historical Gazetteer” in the framework of its SIC “Enrich”. Started in November 2022, the pilot of this project will last two years, and eventually aims to cover the whole country in the coming years. It is hosted at the University of Antwerp and undertaken in liaison with the Belgian National Geographical Institute (IGN/NGI). The aim of this project is to set up a historical gazetteer of toponyms for the whole present-day territory of Belgium, in order to provide researchers with a collection of data that 1) does not stop at Belgian provincial borders 2) goes beyond the level of municipalities and 3) goes back in time as much as possible.

Pilot test

The first phase of the project (pilot 2022/2024) consists in the gathering of an ancient and geographically homogeneous set of toponyms for provinces Antwerp, East Flanders and Liège. This is done by collecting all the toponyms mentioned on the ‘reduced cadastre’, a reduced version of the primitive cadastre drawn between 1847 and 1855 to prepare the creation of the first topographical map of Belgium in the second half of the 19th century.

Workflow of the ‘reduced cadastre’ (©IGN/NGI)
Workflow of the ‘reduced cadastre’ (©IGN/NGI)

From QGis to Linked Open Data

These toponyms are located by using points in QGis and described via the PostGis extension in a PostgreSQL database whose structure is highly adaptable to every source and compatible with Linked Open Data principles. For each toponym we record not only its name, location, type (as described in the source, and a standardized version of it), but also if applicable its matching number in databases of other projects (for instance the “placename” project of the State Archives, the “Dorpskernen” project for Flemish toponyms), its corresponding Wikidata page if it already exists and its corresponding notice in the book Gemeenten van Belgïe[1], where Hervé Hasquin and his team summarized historical information (notably administrative and ecclesiastical belonging during the Ancien Régime) about municipalities.

The ultimate aim of drawing those links with the existing (notably historical) resources is to provide researchers with sufficient information on each toponyms to facilitate the disambiguation of place names. For instance, if one historian wants to locate a place called “heikant” in the primary source and if it appears that there are several “heikant” in Belgium, it may help him/her to identify the correct one among the latter if he knows in which bishopric each of them belongs in the seventeenth century. Finally, thanks to another table of the database (table “relations”), we also describe elementary relations between the toponyms like the administrative ones (“this hamlet belongs to this town” / “this isolated chapel belongs to this town”).

Future integration and accessibility

At the end of the project, this gazetteer will be published on the World Historical Gazetteer, and will progressively be loaded on Druid (a LOD platform), in order to make it easily reachable for everyone as well as potentially extendable. This gazetteer will enable researchers to easily identify place names they find in their sources, and could be used in the future to automatically ‘map’ (e.g. annotated) written sources to a great extent.

Beyond these very practical benefits, we think that this Belgian gazetteer can also be used to open new perspectives in landscape history by making it possible to follow the evolution of the Belgian landscape through the evolution of toponyms (name and location) on the long-term. This is why, in a second phase (which in practice is ran parallel to the first), we compare the nineteenth century toponyms with present-day ones using the database of the IGN/NGI. By establishing links between ancient and actual toponyms, we can explore how a toponym evolved during the nineteenth and the twentieth century in order to research variations of place names through time. In a third phase, toponyms extracted from older material will be added to the database and the same sort of links will be described, in order to make it possible to follow the evolution of place name on a longer term.


[1] Duvosquel, Jean-Marie, Hervé Hasquin, and Raymond Van Uytven. Gemeenten Van België: Geschiedkundig En Administratief-Geografisch Woordenboek. Bruxelles: La Renaissance du Livre, 1980.

DH Benelux 2024: report about the CLARIAH-VL contributions

From June 4-7, the 11th edition of the DH Benelux Conference was held at the Irish College in Leuven, Belgium. With the theme “Breaking Silos, Connecting Data: Advancing Integration and Collaboration in Digital Humanities,” the event gathered a diverse group of Digital Humanities researchers, eager to explore new ways to connect and share knowledge across disciplines. The conference kicked off with pre-conference workshops on June 4, followed by three days of presentations and discussions.

CLARIAH-VL pre-conference workshop

As part of the DH Benelux pre-conference program, CLARIAH-VL hosted a full-day, in-person workshop. The workshop highlighted three tools developed within the CLARIAH-VL infrastructure project: a named entity referencing toolkit, a tool for digital scholarly editing, and a participatory digital asset enrichment platform based on IIIF technology

The workshop was organised into two interactive sessions:

Morning Session: Attendees explored a named entity referencing toolkit (presented by Frederic Pietowski & Tom Gheldof) and a tool for digital scholarly editing (presented by Lamyk Bekius, Vincent Neyt & Nooshin Shahidzadeh Asadi) gaining practical insights into their functionalities and applications.

Afternoon Session: Focused on Madoc, an IIIF-based platform (presented by Lise Foket, Fien Danniau, Rein Debrulle, Julie Birkholz & Annamaria Van Ingelgem) designed for participatory digital asset enrichment, allowing researchers to annotate and collaborate on image-based datasets.

This hands-on workshop offered participants a deeper understanding of these tools’ potential to enhance digital humanities research. The workshop bridged theoretical knowledge and practical application, underscoring CLARIAH-VL’s commitment to advancing digital scholarship.

CLARIAH-VL contributions

Members of CLARIAH-VL played an important role throughout the conference. Tom Gheldof, the CLARIAH-VL coordinator at KU Leuven’s Faculty of Arts, served on the DH Benelux Organising and Program Committee. Supervisors of KU Leuven, Mark Depauw and Fred Truyen acted as the local chairs of the conference, strengthening the event’s ties to the local academic community. Moreover, several CLARIAH-VL coordinators (Lamyk Bekius, Julie Birkholz, Mike Kestemont, and Mark Depauw) chaired sessions, contributing their expertise and fostering discussions throughout the conference.

Several CLARIAH-VL researchers also presented a (joint) paper:

Lamyk Bekius presented in the session Literature and Fiction on day 1, with the paper “What does that little black square store? The contents of Herman de Coninck’s floppy disks in the Letterenhuis”, co-authored with Jordan Thijs.

Mike Kestemont was truly a conference marathoner at DH Benelux 2024! On day 1, he presented in the session Statistics and Patterns, with the paper “Painting a bigger picture: the annotation of emotion in Middle Dutch literature”, co-authored with Cecile Vermaas and Laurent Breeus-Loos. He was back again on day 2 in the session Textual Analysis and Stylometry with the paper Abbreviation Application: A Stylochronometric Study of the Abbreviations in the Oeuvre of Herne’s Speculum Scribe, co-authored with Caroline Vandyck. He also made it to day 3, where he presented in the Disambiguating and Annotating Historical Text session with the paper “Coverage-based Comparisons of Cultural Diversity”, co-authored with Folgert Karsdorp and Melvin Wevers. Three days in a row – talk about dedication!

Tim Van de Cruys shared his expertise on day 2 in the session on NER, with the paper “Named Entity Recognition for a Large Scale Analysis of Individuals in Antiquity“, co-authored with Marijke Beersmans, Evelien de Graaf, Alek Keersmaekers, and Margherita Fantoli.

Julie Birkholz took part in the Computer Vision session on Day 3, with the paper “Finding a Needle in a Haystack: Computer vision and machine learning techniques for extracting comics from Belgian Illustrated Periodicals in the Interwar Period“, co-authored with Benoît Crucifix, Erwin Dejasse, Sébastien Hermans, Krishna Kumar Thirukokaranam Chandrasekar, and Bas Vercruysse.

Sytze Van Herck presented in the Disambiguating and Annotating Historical Text session on day 3, with the paper “PiCo in Practice: An ontology to standardise person reconstructions”, co-authored with Rick Mourits and Ivo Zandhuis.

Styloscope and Toposcope: Towards User-Friendly Digital Text Analysis

Natural Language Processing (NLP) has been one of the fastest-growing research fields in the last decade. Innovations such as pre-trained large language models based on transformer neural networks have not only led to the popularization of AI and NLP in the general public, but also to interdisciplinary research projects in the humanities and social sciences facilitated by the scalability of these methods. In this post, we present two tools that aim to facilitate said interdisciplinary research: Styloscope and Toposcope.  The tools were developed in Python and can be used from the command line or from a user interface. The code, detailed installation instructions, and user guidelines can be found on GitHub:

Styloscope

Styloscope is a tool for automatic writing style analysis. It can be used to test hypotheses about large-scale corpora, parse documents, or detect outliers. Users can provide data by either uploading a local file or by using a publicly available Huggingface dataset. When uploading a corpus, the tool accepts CSV files with one document per row, and ZIP folders in which documents are stored in individual text files. The output contains the parsed documents, raw statistics on various writing style features such as syntactic dependencies, lexical richness,  readability, etc., and visualizations of aggregated results. An example for syntactic dependencies is provided below:

Toposcope

Toposcope can be used to detect topics in unstructured text data. It provides annotations and visualizations of the detected topics, including (changes in) topic frequency over time. The tool features four algorithms: BERTopic (Grootendorst, 2022), Top2Vec (Angelov, 2020), Non-negative Matrix Factorization (Choo et al., 2013), and Latent Dirichlet Allocation (Blei et al., 2003). Users can modify a selection of topic model parameters, and apply a number of built-in preprocessing steps, such as lemmatization and stopword removal. The input format is identical to the Styloscope format: users can upload a local corpus (CSV/ZIP), or use a Huggingface dataset. The output includes visualizations of the topic-document clusters (as shown below) and the most important keywords per topic. The raw results, among other things, consist of annotations, a topic-document matrix, and a topic-term matrix. Topic diversity and topic coherence are also computed in order to support the user during the evaluation of the tool.

How to cite

Jens Lemmens and Walter Daelemans. 2024. Styloscope and Toposcope: Towards user-friendly digital text analysis. CLiPS Technical Report Series (CTRS): 10. https://www.uantwerpen.be/en/research-groups/clips/research/computational-linguistics/compling-resources/clips-technical-repo/

References

  • Dimo Angelov. 2020. Top2Vec: Distributed representation of topics. arXiv:2008.09470.
  • David M. Blei, Andrew Y. Ng, and Michael I. Jordan. 2003. Latent dirichlet allocation. Journal of Machine Learning Research, vol. 3, pp. 993—1022.
  • Jaegul Choo, Changhyun Lee, Chandan K. Reddy and Haesun Park. 2013. Utopian: User-driven topic modeling based on interactive nonnegative matrix factorization. IEEE Transactions on Visualization and Computer Graphics, vol. 19 (12), pp. 1992—2001. Institute of Electrical and Electronics Engineers (IEEE).
  • Maarten Grootendorst. 2022. BERTopic: Neural topic modeling with a class-based TF-IDF procedure. arXiv:2203.05794.

CLARIAH-VL SIC 5 tool descriptions

Reading historical maps in a Digital Era

The promises of Artificial Intelligence technologies to extract historical information from maps are becoming more impressive each day. However, these tools have been developed not only on the basis of but also, for modern maps. Older ones, including hand-drawn maps (early nineteenth century and earlier) therefore remain, as often with AI, the poor siblings of AI revolution, because the models used to extract information from maps do not work on those older documents: older maps have other characteristics than modern ones, often do not display the same sort of information which also means that researchers that are working on this kind of documents do not always have the same research questions than their colleagues working on modern ones.

On the 17th of November, members of the Antwerp Group of CLARIAH-VL (Iason Jongepier and Léa Hermenault assisted by Lamyk Bekius and Rein Debrulle), organized a workshop in Antwerp, with the support of CLARIAH-VL, the University of Antwerp and Ghent University, that aimed to tackle this issue. The underlying idea of this workshop was to facilitate brainstorming around technical solutions that allow older maps to also benefit from AI technology and AI-related workflows.

The workshop gathered 25 participants and welcomed 13 speakers from various horizons. In the morning, researchers from the leading Alan Turing Institute gave presentations and demos regarding two applications from the “Living with machines” project (namely MapReader and Machines Reading Maps). In the afternoon, colleagues from the University of Antwerp, Ghent University and University of Amsterdam presented their own work on AI and AI-related workflows, among others.

Program of the ‘Reading Maps in a Digital Era’ workshop

MapReader, a computer vision pipelines for exploring and analysing images

We first had the great pleasure to hear Katherine McDonough (Lancaster University & Alan Turing Institute) and Daniel Wilson (Alan Turing Institute) who came to introduce the tool MapReader they developed with their team in the framework of the “Living with Machines” project. This tool is open source and can be installed and used by anyone thanks to the instructions available here. Originally developed to automatically browse railways components on the Ordnance survey of England, it can be used to help researchers finding on a raster document any elements they are looking for by automatically identifying the later in pre-defined patches.

MapReader can also simply be used as a way to annotate patches. The tool first needs to be trained on a sample of patches, whose number depend on the size of the patches and the number of the specific characteristics of the browsed element: if the latter is not easily distinguishable from other elements, then the tool will need to be trained on a very large number of patches, but if on the contrary the element has very clear and specific characteristics, then the model only needs to be trained on a relative small number of patches. It should be possible to use the tool to explore old and hand-written maps if elements are easily identifiable, given that enough maps with these elements exist. One of the main problems about older maps indeed remains that we usually do not have enough material to train the models: very few maps collections dated from before the nineteenth century constitute series.  

MapReader patches on the Ordnance Survey

“Machines Reading Maps”, a tool to automatically transcribe texts on maps

After a short break, Katherine McDonough and Valeria Vitale (Sheffield University & Alan Turing Institute) introduced the audience to another tool that has been developed by the University of Southern California Digital Library, the Computer Science & Engineering Department at the University of Minnesota and the Alan Turing Institute, called “Machines Reading Maps”. This tool is trained to identify printed text on maps and to transcribe it. It has been first tested on the Rumsey collection, and then added to the platform that allows to browse it. It enables everyone to look for a toponym not only in the metadata, but also on the map itself. Machines Reading Maps can, therefore, also be used to count the occurrences of a specific place name variant in one collection for instance, or to gather text written with a specific graphic style (Bold, italic, etc.). If the later possibility would only be of interest for maps produced in series, which therefore limits drastically its use for old maps, the tool still looks promising for pre-nineteenth century cartography since writings tend to be more quickly standardized than symbology. It should definitely be tested on maps with non-printed text. 

Example of the results given by the search “Antwerpen” in the Rumsey Collection

Applying computer vision on historical documents

The afternoon was organized in three different sessions with two short papers in each of them. The first session, entitled ‘Computer vision‘ aimed at broadening our scope to the application of computer vision on geospatial and related data in general. José Oramas (University of Antwerp, Imec/IDLab) gave a paper related to his research on computer vision models applied to pictures where he tries to understand how those models really work in order to eventually improve their results. Then Thomas Smits (University of Amsterdam) introduced the audience to a research that he did together with Mona Allaert, Loren Verreyen, Wouter Haverals and Mike Kestemont at the University of Antwerp and which consisted of the use of computer vision, HTR and Large Language Models to transcribe and geo-localize addresses found on 100,000 historical postcards. This research shows how promising HTR technics are but also reveals how important it is to have a solid addresses database that can be used to geo-localize information, which is certainly reachable for modern periods, but which is more challenging for older ones.  

The second session was dedicated to the specific challenges of historical maps. The later have specificities, advantages and inconveniences that we have to be aware of if we want to build efficient and relevant applications and workflows to facilitate their digital use. The aim of this session was to focus on two very different corpus of maps to broaden our knowledge of their specificities. Dieter De Witte (Ghent University and Imec/IDLab) and Iason Jongepier (University of Antwerp and State Archives) introduced us to the specificities of historical maps of Belgium but also to the first attempts that have been done to extract information from them. Then, Katherine McDonough and Daniel Wilson showed how they used MapReader to explore “railway spaces” on the Ordnance Survey and explained the advantages of reflecting on those spaces using patches instead of vector data. 

The audience,at the end of a long day of work

Pipelines and workflows to scale up the digitization of data

The third and last session aimed at pipelines and workflows that can be use for the handling data derived from maps, or historical data with a strong spatial component. Janna Aerts (University of Amsterdam) and Leon van Wissen (University of Amsterdam and UvACreate) presented different projects for which historical data have been gathered and are connected via the linked open data system AdamLink. Next, Vincent Ducatteeuw (University of Ghent) and Léa Hermenault (University of Antwerp) gave a paper related to an article they are currently writing and that aims to show that small-scale/local gazetteers can greatly contribute to the debate regarding the structure of gazetteers by helping to choose information that should be available in a gazetteer to secure its interest for research purposes but also to meet FAIR standards. 

This fruitful day helped each of the participants to get to know new tools and to reflect on new methodological issues. It will without a doubt lead to further explorations and discussions that will hopefully help to unlock the access that historical information that old maps are packed with.

Discover Ghent through historical maps: unveiling the past with Gent Gemapt

Our partners from the Ghent Centre for Digital Humanities are proud to present their newest project: Ghent Mapped. The digital city map ‘Gent Gemapt’ stacks 20 historical maps and connects them to 4,000 places and 10,000 pieces of heritage. Collections and history that over time have become dispersed among museums, libraries, archives, associations and living rooms, come together again virtually. Reunited, they offer a multifaceted view of the city and its past. Go explore for yourself via gentgemapt.be and discover Ghent layer by layer, from the Middle Ages to today.

The cultural heritage project Gent Gemapt is a collaboration of Ghent University LibrarySTAM – Stadsmuseum GentHuis van AlijnIndustriemuseumArchief GentAmsab-Institute for Social HistoryLiberas and Erfgoedcel Gent. It was developed and coordinated by the Ghent Centre for Digital Humanities at Ghent University, with support from CLARIAH-VL.

Example of the enriched map of Ghent Mapped

Want to know more?

  1. De Kaart” is the core of Ghent Gemapt. It is an innovative presentation platform that spatially unlocks the splendor of Ghent’s heritage collections. It allows you to search, zoom and scroll through time and space of the city.
  2. Gent Verrijkt” stands next to De Kaart and is the digital toolbox of Gent Gemapt. Here, museums and archives enlist the help of volunteers to help transcribe, date, describe and identify their collections.
  3. With the project, the heritage partners are innovating in digital technology. Gent Gemapt uses open source technology Omeka SMadocIIIF (International Image Interoperability Framework), Linked Open Data, and a new Ghent place register.
  4. Gent Gemapt is funded through a project grant from the Department of Culture, Youth and Media of the Flemish Government and through Clariah-Vlaanderen.
  5. After this launch, Gent Gemapt continues to grow and the partners continue to build on the heritage presentation and enrichment via GentGemapt.be.
  6. Read About Gent Gemapt and About the project.