User:Trokhymovych/drafts/Multilingual readability model card

Model card
This page is an on-wiki machine learning model card.
A model card is a document about a machine learning model that seeks to answer basic questions about the model.
Model Information Hub
Model creator(s)	Mykola Trokhymovych and Martin Gerlach
Model owner(s)	Martin Gerlach
Code	training and inference
Uses PII	No
In production?	No
This model uses article text to predict how hard it is for a reader to understand it.

How can we understand how difficult is the article for the reader?

While there are off-the-shelve readability-scores for English (and other languages), it is not clear how these approaches can be used to assess readability across the more than 300 language versions of Wikipedia.

As part of the program to address knowledge gaps, the Research team at the Wikimedia Foundation has started to develop a taxonomy of knowledge gaps. One of the goals is to identifying metrics to quantify the size of these gaps. This project focuses on readability-gap in Wikimedia projects (with focus on supporting multiple languages).

We propose using pre-trained multilingual language models as a main component for readability scoring. In particular we use fine-tuned mBERT. It supports not all but about 100 languages with the largest Wikipedias.

One of the main challenges is that for most languages there is no ground-truth data available about the reading level of an article so that fine-tuning or re-training in each language is not a scalable option. Therefore, we train the model only in English on a large corpus of Wikipedia articles with two readability levels (Simple English Wikipedia and English Wikipedia).

Users and uses

Use this model for

• Define the readability score of Wikipedia article revision
• Define the Flesch–Kincaid score of the article in multilingual setup
• Compare readability of different revisions of the same article

Don't use this model for

• Making predictions on language editions of Wikipedia that are not in the listed languages or other Wiki projects (Wiktionary, Wikinews, Wikidata, etc.)

Current uses

Model

Implementation

Model architecture

Output schema

Example input and output

Data

Data pipeline

Training data

Test data

Licenses

• Code: Apache 2.0 License
• Model: Apache 2.0 License

Citation

To be added soon.