MegaRhyme's Wikinews Dataset

Overview

We extracted data from 21,403 articles on the English Wikinews site. As of September 30, 2022, that's roughly 98% of all of Wikinews's English articles. The data has been placed in JSON file and can be downloaded from this page.

Structure of the data

The JSON file contains an array, and each item in the array contains data for a particular article. Each items follows the structure shown below.

{
    "title": "Example title",
    "text": "Example article body\n\nParagraphs are separated by two newline characters.",
    "date": "yyyy-mm-dd",
    "categories": ["example_categories", "are all", "lower case"]
    }

Download

The data at a glance

• Total number of articles: 21,403
• Number of unique categories: 3,219
• Average number of categories per article: 7.2
• Earliest date: 2004-11-13
• Most recent date: 2022-11-18

Extracting information from "unstructured data" doesn't always work perfectly. In some cases, data points are altogether missing from the source. However, for the vast majority of all articles, a datapoints were obtained for all fields. Below is a rough overview.

• articles with a title: 21,403
• articles with a date: 21,397
• articles with at least one category: 21,326
• articles with text: 21,388

License