aboutsummaryrefslogtreecommitdiff
path: root/backend/hist_data/wikidata/README.md
blob: 35dad346c97a0eab00aa165629d43232b540c48e (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
This directory holds files obtained/derived from [Wikidata](https://www.wikidata.org/).

# Downloaded Files
-   `latest-all.json.bz2` <br>
    Obtained from <https://dumps.wikimedia.org/wikidatawiki/entities/> (on 23/08/22).
    Format info can be found at <https://doc.wikimedia.org/Wikibase/master/php/md_docs_topics_json.html>.

# Other Files
-   `gen_events_data.py` <br>
    Used to generate a database holding event information from the dump.
-   `offsets.dat` <br>
    Holds bzip2 block offsets for the dump. Generated and used by
    gen_events_data.py for parallel processing of the dump.
-   `events.db` <br>
    Generated by `gen_events_data.py`. <br>
    Has one table `events`:
    -   Columns: `id INT, title TEXT, start INT, start_upper INT, end INT, end_upper INT, fmt INT, ctg TEXT`
    -   Contains a Wikidata ID, Wikipedia title, start and end dates, and an event category.
    -   If `start_upper` is not NULL, `start` and `start_upper` denote an uncertain range of start times.
        And similarly for 'end' and 'end_upper'.
    -   `fmt` indicates format info for `start`, `start_upper`, `end`, and `end_upper`.
        -   If 1, they denote a Julian date (with 0.5 removed to align with midnight).
            This allows simple comparison of events with day-level precision, but only goes back to 4713 BCE.
        -   If 2, same as 1, but dates are preferably displayed using the Gregorian calendar, not the Julian calendar.
            For example, William Shakespeare's birth appears 'preferably Julian', but Samuel Johnson's does not.
        -   If 3, same as 1, but 'end' and 'end_upper' are 'preferably Gregorian'.
            For example, Galileo Galilei's birth date appears 'preferably Julian', but his death date does not.
        -   If 0, they denote a number of years CE (if positive) or BCE (if negative).