diff options
| author | Terry Truong <terry06890@gmail.com> | 2022-08-30 12:27:42 +1000 |
|---|---|---|
| committer | Terry Truong <terry06890@gmail.com> | 2022-08-30 12:27:42 +1000 |
| commit | e8e58a3bb9dc233dacf573973457c5b48d369503 (patch) | |
| tree | 242500ca304c5afbb7e6506e61da4c4dfff0b175 /backend/tolData/wikidata/README.md | |
| parent | 930c12d33e1093f874a4beb4d6376621e464e8c0 (diff) | |
Add scripts for generating eol/enwiki mappings
- New data sources: OTOL taxonomy, EOL provider-ids, Wikidata dump
- Add 'node_iucn' table
- Remove 'redirected' field from 'wiki_ids' table
- Make 'eol_ids' table have 'name' as the primary key
- Combine name-generation scripts into genNameData.py
- Combine description-generation scripts into genDescData.py
Diffstat (limited to 'backend/tolData/wikidata/README.md')
| -rw-r--r-- | backend/tolData/wikidata/README.md | 18 |
1 files changed, 18 insertions, 0 deletions
diff --git a/backend/tolData/wikidata/README.md b/backend/tolData/wikidata/README.md new file mode 100644 index 0000000..db45b3c --- /dev/null +++ b/backend/tolData/wikidata/README.md @@ -0,0 +1,18 @@ +This directory holds files obtained via [Wikidata](https://www.wikidata.org/). + +# Downloaded Files +- `latest-all.json.bz2` <br> + Obtained from <https://dumps.wikimedia.org/wikidatawiki/entities/> (on 23/08/22). + Format info can be found at <https://doc.wikimedia.org/Wikibase/master/php/md_docs_topics_json.html>. + +# Other Files +- genTaxonSrcData.py <br> + Used to generate a database holding taxon information from the dump. +- offsets.dat <br> + Holds bzip2 block offsets for the dump. Generated and used by + genTaxonSrcData.py for parallel processing of the dump. +- taxonSrcs.db <br> + Generated by genTaxonSrcData.py. <br> + Tables: <br> + - `src_id_to_title`: `src TEXT, id INT, title TEXT, PRIMARY KEY(src, id)` + - `title_iucn`: `title TEXT PRIMARY KEY, status TEXT` |