From 5de5fb93e50fe9006221b30ac4a66f1be0db82e7 Mon Sep 17 00:00:00 2001 From: Terry Truong Date: Sun, 11 Sep 2022 14:55:42 +1000 Subject: Add backend unit tests - Add unit testing code in backend/tests/ - Change to snake-case for script/file/directory names - Use os.path.join() instead of '/' - Refactor script code into function defs and a main-guard - Make global vars all-caps Some fixes: - For getting descriptions, some wiki redirects weren't properly resolved - Linked images were sub-optimally propagated - Generation of reduced trees assumed a wiki-id association implied a description - Tilo.py had potential null dereferences by not always using a reduced node set - EOL image downloading didn't properly wait for all threads to end when finishing --- backend/tol_data/wikidata/README.md | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) create mode 100644 backend/tol_data/wikidata/README.md (limited to 'backend/tol_data/wikidata/README.md') diff --git a/backend/tol_data/wikidata/README.md b/backend/tol_data/wikidata/README.md new file mode 100644 index 0000000..7b3105e --- /dev/null +++ b/backend/tol_data/wikidata/README.md @@ -0,0 +1,18 @@ +This directory holds files obtained via [Wikidata](https://www.wikidata.org/). + +# Downloaded Files +- `latest-all.json.bz2`
+ Obtained from (on 23/08/22). + Format info can be found at . + +# Other Files +- `gen_taxon_src_data.py`
+ Used to generate a database holding taxon information from the dump. +- `offsets.dat`
+ Holds bzip2 block offsets for the dump. Generated and used by + genTaxonSrcData.py for parallel processing of the dump. +- `taxon_srcs.db`
+ Generated by `gen_taxon_src_data.py`.
+ Tables:
+ - `src_id_to_title`: `src TEXT, id INT, title TEXT, PRIMARY KEY(src, id)` + - `title_iucn`: `title TEXT PRIMARY KEY, status TEXT` -- cgit v1.2.3