aboutsummaryrefslogtreecommitdiff
path: root/backend/data/README.md
diff options
context:
space:
mode:
authorTerry Truong <terry06890@gmail.com>2022-05-04 01:17:06 +1000
committerTerry Truong <terry06890@gmail.com>2022-05-04 01:17:06 +1000
commit90a5e15bb824b84e5bb60337d6a57a1394090dc6 (patch)
tree661ea356c8d83b74d16f19d3555b0a1d3eb6eb56 /backend/data/README.md
parentec29e5731136c74a1991e2f93b5e233747f2a230 (diff)
Add scripts for obtaining/sending/displaying wikipedia descriptions
Add backend/data/enwiki/ directory containing scripts and instructive READMEs. Adjust some other scripts to generate 'eol_ids' sqlite table separate from 'names'. Make server respond to /data/desc requests, and have client TileInfo component display response data. Also adjust .gitignore entries to be root-relative.
Diffstat (limited to 'backend/data/README.md')
-rw-r--r--backend/data/README.md36
1 files changed, 23 insertions, 13 deletions
diff --git a/backend/data/README.md b/backend/data/README.md
index e639cb6..8791fb4 100644
--- a/backend/data/README.md
+++ b/backend/data/README.md
@@ -1,21 +1,31 @@
File Generation Process
=======================
-1 Obtain data in otol/ and eol/, as specified in their README files.
-2 Run genOtolData.py, which creates data.db, and adds a 'nodes'
- table using data in otol/*.
-3 Run genEolNameData.py, which adds a 'names' table to data.db,
- using data in eol/vernacularNames.csv and the 'nodes' table.
-4 Run genSpellfixNameData.py, which adds a 'spellfix\_alt\_names'
- table to data.db, using data in the 'names' table.
-5 Use downloadImgsForReview.py to download EOL images into imgsForReview/.
- It uses data in eol/imagesList.db, and the 'names' table.
-6 Use reviewImgs.py to filter images in imgsForReview/ into EOL-id-unique
- images in imgsReviewed/ (uses 'names' to display common names).
-7 Use genImgsForWeb.py to create cropped/resized images in img/, using
- images in imgsReviewed, and also to add an 'images' table to data.db.
+
+1 Tree Structure Data
+ 1 Obtain data in otol/, as specified in it's README.
+ 2 Run genOtolData.py, which creates data.db, and adds a 'nodes'
+ table using data in otol/*.
+2 Name Data for Search
+ 1 Obtain data in eol/, as specified in it's README.
+ 2 Run genEolNameData.py, which adds 'names' and 'eol\_ids' tables to data.db,
+ using data in eol/vernacularNames.csv and the 'nodes' table.
+ 3 Run genSpellfixNameData.py, which adds a 'spellfix\_alt\_names'
+ table to data.db, using data in the 'names' table.
+3 Image Data
+ 1 Use downloadImgsForReview.py to download EOL images into imgsForReview/.
+ It uses data in eol/imagesList.db, and the 'eol_ids' table.
+ 2 Use reviewImgs.py to filter images in imgsForReview/ into EOL-id-unique
+ images in imgsReviewed/ (uses 'names' and 'eol_ids' to display extra info).
+ 3 Use genImgsForWeb.py to create cropped/resized images in img/, using
+ images in imgsReviewed, and also to add an 'images' table to data.db.
+4 Node Description Data
+ 1 Obtain data in enwiki/, as specified in it's README.
+ 2 Run genEnwikiData.py, which adds a 'descs' table to data.db,
+ using data in enwiki/enwikiData.db, and the 'nodes' table.
spellfix.so
===========
+
This file provides the spellfix1 extension for Sqlite, and
is used for responding to fuzzy-search requests.