| Age | Commit message (Collapse) | Author |
|---|
|
- Add unit testing code in backend/tests/
- Change to snake-case for script/file/directory names
- Use os.path.join() instead of '/'
- Refactor script code into function defs and a main-guard
- Make global vars all-caps
Some fixes:
- For getting descriptions, some wiki redirects weren't properly resolved
- Linked images were sub-optimally propagated
- Generation of reduced trees assumed a wiki-id association implied a description
- Tilo.py had potential null dereferences by not always using a reduced node set
- EOL image downloading didn't properly wait for all threads to end when finishing
|
|
Also use consistent quote symbols
Also use 'is None' instead of '== None'
Also use 'if list1' instead of 'if len(list1) > 0'
|
|
Add Wikipedia pageview dumps to enwiki/pageview/
Add scripts to generate viewcount averages
Update backend to sort search suggestions by popularity
|
|
- New data sources: OTOL taxonomy, EOL provider-ids, Wikidata dump
- Add 'node_iucn' table
- Remove 'redirected' field from 'wiki_ids' table
- Make 'eol_ids' table have 'name' as the primary key
- Combine name-generation scripts into genNameData.py
- Combine description-generation scripts into genDescData.py
|
|
|
|
Previously, image files in backend/data/img were moved to, or
symlinked from, public/. This needed to be changed before each
build, otherwise vite would end up copying gigabytes of images.
|
|
|
|
Generate a 'trimmed' reduced tree instead of changing the original.
Generate an 'images-only' reduced tree, and use it as the default.
Combine 'picked' reduced tree code with that of other reduced trees.
Adapt server API to allow selecting between more than 2 trees.
Add client setting for selecting between 3 trees.
|
|
|
|
|
|
|
|
Also enable getEnwikiImgData.py to skip already-processed wiki-ids
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Some nodes had multiple ancestors with over 10k children, and
jump-searching to them could take almost a minute for vue to load.
|
|
Adjust enwiki code to handle single dump file, and add scripts for
'convenient' page-content lookup.
|
|
Add backend/data/dbpedia/ directory containing scripts and README
for obtaining DBpedia data, storing it into a db, converting/adding
description data to data.db, and for resolving tol-node DBpedia-node
association conflicts (via DBpedia relations, manual listing, etc).
Resulted in less (about 3/4 as many) descriptions as with using enwiki,
but with notably less mis-associations (eg: node Thor is described as
a shrimp instead of a god).
|
|
Add backend/data/enwiki/ directory containing scripts and instructive
READMEs. Adjust some other scripts to generate 'eol_ids' sqlite table
separate from 'names'. Make server respond to /data/desc requests,
and have client TileInfo component display response data.
Also adjust .gitignore entries to be root-relative.
|
|
|
|
Adapt otol-data-converting script to generate otol.db, add server.py
script that provides access to that db, and adapt the app to query the
server for tol data when needed.
|
|
Added data_otol/ with script that converts data from 'Open Tree of Life' release 13.4 into a JSON form.
Moved old tree-of-life data and images into data_tol_old/.
Added TolMap type to tol.ts, changed TolNode, and adapted other code to handle it.
Temporarily disabling tile images until image data is added.
|
|
|
|
|
|
|
