fishtraits reads the Wayback capture (2025-02-04) of the ScienceBase item's
whole-item zip and extracts FishTraits_14.3.xls with
download_and_unzip(). ScienceBase now answers with an interactive
Cloudflare Turnstile checkbox, which clears only in a visible Chrome on a
logged-in desktop, so an unattended build cannot fetch it. The archived
file is byte-identical to the one ScienceBase serves (SHA-1
a5d18bc605e1cd5d237e124ebb61abad2f3ed263, uploaded 2018-02-08) and parses
to the same values as the published fishtraits.vtr on all 773 shared
names.
cf_fetch.py clicks an interactive Turnstile checkbox, and on Windows opens
its clearing Chrome on the user's desktop when it runs from session 0 (a
shell under a service or background agent). A Chrome started in session 0
renders on an invisible desktop, and Turnstile re-issued its challenge after
every click there.
normalize_backbone() writes a source's publication reference under one
name, name_published_in (WFO and COL namePublishedIn, WCVP
first_published), plus an integer year, from the source's own year where
it has one and else from the reference. The COL XR reader keeps the export's
namePublishedIn / namePublishedInYear. taxify's homonym pick read these
only on GBIF before, so on every other backbone the step was silent
(#60, gcol33/taxify#91).build_name_lookup() no longer selects bracket_year, which the pick no
longer reads: a recombination is dated by its own publication.resolve_genus_classification()
picks one winning row per genus: accepted before provisional before
synonym, then a row placed in a phylum or family before one naming only a
kingdom, then the kingdom with the most accepted species in any one source,
then priority. Before, the first source by priority decided, so COL XR's
animal synonym rows for Aa and Acaena put the orchid and the rose in
Animalia, and WoRMS, which carries both Olea and counts two sea slugs to
one olive, made the olive a sea slug.NA). It had
read none of them, so GBIF's 535,447 genera contributed no kingdom, and the
parent-key walk meant to recover one never resolved a genus: the backbone
carries no KINGDOM-rank rows. The walk (resolve_kingdom_via_gbif()) is
removed.multi_kingdom: TRUE for the 5,026
genera that accepted records place in more than one kingdom, where the
register's single kingdom is a choice between homonyms (or between
placements). taxify reads the kingdom of the matched record for these.build_backend_coverage() stamps each backbone with the manifest's latest
when the file is the published build (its md5 is the manifest's
content_id), and warns naming any backbone that is not. It read the
version from a <name>.meta sidecar, which only taxifydb's own build output
has, so the published coverage carried no version for 17 of its 19
backbones.read_build_meta() reads a .vtr's build record from either a .meta
sidecar or taxify's installed meta.json, and keeps their two meanings
apart: the sidecar's version is the source's own version, taxify's is the
release tag (returned as release). update_manifest() reads through it.update_enrichment_manifest() writes available_groups as a list, like
every other array in the manifest. A character vector printed on one line
under jsonlite's pretty printer, and a single group collapsed to a scalar.DOUBTFUL and PROVISIONALLY_ACCEPTED into
ACCEPTED. Both stay their own status ("DOUBTFUL",
"PROVISIONALLY ACCEPTED"), which taxify ranks below an accepted record of
the same name. Folded, a doubtful key with no data could win a tie on taxon
ID: Karwinskia mollis resolved to the doubtful Standl. key (0 GBIF
occurrence records) over the accepted Schltdl. one (663).n_occurrences column: the GBIF occurrence
count of each key whose count can decide a pick, i.e. every record sharing a
matching key with a record of another accepted taxon, plus those records'
accepted targets (gbif_count_keys()). Other keys are NA. The counts are
taken by build_gbif_occurrence_counts() through faceted occurrence searches
(count_gbif_occurrences()), frozen as the gbif-occurrence-counts-2026.09
release asset, and read by build_gbif() (recorded as occurrence_counts=
in the .meta). build_name_lookup() passes the column to
taxify::candidate_order(), which ranks by it from taxify 0.6.0..keep_own_concept() drops the re-keyed rows at every key the
own concept reaches, before the reducer runs; a re-keyed row still fills a
region the own concept lacks, under its own authorship. The authorship column
is the one taxify's guard reads (taxify::enrichment_authorship_col(), so
taxify >= 0.5.5.9000). Enrichments without one (every other grouped
enrichment) keep the trait-richest row as before.wcvp republished into enrichment-2026.08 (content id
c940e7e650d79dc6a01e3b52b9d34743, 4,908,476 rows, unchanged): 407,917 of
5,872,271 expanded rows displaced, names carrying more than one authorship
31,694 -> 27,305 (rows 863,624 -> 763,604). The same build without the change
reproduces the previous asset's counts exactly.alien_first_records is one row per species x location, not per species x
country, and its group column is location. Seebens et al. record islands
and other parts of a country as regions of their own, "a unique set of 282
non-overlapping regions (countries and sub-national regions such as
islands)" (Seebens et al. 2017, Methods), so the United States of America
location excludes Hawaii and Alaska and Spain excludes the Canary and
Balearic Islands. The parser mapped every location to its country's ISO code
and took the earliest year across them, which folded 7,807 records from 25
sub-national locations into 14 countries. In v4.0 that dated 665 species x
country first records from an island's earlier year (Hawaii 337, median 25
years; Azores 126, median 41.5; Madeira 71, median 60) and credited 3,541
pairs to a country whose own location holds no record of the species (Hawaii
1,186, Azores 665, Sardinia 362, Corsica 355, Galapagos 303). Peer islands
were merged too: Crozet, Kerguelen and Amsterdam Island into TF, Saint
Helena, Ascension and Tristan da Cunha into SH.
A location that is a whole country keeps its ISO 3166-1 alpha-2 code as its
key (AT, US), and a sub-national one is keyed on its own name (Hawaii,
Canary Islands), listed in .seebens_subnational. Every row carries
country_code, the country the location lies in, so rolling islands up into
their country is an explicit step on that column rather than the default.
A parser that keeps per-record references writes, beside every value column
<col>, a <col>_source column of the reference ids behind the reported
value (|-joined): the records stating a categorical value, the records
entering a numeric aggregate. attach_references() attaches the reference
table, build_enrichment_vtr() writes it as <name>_references.vtr and
declares it in meta.json, publish_enrichment_release() uploads it, and
the manifest entry records it under references. Name resolution ranks
candidate rows without the provenance columns, so every value column stays
as it was.
Carried by AusTraits (dataset_id, cited by the dataset's primary source),
GIFT (the references GIFT_traits() returns beside each aggregate), BROT 2.0
(SourceID, resolved by BROT2_sou.csv) and LEDA (original reference,
else reference). GIFT prefixes some ids with - or + without
documenting either; the table keeps them as listed_negative /
listed_positive rather than naming a meaning. Four GIFT woodiness ids that
GIFT serves nowhere keep an NA citation and a note. A reference a source
lists once per scope (WCSP 2014 over 189 families) is folded to one row.
parse_leda() matched columns by pattern and fell back to a file's last
column, and read.csv() shifted names in files whose rows carry more fields
than the header: leda_seed_mass_mg held the diaspore type code,
seed_length_mm the seed width, and canopy height, leaf mass, terminal
velocity and Raunkiaer life form were empty. .read_leda_table() reads the
export's own structure and .leda_trait_spec() names the one file and field
each output column reads; a missing field stops the parse. Numeric traits
read LEDA's per-record single value, categorical traits report the species
mode, and stem specific density records above LEDA's 0-1.5 g/cm3 range are
divided by 1000. clonal_growth and buoyancy become clonal_growth_organ
and floating_capacity_1week_pct. Seed mass calibrates at 1.00 against
Kew SID (2,198 shared species), GIFT (2,252) and BIEN (1,662).BIEN records leaf dry mass in g, and the parser wrote those values under
leaf_dry_mass_mg; they are now x1000 (Acer campestre 184 mg). The curated
.bien_trait_spec() declares the unit BIEN records for every numeric column,
and a record in an undeclared unit stops the build.
Ecoflora records typical maximum and minimum height in cm, written under
height_max_mm_uk / height_min_mm_uk; they are now x10 (Quercus robur
30,000 mm, Bellis perennis 120).
WFO's originalNameUsageID and COL's dwc:originalNameUsageID map to the
unified original_name_usage_id, which taxify reads to resolve an unplaced
record (WFO UNCHECKED) through its basionym. GBIF's basionym_key is not
mapped, since no GBIF record reaches the unplaced grade.
build_name_lookup() selects the authorship columns and the basionym link,
so a lookup key resolves the way taxify() does: a synonym homotypic with
its accepted name is ranked through candidate_order(), and a key the
backbone leaves unplaced takes its placed basionym's accepted name through
taxify::basionym_placement(). Against WFO 2026.06, 6,546 of 1,568,661 keys
(0.42%) change accepted name. build_name_lookup() takes the candidate order
from taxify rather than a copy of its sort columns. taxifydb requires taxify
(>= 0.5.3.9000).
The reverse hop entered a concept through any backbone filing a key under
the input's forward image. Each hop now counts the backbones placing the key
in the input's species against those placing it in another, and is refused
when the other side outnumbers it: COL XR alone filed
acer negundo f. crispum under Acer pseudoplatanus, so sycamore's rows
were keyed on Acer negundo. A hop onto a name every backbone keeps apart
from the input (brassica macrorhiza under B. napus and B. rapa), or
onto a name recorded under two or more species, is refused too, and forward
resolutions go through the same vote. An even split (Minuartia hybrida)
still resolves.
Source names key on the canonical taxify::parse_name() gives them, so an
author, a sensu-lato note or an unpunctuated rank marker no longer keeps a
record from its species. A record naming two taxa, an open-nomenclature name
and a cut-off hybrid name keep their verbatim spelling.
alien_first_records drops the 1492 and 1500 convention markers to NA
(the raw value stays in the verbatim column) and reduces synonyms to the
earliest year after name resolution, since a first record is a minimum over
records.
The build workflows tagged every backbone with the build month, and that tag
is the version taxify_lock() stamps: col-2026.08 was COL 2025 and
wfo-2026.08 was WFO 2024-12. Each builder now resolves its version from
where it downloads, write_backbone_meta() rejects a version that cannot
name a release, and publishing stops on a version that differs from the
.meta.
col builds from the current COL monthly release and is versioned by it:
col-2026.09 is COL26.9 (5,432,444 rows).
Both build workflows publish through publish_release(), so a CI release
uploads the content-addressed <name>-<content_id>.vtr its manifest
content_url names, and a re-cut reuses its release rather than deleting
the tag (#47).
Genus (Subgenus) epithet had no key for the binomial a
user types, so the fuzzy pass landed on another taxon (Camponotus
herculeanus on Camponotites heracleus). normalize_backbone() drops each
one-word parenthesised group between the genus and the epithet, two of them
in WoRMS' Thoracostoma (Pseudocella) (Corythostoma) filipjevi (COL 325,523
names, WoRMS 102,977), while a parenthesis holding an authorship or a
hybrid formula stays, and
split_scientific_name() uses the same locator. Enrichment source names
written with a subgenus key on the binomial through the same
drop_infrageneric().parse_bet() keeps every BET substrate class as a pipe-delimited set over
all six classes, through the shared .onehot_to_multi(); taxify's
substrate trait takes the primary class.harvest_plazi_dwca() retries requests and stops when a dataset still
cannot be fetched, instead of skipping it (309 of 489 datasets were cached
under load).datasets/ is excluded from the package build and from tracking.create_delta() writes <backend>.xdelta.base beside the patch, holding the
content id (md5) of the base .vtr, and update_manifest() records it as
delta_from_content_id. delta_from is a release tag, and a re-cut reuses
its tag, so it could not tell taxify whether the build a user holds is the
one the patch applies to: taxify tried the patch against other builds and
xdelta3 stopped with "target window checksum mismatch". The field is dropped
with the other delta fields when a release carries no patch./action/downloadSupplement answers the same request with 200 or
403 from one attempt to the next, and cf_fetch.py made one request, so
building chowdhury or clopla failed by chance. On 2026-09-14 chowdhury
failed three times and missed the #52 re-cut, then downloaded on the next
attempt. The wiley and get modes now retry up to five times with
backoff, with a new session each attempt. The article page loaded first to
prime cookies now returns a Cloudflare challenge, and the supplement
request succeeds without it, so its status is no longer relied on.A name lookup keeps one accepted name per spelling, so a homonym such as
Acacia acicularis (R.Br., a synonym of Acacia brownii; Humb. & Bonpl.,
of Vachellia farnesiana) is collapsed onto one of its species by
tiebreak. The reverse hop could enter such a key through one species and
leave through the other, keying the second with the first's traits: the
useful_plants row for Acacia brownii was V. farnesiana's, and
Acacia leptospermoides carried Acacia longifolia's through
Acacia spathulata. Because the tiebreak decides which species comes out,
the contamination also moved between lookup rebuilds, which is how the #50
re-cut appeared to lose keys the August build had.
Lookups now carry n_species, the number of species a spelling's rows point
to (infraspecific ranks of one species count once), and the hop refuses any
key a backbone gives to two or more. Against the current lookups this
removes 22,856 of useful_plants' 134,954 keys and 608 of anage's 13,428.
A sample of 25 refused keys were all homonyms with distinct authors
(Carum aromaticum Salisb. is caraway, Druce's is ajwain).
build_all_name_lookups() rebuilds a lookup that predates its backbone or
has no n_species column even without overwrite = TRUE, and the closure
refuses to read one without the column.
read_wfo() and build_wfo() decoded WFO's classification file as latin1
and converted the result to UTF-8, but the file is UTF-8 (Bañares is
stored as 42 61 c3 b1 61 72 65 73), so every non-ASCII character was
encoded twice: (H.Y.Liu) Bañares came out as (H.Y.Liu) Bañares. In the
2024-12 source that is 111,368 rows of authorship, 131,330 of
namePublishedIn and 109,424 of taxonRemarks. The multiplication sign of
the 6,274 hybrid names was repaired by a substitution after the read, which
is gone with the wrong decode.
Both readers now go through one internal feed, which declares the file
UTF-8, so the whole-file read and the streamed build cannot decode it
differently. delim_chunk_feed(file_encoding = ) drops a byte that does not
form a character: WFO cuts taxonRemarks at a fixed byte width, which
leaves the lead byte of a split character at the end of 8 fields.
Three authorships are double-encoded in the WFO source itself
((Sagorski) Jalás, (Kovács ex Neilr.) Asch. & Graebn., MartÃnez)
and are left as they arrive. The published wfo table carries the fault
until it is rebuilt.
Cross-backbone expansion of a genus-grain source now keeps only accepted
names that are themselves a genus name. A backbone that files a source genus
as a subgenus gave keys like Camponotus (Forelophilus), and one that
resolved it onto a species gave Achelia hispida; no genus of a taxify
result can match either. None of them came from the sources: all 372 such
keys across cefas_btrait, ramond, disperse, freshwater_insects_conus
and blanchard were written by the expansion. A build now stops when a
genus-grain key is not a genus name.
A genus-grain registry entry declares the kingdoms its source covers
(kingdom =), and expansion drops a mapping that some backbone places and
none places inside that set (resolve_name_map(kingdom =)). The consensus
vote could not protect these sources: genus names are homonyms across the
zoological and botanical codes far more often than binomials, and the
vascular-plant backbones carry no kingdom column, so their side of a
collision was never counted. They now count as their fixed kingdom
(taxify::backbone_fixed_kingdom()). Measured against the genus register,
plant genera among the expansion keys of the benthic-invertebrate
cefas_btrait went from 275 to 2, and genera the register files in
Animalia and no plant backbone accepts, among fungalroot's, from 1,363 to
114.
fungalroot, fungal_traits and noddb key their genus column on the
resolved name. Their parsers wrote the source genus there, so every expanded
name sat only in canonical_name and the runtime's genus join never reached
it: reachable keys go from 3,788 to 8,782, 10,592 to 12,071 and 790 to 1,144.
parse_disperse() keys a row naming two genera (Dero / Aulophorus,
Pristina/Pristinella) under each, and parse_ramond() drops Last values
that are clade labels or strain notes rather than a taxon name (lineage,
T/Pf).
update_enrichment_manifest(runtime = TRUE) takes trait_cols from the
build every time. A stored list was kept while every column it named was
still built, so columns a rebuild added never reached it: 40 runtime entries
had fallen behind, most missing the _min / _max / _n spread columns.
New enrichment_trait_cols() is the one definition, excluding the join keys
(canonical_name, accepted_name, genus) and the group column, which is
what taxify's runtime exposes.The cross-backbone expansion resolved each source name forward through every
backbone and kept the union, which is not the set the runtime joins on. A
backbone that keeps a name no other backbone accepts is reachable only by
entering the concept through a name the source never lists, and a forward
image never visits it. WCVP's asset carried no Minuartia hybrida key
because its own names are all Sabulina ..., so a WFO user's add_wcvp()
joined on a key that did not exist and got back what a name the source
genuinely does not cover gets back.
The closure adds one reverse hop and keeps two things from it: a name some
backbone holds ACCEPTED while others synonymise it onto the forward image,
and a re-routing two or more backbones agree on. It does not take the
reverse-discovered names' own forward image, which chains concepts through a
single bad synonym record -- GBIF alone maps sabulina hybrida to
Saponaria officinalis where six backbones keep the name inside the concept,
and following it pulled soapwort into a set of 48 accepted names. Those three
refused pairs are what corroboration buys.
All 108 redistributable enrichments were rebuilt and uploaded to the rolling
enrichment-2026.08 release. Measured on the rebuilt zanne asset, 200
sampled names against every installed backbone: unmatched backbone-name pairs
fall 41 -> 17 of 1,474, and names missing from at least one backbone 10.5% ->
7.5%. Assets grow 1.96x (270 MB -> 529 MB over the 100 with a shipped
counterpart); wcvp, the worst affected at a 22.5% gap, goes 69 -> 141 MB.
Every entry now records resolved_backbones, naming the backbones the
expansion reached. Its absence is what made a partially expanded asset
indistinguishable from a complete one: a missing name_lookup.vtr only
warned. The publishing paths resolve strictly now and a missing lookup stops
the build, while build_enrichment() stays lenient so taxify's
build-from-source fallback still works against whatever a user has installed.
publish-enrichment.yml moves to the self-hosted runner, since a hosted one
has no taxify data dir and resolved against nothing at all.
Three trait_cols lists moved with the republish, none of them from this
change. griis loses recordid, which the GBIF-backed source stopped
returning between builds and which was a per-record id being median-
aggregated across records. avonet loses avibase_id1 and glonaf
presence_uncertain + quest_native: both .vtr schemas are unchanged and
those entries had been advertising columns the built file never carried.
Two defects found while fixing the above. Name lookups gain an
accepted_name index, added in place on existing files, without which the
reverse direction is a full scan. And the schema probe collected each whole
lookup to read one column name (head() after collect(), not before),
which had the audit process at 20 GB resident.
GBIF renders an infraspecific canonical without its rank connecting term
(Erica tenella tenella for Erica tenella var. tenella), a rendering no
other backbone produces, so anything keyed on the name misses and the dropped
marker rides accepted_name into every add_data() and reconcile() join.
gbif_render_infraspecific() reads the marker back from the full
scientific_name, anchored on the epithet so a f. forma marker stays apart
from an L. f. filius author, and inserts it into the canonical. Reading the
marker rather than mapping rank leaves zoological trinomials
(Panthera leo persica, marker-less by code) unchanged. It runs in
normalize_gbif before precompute_keys, so the keys, the embedded
accepted_name and the genus register all pick up the corrected canonical.
gbif-2026.08 is re-cut from a build carrying it: same source snapshot and
row count (6,404,001), with sha, size and content id moving, which is what
signals a runtime to refetch.
Parsers name the column they want as a list of spellings, most specific
first, so a source that renames a field between releases still parses. The
lookup was intersect(names(df), candidates), and intersect() returns its
matches in the order of its FIRST argument: the answer was whichever
candidate sat leftmost in the file, not the preferred one. .first_col()
reads the list as the preference order it was written as, and all 34 lookups
across the two parser files go through it. Three genuine set intersections
(LepTraits' month columns, EltonTraits' diet columns, FloraWeb's labels) are
unchanged.
GloNAF is where that bit. Its flora table carries its own row id before
taxon_wcvp_id, the foreign key into the taxon table, and both are integers
over overlapping ranges, so joining on the row id matched part of the range
instead of failing: glonaf.vtr held 10,238 rows over 84 regions against the
256,454 over 1,343 the source holds, and the rows it did hold paired a name
with another taxon's family (Campanula latiloba filed under Bignoniaceae).
Reading the candidate lists in order then exposed a second wrong key: the
region list put OBJIDsic, the region polygon's GIS object id, ahead of
id. Those ranges overlap too, so it resolved 825 of 1,343 regions.
OBJIDsic is now last, reached only when the region table names its primary
key nothing else.
parse_glonaf() fails the build when either join key does not resolve for
every flora row. Both bugs returned a plausible table rather than an error,
which is what a partly-matching integer key looks like from the inside.
Every candidate list converted was checked against the live source header for
the other thirteen parsers that use one (EltonTraits, AVONET, PanTHERIA,
AmphiBIO, FishMorph, LEDA, Díaz, GRIIS, FUNGuild, LepTraits, AnimalTraits, NW
European arthropod traits, AnAge). All pick the same column as before. The
built meta.json cannot answer this -- .is_bookkeeping_col() strips exactly
the id and name columns at issue -- so the check reads the sources.
The registry was pinned to Zenodo record 13235357 (2024-08-05). The current
release is record 17105725 (2025-09-12), GloNAF v2.02: 16,429 taxa over
1,343 regions from 336 data sources, matched against WCVP v12. Its files moved
from .xlsx to .csv, so building the enrichment no longer needs openxlsx2.
Rebuilt and republished to enrichment-2026.08: 300,094 rows over all 1,343
regions, against 10,238 rows over 84 regions, resolved against all 19
backbones. Both manifests record the new record, row count and content id.
citation is rewritten from the build when the entry moves to a
different source_url or source_doi, so it cannot go on naming a source
nobody read. A versioned deposit moves its URL on every release while the
work being cited stays put, and GloNAF's runtime entry -- a structured block
carrying authors, journal and DOI -- was flattened to the registry's one-line
attribution on the record bump. A stored citation that already names the
entry's source_doi is now kept.TAXIFYDB_PUBLISH_RESUME=1 reuses whatever .vtr sits in an enrichment's
output directory rather than rebuilding it, which is what makes an
interrupted publish cheap to finish. A .vtr left from an earlier source
release is indistinguishable from a current one on disk, so the reuse
uploaded stale bytes and then wrote a manifest entry naming the record that
build never read. assert_built_matches_registry() compares the sidecar's
source_url and version against the registry and refuses the reuse when
they disagree, or when there is no sidecar to compare.Seebens et al. replaced the single freedata xlsx with a relational Darwin Core-style release: a semicolon-delimited dataset table beside location, taxonomy and source tables, every column renamed. The old parser reads an xlsx sheet by its v3.1 headers, so it failed on the new file rather than mis-reading it, and nothing already published was wrong.
The location mapping is what needed care. Locations are keyed on wording and v4.0 renames 24 of them, so a straight port drops 13,104 records without a word ("United States" to "United States of America" alone is 8,050). The lookup folds accents, case and punctuation before matching, which absorbs the cosmetic respellings (Reunion, Curacao, Timor-Leste); a build asserts no two wordings fold together onto different countries, and a non-empty location the map does not hold is a hard error naming it.
The file is UTF-8 except for 679 latin1 lines, which no single fileEncoding
reads -- UTF-8 errors on those, latin1 mojibakes the 2,739 real sequences
elsewhere -- so .read_delim_utf8() decides per line.
Where a species has several records for one country the earliest year still wins, but a record asserting presence now outranks one recording absence, so the retained row does not date an occurrence from a record denying it. That record's own status rides beside its year.
Verified against an independent reimplementation of the parse: 81,747 pairs, identical years, no difference either way. Against v3.1: 72,631 -> 81,731 pairs, 242 -> 245 countries, 23,677 -> 27,251 names, with 6,016 pairs dropping out upstream, mostly ants and consistent with taxonomic cleanup. Published as 93,350 rows over 245 countries, byte-identical to the local build.
Two shared helpers widened for it. Darwin Core writes identifiers in camel
case, so taxonID sanitizes to taxonid with no separator for
.is_bookkeeping_col()'s .*_id rule and the ids were being published as
traits with min/max/n spreads; the DwC spellings are named explicitly rather
than caught by a general .*id, since trait values end the same way (hybrid,
diploid). fold_accents() goes from French lowercase to the whole Latin-1
letter block.
thermofresh was pinned to Zenodo record 14056760, the version deposited
before peer review. The published release is record 16959762 (ThermoFresh
v1.0), described in Bayat et al. (2025), Scientific Data 12,
doi:10.1038/s41597-025-05832-w. The enrichment carried 5,115 tolerance tests
over 828 taxonomy rows against the 6,825 and 931 the published data holds;
parse_thermofresh() now yields 768 species, up from 673.
The v1.0 archive keeps the pre-review submission verbatim under
data/initial_submission/ and puts the peer-reviewed tables beside it as
data/*_final.csv, so bumping the URL alone would have kept reading the old
data under a new version number. The parser matches the _final names, and
its file lookup stops when a pattern matches anything other than exactly one
file rather than taking the first of several.
Rebuilt and republished to enrichment-2026.08: 860 rows against the 750 the
release carried, resolved against all 19 backbones. Both manifests record the
new record, the new row count and the new content id, so a runtime holding the
old file refetches on the content id rather than waiting for a version bump.
The runtime entry went on citing record 14056760 after the build had moved
to the published one. update_enrichment_manifest(runtime = TRUE) fills a
runtime field only when absent, so hand-curated text survives a release --
except citation when the build's source_url or source_doi has changed,
which is the one case where keeping it means citing data nobody read.
check_zenodo_version() queried the pinned record. A Zenodo record is
immutable, so it reported the day that record was published no matter how
many newer versions its concept had gained, and no Zenodo-sourced enrichment
could ever be reported as outdated -- which is how the ThermoFresh release
above went unnoticed for a year. It resolves versions/latest and compares
record IDs. It also reads the legacy /record/ path, which animaltraits
uses and the pattern did not match.
check_enrichment_source_version() picked its checker with
switch(entry$source_format, ...). That field is absent from 97 of the 108
manifest entries, so the call errored and every one of them was reported as
"could not determine upstream version" -- an error the weekly workflow
records and does not open an issue for. The checker is chosen from the host
the source URL names, which every entry carries.
check_figshare_version() read versions[[1]] as the newest. Figshare lists
versions ascending, so it reported version 1 for every article however many
versions had been published. It takes the highest. It also accepts an article
URL, not only a file URL.
Freshness a checker cannot settle is now reported as unknown with a note. The recorded version is this package's own release string and the upstream version is whatever the host counts in, so comparing the two answered a different question than the one asked; on Figshare-sourced entries it read as outdated every week.
build_enrichment() asks the source host for its identity for the version
being downloaded -- a Zenodo record number, a Figshare or Dryad version
counter, a Last-Modified stamp -- and build_enrichment_vtr() writes it to
meta.json as upstream_id, which update_enrichment_manifest() carries
into the manifest entry. The weekly check compares that against what the host
offers now, so both sides of the comparison are the host's own counting.
Zenodo entries were already answerable from the record number in the URL;
this is what makes the Figshare, Dryad and GBIF entries answerable too.
An entry built before upstream_id existed records none, and reports unknown
rather than a guess until its next rebuild. Filling the field in from what
upstream offers today would assert every enrichment is current, which is the
claim the check exists to test.
The probes report identity and nothing else; check_enrichment_source_version()
decides freshness in one place from what they report. Which probe answers for
a source is a table keyed on the host, so covering a new one is a row.
The first sweep over all 108 entries reports six behind upstream, not one:
thermofresh, glonaf, alien_first_records, gwdd, pottier and
zooplankton, each pinned to a Zenodo record whose concept has since gained
a newer one. Twenty-five report current. The remaining 77 report unknown --
12 awaiting a rebuild to record an identity, 15 naming a URL the host cannot
answer for, and 50 on hosts no probe covers.
A host with no probe is now reported apart from a host whose probe came back
empty. Reading both as "no version check for source host" is how a source URL
that no longer resolves passes for a source nobody checks -- amphibio and
elton_traits name ndownloader.figshare.com download ids the Figshare API
answers 404 for, and they read as unchecked rather than unreadable.
jsonlite::write_json() writes through a text connection, so publishing from
Windows ended every line of manifest.json CRLF where CI ends it LF: the one
entry a release changed arrived inside a rewrite of all 5,800 lines. The
backbones too large for the hosted runner are published by hand from Windows,
so that was every COL, COL XR and GBIF release. write_json_lf() serializes
to a string and writes it through a binary connection; both manifest writers,
the meta.json sidecar and sync_manifest.R go through it.Runtime-only manifest fields are written only when absent, so hand-curated
values survive a republish. trait_cols is not merely accompanying text
though: it names the columns the .vtr carries, and preserving it through a
source that renames its fields leaves the runtime advertising columns that
are not there. alien_first_records is where that showed -- the republished
entry went on naming thirteen columns the built file no longer had while
hiding the nine it did.
A stored list is kept whole while every column it names is still built, so a curated subset or ordering is untouched; once one is gone the list describes a file that does not exist and is taken from the build. Whether the group column belongs in the list is a per-entry choice -- five of the seven group-keyed entries carry it, two do not -- so that choice is preserved across the rewrite rather than settled here. This is the same shape as the citation rule beside it, and for the same reason: both describe the source rather than merely accompany it, so both go stale when it moves.
check_manifest_coverage.R walked a BACKENDS vector kept by hand, so a
backbone wired into the build but not added there was audited by nothing and
reported as clean. COL XR had been in that position since it was wired. The
vector is gone: the set is parsed out of .register_extractors in
R/register.R, and the script stops with an explanation rather than auditing a
guessed set when it cannot read that file.
The audit also checks what the published register was built from. Both
register artifacts already record the backbones they unioned in their .meta
sidecar and nothing compared it, so a register going stale as backbones were
added had no signal at all -- no version moves and no bytes change in anything
that is checked. register_incomplete and register_unreadable join the drift
states the weekly workflow opens an issue for.
Reading a release asset over the API works with a token present.
curl::handle_setheaders() replaces the header set rather than adding to it,
so setting Authorization in a second call dropped the
Accept: application/octet-stream that asks for the bytes, and the API
answered with the asset's JSON description instead. The same defect had been
dropping User-Agent from every authenticated request. One builder now
assembles the whole header set in a single call.
build_backend("wgsrpd") and build_backend("meow") work, and both build in
build-light.yml alongside the backbones. The boundary geometry behind
region = and the coordinate lookup had no entry in any builder registry, so
it could only be rebuilt by calling the internal function directly.
They are registered in .geometry_builders, not .backend_builders, and
list_geometry() names them. list_backends() is what
build_all_name_lookups() and resolve_name_map() iterate to build the
per-backbone accepted-name lookups that every enrichment is resolved through;
a vertex table carries no taxonomic names, so a geometry asset listed there
would yield an empty lookup and quietly cost the enrichments resolved against
it their matches.
Builds stamp date +%Y.%m, which records when a build ran rather than what
it read. Several backbones read a pinned source and rebuild to identical
bytes every time -- Euro+Med from a frozen snapshot release, WFO from a fixed
Zenodo record, COL from the pinned annual archive -- so releasing them again
minted a version whose only difference was its name. taxify's runtime treats
a fresh version as reason to refetch, so every user downloaded a file they
already held: 740 MB of it for WFO.
New vtr_changed() compares the built .vtr against the full_sha256 the
manifest already records, and both workflows gate the delta, release,
manifest commit and runtime sync behind it. It fails open -- no manifest, no
entry, no recorded hash, a first-ever build all count as changed -- so an
undecidable case still publishes. A skipped release is reported as a run
notice rather than passing silently.
build-heavy.yml targets self-hosted and no runner is registered on the
repository, so no job of its has ever been picked up. WFO, WCVP, LCVP,
Fungorum, AlgaeBase and Euro+Med all sit below OTT (3.7M rows) and NCBI
(2.8M), which build on the hosted runner today, and need no dependency the
light workflow lacks. They move to build-light.yml, leaving COL, COL XR and
GBIF. Euro+Med built there in 2m41s.
Both build-heavy.yml crons are held while it has no runner. A schedule that
queues a job nothing can pick up produces a stuck run a month, which the COL
XR monthly cron had been doing since it was added.
update_manifest() cleared the extras block whenever no sidecar was
passed, by analogy with the stale delta it drops on the line above. The two
are not alike: a delta URL names the release being written, so an old one
would 404, while a sidecar records the tag it was published under and stays
reachable across releases that do not carry one. The manifest is the only
record of that URL, so clearing it is how the runtime stops downloading the
file. Passing no extras now leaves the block alone; character(0) removes
it.
Both build workflows upload whatever sidecar the build wrote beside the
backbone, and record it. Neither had, which is why
worms_species_profile.vtr sat at worms-2026.05 while its own backbone
reached 2026.08 -- and 14,227 rows short, since it is read by the same fixed
reader. Republished at 1,562,065 rows, one per taxon.
The light build, the heavy build and the taxify runtime sync each rebuilt the
release's artifact paths in their own embedded R one-liner, which is how they
came to disagree about what a release carries. They now share
scripts/update_manifest_entry.R.
read_worms() no longer uses utils::read.delim(), which cannot read the
WoRMS taxon core at all. Taxon.tsv carries 4,626 newlines and 59 lone
carriage returns inside quoted fields; R reads a lone CR as the end of a
line, and in text mode loses bytes doing it, so the file's 15,419,038 quote
characters come back as 15,418,989. The quoting stops balancing, scan()
reaches a record it cannot close, and it returns 1,363,240 of 1,562,065 rows
-- 198,825 marine taxa missing, the last few hundred filled with fragments of
the citation that broke the parse -- reporting it only as a warning. The
reader is now data.table::fread() followed by unescape_quotes(), since
fread() leaves RFC 4180's doubled quote as it found it
(Rdatatable/data.table#1109) where read.delim() collapses it.
Published as worms-2026.08: 1,562,065 rows, against 1,557,860 in
worms-2026.07.
assert_worms_taxon_core() fails the build when a taxonID is missing or
repeated. A reader stopping partway through leaves the rows it did return
well formed, so the identifiers are what show that a record was cut in two.
build_worms() streams. The taxon core is over a gigabyte unpacked and was
the largest backbone still assembled in memory.
delim_block_feed() no longer parses with read.delim(), and the choice of
reader no longer decides what a value ends up as. Blocks are parsed with
fread() and the RFC 4180 unescaping is done in unescape_quotes(), so it
holds whichever source is read. delim_lf_reader() splits blocks on LF or
CRLF and never on a bare CR, and a block reaches the parser as a file
rather than as text, since fread(text=) splits its input into lines itself
and so cannot carry a record that spans them.
Verified against both real sources: WFO is identical to its previous read across all 1,638,552 rows and all 20 columns, and WoRMS streams to the same 1,562,065 rows and 14 columns as the whole-file read, building to 115,741 genera in 115,741 contiguous runs with all 857,427 synonyms resolved.
R/betsi-recovery.R rebuilds published BETSI-derived trait matrices into
per-taxon enrichment assets, since BETSI's live portal is offline and no
complete export is recoverable. build_betsi_recovery() /
list_betsi_recovery() scope the recovery subset; parse_betsi_recovery()
dispatches on each source's matrix shape; gen_spe() builds the six-letter
species code the code-keyed matrices use.
betsi_earthworm_traits (gap G2): Pelosi et al. (2014) Appendix 1, 11
earthworm species x 7 fuzzy-coded traits. Each (species, trait) affinity block
is checked to sum to 100 on ingest, and the full fuzzy vectors are kept as one
numeric column per <trait>__<modality> bin.
betsi_collembola_traits (gap G1): Lu et al. (2025) Appendix S1, 26 Collembola
species. Fills pigmentation, number of ocelli and furcula, which had no prior
Collembola coverage. Six traits are BETSI-derived; vertical distribution,
trophic guild and life form were measured by the source study.
Provenance is recorded per column in each asset's meta.json provenance
block (betsi_export / betsi_derived / literature_reconstruction /
source_study), never flattened onto rows a tier does not describe.
build_enrichment_vtr() gains a provenance argument that the registry
supplies through a per-enrichment provenance_fn.
resolve_enrichment_names(), resolve_name_map(), .resolve_species_names()
and build_all_name_lookups() now default to every taxify backbone
(list_backends()), up from the seven broadest. An enrichment .vtr built
after this change carries the accepted-name variants from the domain-specific
backbones too (Euro+Med, Species Fungorum, AlgaeBase, FishBase, SeaLifeBase,
Reptile Database, LCVP, WCVP), so an enrichment join through one of those
backbones no longer falls through.
resolve_name_map() warns when a requested backbone lacks a
name_lookup.vtr, so a production build cannot silently resolve against a
narrower backbone set than intended.
build_enrichments.R no longer publishes or writes a manifest entry for a
build-only enrichment (ccdb, gmpd, plantatt, bryoatt, clopla): publish all
and all skip them, and publish <name> refuses a build-only name. These
sources carry a citation-only or unstated licence and are built locally only.build_register() was added to 0.1.18 after 0.1.18 had already been built
and cached, so anything resolving taxifydb by version string kept serving the
earlier code and taxifydb::build_register was missing from it. taxify's
register fallback calls that function, so the call failed wherever the older
0.1.18 was installed. The version now identifies the code that carries it.New marine_distribution enrichment: the marine analogue of the WCVP range
table, so taxify's region=/coords= filter can constrain animal and marine
matches, not just vascular plants. It keys canonical_name on a MEOW
ecoregion region_code with a native/introduced status.
WoRMS distributions are not in the ChecklistBank/GBIF export (the per-taxon
distribution endpoint returns nothing there), so they are harvested per taxon
from the WoRMS REST API by inst/py/crawlers/crawl_worms_distributions.py,
keyed on Marine Regions localities (MRGID).
inst/py/crawlers/crosswalk_mrgid_meow.py rolls each MRGID up to the Marine
Ecoregions of the World (Spalding et al. 2007) it falls in by point-in-polygon
against the frozen MEOW GeoJSON, and parse_marine_distribution() joins the
two frozen snapshots, dropping records that report non-presence or that WoRMS
flags as doubtful or inaccurate.
New reference-geometry backend build_meow() writes meow.vtr (MEOW
ecoregion boundary polygons), the marine counterpart of build_wgsrpd(),
indexed on the same ECO_CODE the enrichment's region_code uses so the
runtime coordinate-to-region path is a drop-in.
build_meow_geojson.py reads each ecoregion's province and realm from the
Marine Regions gazetteer hierarchy instead of inferring them by point-in-
polygon on the ecoregion centroid. MEOW nests an ecoregion in exactly one
province and a province in exactly one realm, but assigning the two
independently produced combinations that cannot co-occur (a Cold Temperate
Northwest Pacific province under a Central Indo-Pacific realm) and misplaced
any ecoregion whose centroid falls outside its own polygon or spans the
antimeridian -- the Aleutian Islands were filed under the Lusitanian province
of the Temperate Northern Atlantic, and the Chukchi and Beaufort Seas under
Tropical Eastern Pacific. 39 of the 232 ecoregions carried a wrong province
or realm. Reading the stated hierarchy yields exactly MEOW's published 232
ecoregions / 62 provinces / 12 realms, with every province resolving to a
single realm. The geometry is unchanged, so meow.vtr is unaffected.
parse_marine_distribution() counts every WoRMS establishmentMeans variant
that states nativeness as native. WoRMS qualifies the value where it can, so
matching "Native" exactly scored the 16,663 "Native - Endemic" and 4,375
"Native - Non-endemic" records as unknown origin and left range_mode = "native" missing a sixth of its evidence.
crosswalk_mrgid_meow.py handles ecoregions that cross the antimeridian.
Thirteen of the 232 run to both -180 and +180, so in the raw longitude framing
they measure 360 degrees wide and their centre lands at longitude 0: the
Aleutian Islands centred on the English Channel, Kamchatka and the Eastern
Bering Sea on the North Sea and Norway. Every coarse bounding box over Europe
therefore swept them in, attaching Pacific and Arctic ecoregions to European
records -- and "European Marine Waters" alone carries 19,763 of them. Such a
region is now held in 0..360 space, where it is contiguous, and query points
are shifted to match. An MRGID bounding box that crosses the antimeridian is
read as the union of its two halves rather than one interval.
crosswalk_mrgid_meow.py splits into a fetch phase that caches each MRGID's
gazetteer record and an assign phase that recomputes the crosswalk from that
cache. Revising how a locality maps to an ecoregion is now seconds of local
work instead of another throttled pass over 7,112 gazetteer records.
crosswalk_mrgid_meow.py discards a locality that covers every ecoregion.
"World", "World Oceans" and "High Seas" all roll up to the complete set of
232, which records a species as present everywhere and constrains nothing
while costing a row per ecoregion per species; "High Seas" alone accounted for
49,184 of them. This is the degenerate case rather than a threshold, so
regions that cover much of the globe without covering all of it are kept: the
hemispheres at 119 and 113 of 232, the ocean basins, and "European Marine
Waters" still carry information, and their breadth is a fact about the record.
parse_marine_distribution() also drops the occurrence values that report
non-presence rather than only "Absent": a record retracted as "Recorded in error", a population that is "Extirpated" or "Eradicated", and one held
only "In captivity/cultivated" are not evidence that a species lives in a
region. Borderline values ("Uncertain", "Sometimes present") still count
as presence, since the filter is a soft disambiguation aid.
Added a package README covering the build API, the two-repo split with
taxify, data hosting, requirements, and how to add a backbone or enrichment.
The DESCRIPTION Description field now names all 15 backbones; it had listed
only the original 12 and omitted FishBase, SeaLifeBase, and the Reptile
Database.
.cf_python() no longer relies on PATH alone to locate an interpreter with
curl_cffi. Under Rscript, Rtools prepends its own usr/bin/python (which
lacks curl_cffi) to PATH, so the previous PATH-only probe picked the wrong
interpreter and errored even when a suitable python was installed -- forcing a
manual TAXIFYDB_PYTHON. Discovery now also scans pyenv-managed versions
(newest first, honouring PYENV_ROOT, both pyenv-win and unix layouts) and
interpreters registered with the Windows py launcher, then falls back to
PATH. The first candidate that can import curl_cffi wins, and the error
lists everything it tried. Building hosts, usda_fungus_host and clopla
now works out of the box on a pyenv or py-launcher setup.A new enrichment's runtime manifest entry is now complete straight from the
build, with no manual curation step. build_enrichment_vtr() records
trait_cols (the built .vtr columns, minus the join key and group column),
static, and source_format in meta.json, and
update_enrichment_manifest(runtime = TRUE) populates trait_cols,
species_col, static, source_format, and citation (from the registry
attribution) on the runtime entry.
Runtime fields are filled only when absent, so a hand-curated value in an
existing entry is preserved; only the build-side manifest is written without
them (runtime = FALSE, the default), keeping it lean. The enrichment publish
path and the taxify-enrichment-manifest-sync action pass runtime = TRUE
when writing taxify's inst/manifest.json.
build_enrichment_vtr() gains species_col, static, and source_format
arguments; the registry may set them per enrichment (defaults: species-grain,
frozen snapshot).
New publish_enrichment_release() uploads enrichment .vtr files to the
shared, rolling enrichment-<version> release. The release is created only
when absent and never deleted; assets upload with --clobber, replacing only
the same-named file so other enrichments' assets stay in place.
New build_enrichments.R publish <name|all> <version> action builds the
enrichment(s), uploads to the release, and updates both manifests in one
step -- this repo's build-side manifest/manifest.json and, when ../taxify
is checked out alongside, taxify's runtime inst/manifest.json. This closes
the root cause behind #20: previously the publish path had no single step that
moved both manifests together, so they could drift.
New publish-enrichment.yml workflow (dispatch) does the same in CI for
headlessly-buildable enrichments: build, upload, commit the build-side
manifest, and sync taxify's runtime manifest via a PR (new
taxify-enrichment-manifest-sync action, mirroring the backbone sync).
update_enrichment_manifest() now records the rolling release version in
latest (matching the enrichment-<latest> download tag) and the dataset's
own version in a new source_version field. Previously latest held the
dataset version, so the build-side manifest disagreed with taxify's runtime
manifest on every entry whose dataset version differs from the release.
Regenerated manifest/manifest.json so all 89 enrichment entries match the
current enrichment-2026.07 release (they had lagged enrichment-2026.05;
#20). The enrichment publish path had updated taxify's runtime manifest but
not this build-side copy, leaving it a stale second source of truth.
scripts/check_manifest_coverage.R (the weekly coverage cron) now also
compares the two manifests' enrichment entries directly, so a publish that
updates one manifest but not the other surfaces the same day instead of
lying dormant. Enrichments share one rolling tag, so there is no per-entry
release tag to compare against as with backbones.
gidias, CC BY 4.0).
Only derived aggregates are distributed, not the raw impact records:
parse_gidias() reduces each species' impact records to its IUCN EICAT
environmental-impact category and SEICAT socio-economic-impact category (each
the most severe magnitude among the species' negative impacts; the per-record
global-extinction flag splits EICAT magnitude 3 into Major and Massive), plus
the driving mechanism, affected well-being constituents, realms, and
record/source counts. Names are resolved to the accepted grain inside
parse_gidias() (the InvaCost / GloBI rollup pattern), so a species whose
records are split across a synonym and its accepted name keeps its full
evidence (#10).invacost, CC BY 4.0). Only derived
aggregates are distributed, not the raw estimate rows: cost_total_usd (each
estimate's standardised annual 2017-USD cost expanded across its documented
impact period and summed, overlapping estimates not de-duplicated), cost_n,
and the dominant cost_type. Names are resolved to the accepted grain inside
parse_invacost() and the cost is summed there (the GloBI rollup pattern), so
a species whose costs are split across a synonym and its accepted name keeps
its full total (#9).parse_phylacine() keeps the source Mass.Method flag as mass_method and
adds a coarse mass_method_class (reported / estimated / imputed), so a
phylogenetically imputed or allometrically estimated body mass is not served as
a measurement where PHYLACINE is a species' only source (#11).update_enrichment_manifest() now records the .vtr content_id (its md5),
matching update_manifest() for backbones, so a same-tag enrichment republish
refreshes an otherwise version-locked runtime cache.inst/py/crawlers/crawl_euromed.py harvests the full
classification through the per-taxon portal API (the CDM bulk-export endpoints
time out through the public proxy and no bulk Euro+Med exists on
GBIF/COL/ChecklistBank) and freezes it as an euromed.jsonl + nodes.tsv
snapshot, the same pattern as the ecoflora/floraweb scrape snapshots.
read_euromed() builds the unified backbone from the snapshot: accepted taxa
with nested synonyms resolved to their accepted concept, authorship and
epithets parsed from the rendered name, and family mapped to each genus from
the node treeIndex materialised path. Taxa the portal does not publish
(publish = false) are skipped. The version pin moves from 2020.1 to
2026.07.lcvp-3.0.1, 1,337,891 names), WCVP (wcvp-2026.06, 1,448,984
names), and Reptile Database (reptiledb-2026.06, 50,043 names) backbones
are now in manifest/manifest.json. LCVP and WCVP had no published .vtr
release at all, so they could previously only be built from source; both are
now built and released, and all three entries carry the full metadata
(nrow, full_size, full_sha256, content_id, source_url, citation,
license). The manifest now lists every one of the fifteen backbones.update_manifest() now fills a backbone's source_url from the url field
of the .meta sidecar that build_vtr() writes next to every .vtr, so the
manifest URL always matches what the build downloaded from (each backend's
single .<name>_url constant). Previously source_url was an optional
argument that no build path passed, so it was hand-curated and present for
only some backbones. The euromed, fungorum, and algaebase entries gain
their source_url, and ncbi's stale URL (.../new_taxdump.tar.gz, which
the server does not serve) is corrected to the path the builder actually
downloads (.../new_taxdump/new_taxdump.tar.gz). Adds an internal
read_meta() reader for the sidecar; this mirrors the enrichment path, which
already derives source_url from the built meta.json.build_wcvp() builds the World Checklist of Vascular Plants backbone
(wcvp) from Kew's pipe-delimited wcvp_names.csv (CC BY). The canonical
name is taken straight from taxon_name (hybrid signs and infraspecific
markers already rendered), and acceptance is derived from the
accepted_plant_name_id link (a name pointing at another is a synonym; a
self-pointing or unplaced name is its own accepted concept) rather than the
nine taxon_status spellings. read_wcvp() reads with quote = "" so the
0.06% of names carrying a genuine embedded double-quote (informal epithets
like f. "A") stay literal; it uses data.table::fread() when available and
falls back to utils::read.delim().build_lcvp() builds the Leipzig Catalogue of Vascular Plants backbone
(lcvp) from the idiv-biodiversity/LCVP tab_lcvp.rda (MIT), loaded with
base load() so no LCVP package dependency is needed. The canonical name is
assembled from Input.Genus / Input.Epitheton / Rank /
Input.Subspecies.Epitheton (nil marks species rank; forma renders as
f.); the synonym to accepted link comes from globalId.of.Output.Taxon,
and unresolved names are kept as their own accepted concept.build_backend(), list_backends(), and the heavy
build workflow, bringing the backbone count to 15, and carry
check_wcvp_version() / check_lcvp_version() upstream-version checks. Adds
data.table to Imports.read_worms() (and the SpeciesProfile reader) now parse ChecklistBank's
double-quoted Taxon.tsv / SpeciesProfile.tsv with read.delim(quote = "\"") instead of quote = "". Previously the field-wrapping quotes were
kept as literal characters, so 90.2% of canonical_name, key_ci,
key_normalized, and authorship came out wrapped in " and some
quoted-field rows were mis-split, which broke exact matching (falling to
fuzzy) and every marine enrichment join. After the fix, wrapped
canonical_name quotes drop from 1,406,915 to 11 (the genuine embedded
ones), and only " is treated as a quote so authorship apostrophes
(d'Orbigny) stay intact. WoRMS rebuilt under worms-2026.07.publish now stamps a content_id (md5 of the built .vtr) for each
backbone in the manifest, mirroring the enrichment content_id, so taxify's
runtime can detect a same-tag republish and refresh a version-locked static
cache instead of serving stale data.bacdive, globi, italic (lichens), hosts,
usda_fungus_host, edwards_phyto, kew_sid, thermofresh, ramond,
fw_insects_conus, eurobat, copepod_traits, fishtraits,
cefas_btrait, kew_cvalues, epa_freshwater, ccdb, gmpd, plantatt,
bryoatt, and clopla. Run taxifydb::list_enrichments() for the full set;
pre-built .vtr artifacts are published under the enrichment-2026.07 tag.
The GloBI, BacDive, and ITALIC builds use resume-safe crawlers.parse_elton_traits() now derives a diet_guild column from the ten
EltonTraits diet-fraction columns: fractions are summed within each guild
(the four vertebrate/fish columns are all carnivory), the dominant guild
wins when it reaches 50 percent, otherwise the species is omnivore. The
derived label agrees 93% with EltonTraits' own diet_5cat and 83% with
AVONET's independent trophic_niche on shared species. This lets the taxify
runtime coalesce EltonTraits (birds and mammals) into the cross-source
diet_guild trait alongside AVONET and ReptTraits. Requires rebuilding
elton_traits.vtr (enrichment-2026.07).parse_birdbase() now derives a clutch_mean column as the NA-safe mean of
the reported clutch min and max, so the taxify runtime can coalesce Birdbase
bird clutch sizes into the cross-source clutch_litter_size trait (calibrated
1:1 against the Amniote database on 6781 shared species). Requires rebuilding
birdbase.vtr (enrichment-2026.07).build_enrichment_vtr() now records a content_id (md5 of the built .vtr)
in each enrichment's meta.json, and sync_manifest.R propagates it into
taxify's manifest. This lets taxify's runtime detect a same-tag republish (a
rebuilt asset re-uploaded under an unchanged release tag) and refresh an
otherwise version-locked static cache offline, instead of serving stale data
forever. Every rebuilt enrichment carries the id from this release onward.parse_groot() now repairs GRooT's specific_root_area column. GRooT's
species aggregate carries three source papers (Quanquan 2011, Mokany & Ash
2008, Chanteloup & Bonis 2013) at ~1000x below GRooT's documented cm2 g-1
standard (data paper Table 1/S1, median 385.8) -- physically impossible for
fine-root SRA -- which corrupts the aggregate mean for any co-measured species.
This is a compilation unit error, not GRooT's conversion (AusTraits carries the
identical Mokany 2008 data equally low). The x1000 correction is grounded in the
primary literature: Mokany & Ash 2008's own SRA-SLA regression (Fig 1B) puts the
real magnitude at ~10 m2/kg = ~100 cm2/g, and the stored values x1000 land in
that range. The build now also fetches the full per-record version
(download_groot()) and recomputes specific root area from it with the three
papers rescaled x1000, standardized sources winning per species -- a species
with any standardized record keeps its clean median (Mokany's paper itself
cautions its pot-grown values differ from the field), and the rescaled papers
fill only species no standardized source covers. The repaired column has median
386 cm2 g-1 (matching the data paper), full species coverage, and no physically
impossible sub-1 values. Every other GRooT trait is unchanged.gift enrichment: parse_gift() fetches GIFT's (Global Inventory of
Floras and Traits; Weigelt et al. 2020) species-level plant traits from the
live API at build time -- batched over all trait IDs -- and writes them to a
.vtr so the taxify runtime joins them offline. Only the redistributable
subset the API returns is included (CC BY 4.0; restricted references are
excluded by the default call). Registered in the enrichment build registry;
requires the GIFT package to build from source.build_reptiledb() builds the Reptile Database backbone (reptiledb): the
reptarium taxa export plus the synonym snapshot and checklist, normalized to
the unified backbone schema (~12.6k accepted reptile species + ~34k
synonyms, stamped Animalia/Chordata/Reptilia). Registered in
build_backend() and the build-light workflow. License CC-BY 4.0.lizard_traits -> repttraits to match
its source (it covers all reptiles, not lizards), with the citation corrected
to Oskyrko et al. (2024). parse_repttraits() maps source headers explicitly
and adds the distribution block (biogeographic realm, microhabitat, habitat
type, elevation range, mean annual temperature, insular/endemic) that the
previous morphology-only parser dropped.aggr. form at build time,
so taxify recognizes species aggregates uniformly across every backbone and
enrichment. precompute_backbone() folds backbone aggregate names and
aggregate-rank rows (via taxify::normalize_aggregate_name());
resolve_enrichment_names() keeps aggregate source rows out of cross-backbone
expansion (which would otherwise leak an aggregate's traits onto the binomial
species key); build_enrichment_vtr() normalizes the enrichment join key.
Requires taxify (>= 0.3.0).parse_fungalroot() builds a genus-level mycorrhizal type enrichment from the
FungalRoot database (Soudzilovskaia et al. 2020), published on GBIF as a
Darwin Core Archive (doi:10.15468/a7ujmj, CC BY-NC 4.0). It reads the
occurrence core plus its MeasurementOrFact extension, standardizes the
per-observation Mycorrhiza type labels to AM / EcM / ErM / OM / NM
(plus dual types, Other, uncertain), and reduces them to one row per plant
genus by majority consensus (mycorrhizal_type, mycorrhizal_status,
mycorrhizal_records). Registered in .enrichment_build_registry as a
genus-keyed enrichment (name_col = "genus").parse_ecoflora() and parse_floraweb() build the Ecoflora (British Isles)
and FloraWeb (German flora) plant-trait enrichments from frozen scrape
snapshots. Both are registered in .enrichment_build_registry and produce
bundled .vtr files (Ecoflora 18 _uk columns; FloraWeb 59 _de columns
spanning morphology, reproductive biology, the nine Ellenberg indicator
values, ploidy and chromosome number, and chorological distribution).
Ecoflora is redistributed under CC BY-NC-SA 4.0; FloraWeb carries the
BiolFlor data (Klotz, Kuehn & Durka 2002), usable with acknowledgement and
citation per the BioFresh metadata statement. The raw per-species snapshots
are hosted in the scrape-snapshots-2026.06 release; the access date is the
dataset version.resolve_enrichment_names() now keeps the best-populated source record when
several taxa resolve to the same accepted name (subspecies and synonyms
collapsing onto a species) rather than an arbitrary first row. This recovers
trait data that could previously be discarded for widely-circumscribed
species.
Ecoflora and BiolFlor are removed from .enrichment_scrape_only; only
Pignatti (copyrighted, not redistributable) remains there.
.vtr
files: Pignatti (copyrighted) cannot be redistributed; BiolFlor is usable
with acknowledgement + citation (BioFresh metadata statement) but has no
obtainable bulk copy while the UFZ site is offline; Ecoflora's CC BY-NC-SA
licence would allow it but ecoflora.org.uk has no bulk download.
list_scrape_only_enrichments() lists them. taxify accesses these on demand
via the TR8 package (add_ecoflora(), add_biolflor(), add_pignatti());
nothing is redistributed by taxify.baseflor builder added (parse_baseflor(), registry entry). Reads Julve's
Baseflor spreadsheet (Programme Catminat; ODbL 1.0 / CC BY-SA 2.0) and emits
flowering months, pollination vector, dispersal mode, breeding system, flower
colour, fruit type, woody growth form, and the continentality and salinity
indicator-value axes for ~7,000 vascular plant taxa of France and
neighbouring regions. Source is the archived 2023-10 snapshot of the file
(the Orange personal host has since been retired). Built .vtr published in
the enrichment-2026.06 release; manifest updated.