Package: BORG 0.3.2

BORG: Bounded Outcome Risk Guard for Model Evaluation

Comprehensive toolkit for valid spatial, temporal, and grouped model evaluation. Automatically detects data dependencies (spatial autocorrelation, temporal structure, clustered observations), generates appropriate cross-validation schemes (spatial blocking, checkerboard, hexagonal, KNNDM, environmental blocking, leave-location-out, purged CV), and validates evaluation pipelines for leakage. Includes area of applicability (AOA) assessment following Meyer & Pebesma (2021) <doi:10.1111/2041-210X.13650>, forward feature selection with blocked CV, spatial thinning, block-permutation variable importance, extrapolation detection, and interactive visualizations. Integrates with 'tidymodels', 'caret', 'mlr3', 'ENMeval', and 'biomod2'. Based on evaluation principles described in Roberts et al. (2017) <doi:10.1111/ecog.02881>, Kaufman et al. (2012) <doi:10.1145/2382577.2382579>, Kapoor & Narayanan (2023) <doi:10.1016/j.patter.2023.100804>, and Linnenbrink et al. (2024) <doi:10.5194/gmd-17-5897-2024>.

Authors:Gilles Colling [aut, cre, cph]

BORG_0.3.2.tar.gz
BORG_0.3.2.zip(r-4.7-x86_64)BORG_0.3.2.zip(r-4.7-arm64)BORG_0.3.2.zip(r-4.6-x86_64)BORG_0.3.2.zip(r-4.6-arm64)BORG_0.3.2.zip(r-4.5-x86_64)
BORG_0.3.2.tgz(r-4.6-x86_64)BORG_0.3.2.tgz(r-4.6-arm64)BORG_0.3.2.tgz(r-4.5-x86_64)BORG_0.3.2.tgz(r-4.5-arm64)
BORG_0.3.2.tar.gz(r-4.7-arm64)BORG_0.3.2.tar.gz(r-4.7-x86_64)BORG_0.3.2.tar.gz(r-4.6-arm64)BORG_0.3.2.tar.gz(r-4.6-x86_64)
BORG_0.3.2.tgz(r-4.6-emscripten)
manual.pdf |manual.html✨
DESCRIPTION |NEWS
card.svg |card.png
BORG/json (API)

# Install 'BORG' in R:
install.packages('BORG', repos = c('https://gcol33.r-universe.dev', 'https://cloud.r-project.org'))

Bug tracker:https://github.com/gcol33/borg/issues

Pkgdown/docs site:https://gillescolling.com

Uses libs:
  • c++– GNU Standard C++ Library v3

On CRAN:

Conda:

dependencemodel-evaluationriskvalidationcpp

6.01 score 15 scripts 413 downloads 82 exports 1 dependencies

Last updated from:9b203ee400. Checks:15 OK. Indexed: yes.

TargetResultTimeFilesSyslog
linux-devel-arm64OK168
linux-devel-x86_64OK172
source / vignettesOK360
linux-release-arm64OK159
linux-release-x86_64OK200
macos-release-arm64OK145
macos-release-x86_64OK305
macos-oldrel-arm64OK162
macos-oldrel-x86_64OK376
windows-devel-arm64OK327
windows-devel-x86_64OK242
windows-release-arm64OK305
windows-release-x86_64OK149
windows-oldrel-x86_64OK194
wasm-releaseOK124

Exports:audit_importanceaudit_predictionsborgborg_adversarialborg_aoaborg_assimilateborg_auto_checkborg_best_subsetborg_block_sizeborg_bootstrapborg_cache_clearborg_cache_getborg_cache_infoborg_cache_setborg_calibrationborg_certificateborg_checkborg_check_coverageborg_check_nested_cvborg_check_residualsborg_clustered_sampleborg_compare_cvborg_compare_modelsborg_conformalborg_cvborg_debiasborg_diborg_diagnoseborg_disc_cvborg_driftborg_ensembleborg_error_profileborg_explain_riskborg_exportborg_extractborg_extrapolationborg_fairnessborg_fold_performanceborg_fold_similarityborg_forward_selectionborg_geodistborg_global_validationborg_group_vfold_cvborg_importanceborg_initial_splitborg_inspectborg_leafletborg_literature_checkborg_local_moranborg_metricsborg_multiscaleborg_null_testborg_optionsborg_pipelineborg_powerborg_predict_rasterborg_prediction_mapborg_register_hooksborg_repeated_cvborg_reportborg_rsetborg_sample_designborg_shapborg_simulateborg_spatial_cvborg_spatial_looborg_stabilityborg_stability_mapborg_temporal_cvborg_thinborg_to_biomod2borg_to_enmevalborg_to_mlr3borg_trainControlborg_transferabilityborg_unregister_hooksborg_validateborg_vfold_cvborg_willmottborg_workflowcv_leakage_reportshow

Dependencies:Rcpp

Area of Applicability and Extrapolation Detection
Installation | Introduction | The Dissimilarity Index | What it measures | Basic usage | Variable importance weighting | Area of Applicability | From DI to a binary mask | Full worked example | Cross-validated threshold | Manual threshold | Extrapolation Detection | Univariate and multivariate checks | Worked example | Comparing extrapolation and AOA | Transferability Assessment | Quantifying performance decay with distance | Interpreting transfer ratios | Prediction Maps | Spatial uncertainty from cross-validation | Stability Maps | Where predictions change with the training set | Stability maps versus prediction maps | Combining AOA with Blocked Cross-Validation | The full pipeline | Diagnose dependencies | Generate blocked CV folds | Fit and evaluate with honest folds | Compute spatially-aware importance | Determine the area of applicability | Generate predictions with AOA mask | Check for extrapolation in the reliable predictions | Practical Guidance | When to use the AOA | When the AOA is less useful | Interpretation guidelines | Mahalanobis distance and its limitations | Reporting AOA results in publications | Connecting to the broader BORG workflow | References

Last update: 2026-10-10
Started: 2026-03-31

Feature Selection with Blocked Cross-Validation
Installation | Introduction | Simulated data | Block-permutation importance | The problem with standard permutation importance | How block-permutation importance works | Worked example | Comparing block vs. row permutation | Forward feature selection | Why greedy search with blocked CV | Step-by-step example | Using a custom model | Interpreting the selection path | Best subset selection | SHAP values | Beyond global importance | Spatial SHAP | When to use SHAP vs. permutation importance | Comparing methods | Full pipeline | Diagnose dependencies | Generate blocked folds | Screen with permutation importance | Forward selection on screened candidates | Fit the final model | Interpret with SHAP | Practical guidance | How many features relative to sample size? | Stability of selection | Collinearity and importance | When NOT to do feature selection | Reporting feature selection in publications | Temporal and grouped data | References

Last update: 2026-10-10
Started: 2026-03-31

Framework Integration
Installation | Why This Guide Exists | Quick Reference | Base R | Safe Preprocessing Pattern | caret | When to Use BORG with caret | Validating preProcess Objects | Guarded trainControl | Pipeline-Level Validation with borg_pipeline() | tidymodels (rsample + recipes) | When to Use BORG with tidymodels | Validating Recipe Objects | Guarded rsample Functions | Converting BORG Folds to rset Objects | Convenience Constructors: borg_spatial_cv() and borg_temporal_cv() | Validating Existing rsample Objects | mlr3 | When to Use BORG with mlr3 | Validating mlr3 Tasks and Resamplings | Generating mlr3 Resamplings from BORG Folds | Species Distribution Models | ENMeval | biomod2 | Temporal Data Workflows | Basic Temporal Validation | Rolling Origin with rsample | Spatial Data Workflows | Spatial Block Validation | Automatic Spatial CV Generation | Grouped Data Workflows | Complete Pipeline Validation | borg_validate(): Workflow-Level Checks | Catching Overlap | borg_pipeline(): Stage-by-Stage Decomposition | Automatic Repair with borg_assimilate() | Migration Checklist | See Also

Last update: 2026-10-10
Started: 2026-01-07

Model Comparison with Blocked Cross-Validation
Installation | Introduction | Simulating spatial data | Quantifying inflation | Fair model ranking | Confidence intervals on metrics | Is the signal real? | Per-fold performance | Full comparison pipeline | Practical guidance | How many repeats for borg_compare_cv() | How many permutations for borg_null_test() | How many bootstrap replicates | When to report which metric | When model comparison is misleading | Reporting for publications | Number of folds | Minimum sample size | References

Last update: 2026-10-10
Started: 2026-03-31

Post-Fit Diagnostics
Installation | Setup: simulating spatial data and fitting a model | Quick diagnostic check | Residual diagnostics | Calibration | Null hypothesis testing | Model comparison | Quantifying inflation with borg_compare_cv() | Ranking models with borg_compare_models() | Conformal prediction | Bootstrap uncertainty | Power analysis | Per-fold performance | Diagnostic workflow | Practical guidance | When to run which diagnostic | Common pitfalls | Integration with external frameworks

Last update: 2026-10-10
Started: 2026-03-31

Quick Start
Installation | Why your test accuracy might be wrong | Simulating data with known dependencies | Diagnosing data dependencies | Temporal and grouped data | Working with CV folds | Exporting folds to other frameworks | Validating existing splits | Overlapping indices | Target leakage | Preprocessing leakage | Accessing risk details | The borg_check() shortcut | Fitting a model with borg_workflow() | Comparing random vs blocked CV | Power analysis after blocking | Putting it all together | Taste tests: more features | Area of Applicability | Block-permutation variable importance | Residual diagnostics | Calibration | Null test | Extrapolation detection | Geographic distance analysis | Interactive maps | S3 methods | Methods text for publications | Practical guidance | When to use BORG | When NOT to use BORG | Rules of thumb | Validation certificates | Spatial input formats | Function reference | Core | Evaluation | Spatial analysis | Feature selection and importance | Reporting | Simulation | Next steps | References

Last update: 2026-10-10
Started: 2026-01-07

Risk Taxonomy
Installation | Risk Classification | Hard Violations | 1. Index Overlap | 2. Duplicate Rows | 3. Preprocessing Leakage | 4. Target Leakage (Direct) | 5. Group Leakage | 6. Temporal Ordering Violation | 7. CV Fold Contamination | 8. Model Scope | Soft Inflation Risks | 1. Target Leakage (Proxy) | 2. Spatial Proximity | 3. Spatial Overlap | 4. Random CV on Dependent Data | Risk Interactions | Spatial + Target Leakage | Preprocessing + Spatial Structure | Groups + Temporal Ordering | Temporal + Spatial | Decision Tree: Which Risks to Check | Practical Guidance | Hard vs. Soft: Triage Order | How Risk Severity Scales with Dataset Size | Rules of Thumb | Quick Reference | Accessing Risk Details | See Also

Last update: 2026-10-10
Started: 2026-01-07

Spatial Cross-Validation
Installation | Introduction | The problem: spatial autocorrelation | Simulating spatial data | Spatial block cross-validation | Block size optimization | Alternative spatial CV strategies | Environmental blocking | Leave-disc-out (buffered spatial CV) | Buffered leave-one-out | Spatial+ CV | KNNDM | Distance distribution matching | Comparing random vs blocked CV | Practical guidance | Choosing a spatial CV strategy | Rules of thumb | When NOT to use spatial CV | A practical workflow | References

Last update: 2026-10-10
Started: 2026-03-31

Temporal Cross-Validation
Installation | Introduction | The problem: temporal autocorrelation | Simulating temporal data | Temporal block CV | Expanding window CV | Embargo periods | Comparing random vs temporal CV | Concept drift detection | Practical guidance | Choosing a strategy | Rules of thumb with specific numbers | When NOT to use temporal CV | Panel data | End-to-end workflow | References

Last update: 2026-10-10
Started: 2026-03-31

Readme and manuals

Help Manual

Help pageTopics
Coerce BorgDiagnosis to Data Frameas.data.frame.BorgDiagnosis
Coerce BorgRisk to Data Frameas.data.frame.BorgRisk
Audit Feature Importance Calculationsaudit_importance
Audit Predictions for Data Leakageaudit_predictions
Autoplot Method for borg_comparison Objectsautoplot.borg_comparison
Autoplot Method for borg_cv Objectsautoplot.borg_cv
Autoplot Method for borg_fold_perf Objectsautoplot.borg_fold_perf
Autoplot Method for borg_result Objectsautoplot.borg_result
Autoplot Method for BorgDiagnosis Objectsautoplot.BorgDiagnosis
Autoplot Method for BorgRisk Objectsautoplot.BorgRisk
BORG: Guard Your Model Evaluationborg
Adversarial Validationborg_adversarial
Area of Applicabilityborg_aoa
Assimilate Leaky Evaluation Pipelinesborg_assimilate
Enable/Disable BORG Auto-Check Modeborg_auto_check
Best Subset Variable Selection with Blocked CVborg_best_subset
Optimize Spatial Block Sizeborg_block_size
Block Bootstrap Confidence Intervals for CV Metricsborg_bootstrap
Cache and Retrieve BORG Diagnosesborg_cache borg_cache_clear borg_cache_get borg_cache_info borg_cache_set
Model Calibration Diagnosticsborg_calibration
Create Validation Certificateborg_certificate
Quick Leakage Check (Pipe-Friendly)borg_check
Check Geographic Representativeness of CV Foldsborg_check_coverage
Check Nested CV for Leakageborg_check_nested_cv
Check Residual Spatial Autocorrelationborg_check_residuals
Draw a Clustered Spatial Sampleborg_clustered_sample
Compare Random vs Blocked Cross-Validationborg_compare_cv
Compare Multiple Models with Spatial CVborg_compare_models
Conformal Prediction with Spatial Dependenceborg_conformal
Generate Valid Cross-Validation Schemeborg_cv
Spatial+ Debiasing for Spatial Confoundingborg_debias
Dissimilarity Indexborg_di
Diagnose Data Dependency Structureborg_diagnose
Leave-Disc-Out Cross-Validationborg_disc_cv
Detect Distribution Shift Between Training and Deployment Databorg_drift
Ensemble Predictions from CV Fold Modelsborg_ensemble
Calibrate Dissimilarity Index to Prediction Errorborg_error_profile
Explain Risks in Plain Language with Actionable Recommendationsborg_explain_risk
Export Validation Certificateborg_export
Extract Raster Values at Point Locations for BORGborg_extract
Detect Model Extrapolationborg_extrapolation
Detect Performance Disparities Across Subgroups Under Blocked CVborg_fairness
Evaluate Per-Fold Model Performanceborg_fold_performance
Per-Fold Environmental Similarity (MESS)borg_fold_similarity
Forward Feature Selection with Spatial/Blocked CVborg_forward_selection
Distance Distribution Diagnosticsborg_geodist
Global (Pooled) Cross-Validation Metricsborg_global_validation
BORG-Guarded group_vfold_cvborg_group_vfold_cv
Block-Permutation Variable Importanceborg_importance
BORG-Guarded initial_splitborg_initial_split
Inspect R Objects for Evaluation Risksborg_inspect
Interactive Leaflet Map of Spatial CV Foldsborg_leaflet
Scan Methods Text for Common Leakage Patternsborg_literature_check
Local Moran's I for Residualsborg_local_moran
Available Classification Metricsborg_metrics
Multi-Scale Performance Assessmentborg_multiscale
Null Model Significance Testborg_null_test
Get Current BORG Optionsborg_options
Validate an Entire Modeling Pipelineborg_pipeline
Estimate Statistical Power After Blockingborg_power
Predict onto a SpatRaster with AOA Maskingborg_predict_raster
Spatial Prediction Uncertainty Mapborg_prediction_map
Register BORG Hooksborg_register_hooks
Repeated Blocked Cross-Validationborg_repeated_cv
Generate BORG HTML Diagnostic Reportborg_report
Convert BORG Folds to an rsample rset Objectborg_rset
Suggest Sampling Locations to Improve AOAborg_sample_design
Spatial SHAP Valuesborg_shap
Generate Synthetic Data with Known Leakageborg_simulate
Spatial Block Cross-Validation (rsample-compatible)borg_spatial_cv
Buffered Leave-One-Out Cross-Validationborg_spatial_loo
Analyze CV Fold Stability Across Repeatsborg_stability
Prediction Stability Mapborg_stability_map
Temporal Block Cross-Validation (rsample-compatible)borg_temporal_cv
Spatially Thin Occurrence Databorg_thin
Convert BORG Folds to biomod2 Formatborg_to_biomod2
Convert BORG Folds to ENMeval Partition Formatborg_to_enmeval
mlr3 Resampling for BORG Cross-Validationborg_to_mlr3
BORG-Guarded trainControlborg_trainControl
Assess Geographic Transferabilityborg_transferability
Unregister BORG Hooksborg_unregister_hooks
Validate Complete Evaluation Workflowborg_validate
BORG-Guarded vfold_cvborg_vfold_cv
Willmott's Index of Agreementborg_willmott
Create a BORG Validation Workflowborg_workflow
BORG-Guarded Cross-Validation Functionsborg-wrappers
BorgDiagnosis S4 ClassBorgDiagnosis BorgDiagnosis-class show,BorgDiagnosis-method
BorgRisk S4 ClassBorgRisk BorgRisk-class show,BorgRisk-method
Generate CV Leakage Reportcv_leakage_report
Plot CV Comparison Resultsplot.borg_comparison
Plot BORG Result Objectsplot.borg_result
Plot BORG Objectsplot.BorgRisk
Print CV Leakage Reportprint.borg_cv_report
Summarize BORG Cross-Validationsummary.borg_cv
Summarize BORG Pipeline Validationsummary.borg_pipeline
Summarize BORG Power Analysissummary.borg_power
Summarize BORG Resultsummary.borg_result
Summarize BORG Diagnosissummary.BorgDiagnosis
Summarize BORG Risk Assessmentsummary.BorgRisk