Evaluating and validating species distribution models with R

Species distribution models are commonly evaluated with metrics such as the area under the receiver operating characteristic (ROC) curve (AUC), the true skill statistic (TSS), Cohen’s kappa, sensitivity, specificity, or the overall classification accuracy or correct classification rate.

These metrics assess discrimination and/or classification capacity, i.e., the ability of the model to distinguish and classify localities with and without presence records. However, this is only one limited aspect of model performance. Other aspects, such as calibration or reliability (i.e., how predicted presence probabilities relate to observed occurrence frequencies), are also very important, albeit often overlooked, aspects of model performance. In addition, common discrimination metrics are often deceivingly high for rare species and naturally lower for generalist species, which do not clearly discriminate “good” from “bad” habitat.

Model quality should thus be assessed with a comprehensive set of evaluation measures, appropriate for species with different ecological and biogeographical traits.

In this short course, we will use R to compute a wide range of model evaluation metrics, encompassing discrimination, classification, explanatory power and calibration. We will also see different ways to validate models outside their training area, including spatial block cross-validation.

The course will be highly interactive and will include both theoretical and practical sessions. Participants are encouraged to bring their own models with observed and predicted values, though they can also work with example models that will be provided.

This course will not cover data preparation and model building, participants are expected to be familiar with how to build species distribution models.