The logic of empirical testing of accident prediction models

This paper explains the logic of empirical testing of accident prediction models. The key element of empirical testing is to make out-of-sample predictions of the number of accidents. This means that a model developed in sample A is applied, without modification, to predict the number of accidents in sample B. The procedure is illustrated in two samples formed by randomisation. A model fitted to the first sample was applied to predict the number of accidents in the second sample. The model was only partly supported. In general, any accident prediction model is likely to be merely a local statistical description of a particular data set. If tested by means of out-of-sample predictions, the model is very likely to be falsified. This does not mean that accident prediction models do not show general tendencies, but these tendencies are likely to be empirically supported only at a qualitative level, or at best an ordinal level of numerical measurement. In this sense accident prediction models are similar to many models developed in economics. The models predict the direction, and in some cases the relative strength of statistical relationships, but not their precise numerical values.