Cross-validating causal discovery via Leave-One-Variable-Out

Abstract

We propose a new approach to falsify causal discovery algorithms without ground truth, which is based on testing the causal model on a pair of variables that has been dropped when learning the causal model. To this end, we use the "Leave-One-Variable-Out (LOVO)" prediction where Y is inferred from X without any joint observations of X and Y, given only training data from X,Z_1,,Z_k and from Z_1,,Z_k,Y. We demonstrate that causal models on the two subsets, in the form of Acyclic Directed Mixed Graphs (ADMGs), often entail conclusions on the dependencies between X and Y, enabling this type of prediction. The prediction error can then be estimated since the joint distribution P(X, Y) is assumed to be available, and X and Y have only been omitted for the purpose of falsification. After presenting this graphical method, which is applicable to general causal discovery algorithms, we illustrate how to construct a LOVO predictor tailored towards algorithms relying on specific a priori assumptions, such as linear additive noise models. Simulations indicate that the LOVO prediction error is indeed correlated with the accuracy of the causal outputs, affirming the method's effectiveness.

Tasks

Reproductions