Free energy minimization and information gain: The devil is in the details

Abstract

Contrary to Friston's previous work, this paper describes free energy minimization using categorical probability distributions over discrete states. This alternative mathematical framework exposes a fundamental, yet unnoticed challenge for the free energy principle. When considering discrete state spaces one must specify their granularity, as the amount of information gain is defined over this state space. The more detailed this state space, the lower the precision of the predictions will be, and consequently, the higher the prediction errors. Hence, an optimal trade-off between precision and detail is needed, and we call for incorporating this aspect in the free energy principle.