Abstract We wish to minimize the information that a robot maintains to carry out its task. Filters are one way to keep stored state consistent with sensed values, though they may also capture some information about the structure of the world that the robot inhabits. This paper builds on prior work on (improper) filter minimization, but considers a new way to characterize structure in the world. By introducing a probabilistic model, one can define a notion of expected distance between two filters. Then, with such a measure, we pose the question of optimal lossy compression in the sense of having minimal expected distance. The problem retains the NP- hardness of the non-probabilistic worst-case minimization and, consequently, in this paper we focus on developing an effective heuristic algorithm. Our results illustrate that, in settings where the probabilities describe evolution of the world's state, the algorithm can do substantially better than existing worst-case minimization techniques oblivious to such structure.