Semantic criteria for the assessment of uncertainty handling fusion models

Abstract

This paper proposes an illustration of the Uncertainty Representation and Reasoning Evaluation Framework (URREF) for the comparison of two classical fusion schemes. We revisit the classical works comparing Bayes' rule and Dempster's rule for fusion, and identify the criteria that have been used for both semantic theoretical and algorithmic implementation comparisons. The discussion is illustrated on a practical maritime use case previously experimented as a game with experts during a Table Top eXercise (TTX). The maritime use case described is simple enough to clearly highlight the differences and similarities between the different approaches, but complex enough to reflect real practical situations where sources of information of various nature are involved, the information they provide reflect different type of imperfection, several features are measured or observed, concurrent events happen. We highlight how the elements of the URREF ontology of UNCERTAINTYNATURE, UNCERTAINTYDERIVATION, UNCERTAINTYTYPE and SOURCE Quality influence the assessment of UNCERTAINTYMODELS through REPRESENTATION and REASONING evaluation criteria. We propose a list of additional elements which could be considered in the URREF ontology: Type of Problem (revision, prediction, fusion), Information Type (generic vs singular), Uncertainty Supports (variable, link, uncertainty statement) and Measurement Scale. We illustrate the different semantics of the two rules and how they may use different information.