A decision-making (-support) system consists of three basic steps: sample extraction, characterization, and classification. In sample extraction, the system needs to isolate the content of interest for image analysis, which may be a superpixel, object, or subimage. Each image may contain from one to multiple samples and the characterization of each sample by a mathematical representation (a vector, a graph) allows the sample annotation (labeling) by a pattern classifier. In this project, we investigate interactive machine learning methods for sample extraction, characterization, and classification. That is, object shape models for image segmentation, feature learning methods, such as visual dictionaries and deep learning techniques, for sample characterization, and active learning methods to train pattern classifiers.

We aim at answering the following questions when building decision-making (-support) systems: What is the most intuitive way specialists can teach machines to annotate images? What are the tasks and challenges involved in this process? How to minimize human effort with maximum efficacy in machine learning? What can machines and specialists learn from their interaction?