Highlights A fuzzy-knowledge based intelligent system for multilayered image annotation Novel merged statistical and knowledge-based approach for image interpretation Automatic acquisition of facts and rules about the concepts, and their reliability. Inconsistency checking of image segments classification. Automatic knowledge-based scene recognition and inference of more abstract classes.