Traditional image retrieval aims at bridging visual images and human concepts through visual or textual descriptions. However, it is still a challenging issue to reduce the gap between the images and user’s intentions. To this end, a considerable number of studies in the field of multimedia mining have been conducted on how to effectively meet the user’s requirements for image retrieval over the past few decades. However, there remain some problems unsettled. For content-based image retrieval, it is not easy to identify the user’s interest by using visual descriptions only. For textual-based image retrieval, the modern search engines incur the problems of high manual tagging cost and low automated tagging precision. Moreover, precise image retrieval also leads unsatisfied results because of the rough human concepts. To catch user’s concept well, we propose a novel approach, namely Intelligent SeMantic Image explorER (iSMIER), which considers the requirements of usability, intelligence and effectiveness simultaneously. Based on the proposed web image annotation, concept matching and fuzzy ranking methods, the users can obtain the desired images from the image collection easily and effectively. Through empirical evaluations, our annotation models can deliver high accuracy for serving semantic image retrieval.