This paper presents an ontological intelligent agent to remove impulse noise from highly corrupted images. It contains an image noise ontology to represent the image noise knowledge for the agent, a fuzzy inference mechanism for noise detection and removal, and an intelligent learning process for automatically generating the fuzzy numbers of the agent. The working environment for the intelligent agent is defined and the image noise ontology referred by the fuzzy inference mechanism is utilized to perform the task of noise removal. Then, using orthogonal array and factor analysis, a genetic algorithm is applied to the intelligent learning process. Finally, the fuzzy numbers of the image noise ontology are adjusted via the intelligent learning process to increase the performance of the intelligent agent. Experimental results show that the proposed approach can achieve better results than the state-of-the-art filters based on the criteria of Mean-Absolute-Error and Mean-Square-Error. Besides, on the subjective evaluation of those filtered images, the proposed approach can also generate a higher quality of global restorations.