In order to enhance the immune algorithm (IA) performance and find the optimal solution when dealing with difficult problems, we propose an efficient immune-based particle swarm optimization (IPSO) for neuro-fuzzy classifiers to solve the skin color detection problem. The proposed IPSO combines the immune algorithm (IA) and particle swarm optimization (PSO) to perform parameter learning. The IA uses the clonal selection principle, such that antibodies between others of high similar degree are affected, and these antibodies, after the process, will have higher quality, accelerating the search and increasing the global search capacity. The PSO algorithm has proved to be very effective for solving global optimization. It is not only a recently invented high-performance optimizer that is easy to understand and implement, but it also requires little computational bookkeeping and generally only a few lines of code. Hence, we employed the advantages of PSO to improve the mutation mechanism of the immune algorithm. Simulations have shown the performance and applicability of the proposed method.