The automatic quality inspection and grading application of mangoes, a widely cultivated fruit, is of paramount importance in post-harvest processing. Fruit features such as size, surface defects, and color are extracted using machine vision algorithms. A novel curve-fitting route algorithm CFR is proposed to improve the size measurement in image preprocessing. Surface defects are inspected and computed by utilizing image binarization techniques. Mango ripeness is analyzed and calibrated by utilizing the RGB color model. Finally, the Random Forests Classification technique is adopted in the machine learning layer to resolve the limitations in traditional multi-feature grading algorithms. Experimental results show that the accuracy of size calculation is nearly 98.4%, and the accuracy of quality grading received is up to 94.4%. Therefore, the proposed multi-feature grading method of mango (MFG) proved to be efficient and reliable.