High-frequency capacitors of high-quality are widely used in military and high-end consumer electronic products, it’s particularly important to measure Q value and other performance parameters of high-frequency capacitance. The fixture effects error of measurement system is particularly acute under the influence of capacitance for high frequency band. Aiming at this issue, a way to eliminate errors of the testing system based on modeling of telomeric extension and auto telomeric extension technology combines with de-embeding technologies is proposed to eliminate errors produced by fixture influence, and then using polynomial fitting algorithm that based on least square method to compute performance parameters and equivalent circuit parameters accurately. Experimental results indicate that the test data of proposed algorithm are more accurate compared with that of using the de-embeding technique simply for measuring capacitance at 10GHz or higher frequencies.