This paper describes the critical study of open ended coaxial sensor using simulation technique by finite element method. The simulation includes the finding of cutoff wavenumber at each mode and was compared to analytical method. The simulation was also performed in finding the minimum thickness requirement for sample and the influencing parameters of coaxial probe performance. The simulation was carried out using COMSOL software.