In recent years, L2 pragmatics has increasingly gained its importance in second language acquisition, but the assessment in this field still lags behind. This study1 aims to highlight the limitations of a holistic scoring method used in L2 pragmatics through raters' eyes. Two native speakers of English were invited to rate five compliment exchanges produced by Taiwanese college students based on Sasaki 's (1998) appropriateness-grammar scales. The results demonstrate that pragmatic competence involves multiple components and that a learner may succeed in one aspect, but fail in another. This study closes by advocating the development of an analytic scoring instrument in L2 pragmatics since such assessment can reflect the componential nature of pragmatic competence and provide teachers with more pedagogical implications.