The assumption of unidimensionality is central to the interpretation of test score results. The most common method for assessing unidimensionality may, however, be demanding in terms of resources and expertise in many settings. The purpose of this study is to illustrate the use of a Rasch-based procedure for evaluating unidimensionality and to evaluate it through comparison with an accepted technique. Two procedures were used to assess the unidimensionality of an English listening test given to first year students in a practical English course at a university in Taiwan. The first procedure was the commonly used factor analysis approach. The second procedure was a principal components analysis of the residuals from a Rasch (IRT IPL) analysis of the test results. Both methods were found to produce similar results. Specifically, both isolated two off-construct dimensions in the data exceeding the level of "random noise", and an item-level investigation suggested that they had both isolated the same source of disturbance. The finding that both methods produced nearly identical results supports the contention that the Rasch-based procedure offers a viable alternative to traditional factor analysis and can be thought of as another tool available to language testers.