篇名 | 標準設定之效度評估-以2010年臺灣八年級學生國語文學習成就資料為例 |
---|---|
卷期 | 22下 |
並列篇名 | Exploring Standard Setting in Chinese for the 2010 Taiwan Assessment of Student Achievement in Grade Eight |
作者 | 吳慧珉 、 蘇珊玉 |
頁次 | 001-021 |
關鍵字 | Yes/No Angoff方法 、 八年級 、 臺灣學生學習成就評量資料庫 、 標準設定 、 國語文 、 chinese 、 grade eight 、 standard setting 、 TASA 、 Yes/No Angoff 、 TSCI |
出刊日期 | 201412 |
本研究是使用臺灣學生學習成就資料庫2010年收集之臺灣八年級學生實徵 數據,進行八年級國語文學習成就的標準設定。本研究經學科專家確立政策定義 及國語文表現標準描述後,邀請北、中、南、東及離島等區之19名標準設定成 員,使用Yes/No Angoff方法執行標準設定,從效度的内部證據、效度的過程證 據和效度的外部證據,探討TASA國語文八年級標準設定之適切性。本研究從極 端判斷值、標準設定成員内設定結果的一致性和標準設定成員間設定結果的一致 性評估效度的内部證據,本研究内部證據來源顯示本研究之標準設定結果具有一 致性。從標準設定技術的選擇及執行,包含會議執行方式之評估、標準設定成員 的選擇、標準設定方法、回饋訊息的影響等面向進行效度的過程證據評估。整體 而言,八年級國語文科標準設定具有相當水準的過程效度。效度的外部證據檢視 方面,本研究從切截分數設定與分類的一致性,Angoff法對於基礎水準切截分數 設定較低,而在進階水準切截分數設定是較高的,但以Angoff方法為基礎,其 他三種方法與Angoff方法之分類具有一致性。本研究能為實務應用者提供良好參考範例。
The purpose of this study is to conduct standard setting in Chinese for the 2010 Taiwan Assessment of Student Achievement (TASA) of grade eight. The specific definitions of the basic, proficient, and advanced achievement levels are defined by the subject matter experts. The nineteen panelists coming from north, middle, south, and east areas of Taiwan are invited to conduct the standard setting procedures with Yes/No Angoff method. This study evaluates the internal evidence for validity using the extreme value method, standard setting consistency within and between panelists. Such internal evidence then reports that the results of the standard settings in this study are consistent. Meanwhile, we also evaluate the procedural evidence of validity in the choice and execution of the standard settings which collect the assessment of panel meeting, the recruitment of panelists, standard setting methods, and the impact of feedbacks. And the procedural validity of the standard setting in Chinese for grade eight performs quite well as an entirety. Finally, the external evidence of validity is collected from the consistency between the cut-scores setting and the classification. Yes/No Angoff’s method may tend to set low cut-scores for the basic level while it poses high cut-scores for the advanced level. Based on the Yes/No Angoff method, its classification would be consistent with the three other methods. Based on the internal, procedural, and external evidences, the standard setting procedures for the 2010 TASA of grade eight has its validity. This study shall provide a great reference for those who wish to conduct standard setting methods into practical use.