Informacja

Drogi użytkowniku, aplikacja do prawidłowego działania wymaga obsługi JavaScript. Proszę włącz obsługę JavaScript w Twojej przeglądarce.

Przeglądasz jako GOŚĆ
Tytuł pozycji:

Examining the Calibration Process for Raters of the "GRE"® General Test. ETS GRE® Board Research Report. GRE®-19-01. Research Report Series. ETS RR-19-09

Tytuł :
Examining the Calibration Process for Raters of the "GRE"® General Test. ETS GRE® Board Research Report. GRE®-19-01. Research Report Series. ETS RR-19-09
Autorzy :
Wendler, Cathy; Glazer, Nancy; Cline, Frederick
Źródło :
ETS Research Report Series, Dec 2019. 19 pp.
Dostępność :
Educational Testing Service. Rosedale Road, MS19-R Princeton, NJ 08541. Tel: 609-921-9000; Fax: 609-734-5410; e-mail: ; Web site: https://www.ets.org/research/policy_research_reports/ets
Recenzowane naukowo :
Y
ISSN :
2330-8516
Deskryptory :
College Entrance Examinations, Graduate Study, Accuracy, Test Reliability, Scoring, Interrater Reliability, Examiners, Quality Control, Essays, Writing Evaluation
Abstractor :
As Provided
Liczba referencji :
-1
Język :
English
Liczba stron :
19
Education Level :
Higher Education; Postsecondary Education
Typ publikacji :
Journal Articles; Reports - Research; Tests/Questionnaires
Kod czasopisma :
JAN2021
Data wpisu :
2020
Numer akcesji :
EJ1238527
Czasopismo naukowe
One of the challenges in scoring constructed-response (CR) items and tasks is ensuring that rater drift does not occur during or across scoring windows. Rater drift reflects changes in how raters interpret and use established scoring criteria to assign essay scores. Calibration is a process used to help control rater drift and, as such, serves as a type of quality control during CR scoring. Calibration sets are designed to provide sufficient evidence that raters have understood and internalized the rubrics and can score accurately across all score points of the score scale. This study examined the calibration process used to qualify raters to score essays from the "GRE"® Analytical Writing measure. A total of 46 experienced raters participated in the study, and each rater scored up to 630 essays from 1 of 2 essay prompt types. Two research questions were evaluated: "Does calibration influence scoring accuracy?" and "Does reducing the frequency of calibration impact scoring accuracy?" While the distribution of score points represented by the essays used in the study did not necessarily reflect what raters see during operational scoring, results suggest that the influence of calibration on Day 1 remains with raters through at least 3 scoring days. Results further suggest that scoring accuracy may be moderated by prompt type. Nevertheless, study results indicate that daily calibration for GRE prompt types may not be necessary and that reducing the frequency of calibration is unlikely to reduce scoring accuracy.

Ta witryna wykorzystuje pliki cookies do przechowywania informacji na Twoim komputerze. Pliki cookies stosujemy w celu świadczenia usług na najwyższym poziomie, w tym w sposób dostosowany do indywidualnych potrzeb. Korzystanie z witryny bez zmiany ustawień dotyczących cookies oznacza, że będą one zamieszczane w Twoim komputerze. W każdym momencie możesz dokonać zmiany ustawień dotyczących cookies