-
Tytuł:
-
An exploration of automated narrative analysis via machine learning.
-
Autorzy:
-
Jones S; Department of Mathematics and Statistics, Utah State University, Logan, Utah, United States of America.
Fox C; Department of Special Education and Rehabilitation, Utah State University, Logan, Utah, United States of America.
Gillam S; Department of Communication Disorders and Deaf Education, Utah State University, Logan, Utah, United States of America.
Gillam RB; Department of Communication Disorders and Deaf Education, Utah State University, Logan, Utah, United States of America.
-
Źródło:
-
PloS one [PLoS One] 2019 Oct 31; Vol. 14 (10), pp. e0224634. Date of Electronic Publication: 2019 Oct 31 (Print Publication: 2019).
-
Typ publikacji:
-
Journal Article
-
Język:
-
English
-
Imprint Name(s):
-
Original Publication: San Francisco, CA : Public Library of Science
-
MeSH Terms:
-
Reproducibility of Results*
Educational Measurement/*methods
Child ; Female ; Humans ; Machine Learning ; Male ; Narration ; Observer Variation
-
References:
-
Lang Speech Hear Serv Sch. 2016 Jul 1;47(3):246-58. (PMID: 27380004)
Int J Speech Lang Pathol. 2014 Jun;16(3):242-9. (PMID: 24447161)
Brain Lang. 2004 Feb;88(2):229-47. (PMID: 14965544)
PeerJ Comput Sci. 2019 Aug 12;5:e208. (PMID: 33816861)
J Speech Lang Hear Res. 2000 Feb;43(1):34-49. (PMID: 10668651)
Behav Res Methods Instrum Comput. 2004 May;36(2):193-202. (PMID: 15354684)
J Speech Hear Res. 1987 Dec;30(4):539-52. (PMID: 3695446)
-
Entry Date(s):
-
Date Created: 20191101 Date Completed: 20200313 Latest Revision: 20231014
-
Update Code:
-
20240104
-
PubMed Central ID:
-
PMC6822746
-
DOI:
-
10.1371/journal.pone.0224634
-
PMID:
-
31671140
-
The accuracy of four machine learning methods in predicting narrative macrostructure scores was compared to scores obtained by human raters utilizing a criterion-referenced progress monitoring rubric. The machine learning methods that were explored covered methods that utilized hand-engineered features, as well as those that learn directly from the raw text. The predictive models were trained on a corpus of 414 narratives from a normative sample of school-aged children (5;0-9;11) who were given a standardized measure of narrative proficiency. Performance was measured using Quadratic Weighted Kappa, a metric of inter-rater reliability. The results indicated that one model, BERT, not only achieved significantly higher scoring accuracy than the other methods, but was consistent with scores obtained by human raters using a valid and reliable rubric. The findings from this study suggest that a machine learning method, specifically, BERT, shows promise as a way to automate the scoring of narrative macrostructure for potential use in clinical practice.
Competing Interests: The authors have declared that no competing interests exist.
Zaloguj się, aby uzyskać dostęp do pełnego tekstu.