An Exploration of Prediction Accuracy of Developmental Degrees of Music Expression in Early Childhood Based on A Simultaneous Analysis of Both Eye Movement and Full-Body Movement
DOI:
https://doi.org/10.14738/tecs.113.14682Keywords:
feature quantities, body movement in musical expression, eye movement, quantitative analysis, machine learning, classification accuracyAbstract
This article explores validity to introduce eye movement of children in association with body movement as feature quantities of machine learning, in particular, to assess children’s maturity level of musical expression. There is a growing awareness of applying machine learning technique to yield practical benefit on predicting developmental degrees of music education by observing body movements with motion capture technology. From professional teaching perspective, it is believed only experienced teachers can evaluate children’s music achievement because coordinated functioning of body movements need carefully be observed in connection with various music factor such as rhythm, beat strength, tones, etc. If new teachers can take advantage of having objective results of evaluation of children’s expression level by machines, it will enhance educational efficiency to certain levels which experts could attain. The author aims to improve classification accuracy of machine learning by focusing on eye movements data simultaneously recorded with body movements. In this study, children (n=43) at two child facilities participated in the data capture of both eye movement and body movement during musical expression. Feature quantities were extracted from the results of a three-way of ANOVA, and applied to machine learning to improve the classification accuracy of developmental degree in musical expression. Specifically, when using several classifiers such as NN (Neural Network model), the classification accuracy of developmental degree of musical development was more precisely in the case of both body movement data and eye movement data included in feature quantities than in the case of only body movement data as feature quantities.
References
Jaques-Dalcroze, E., Rhythm, Music and Education, 1921, G. P. Putnam's Sons.
Ando, A. and Sumikawa, Y., Development and functional evaluation of teaching materials for observing sawing motion using motion capture and virtual space, Journal of the Japan Society for Educational Technology, 2012, 36 (2): p.103-110. https://doi.org/10.15077/jjet.KJ00008274459
Sato, K., Kaiga, T., and Watabe, S., Utilization of motion capture to support the mastery of dance, Journal of the Japan Society for Educational Technology, 2010, 34: p.133-136.
Thompson, M., & Luck, G., Exploring relationships between pianists’ body movements, their expressive intentions, and structural elements of the music. Musicae Scientiae, 2012, 16: p.19–40. https://doi.org/10.1177/1029864911423457
Burger, B., Move the way you feel: Effects of musical features, perceived emotions, and personality on musicinduced movement. 2013, Department of Music, University of Jyväskylä.
Burger, B., Thompson, M., Saarikallio, S., Luck, G., & Toiviainen, P., Influences of rhythm and timbre related musical features on characteristics of music induced movement, Frontiers in Psychology, 2013, 4 (183): p.1-
Sano, M., Statistical Analysis of Elements of Movement in Musical Expression in Early Childhood Using 3D Motion Capture and Evaluation of Musical Development Degrees through Machine Learning, World Journal of Education, 2018, 8 (3): p.118-130.
Fernandez-Delgado, M., Cernadas, E., Barro, S., Amorim, D., Do we need hundreds of classifiers to solve real world classification problems? Journal of Machine Learning Research, 2014, 15(90): p.3133−3181.
Sano, M., Predicting developmental degrees of music expression in early childhood by machine learning classifiers with 3D motion captured body movement data, Journal of Educational Research and Reviews (International),2019, 7(7): p.155-168.
Sano, M., Verification of a classification prediction method for the development of musical expression in early childhood using a machine learning method based on 3D motion capture data, Advances in Social Sciences Research Journal (International), 2020, 7(9): p.338 -358.
Kodama, U., Oba, N., and Ishi, N., Examination of behavior recognition method in general living environment using machine learning, IEICE technical report, 2015, 114(502): p.73-78.
Takada, K., Kitasuka, T., and Aritugi, M., Examination of personal identification method by gait using markerless motion capture device, Research Report Entertainment Computing(EC)2012: p.1-7.
Matumoto, A., Mikami, H., Kawamura, H., and Kojima, A., Examination of form classification method for supporting motor learning of elementary school students who performed motion analysis using image processing and machine learning techniques, Video Information Media Society Technical Report, 2014, 38, 51(0): p.9-12.
Higuchi, H., Okumura, Y., and Kobayashi, T., Development of toddler's hiragana-sound knowledge: An eyetracking study, IEICE technical report, 2019, 118(437): p.83-88.
Watabe, M., Matumura, A., and Uda, N., An attempt to extract children's tastes from gaze movements and facial expressions in storytelling of picture books, Proceedings of the 81st National Convention, 2019, (1): p.563-564.
Kusunoki, K., Ozawa, W., and Kanemori, Y., Evaluation of Immigrant Children’s Impediments in Reading Japanese: Comparative Analysis on Reading Horizontal and Vertical Writing Texts, 2017, 6(0): p.7-8.
Mpofu,B., University Students Use of Computers and Mobile Devices for Learning and their Reading Speed on Different Platforms, Universal Journal of Educational Research, 2016, 4(4): p.926-932. https://doi.org/10.13189/ujer.2016.040430
Seong-un Kim, Sung-man Lim, Eun-ae Kim, Il-ho Yang, An Analysis of Eye Movement and Cognitive Load about the Editorial Design in Elementary Science Textbook, Universal Journal of Educational Research, 2016, 4(3): 570-575, https://doi.org/10.13189/ujer.2016.040314
Moreno-Estevaa, E., White, S., Woodc,J., and Blackc, A., Application of mathematical and machine learning techniques to analyze eye tracking data enabling better understanding of children’s visual cognitive behaviours, Frontline Learning Research, 2019, 6(3): Special issue: p.72 – 84. https://doi.org/10.14786/flr.v6i3.365
Remount,F., and Budke,A., Strategies for Successful Learning with Geographical Comics: An Eye-Tracking Study with Young Learners, Education Sciences, 2020, 10: p.293, doi:10.3390/educsci10100293
Valtakari,N., Hooge1,I., & Viktorsson,C., & Nyström,P., Falck-Ytter,T., and Hessels, R., Eye tracking in human interaction: Possibilities and limitations, Behavior Research Methods, 2021, 53: p.1592–1608.https://doi.org/10.3758/s13428-020-01517-x/ Published online: 6 January 2021
Plö chl, M., Obleser, J., Do auditory rhythms influence eye movement statistics? Conference on Music & Eye-Tracking, 2017, Max Planck Institute for Empirical Aesthetics, p.19.
Lörch, L., Fehringer, B., Münzer, S., Reading music. How tonality and notation influence music reading experts’ eye movements, pupil dilation and performance in a pattern matching task. Conference on Music & Eye-Tracking, 2017, Max Planck Institute for Empirical Aesthetics, p.17.
Fusase, E., In “Childcare Expression Technology" Music I Consideration of specialized skills-line of sight when playing the piano: Through the difference in behavioral proficiency, Nagoya Yanagishiro Junior College Lab Bulletin of Research, 2017, (39): p.313-331.
Vandemoortele, S., Feyaerts,K., Reybrouck, M., De Bièvre, G., Brône, G., & De Baets, T., Gazing at the partner in musical trios:amobileeye-trackingstudy. Journal of Eye Movement Research, 2018, 11(2):p.6. DOI: 10.16910/jemr.11.2.6
Marandola, F., Eye-Hand synchronisation and interpersonal interaction in xylophone performance: A comparison between African and Western percussionists, Conference on Music & Eye-Tracking, 2017, Max Planck Institute for Empirical Aesthetics, p.18.
Drai-Zerbib, V. & Baccino, T., Cross-modal music integration in expert memory: Evidence from eyemovements. Journal of Eye Movement Research, 2018, 11(2):4, p.1-21. DOI: 10.16910/jemr.11.2.4
Puurtinen, M., Learning on the job: Rethinks and realizations about eye tracking in music-reading studies, Frontline Learning Research, 2018, 6 (3): p.148- 161. Sano, M. (2023). An Exploration of Prediction Accuracy of Developmental Degrees of Music Expression in Early Childhood Based on A Simultaneous
Orman, K., Price, H., Russell, R., Feasibility of Using an Augmented Immersive Virtual Reality Learning Environment to Enhance Music Conducting Skills, Journal of Music Teacher Education, 2017, 27 (1): p.24-
Orman, K., Effect of Virtual Reality Exposure and Aural Stimuli on Eye Contact, Directional Focus, and Focus of Attention of Novice Wind Band Conductors, International Journal of Music Education, 2016, 34(3): p.263-
Sano, M., Quantitative Analysis of Eye Movements during Singing of Early Childhood Children, Research Journal of Education, 2021, 7(3): p.125-140.
Sano, M., Characteristics of Eye Movements during Musical Expression in Early Childhood by Tonality: Through Quantitative Analysis by Eye Tracking, Frontiers in Education Technology, 2022, 5(2): p.67-93.
Nishina, M., Kume, M., Yoshida, T., Takai, Y., and Goto, A., Process Division of Soldering Work by Skilled Workers and Cooperative Relationship between Motions and Eye Movements in Making Metal Fittings of Flags, J. Science of Labour, 2013, 89(1): p.1-11.
Bishopa,L., Cancino-Chacón, C., and Goebl, W., Eye gaze as a means of giving and seeking information during musical interaction, Consciousness and Cognition, 2019, 68: p.73-96.https://doi.org/10.1016/j.concog.2019.01.002
Burger, B., Puupponen, A., & Jantunen, T., Synchronizing eye tracking and optical motion capture: How to bring them together, Journal of Eye Movement, 2018, 11(2): p.1-16.
Fink, L., Lange, E., & Groner, R., The application of eye-tracking in music research, Journal of Eye Movement Research, 2018, 11(2)1: p.1-4. doi: 10.16910/jemr.11.2.1
Jantunen, T., Puupponen, A., & Burger, P., What Comes First: Combining Motion Capture and Eye Tracking Data to Study the Order of Articulators in Constructed Action in Sign Language Narratives, Proceedings of the 12th Language Resources and Evaluation Conference, 2020, p.6003–6007. https://aclanthology.org/2020.lrec-1.735
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2023 Mina Sano

This work is licensed under a Creative Commons Attribution 4.0 International License.