Part 5: Myth: Learners Learn More
 | |||||||||||
The
foundation of all learning, or child's first book, by which a child
will learn more in one month than by many others in twelve, Author Unknown, Date 1800. Source: Compositor, University of Birmingham Myth: Learners learn more in immersive experiencesThis myth shrouds itself within a cloak of research. Citations will state that learning in immersive experiences is somehow greater when pitted against an implied traditional learning approach. The claim could appear as retention, but it is related to how well the learning was accomplished when measured up against learning objectives or a final goal. When referring to the PricewaterhouseCooper (PwC) report results mentioned earlier in this article series, Scott Likens claimed, “We found the realism and performance feedback in virtual reality simulations helped people learn faster and retain more information around soft skills,” (Zielinski, 2021, para. 9). He claimed they retained more information. This specific claim has been repeated in academic literature, which stated, “Studies have found that students who use XR training are more engaged with the content, display more confidence with the material, and retain more information than students who use traditional training methods.” (Rode, 2024, para 2.) A close examination of the PwC report, however, reveals that the claim was unsupported within the report’s own data. When comparing information retention in VR versus an e-learning course, the authors “quickly discovered retention scores were inconclusive, as the delta between pre-and post-assessments in each modality was not significant” (Eckert & Mower, 2020, p. 44). Thus, there was no statistical difference between VR-based, e-learning, and traditional classroom learning.
The report therefore does not provide statistical evidence of more or greater learning within VR, yet it has been cited in academic publications (O'Dwyer, 2021, Etienne et al., 2022, Jelki et al., 2022; Bäckelin, 2023; Etienne et al., 2023; Lønne et al,. 2023) and touted in media outlets (Murad, 2023; Schwantes, 2020). For the dubious claims, the report has been debunked as untrustworthy (Neelen & Kirschner, 2020). There are similar claims about greater learning retained from immersive experiences. Advocates for digital twin campus environments claimed that they “create greater retention of the information that is learned” (D’Agustino, 2022, para. 5) and “students' grades go up” (Victory XR, 2024, Who We Are).
In another example, the CEO of the Miami Children’s Health System touted that learners had 80% retention after one year after using VR, but traditional learners had 20% retention. A close look at the supporting documentation shows that the CEO actually said that the difference between VR learners and traditional learners can be the 80% to 20% difference.
 However, the CEO’s statement did not refer to any published results; it was opinion. The CEO explained their claim by saying that, “The level of understanding through VR is great because humans are primarily visual, and VR is a visual format” (Gaudiosi, 2021, para. 4). Cue learning styles!  Nonetheless, the quote of 80% retention has made its way into academic research (Iacono & Vercelli, 2019; Mathew & Pillai, 2020; Ternès, 2018). Some claims are extreme. One keynote speaker, Alvin Graylin, speaking as a leader at HTC (a VR headset maker) declared that as a result of VR use in the classroom, “Every single child has the potential to be a genius” (Educators in VR, 2020, 23:33).
Justification for the greater learning or retention claims seems to be conjecture. Claims refer to how real an immersive experience feels to a learner. Returning to the PwC report, Likens credited “the realism and performance feedback…helped people..retain more information.” (Zielinski, 2021, para. 9). But long term studies measuring retention are hard to find within the body of academic literature. Some studies measure retention three weeks after the immersive experiences. Given that many of the examples of immersive experiences relate to hands-on disciplines like nursing or construction, things learned in immersive experiences would be needed on the job more than three weeks after the training. Hamilton et al. (2020) stated that finding “learning outcomes, intervention characteristics, and assessment measures associated with immersive virtual reality has been sparse” (p. 1). Beck, Morgado, and O’Shea (2023) pointed out that details of methods are missing so that outcomes become questionable, “Very few literature reviews focus on the educational practices and strategies used in immersive learning environments. Thus, the problem is that we are evaluating outcomes without a comparable way to describe the educational approaches that led to those outcomes” (p. 2). Lawson et al. (2024) completed a systematic review of immersive experiences and found that research studies rarely isolate instructional methods and conditions when describing research studies and thus impact real world classroom decisions. Meta analyses are starting to illuminate this area. Akgün and Atıcı (2022) observed that there was only a moderate effect on learner achievement after surveying 31 studies. Kaplan, Cruit, Endsley, Beers, Sawyer and Hancock found that “XR does not express a different outcome than training in a non-simulated, control environment. It is equally effective at enhancing performance” (2020, p. 1) Some researchers are starting to incorporate machine learning and artificial intelligence into this challenge in order to determine what the published records states about immersive experiences. Markowitz et al. (2024) recently surveyed 196,734 paper abstracts with this method.
Myth: Learners learn more in immersive experiencesI find this myth is misleading. If immersive learning is compared to other forms of learning, all other variables must be held the same in order for the results to be considered valid. More on the faults of comparative media studies will be covered in another section of this series. ReferencesAkgün, M., & Atıcı, B. (2022). The effects of immersive virtual reality environments on students' academic achievement: A meta-analytical and meta-thematic study. Participatory Educational Research, 9(3), 111-131. Bäckelin, J. (2023, August). Afraid or prepared-using dialogue training to practice crisis leadership in municipalities. In International Symposium on Digital Transformation. Beck, D., Morgado, L., & O'Shea, P. (2023). Educational practices and strategies with immersive learning environments: Mapping of reviews for using the metaverse. IEEE Transactions on Learning Technologies, 17, 319-341. D’Agustino, S. (2022, August 3). College in the metaverse is here. Is higher ed ready? Inside Higher Ed. https://www.insidehighered.com/news/2022/08/03/college-metaverse-here-higher-ed-ready Eckert, D., & Mower, A. (2020). The effectiveness of virtual reality soft skills training in the enterprise: a study. PwC Public Report. https://www.pwc.com/us/en/services/consulting/technology/emerging-technology/assets/pwc-understanding-the-effectiveness-of-soft-skills-training-in-the-enterprise-a-study.pdf Educators in VR. (2020, February 20). Alvin Wang Graylin – Keynote (Alvin Wang Graylin) [Video]. YouTube. https://www.youtube.com/watch?v=-xT0sCUkJ-U Etienne, E., Leclercq, A. L., Remacle, A., & Schyns, M. (2022). Validation of a virtual framework for public speaking training. [Paper presentation]. 7th International XR Conference, Lisbon, Portugal. https://orbi.uliege.be/bitstream/2268/290274/1/Revised%20AR_VR%20abstractdec2021-22.pdf Etienne, E., Leclercq, A., Remacle, A., Dessart, L., & Schyns, M. (2023). Perception of avatars nonverbal behaviors in virtual reality. Psychology and Marketing, 40(11), 2464–2481. https://doi.org/10.1002/mar.21871 Gaudiosi J. (2015, August 17). Here's why hospitals are using virtual reality to train staff. Fortune. https://fortune.com/2015/08/17/virtual-reality-hospitals/ Hamilton, D., McKechnie, J., Edgerton, E., & Wilson, C. (2021). Immersive virtual reality as a pedagogical tool in education: a systematic literature review of quantitative learning outcomes and experimental design. Journal of Computers in Education, 8(1), 1-32. Iacono, S., & Vercelli, G. (2019). Lessons learned about language learning and extended reality frameworks. In the 5th International Conference on Virtual and Augmented Reality in Education (pp. 51-55). Jelke, B., Tran, L., Wagner, L., & Baus, L. (2022). Developing a trauma-informed interviewing training module prototype for law enforcement using virtual reality. Immersive Learning Research-Practitioner, 28-31. Kaplan, Alexandra D., Jessica Cruit, Mica Endsley, Suzanne M. Beers, Ben D. Sawyer, and Peter A. Hancock. "The effects of virtual reality, augmented reality, and mixed reality as training enhancement methods: A meta-analysis." Human factors 63, no. 4 (2021): 706-726. Lawson, A. P., & Mayer, R. E. (2024). Effect of pre-training and role of working memory characteristics in learning with immersive virtual reality. International Journal of Human–Computer Interaction, 1-18. Lønne, T. F., Karlsen, H. R., Langvik, E., & Saksvik-Lehouillier, I. (2023). The effect of immersion on sense of presence and affect when experiencing an educational scenario in virtual reality: A randomized controlled study. Heliyon, 9(6). Markowitz, D. M., Bailenson, J. N., & Markowitz, D. M. (2024, preprint) A looking glass into a research wonderland: 40 years of virtual reality scholarship explicated via natural language processing. Mathew, P. S., & Pillai, A. S. (2020). Role of immersive (XR) technologies in improving healthcare competencies: a review. Virtual and augmented reality in education, art, and museums, 23-46. Murad, A. (2023, August 21). Virtual reality brings new vision to workplace training. BBC News. https://www.bbc.co.uk/news/business-66393315.amp Neelen, M. & Kirschner, P. (2020). Truth or truthiness? Analysing a VR study using Gorard’s sieve. https://3starlearningexperiences.wordpress.com/2020/07/07/truth-or-truthiness-analysing-a-vr-study-using-gorards-sieve/ O'Dwyer, L. (2021). Review of employment-based training models. National Centre for Vocational Education Research (NCVER). https://files.eric.ed.gov/fulltext/ED615316.pdf Rode, R. & Spasojevic, M. (2024, April 2). From XR pilots to full-scale deployments: considerations. Educause Review. https://er.educause.edu/articles/2024/4/from-xr-pilots-to-full-scale-deployments-considerations Schwantes, M. (2020, July 22). 3 ways to train your workers: virtual reality, classroom, and e-Learning (One works best). Inc. https://www.inc.com/marcel-schwantes/how-to-train-your-workers-virtual-reality.html VictoryXR. (2024). About us. https://www.victoryxr.com/about-us/ Zielinski, D. (2021). The growing impact of virtual reality training. https://www.shrm.org/hr-today/news/hr-magazine/spring2021/pages/virtual-reality-training-spreads-its-wings.aspx |
The content cannot be used to train or be reviewed by AI. All copyrights retained.
Did you miss the other parts of this series? Here they are!
Part 1: From Myths To Principles: Navigating Instructional Design in Immersive Environments
Part 2: The Immersive Environment Delusion
Part 3: The Case Against Virtual Campuses
Part 4: Myth: Learners Learn Faster
Part 5: Myth: Learners Learn More
# The example for img2dataset, although the default is *None* User-agent: img2dataset Disallow: / # Brandwatch - "AI to discover new trends" User-agent: magpie-crawler Disallow: / # webz.io - they sell data for training LLMs. User-agent: Omgilibot Disallow: / # Items below were sourced from darkvisitors.com # Categories included: "AI Data Scraper", "AI Assistant", "AI Search Crawler", "Undocumented AI Agent" # AI Search Crawler # https://darkvisitors.com/agents/amazonbot User-agent: Amazonbot Disallow: / # Undocumented AI Agent # https://darkvisitors.com/agents/anthropic-ai User-agent: anthropic-ai Disallow: / # AI Search Crawler # https://darkvisitors.com/agents/applebot User-agent: Applebot Disallow: / # AI Data Scraper # https://darkvisitors.com/agents/applebot-extended User-agent: Applebot-Extended Disallow: / # AI Data Scraper # https://darkvisitors.com/agents/bytespider User-agent: Bytespider Disallow: / # AI Data Scraper # https://darkvisitors.com/agents/ccbot User-agent: CCBot Disallow: / # AI Assistant # https://darkvisitors.com/agents/chatgpt-user User-agent: ChatGPT-User Disallow: / # Undocumented AI Agent # https://darkvisitors.com/agents/claude-web User-agent: Claude-Web Disallow: / # AI Data Scraper # https://darkvisitors.com/agents/claudebot User-agent: ClaudeBot Disallow: / # Undocumented AI Agent # https://darkvisitors.com/agents/cohere-ai User-agent: cohere-ai Disallow: / # AI Data Scraper # https://darkvisitors.com/agents/diffbot User-agent: Diffbot Disallow: / # AI Data Scraper # https://darkvisitors.com/agents/facebookbot User-agent: FacebookBot Disallow: / # AI Data Scraper # https://darkvisitors.com/agents/google-extended User-agent: Google-Extended Disallow: / # AI Data Scraper # https://darkvisitors.com/agents/gptbot User-agent: GPTBot Disallow: / # AI Data Scraper # https://darkvisitors.com/agents/omgili User-agent: omgili Disallow: / # AI Search Crawler # https://darkvisitors.com/agents/perplexitybot User-agent: PerplexityBot Disallow: / # AI Search Crawler # https://darkvisitors.com/agents/youbot User-agent: YouBot Disallow: /