Embodied cognition and immersive virtual environments: doing, being, seeing and language learning

Thesis

Exploring embodied cognition inside immersive virtual environments (IVEs) and the implications for language learning, by examining the impact of “doing” an embodied action, the experience of “being” in an embodied space, and “seeing” embodied agents.

Sections

1) Embodied cognition and immersive virtual environments: Introduces embodied cognition and expands on the aspects of embodied cognition particularly affordanced by immersive virtual environments; and introduces the impact of embodiment, and embodiment in immersive virtual environments, on learning and language learning.

2) Evidence for the efficacy of embodied cognition in an immersive virtual language-learning environment Study design and results of an experiment to understand the impact of embodied actions in immersive virtual environments on language memorisation in a scenario designed to find embodiment benefits.

3) Kinesthetic controls in immersive virtual environments: simulated gesture or bodily action? Study design and results of an experiment to understand how our brains understand kinesthetic action in immersive virtual reality, as real-action on virtual space, or gesture simulation affecting a digital world, and its relationship to real-world cognition.

4) Physical objects vs. virtual objects in immersive virtual reality: are there differences in memory associations? Study design and results of an experiment to see if memorisation of nouns is different between interactive objects in real and virtual environments

5) Situatedness in immersive virtual environments: does environment impact second learning performance in a language-learning game? Study design and results of an experiment to see if a contextually-accurate and high-quality environment encourages better language learning than a contextually-inaccurate gaming environment for language memorisation

And either (a) or (b):

6a) What does a second-language acquisition syllabus look like for an immersive virtual environment designed to leverage embodied cognition? Outlines the challenges of designing a syllabus for an embodied cognition-rich immersive virtual learning environments, proposes a process, design (corpus-informed, multimodal interaction), and demonstrates the syllabus.

7a) Does an embodied, immersive virtual learning environment promote language acquisition better than classroom or less-rich alternatives? Outlines the design and outcome of an experiment created to understand whether leveraging embodiment in an immersive virtual environment affects learning, and learning a second language. Controls against classroom and less-rich immersive systems.

OR

6b) Codifying embodiment for a language learning in an immersive virtual environment Outlines the process of observing and codifying embodied interactions for language learning design inside an immersive virtual learning environment; proposes a process, design (corpus-informed, multimodal interaction).

7b) How do leaners respond to highly embodied agents compared with less embodied ones? Study design and results of an experiment to understand how learners responses differ from highly embodied-presenting non-player agents to static ones.

8) Conclusions

Cancelled Section

Why spoken-dialogue, and immersion, for second language acquisition? Outlines why spoken-dialogue and immersion are beneficial for second language acquisition, with theoretical groundings from linguistics and cognitive science, and examples of existing computer-aided spoken-dialogue and immersive investigations.

How do aspects of non-dialogue immersion affect second language acquisition? Outlines the justification, purpose, design, and outcome of an experiment created to understand how widely-used VR paradigms affect learning, and learning a second language.

Fields

  • Cognitive science
  • Human computer interaction
  • Immersion
  • Virtual reality
  • Virtual reality learning
  • Serious Games / Digital Game-Based Learning
  • Applied linguistics
  • Computer-aided language learning
  • Dialogue systems

Hypotheses

Other

Research Management + Schedule

Tasks

Outstanding task list (powered by Trello)

Research plan
JanuaryFebruaryMarchAprilMayJuneJulyAugustSeptemberOctoberNovemberDecember
TWTFMTWTFMTWTFMTWTFMTWTFMTWTFMTWTFMTWTFMTWTFMTWTFMTWTFMTWTFMTWTFMTWTFMTWTFMTWTFMTWTFMTWTFMTWTFMTWTFMTWTFMTWTFMTWTFMTWTFMTWTFMTWTFMTWTFMTWTFMTWTFMTWTFMTWTFMTWTFMTWTFMTWTFMTWTFMTWTFMTWTFMTWTFMTWTFMTWTFMTWTFMTWTFMTWTFMTWTFMTWTFMTWTFMTWTFMTWTFMTWTFMTWTFMTWTFMTWTFMT
Research computer-aided language learning
Research computer-aided language learning
2019/01/01
2019/01/20
Research immersive virtual environments
Research immersive virtual environments
2019/01/21
2019/02/09
Research dialogue management approaches
Research dialogue management approaches
2019/01/21
2019/02/28
Research immersion, embodiment + learning
Research immersion, embodiment + learning
2019/02/10
2019/03/20
Design immersion + learning study
Design immersion + learning study
2019/03/21
2019/04/30
Build embodied learning IVE
Build embodied learning IVE
2019/04/30
2019/07/30
Organise + run study
Organise + run study
2019/08/01
2019/11/21
Write-up study
Write-up study
2019/11/21
2019/12/31
TWTFMTWTFMTWTFMTWTFMTWTFMTWTFMTWTFMTWTFMTWTFMTWTFMTWTFMTWTFMTWTFMTWTFMTWTFMTWTFMTWTFMTWTFMTWTFMTWTFMTWTFMTWTFMTWTFMTWTFMTWTFMTWTFMTWTFMTWTFMTWTFMTWTFMTWTFMTWTFMTWTFMTWTFMTWTFMTWTFMTWTFMTWTFMTWTFMTWTFMTWTFMTWTFMTWTFMTWTFMTWTFMTWTFMTWTFMTWTFMTWTFMTWTFMTWTFMTWTFMT

See full schedule + year 2

Milestones

Technical Development

See technical development.

Glossary

  • VR: Virtual reality
  • AR: Augmented reality
  • CALL: Computer-assisted language learning
  • SAL: Second-language acquisition
  • DGBL: Digital game-based learning
  • Second: Language learned after early childhood
  • Language: Knowledge system that contains all information about language
  • Acquisition: Ability to effortlessly or near-effortlessly recall
  • Immersive Virtual Environment: An environment experienced using a total immersion headset (Rift/Vive)
  • Dialogue-system CALL: dialogue-based CALL corresponds to a consistent undertaking, that is, allowing a learner to practice an L2 autonomously in meaningful conversations (also Conversational Agent)
  • SDS: Spoken dialogue system
  • ASR: Automatic speech recognition

Research Trips

Funding opportunities and grants

Relevant Conferences & Calls

Conferences + calls

Resources

Immersive Virtual Reality Learning Environment Database