Virtual reality in the service of psychology

In the current issue of the research magazine "Forschung Frankfurt", journalist Jessica Klapp reports on her virtual trip to Italy and explains why we don't look for the milk under the bed or the pillow in the bathtub.

"When we look for a particular object in a scene, we seem to have developed precise ideas about which objects to look for and where to find them," explains Melissa Võ. When researching these natural scenes, she is particularly interested in how we perceive our surroundings. What circumstances particularly catch our eye? And what would we remember later? To find out, the psychologist uses eye tracking and virtual reality scenes in addition to brain potential measurements in her lab.

Eye movements are recorded "With eye tracking, we measure which parts of an image the viewer finds interesting or important, how quickly the gaze falls on certain objects in scenes and how long the gaze remains there," explains Dr Dejan Draschkow. Because of the close relationship between eye movement and cognitive processes, eye tracking is of great importance. With the mobile system, the test subjects can move around the room, search for objects and interact with them.

Simulated environments

With the virtual reality headset, a virtual 3D world is simulated via the computer through which the test person moves. With simulated environments such as an Italian piazza, in the middle of which unexpected brown boxes float, the researchers test whether the results they determine on two-dimensional screens also apply in a realistic, three-dimensional environment. They want to understand laws that help people construct their environment and interact with the objects in it.

The research of scene knowledge in childhood is one of the fields that the working group is dealing with very intensively. The aim of the SCESAM project is to identify and treat possible cognitive deficits such as dyslexia at an early stage. Using a mobile research laboratory, the studies take place directly in front of the KiTa: The researchers show the children "ungrammatical" pictures, such as a shoe instead of a pot on the cooker, and observe their reactions with the help of an eye-tracking camera. If one child among many behaves differently, they are interested in whether there is a connection to linguistic development and attention behaviour.

Research into scene knowledge helps in many areas

Areas such as medicine also benefit from the results. For example, the researchers measured the eye movements of radiologists when viewing X-ray images and investigated which strategies they use to detect tumours and with what success these strategies are associated. The research results are equally important for hand luggage security checks at airports. How do employees decide which pieces of luggage need to be checked more closely? Why was a dangerous item not found? Was the screener not looking at that area? Or did he look at it but not consider that part important?

Finally, people with dementia could also benefit from research into scene knowledge. This is because Võ and her colleagues have found that memory performance for images in a scene increases if the test subjects have previously searched for and found individual objects. In a surprise memory test, they performed significantly better than people who were explicitly asked to remember objects. "For us, this means that a strong engagement with the scene takes place during the visual search and that objects are better remembered," explains the psychologist.

Source: Win-Verlag / Johann Wolfgang Goethe University Frankfurt