Data Capture

Scientists and engineers are increasingly generating large amounts of complex 3D data thanks to advanced data capture, 3D graphics, and simulation technologies. Seeing and working with such 3D data is becoming more common and difficult as a part of many research processes, and systems that support these tasks are receiving more attention than ever before.

Virtual reality (VR) research in psychology is expanding, offering unique methods for collecting a variety of data. The types of data that can be collected using VR tools are diverse, ranging from quantitative data (such as spatial coordinates and completion times) to qualitative data (like subjective user experiences and behaviours). Here are the most common data types:

Spatial Data: These include the three-dimensional coordinates of actions within the virtual space. This data provides information on participants' movements, gestures, and spatial interactions.

Temporal Data: Time-based measurements such as the time taken to complete tasks or the temporal sequence of actions.

Behavioural Data: Observable actions within the VR environment, such as choices participants make when interacting with virtual objects.

Physiological Data: Metrics such as heart rate or eye movements, which can be collected using additional hardware integrated into the VR system.

Subjective Data: Participants’ self-reported experiences, often collected post-experiment via interviews or questionnaires.

Example of Data Extraction Using 3D Drawings

One of the key contributions of VR to psychological research is the ability to collect and analyse 3D spatial data. In studies like the one outlined in the thesis, where participants recreated the Rey-Osterrieth Complex Figure (ROCF) in a virtual environment using tools like Tilt Brush, novel techniques were employed to extract this three-dimensional data.

The data extraction process involves:

Data Export: After participants draw in VR, the sketches are exported as .fbx or .obj files, which contain three-dimensional coordinates.

Conversion and Analysis: Using software such as MeshLab, these files are converted into a format that can be analysed. Key metrics such as surface area, perimeter, and depth can be extracted for each drawing.

Visualization: The data can be visualized as 3D models to explore drawing strategies, as illustrated in Figure 1. This allows researchers to analyse how participants approach the task in VR and compare performance across conditions.

Applications to Experimental Design and Data Analysis

The ability to extract detailed 3D data from virtual environments opens up new avenues for experimental designs in psychology. By using VR, researchers can manipulate variables in ways not possible in traditional settings, such as creating controlled environments for studying perception or social interaction. Here’s how these types of data help in designing experiments:

Controlled Manipulation: VR allows precise control over variables such as spatial layout, lighting, and objects, providing a level of experimental control that is difficult to achieve in the physical world.

Reproducibility: Experiments conducted in VR can be exactly replicated, as the virtual environment can be programmed to be identical for every participant.

Rich Quantitative Data: The three-dimensional data extracted provides researchers with precise measures of behaviour, such as drawing strategies, movement patterns, or task completion rates. This is useful for statistical analysis, allowing researchers to detect patterns in how participants interact with VR.

Behavioural Insights: VR facilitates the collection of behavioural data in real time. For example, studies can measure how many times participants reference a figure or adjust their drawing, as shown in Chapter 5 of the thesis​

Linking to Psychological Constructs: By correlating spatial data with psychological measures like attention or memory, researchers can explore deeper insights into cognitive processes.

Weighing the Contribution of Quantitative Data

Quantitative data from VR studies is immensely valuable because it allows researchers to precisely measure behaviours and interactions in ways that would be impossible in real-world settings. For example, in the study mentioned in Chapter 5, participants’ spatial data during drawing tasks provided insights into local versus global processing styles.

While quantitative data is robust for statistical analysis, it often misses the subjective experience of participants. This is where qualitative data becomes essential, particularly in fields like neurodiversity research.

Because computer representations must be finite, models are defined in terms of primitives, each of which represents an infinite set of points. A 3D triangle is the most basic and useful primitive. The coordinates of the triangle vertices fully specify a planar surface patch that corresponds to all points 'inside' and on the boundary of the triangle.

((x1, y1, z1), (x2, y2, z2), (x3, y3, z3))

Numerous triangles can be arranged into a mesh to model a complex object in the virtual world. The doubly connected edge list is a popular and useful way to represent 3D data structures. Faces, edges, and vertices are the three types of data elements in this and similar data structures. Every 3D model has a unique geometry, and the ability to store this geometry is the most basic feature of any 3D file format.

The coordinates of the 2D image have attributes like colour and texture, and when rendering the 3D model, every surface point is assigned a coordinate. The vertices of the mesh are mapped first, and the other points are then assigned coordinates by interpolating between the coordinates of the vertices. Most 3D file formats support texture mapping, though the 2D image containing texture information is sometimes stored in a separate file, depending on the format.

A 3D file’s basic function is to store information about a 3D model in a format that a computer can understand, either plain text or binary data. Specifically, they can store details about four key features of a 3D model, though not all four features are always used. The four key features a 3D file can store include the model’s geometry, the model’s surface texture, scene details, and animation of the model. OpenBrush drawings are saved as .TILT files. The sketch is normally exported via the ‘save’ feature in OpenBrush. The folder contains all the information associated with the virtual sketch, including coordinate information, brushes chosen and any animations. As a result, most of this information is captured in a .FBX file format.

FBX is a proprietary file format that is widely used in the film industry and in video games. It was originally developed by Kaydara (established 1993) but was bought by Autodesk (established 1982) in 2006. Ever since the acquisition, Autodesk has used FBX as an interchange format for its own portfolio, which includes AutoCAD, Fusion 360, Maya, 3ds Max, and other software packages. The file extension for the format is .fbx. The FBX file format supports geometry and appearance-related properties like colour and textures. It also supports skeletal animations and morphs. FBX is one of the most popular choices for animation. In addition, it is also used as an exchange format that facilitates high fidelity exchange between 3ds Max, Maya, MotionBuilder, Mudbox and other proprietary software. In the example from my thesis where we are looking at three dimensional drawings, we were interested in the coordinates of the drawings and therefore the .FBX files had to be converted to .OBJ, which produced coordinates of the sketch for further analysis. Autodesk’s FBX Converter (Autodesk, 2006) was used for the file conversion. The OBJ file format is a neutral17 one commonly used in the field of 3D printing. It is also widely used in 3D graphics. It was first developed by Wavefront Technologies for its Advanced Visualization animation package. The 3D file format has the extension .obj. The OBJ file format supports both approximate and precise encoding of surface geometry. The OBJ format can encode colour and texture information as well. This information is stored in a separate file with the extension .mtl (Material Template Library). It does not support any kind of animation. The OBJ file format, by virtue of being neutral, is one of the most popular interchange formats for 3D graphics. It is also gaining traction in the 3D printing industry as the industry moves towards full-colour printing.

However, note that drawing in VR allows for selection of fully three-dimensional brushes or markers participants can draw with. These brushes can be more ribbon-like rather than just a straight 2D line drawing. Some have textures and animations associated with them as well. All of this data is captured in the data files discussed above. However, due to the complexity of analysing such dimensional data, coordinates were treated as a simple line in a three-dimensional space. Investigation of how additional features of OpenBrush tools can be interpreted and visualized should be one of the key foci for future research.

Adding Qualitative Data: Case Study

Qualitative data provides a contextual richness that quantitative data alone cannot offer. This method is particularly valuable in psychology, where subjective experience is often as important as measurable behavior. See Chapters 6 and 7 of my thesis for full overview of this case study.

Why Mixed Methods are Ideal for VR Research

Mixed-methods research, as discussed in Chapter 1 of the thesis, is often the best approach for exploring interdisciplinary topics like VR in psychology. By combining both quantitative and qualitative data, researchers can triangulate their findings, ensuring a more comprehensive understanding of the phenomena under study. This is especially relevant in emerging fields like VR, where traditional methods might overlook the nuanced ways participants engage with immersive environments​.

Virtual reality offers exciting new possibilities for psychology research, providing a controlled, immersive environment that can capture both precise quantitative data and rich qualitative insights. By using a mixed-methods approach, researchers can better understand complex phenomena, from perceptual processing to emotional and social behaviors.

Most of the data examples are from my thesis and this preprint.