hardware.knit

💡DID YA KNOW?

To ensure safe experimentation settings for human-subjects, driving studies often use a combination of hardware and software tools to develop simulated road settings for controlled experimentation. Simulated environments serve as experimental testbeds that allow researchers to systematically manipulate traffic conditions, road infrastructure, and interaction scenarios while ensuring participant safety during potentially hazardous driving situations. This is particularly crucial when studying mixed-traffic interactions in shared urban spaces, where multiple types of road users—motorized and nonmotorized- navigate complex social and physical environments together.

Most my experimentation is done in a virtual environment. I use Strangeland - a Unity-based multiplayer driving simulator where participants can be drivers, pedestrians, or cyclists, all interacting in the same virtual space. These tools allow us to simultaneously capture and cast behavior from multiple road users to eachother in commmon virtual worlds. This real-time synchronization between co-located human-subjects contributes to current novel representations of mixed-traffic driving environments under controlled experimentation settings.

Simulation

The Strangeland environment is the backend of our multiplayer studies, which was developed in the Unity game engine with a .NET codebase for multi-platform deployments. This virtual environment facilitates real-time multi-agent interaction in shared urban spaces, enabling natural interactions between drivers, pedestrians, and other road users. Instantiations of customizable urban environments allow for the re-creation of culturally-specific urban scenarios, enabling comparative studies across different cultural contexts.

Strangeland simulator — Strangeland, a Unity-based VR driving simulator.

︎💡 Fun fact: Motion sickness occurs when your body feels movement but your eyes don't match it. Cybersickness is the opposite - your eyes see movement on screens (like in VR) but your body isn't actually moving. Both can cause nausea and dizziness, but they're triggered differently.^[1]

Vehicle simulators

The NADS miniSim Quarter Cab vehicle simualtor is the largest driving simulator I work with. The system incorporates OEM vehicle components and a tri-screen display configuration, delivering a 180-degree field of view that can ideally mimic real-world driving conditions. The platform's advanced physics engine accurately models various environmental conditions, while integrated eye-tracking and behavioral monitoring systems capture rich data about driver behavior and attention patterns. In addition to the miniSim we also use a Logitech G29 force-feedback steering pedal and wheel set, which provides a flexible configuration for 2-driver studies.

Free-motion simulators

The integration of Meta Quest 2 and Meta Quest Pro systems extends our capability to provide more immersive experimentation solutions. For example, becoming a pedestrian requires the construct of a road space and traffic environment. These advanced HMDs feature high-resolution immersion that allows us to design environments to target certain rare traffic scenarios. Integrated eye-tracking and facial expression monitoring systems provide detailed data about participant attention and ~emotional states during interactions.

Quarter cab specifications — Quarter cab technical specifications.

Viewport — Simulator viewport configuration.

Motion capture

Motion capture helps to precisely measure, model, and reproduce operational human movement. In my research, motion capture streams are useful for capturing behavior of pedestrians and other vulnerable road users with incredibly granularity.

OptiTrack (outside-in tracking)

Our OptiTrack setup is like having 24 super-precise cameras all working together. Operating at up to 120hz and integrates directly with Strangeland! This system also tracks multiple people at once - perfect for studying group dynamics such as herd behavior and in- and out-group biases. Addtitionally, optiTrack supports a .NET base, allowing participants to interact on-line.

Sony Mocopi (inside-out tracking)

The Sony Mocopi is our newest toy, and it lets us catpure movement on the go! As a portable motion capture solution, six wireless IMU-based sensors enable natural movement studies in both lab and field settings. This dual-system approach provides flexibility in studying pedestrian behavior across various environmental contexts while maintaining high data fidelity.

OptiTrack camera system — OptiTrack motion capture system

Eye tracking

Where someone looks is the closest thing we have to a direct readout of what they are attending to — and for a pedestrian deciding whether to step off a curb, that readout is the whole story. Head direction is a decent proxy indoors; on a street it is not.

Meta Project Aria (Gen 1)

The Meta Project Aria glasses carry the sensor suite for all of my outdoor pedestrian work. They look like ordinary glasses, which matters: participants walk their own neighborhoods without being read as an experiment. Running the all-sensor profile, a single pair captures eye tracking, SLAM and mono cameras, an RGB scene camera, microphone, IMU, GPS, magnetometer, and barometer — all time-aligned, at 10 fps.

That combination is the reason we use them. The SLAM stream reconstructs the walker's path and the geometry around it, so gaze can be scored against the visible field the person actually had — per-step isovists built from the real world rather than from a model of it. The microphone picks up think-aloud narration in the same clock, so a spoken moment of hesitation lands on the same timeline as the fixation that preceded it.

In the lab

For fixed-base work the miniSim's integrated eye tracking and the Quest Pro's built-in gaze and face tracking cover the same questions under controlled conditions, where the scene geometry is already known because we built it.

Gaze tracked from the Aria glasses during an outdoor walk.

︎💡 Fun fact: the Aria profiles with faster eye tracking drop either SLAM or GPS. Since the whole analysis depends on reconstructing where the walker was and what they could see, 10 fps with everything on beats a higher gaze rate with a hole in it.

A note on networking

A humanFUELed hot-pot: Here's where it all comes together! Our lab network is a carefully crafted stew of different technologies (hence hot-pot). Here's what's simmering in our digital kitchen:

Real-time data synchronization (and logging!) between all devices and participants.
Exact positioning and operation movement of participants throughout experimentation.
HMDs for pedestrians, cyclists, and/or desktop vehicle simulator participant.
NADS MiniSim for driving participants.