Paul McLaughlin

When virtual reality first became mainstream, I was captivated by its potential to create new high-end experiences. At the same time, I was fascinated by industrial robots—however I had no experience with either developing VR or programming industrial robots.

As I delved deeper into both fields, I realized that the integration of VR and robotics could revolutionize industries such as manufacturing, entertainment, and education.

I couldn't help but wonder: What if we could combine the immersive power of VR with the dynamic motion of an industrial robot?

Accessing a KUKA robot was the first major challenge. These machines cost around $500,000 and are typically found only in theme parks like Disney or research institutions such as Deakin University.

I was able to gain access to the robot at Deakin's research lab which was used to simulate helicopters and tanks for training military personnel. However, my progress was slow due to limited time with their robot. The turning point came when I connected with the only privately owned KUKA robot in the world, just a 45-minute drive away. This unique opportunity finally allowed me unrestricted access, which enabled rapid testing and iteration—a critical factor in development.

The default programs that came with the robot were unenjoyable, focusing on extreme motions. I realized that to create an enjoyable ride, I needed to design custom motion paths.

This involved extensive experimentation to find movements that gave the "drop" sensation of a rollercoaster while remaining comfortable. By focusing on smooth accelerations and minimizing jerky motions, I eventually created a sequence of about ten movements that provided an exhilarating yet pleasant ride.

To accurately capture the robot's motion for VR integration, I used the first-generation HTC Vive setup. Its lighthouse technology uses fast-spinning lasers to track the headset and controllers 60 times per second, crucial for avoiding motion sickness in VR.

After overcoming challenges with signal interference, I finally achieved accurate motion capture during a midnight session, providing the precise data needed to synchronize the robot's physical movements with the virtual environment.

Importing the captured motion data into Unreal Engine was straightforward, but to create a thrilling experience, I added virtual acceleration that correlated with the robot's movements.

Whenever the robot accelerated beyond a certain threshold, the virtual environment would simulate increased speed. This approach created a seamless and immersive experience that felt both exciting and natural, without inducing motion sickness.

With the synchronized motion in place, I crafted an action-packed virtual experience complete with special effects, explosions, and near-miss encounters with virtual obstacles. By building a narrative around the motion path, I transformed it from a VR video and some robot motion to an exciting attraction.

To address immersion-breaking pauses caused by the robot's controller setup, I incorporated a SubPac into the seat—a wearable device that converts audio signals into low-frequency vibrations.

I created a separate audio file focused solely on low-frequency effects like explosions and engine noises. This small addition was a game-changer and elevated the experience to another level.

SEEit was the first complex project I had led, requiring the building and integration of various technologies. From overcoming access limitations to refining the minutiae that make an experience truly immersive, I learned invaluable lessons about product development, robotics, and perseverance.

While many technical details have been omitted, this project solidified my passion for pursuing challenging projects.