Designing large AR requires pinning virtual objects with extreme accuracy

Designing large AR requires pinning virtual objects with extreme accuracy. Why? Because being even 1 degree off will cause MASSIVE problems and break player immersion.

@meshmapxyz solution = professional scans + clear localization spot + QR code marker + easy calibration toolCombining these, we have created and operated AR game tours that span several acres (the equivalent of 3 football fields)!

First, let’s take a step back.This is the map of our Halloween event. The path was almost a half-mile long (700m) and covered over 4.6 acres (18,700 sq m) of Washington Sq Park.

I placed virtual decorations, enemy monsters, and minigames all over the park. Here are some of their locations:

At the time, we weren’t yet using QR codes and manual calibration. Just extremely accurate 3D scans and crafty game design to position the start point directly in front of the arch.

Assuming every player’s device localized in the perfect POSITION to start, there was still the possibility that ROTATIONAL error would mess things up. If the device was turned 1 degree to the left at launch, then the position of every object would be skewed to the left.

The furthest object I placed was 150 meters away from the tour’s start location. At 1 degree of accuracy, that object would be off by 2.62 meters. At 5 degrees, this error increases to 13.1 meters.

The formula for calculating the distance between two points on the perimeter of a circle based on the interior angle is L = 2rsin(degree/2)

Given this potential for error, we were impressed that in most cases the content was all within 1 to 2 meters of accuracy during our live tours.

We wanted a more reliable solution for our Christmas event

I began by adding QR code localization at the starting point. This ensured we would always have the correct position.I also froze the global x and z rotations of our objects. This reduced the potential of rotational error along those axes.

But rotational error will persist for several reasons:incorrect QR code placement, the device camera’s depth perception in different lighting conditions, user error…

Although this part of south Central Park was a smaller total area than Wash Sq Park (*just* 7,300 sq m, or 1.81 acres), it had a 43% longer max distance: 215 meters.

This meant that at 1 degree of accuracy, the furthest object would be off by 3.75 meters. At 5 degrees, this error would increase to 18.75 meters. So if our QR code was turned in slightly the wrong direction when placed on the ground, then the candy canes, Christmas trees , and snowmen would be a swimming pool’s length away.

Adding more QR code spots also won’t solve the root problem because they are all equally susceptible to error. More QR codes may have you think that the localization is better, but it’s only decreasing the distance to the furthest object in exchange for potentially re-introducing rotational error. QR codes are also limited by their requirement of being at pre-planned locations.

Creating a manual calibration tool was the unlock.

Players can now easily tweak the placement of content as they explore anywhere on the map.And we can design experiences that *theoretically* could cover miles.

In our experience, VPS is not good enough yet and I’m unsure how long it will take to reach good enough. It may work well enough to place a couple objects in easily identifiable areas. But VPS is still extremely inaccurate when your play area is as large as an aircraft carrier and has tons of temporary variables, like foliage, people, furniture…(this means that spatial anchor systems that rely on VPS also miss the mark.)

So far, our method of combining professional scans with manual calibration has produced great results and really fun player experiences.

Check out the full video (7 minutes) if you want to see our simple AR solutions in action!