3-D Mapping Your World with a Backpack

Article written by Lisa M. Krieger

Digital 3-D renderings of Earth are the backdrop for everything from computer games like "Grand Theft Auto" and Hollywood films like "Enemy of the State" to mortal combat simulation in Iraq.

This new technology permits astonishingly precise measurements of buildings, roads, waterways, coastlines and even vegetation, right down to individual plants. Its attention to detail is so accurate that thousands of tiny windows on skyscrapers seem to show "reflections" and the bricks holding the Statute of Liberty show detailed textures.

Avideh Zakhor, a UC Berkeley professor of electrical engineering, is the brains behind these realistic visualizations.

"I call it 'reality capture,' " she said. "You're trying to capture what's out there in the real world...and create a representation of that so others can feel they're in that same space."

To test her technology, Zakhor had a novel idea. "Why not put all these sensors on top of a truck, drive around, acquire all the signals, images and videos and laser scans and make 3-D models of the real world of everything that is outside?" she wondered. So her team's sensor-equipped car drove the streets of Berkeley, creating a 3-D model of the city. Her technology was licensed by Google in 2007 to help produce its 3-D Google Earth.

Now anyone can navigate through lifelike "virtual" settings - prowling Parisian parks, exploring ruins in Cambodian jungles or fighting zombies in alien landscapes.

But there's a big problem in this approach to imaging: it only works outdoors.

It uses GPS, which relies on a series of satellites to identify where it's physically located - and satellite signals are carried through waves at a frequency that doesn't pass through solid objects like walls. That's why, when you use a GPS inside a building, the device can't pinpoint your location accurately.

There are other ways to image interiors, like using laser devices that measure distances across walls and then assembles them. But those take time - 30 to 40 minutes for a small room, said Zakhor. "A whole building could take a week to scan."

Zakhor had a different idea. She created a portable, laser backpack for 3-D mapping that creates fast, automatic and realistic inside images. It collects thousands of data points, then stitches them together with the system's custom-built software, using fancy mathematical algorithms, into a 3-D model.

Inside the 30-pound backpack are high-tech laser cameras and scanners, which collect all of the information.

"It's a way of documenting a building in a very fast way," she said.

She was stymied by how to identify the position of the backpack indoors - an essential part of image localization. They ended up relying on a technique used in robotics, called SLAM (for "simultaneous localization and mapping".) A combination of good sensors and good algorithms, it can find the backpack, construct a map and then build the 3-D models.

"Localizing the backpack - that took awhile," she said. "We had to try many different things. You tried one thing doesn't work, what if you try that other thing."

They also had to tweak the backpack so it wasn’t so big and bulky, removing some components.

Now she dreams of many different ways her stripped-down 3-D backpack could be used. It could help during construction, for instance, if a plumber sees that an electrician's wires are in the way of his pipes. It could be used in commercial real estate sales - agents could explore a property without actually visiting it. If the backpack has an energy sensor, it could measure everything emitted by lights. It could create drawings of ancient buildings that no longer have blueprints.

In fact, companies like Google are putting simplified, consumer-grade versions of the backpack's 3-D mapping algorithm into phones, allowing users to create 3-D maps of rooms and other interior environments. It doesn't have the customizability or the accuracy of the backpack, but it's still an impressive feat to have that 3-D mapping power in your mobile phone.

Zakhor is particularly excited by the backpack's potential to save lives during fires or earthquakes.

"First responders can figure out, when they get to a site, their plan of attack in terms of rescuing people," she said. "It allows you to do much more methodical planning of how you send your people inside a collapsed building to rescue people...which staircases are connected to floors, where all the sprinklers are, or where the emergency exits are."

Someday it could test the health of our interior spaces, if loaded with air-quality sensors that measure things like atmospheric pressure, carbon dioxide, air temperature and particulate levels.

"The beauty of the backpack," she said, "is that, as I'm walking through, I collect as many signals as I possibly can."