Software Models Physical Structures Automatically

A technology called ProForma (Probabilistic Feature-based On-line Rapid Model Acquisition) from researchers at Cambridge Univ. in England lets users simply rotate a physical item in front of a video camera to turn the item into a 3D virtual object that can be used in graphic applications. ProForma uses augmented reality (AR) — techniques for tracking and registering computer graphics to physical objects — to guide users in how to rotate the object in front of the camera. AR has been limited in the past because reconstruction techniques, such as laser scanning, are too slow.

In contrast, ProForma is said to let users build complete 3D virtual objects in about a minute. The software passes live video to a module that tracks the object’s position at the frame rate. When the software detects rotation, the tracker sends camera-orientation data with associated tracking information to another module that updates the digital object and its current pose. The software continually augments the video with transparent overlays containing a large 3D arrow that points users towards the best direction to rotate the object for a more-complete digital representation.

Keyframes contain 2D feature tracks: For each input video sequence, ProForma extracts object feature points from each pair of adjacent frames and then a matching algorithm evaluates common features between the two sets of points. The software tracks all the information and uses it to calculate 3D landmarks. A technique called bundle adjustment obtains an accurate point cloud based on camera positions and 3D landmarks. This generates a sparse model that needs further processing to become a surface model. The program does this by performing a Delaunay tetrahedralization on the point cloud to get a shape portioned into tetrahedra. The program “carves” away tetrahedra probabilistically to obtain the shaped surface mesh, and then maps a texture onto it to form the 3D virtual object.

ProForma also uses a discretized form of the object represented as an icosahedron (a shape with 12 vertices, 20 faces, and 30 sides) placed at the center of its mass. The program assigns a “score” to each face representing the uncertainty of what the object looks like from the orientation corresponding to the face normal. Higher scores represent orientations the physical object needs to undergo (rotations the user must perform) to get a refined digital representation. When all faces of the icosahedron have a score below a threshold value, the model-building process is complete and keyframe acquisition stops.