3D Visualization

In this project, I implement an interactive MATLAB application that transforms single photographs into explorable 3D scenes, based on the "Tour Into the Picture" (TIP) technique by Youichi Horry, Ken-ichi Anjyo, and Kiyoshi Arai from Hitachi, Ltd. Their innovative approach uses a "spidery mesh" interface to model scene geometry from a single image. Through user-defined vanishing points and perspective planes, the program computes homography transformations to extract and rectify textured surfaces, creating a virtual 3D environment that allows for dynamic camera movement and scene exploration.

Implementation Details

I developed a method that turns regular 2D photos into 3D scenes you can navigate through. Users first interact with the image by placing a vanishing point and drawing out perspective planes using a web-like interface, similar to what was described in the TIP paper. For each plane, the program computes homography matrices using the Direct Linear Transform (DLT) algorithm, mapping the user-defined quadrilaterals to rectangles in 3D space.

The rectification process involves:

• Computing the homography matrix H that maps source points to destination points using SVD-based DLT
• Applying perspective warping to extract and rectify textures from the original image
• Creating proper UV coordinates for texture mapping onto 3D geometry
• Handling foreground objects through separate billboard-style projections

For navigation, I implemented standard 3D camera controls that let users move around the scene. Everything is set up in a right-handed coordinate system, allowing for smooth camera movement and rotation while maintaining proper perspective. The rendering pipeline uses MATLAB's built-in graphics tools for real-time visualization of the textured 3D geometry.

Examples

Jerome of Stridon

Palace Skateboards, New York

Supreme, Chicago