Reassessing 2D photo to 3D model processing, using free tools

Previously, I wrote a blog post on using ReCap Photo to create a 3D model from a set of 2D pictures. Some of my friends that are interested in the topic let me know they would rather perform the process using open source (free) tools.

I researched several options and came upon:
Meshroom
: a free and open source photogrammetry software from AliceVision

The tool appears to be very tunable with many knobs to turn at every level of the process. It also has a solid user interface that is relatively easy to use.

Unfortunately, detailed documentation about what all those knobs can do is a bit lacking, though there is a post with instructions on running through the process of using Meshroom, using a photoset available here. There is also a wiki about Meshroom available. There is also a good YouTube overview of Meshroom and a discussion of the process.

When you install Meshroom, place the installation folder on the top level of your hard drive. It seems to be overly sensitive about non-alphanumeric letters in the file structure (including spaces). Meshroom will only run on systems with a CUDA (nVidia) GPU. The rest of this post is a walkthrough of the process.

  1. Once Meshroom is installed, you can drag your photos to the Images window on the left hand side of the screen.
Images window

Note that each of my images has a little red circle beside it. This indicates that Meshroom cannot determine the type of camera used to take the picture. It gets the camera information from the metadata inside the picture. If you really want to get serious about this, you can define the focal length… for the pictures and the tool will do a better job. The better the camera the better the results. I just used my cell phone.

  • Once you have loaded up your pictures and are ready to begin, pull down the File menu and define the location you would like to have the files stored. I recommend you do not try to place them in a cloud-based folder, since that will really slow down the process. Also try to make the file path contain only alphanumeric letters.
  • Once the folder location is defined, click the green Start button at the top of the screen.
image
Start button

There is a progress bar that runs along the top of the screen to show you how the process is progressing.

Progress bar

The yellow section indicates how far along the process has proceeded. If the program encounters an error, a red section will be displayed. You’ll need to dig into the logs to see what happens at that point –more on that later.

You can also see a progress indicator at the actual stage being executed:

DepthMap node

In this case, we are about 1/5th of the way through the DepthMap step. This stage will take a very long time to complete, so expect the process to thrash your computer for possibly hours. By the way, don’t let your computer fall asleep until the process completes.

All the steps in the process are displayed in the Graph Editor window of the screen. These various processes pipe their output to each other.

Process steps display

For each of these nodes, you can click on them to see the log of what was being worked on, if a problem occurs. You also have a real-time log window that tracks the entire process.

As the process continues, the program will evaluate all the photos and acceptable/usable photos will be marked with a green checkmark in the Images window.

Images window

If any images are determined to be flawed, remove them, and start the process again.

As the process continues, you can see work in progress in the 3D viewer window:

3D viewer window

In this window, you can pan, zoom, and orbit the work in progress model using the mouse. More detail in the point cloud will appear as the process runs. The white boxes on the screen represent where the program thinks the cameras were located.

Once the process completes successfully, you will see a button titled Load Model.

Load Model button

Use this button to preview the results.

To work on the model that was generated, a folder is created where you told the program to store the model (way back at step 2). The folder called MeshroomCache.

MeshroomCache folder for in-process files

Within that folder will be a hierarchy of files, containing work-in-progress data that was transferred between the various stages of the recognition process:

Work in process file heirarchy

Though the various intermediate files may be useful, you are likely most interested in is the .obj, .mtl and texture file in the Texturing folder.  To access those files, you will need to go one level deeper in the Texturing folder, past a randomly named directory.

Texturing folder contents

The .obj The OBJ file format is a simple data-format that represents 3D geometry alone — namely, the position of each vertex, the UV position of each texture coordinate vertex, vertex normals, and the faces that make each polygon defined as a list of vertices, and texture vertices. This format can be imported into many (most) 3D modeling software.  This mesh will be high a high definition model. You can use a tool like Instant Meshes to view the results and simplify the model (here is a video explaining Instant Meshes use).

The results of my efforts were not quite as precise as the results from Recap Photo, but the process was still effective. I am sure if I experimented more, I could improve the results. Here is what the generated model looks like in Meshlab:

Load of initial model into Meshlab

There are over 1.2 million faces and 621,000 vertices. After running it through Instant Meshes, the model looked like:

Post simplification processing model in Meshlab

It now has 249,000 faces and 95,000 vertices. We lost the color (which we didn’t need anyway) and a much more simplified result was generated. We can now crop out the areas we don’t want and end up with a mesh we can work with.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s