Reassessing 2D photo to 3D model processing, using free tools

Previously, I wrote a blog post on using ReCap Photo to create a 3D model from a set of 2D pictures. Some of my friends that are interested in the topic let me know they would rather perform the process using open source (free) tools.

I researched several options and came upon:
Meshroom
: a free and open source photogrammetry software from AliceVision

The tool appears to be very tunable with many knobs to turn at every level of the process. It also has a solid user interface that is relatively easy to use.

Unfortunately, detailed documentation about what all those knobs can do is a bit lacking, though there is a post with instructions on running through the process of using Meshroom, using a photoset available here. There is also a wiki about Meshroom available. There is also a good YouTube overview of Meshroom and a discussion of the process.

When you install Meshroom, place the installation folder on the top level of your hard drive. It seems to be overly sensitive about non-alphanumeric letters in the file structure (including spaces). Meshroom will only run on systems with a CUDA (nVidia) GPU. The rest of this post is a walkthrough of the process.

  1. Once Meshroom is installed, you can drag your photos to the Images window on the left hand side of the screen.
Images window

Note that each of my images has a little red circle beside it. This indicates that Meshroom cannot determine the type of camera used to take the picture. It gets the camera information from the metadata inside the picture. If you really want to get serious about this, you can define the focal length… for the pictures and the tool will do a better job. The better the camera the better the results. I just used my cell phone.

  • Once you have loaded up your pictures and are ready to begin, pull down the File menu and define the location you would like to have the files stored. I recommend you do not try to place them in a cloud-based folder, since that will really slow down the process. Also try to make the file path contain only alphanumeric letters.
  • Once the folder location is defined, click the green Start button at the top of the screen.
image
Start button

There is a progress bar that runs along the top of the screen to show you how the process is progressing.

Progress bar

The yellow section indicates how far along the process has proceeded. If the program encounters an error, a red section will be displayed. You’ll need to dig into the logs to see what happens at that point –more on that later.

You can also see a progress indicator at the actual stage being executed:

DepthMap node

In this case, we are about 1/5th of the way through the DepthMap step. This stage will take a very long time to complete, so expect the process to thrash your computer for possibly hours. By the way, don’t let your computer fall asleep until the process completes.

All the steps in the process are displayed in the Graph Editor window of the screen. These various processes pipe their output to each other.

Process steps display

For each of these nodes, you can click on them to see the log of what was being worked on, if a problem occurs. You also have a real-time log window that tracks the entire process.

As the process continues, the program will evaluate all the photos and acceptable/usable photos will be marked with a green checkmark in the Images window.

Images window

If any images are determined to be flawed, remove them, and start the process again.

As the process continues, you can see work in progress in the 3D viewer window:

3D viewer window

In this window, you can pan, zoom, and orbit the work in progress model using the mouse. More detail in the point cloud will appear as the process runs. The white boxes on the screen represent where the program thinks the cameras were located.

Once the process completes successfully, you will see a button titled Load Model.

Load Model button

Use this button to preview the results.

To work on the model that was generated, a folder is created where you told the program to store the model (way back at step 2). The folder called MeshroomCache.

MeshroomCache folder for in-process files

Within that folder will be a hierarchy of files, containing work-in-progress data that was transferred between the various stages of the recognition process:

Work in process file heirarchy

Though the various intermediate files may be useful, you are likely most interested in is the .obj, .mtl and texture file in the Texturing folder.  To access those files, you will need to go one level deeper in the Texturing folder, past a randomly named directory.

Texturing folder contents

The .obj The OBJ file format is a simple data-format that represents 3D geometry alone — namely, the position of each vertex, the UV position of each texture coordinate vertex, vertex normals, and the faces that make each polygon defined as a list of vertices, and texture vertices. This format can be imported into many (most) 3D modeling software.  This mesh will be high a high definition model. You can use a tool like Instant Meshes to view the results and simplify the model (here is a video explaining Instant Meshes use).

The results of my efforts were not quite as precise as the results from Recap Photo, but the process was still effective. I am sure if I experimented more, I could improve the results. Here is what the generated model looks like in Meshlab:

Load of initial model into Meshlab

There are over 1.2 million faces and 621,000 vertices. After running it through Instant Meshes, the model looked like:

Post simplification processing model in Meshlab

It now has 249,000 faces and 95,000 vertices. We lost the color (which we didn’t need anyway) and a much more simplified result was generated. We can now crop out the areas we don’t want and end up with a mesh we can work with.

3D model creation using my cell phone’s camera

Recently, I was in a discussion with some folks in our Technology Enthusiasts group about 3D model creation. I dabbled with this several years back, purchasing a Microsoft PC interface for the Xbox One camera. It worked, but only on fairly large objects (human size).

I wanted to be able to create models of smaller things, so I investigated Photogrammetry. This is the technique of taking multiple overlapping photographs and deriving measurements from them to create 3D models. The basic principle is like the way many digital cameras allow you to create a panoramic photograph by stitching multiple shots together.

I have been creating 3D printed buildings in HO gage for the model train group here, so I thought I would work on a model that size. My wife has a miniature Christmas village that has a cinema building, so I thought I would try and scan that in. It took 3 tries but I think I ended up with something viable. To get started you need to understand the process:

  1. Place the model on a flat surface you can move around. I tried placing it in an isolated environment with a bland background (e.g., surrounded by cardboard) but that did not work as well as just taking many photographs as I moved around the box. In my case, I used a table on my porch, since it had good lighting and I could move around smoothly.
  2. Take a series of pictures around the object and from various angles. In this case I took a straight on shot and one slightly elevated shot and moved around the object about 10 degrees each time. I did not care about the bottom of the model; I knew it was flat. My goal was to get 100 good shots (well-lit and not blurry) for the software to work with.
  3. Once all the shots were done, I loaded them into Autodesk Recap Photo. There are many tools out there but since I use Fusion 360 and there is a hobbyist license, I settled on this one:
    • Start by loading the software and creating a new project.
Creating a project
  • Load the images into the software, using the button on the left.
Loading pictures into the project
  • Once all the pictures are loaded, and you have made sure you have only the good ones you want selected, click on the Create button at the bottom of the screen to start the process of creating a 3D model.
Creating a model
  • This is a cloud-based process that runs on Autodesk managed servers. All the images need to be uploaded and take can take a while (3-5 minutes). Progress is shown in the icon at the bottom of the screen.
Uploading to the server
  • Once the upload is complete, the job will be placed in a queue with everyone else wanting to create a 3D model and the waiting really begins. The 3 models I loaded took at least 4 hours to complete. I was not there watching to see the exact time they completed.
Waiting for the model to be created
  • Once the job is complete, a small arrow will appear, allowing you to download the model to your PC. After you click download, a dialog will appear asking you where to store the model. A new folder will be created in that location with the model.
Downloading the model
  • Once you have downloaded the model, a new model icon will appear in ReCap Photo in the My Computer section of the screen.
Opening a new model
  • When you open the model, you should see a 3D rendering of the object. Note, there will likely be some anomalies. You can use the tools on the left of the screen to address many of these. This model can be rotated and manipulated at this point.
The 3D model
  1.  You can export the model in numerous formats:
Exporting to a model other software can use
  • In my case, I saved the model as a .STL file and loaded it into 3D builder to scale and clean up further. You will likely need to use a mesh editor to simplify the file before Fusion 360 can turn it into a solid.
The model loaded into Microsoft 3D builder

Hopefully, you will find this tutorial useful. There is also a great video on how to manually create 3D models from photographs out on YouTube using Fusion 360.

Creating a temperature tower

When you really want to print some detail in a 3D printed part using plastic filament, you need to understand what temperature the material prints best. Granted, PLA typically likes 190 degrees C and ABS likes about 230 degrees C, but each spool can vary on its optimal print temperature.

That is where a temperature tower comes in. This model has multiple strata of identical elements. Each one is printed at a different temperature. There are several designs out in various repositories, but the one I created looks like:

Temperature tower design

This is shown with an X-ray view so you can see some of the internal components.

This design was based on one I found but I added two horizonal elements on each ‘layer’. One that in 0.5 mm and one that is 1.0mm thick. There is also the temperature value embossed in the layer that it will be printed at – to do this, you need to go in and modify the G-code. Each layer is 5 mm tall. This model demonstrates printing ranging from 220 – 185 (for PLA).

Below are some that I actually printed. One from some white PLA and another in black. You cannot really see the quality differences in the picture, but the white likes to be printed at a slightly higher temperature.

I can tell by the detail loss of the small cones designed into the ends. When it is too hot, they sort of sag. When it is too cold, the layers become more distinct.

I have another temperature tower design for ABS and PETG that covers the range from 260-230.

Once I figure out the right temperature for the material, I write it on the spool.

Now that the printer is working… it is time to give it a real test

I thought I would create a model that really tests it out the capabilities. I designed a country store for this purpose (and placed it out on Thingiverse). The model is HO gage and I will paint it up for the local model train club.

Front of the country store

I figure if the printer can reliably print that fire escape, its working well.

Back of the country store

I printed it, but the fire escape was not as clean as I wished.

The printed result

There were a number of “strings” between horizontal elements that really cause problems. That is probably because I did not have the printer at the optional temperature and retraction enabled.

In order to address this issue, I will need to print a temperature tower, to identify the optimal temperature for this material. That is a whole other post though.

My 3D printer wasn’t done with me yet… replacing the BLTouch probe pin

I mentioned in yesterday’s post that I was having some problems with my 3D printer and needed to replace the hot end. Well today when I was calibrating the Z axis, I figured out that in the catastrophic ‘meltdown’ that happened, the printer managed to snap off the end of the BLTouch, Z axis zero sensor probe. Below is what the probe is supposed to look like. Note the knob on the end of the probe.

The probe extended

When I received my BLTouch, it came with a spare probe. At least I had something to compare the probe against. Below is what the broken one looks like. There was a good 4 mm missing.

The broken sensor pin

There are videos out there that talk about replacing the probe. It was quick and easy to do. If you need to buy a probe tip, it looks like they are about $15 apiece.

This is an interesting technology. You can see a metal on one end of the probe. I assume this is an inductor that when moved causes the device to recognize when it has touched the printing surface.

Now, I have gone through the Z axis calibration and everything seems to be working again.

Replacing the hot end on a Creality Edge 3

Since I have been doing quite a bit of 3D printing, all that load has been placing quite a bit of wear and tear on my printer. One of the common things that can happen is that the hot end of the printer comes loose from the tube that feeds the filament (this video goes into more detail about the issue and hot end repair than anyone likely would ever need). Having the hot end fail can be catastrophic — I had molten material spew out all over the hot end and the surrounding structure. By the time I got around to looking at it, it had solidified into a plastic solid.

I had another hot end lying around so I decided I’d just replace it. There is a video from Creality that goes into a great deal of detail about replacing the hot end and most of it is subtitled in English. One thing about the video that made me a bit worried was the number of wires I would have to detach from the control board, to connect the new head. There are quite a few of them even though the head and the temperature sensor only have four wires. This is because the wiring harness should be removed and taken apart. You could probably get away without doing disassembling and reassembling the harness, but then your printer would start to resemble a spaghetti of wires. If you watch the video it is actually fairly straightforward, though I wish there was a bit more detail provided.

I thought I would provide a picture and explanation, hopefully this will help:

On my machine, the connector wires were hot glued to the PC board connectors, so the glue had to be cut free before I could remove the original connections. I suppose this gluing was to prevent them from coming loose due to vibration.

It is all back together now. I just need to perform the wonderful task of calibrating the print height. I should also calibrate the flow of material through the extruder – in order to get the best result.

Radio Direction Finding as Entertainment (Part 2)

Our ham radio group decided to have a real radio foxhunt last week and this time I was successful using the techniques described in the presentation I put together previously. I used the technique of using different length antennas this time and driving around looking for stronger signal strength (since I was close enough, I did not need a directional antenna).

From the time I left my house until the time I found the transmitter was about 30 minutes.

I 3D printed an SMA socket to hold the paper clip in place as the antenna when I got close. Once, I had it printed, I just warmed up the paper clip and pushed it through the PLA plastic.

3D printed SMA connection

When I was using that small antenna, I got within a couple hundred meters. Then had to resort to looking for the 3rd harmonic to get close enough. I could not pick up the 3rd harmonic until I was within 100 meters of the transmitter. Finally, I removed the antenna altogether when I was within 10 meters and listened for the 3rd harmonic. It was well hidden, so I needed that level of attenuation.

It was a fun exercise and did not have to get within 6 feet of anyone else in the whole process. Though I did get within about 15 feet of an alligator at one point.

3D printing replacement parts for a Sunbeam toaster

During this time of isolation there is time to tackle some of the annoying things that have never been annoying enough to correct. This is a brief story about addressing one of those.

I have a great Model 20-3 AG Sunbeam automatic toaster. This was a wedding present my wife and I received long ago. They don’t manufacture this device anymore, unfortunately. This toaster is a descendant of a long line of Sunbeam toasters that are probably the best toasters in the world.

It has always had a slight problem in that it did not go down and start reliably. This is easily corrected by adjusting the tension on the toaster.

The annoying problem we’ve had is that the control knob on the front of the toaster cracked, fell apart and was lost long ago. Now that I have a 3D printer though, that issues should easily be addressed. The design it may not be the same as the original, but I was hoping for something close enough to pass inspection by my wife.

First, I had to find some pictures of the toaster showing the control, from th

First, I had to find some pictures of the toaster showing the control, from the Internet.

Then I had to measure the post that fits into the control. It turns out it was about 5 mm wide, 1.5mm mm tall and 8 mm deep.

Next, I created geometry in Microsoft 3D builder to mimic my impression of the ellipses removed from a rectangle to create the shape, as well as the small rectangle used to represent the post that also needs to be removed. The black objects are the removed geometry.

I then subtracted those design elements and ended up with:

Finally, I printed out the resulting model in white PLA and tried it out. For the final version I used a 0.12 mm layer height and 20% infill. The final result came out looking like:

I could have made the knob a bit narrower, but my wife liked the feel of the wider knob. I uploaded the knob on Thingiverse.

An Intro presentation to CNC

I recently gave an Introduction to CNC presentation to the Woodshop here in Sun City. So I thought I’d share it here.

The presentation and demonstration was fairly well received, though I did make a few tweeks, this presentation is the revised version. I am going to give it again next month, so we’ll see if it needs to be improved again.

I don’t claim to be an expert. I am just sharing what I’ve learned along the way and try and encourage others to share their knowledge and experiences.

I also continue to give an Introduction to 3D printing presentation that I pulled together a while back.