Photogrammetry 101 with Harman

Goal

Using open source software and accessible hardware like a cell phone, successfully generate a 3D model from images taken of an object, a process known as photogrammetry.

Create an Image Set

When taking photos of an object for use in photogrammetry, it is best to take the photos in diffuse lighting (i.e. outside on a cloudy day). Move around the object, taking photos at every 10-15 degrees such that approximately 36 photos are taken. Every detail of the object should be captured in 2-3 of the photos, so there should be significant overlap between the photos. Photos of the top or bottom of the object (as applicable) should also be captured. Save these images as .JPGs.

Alternatively, a high-framerate video can be taken with a cell phone and then use KDENLive to extract frames. With this process, a selection of 50-100 photos can be easily attained within a few minutes.

Generate a Sparse Point Cloud

COLMAP is open source software that can be used to generate a sparse point cloud without any specialized hardware (see note at end of section). Install COLMAP: https://colmap.github.io/install.html

Run COLMAP then go to Reconstruction > Automatic Reconstruction. Choose a Workspace folder, to which the output files will be saved. Choose the Image folder that contains your Image Set. Set Quality to "Medium" to avoid crashing due to memory limits. REMOVE the checkboxes in Dense Model and GPU. Click Run.

After some minutes, a sparse point cloud will appear in the main window. Go to File > Export Model as. Save the model as a .nvm file for use in the next step.

NOTE: if running COLMAP on a CUDA-enabled GPU, a dense point cloud can also be generated from the same image set. Make sure COLMAP with CUDA is installed and leave the Dense Model and GPU options checked during Automatic Reconstruction.

Generate a Dense Point Cloud and Final Mesh

OpenMVS generates a dense point cloud and final mesh from a sparse point cloud (which was saved as a .nvm file in the previous step).

Build the OpenMVS executable binaries https://github.com/cdcseacave/openMVS/wiki/Building or download them https://github.com/cdcseacave/openMVS_sample

Then the steps listed here can be used to generate a dense point cloud and a final mesh: https://groups.google.com/forum/#!topic/vsfm/VcXGvP1HuMo (in this post VisualSFM is used to create the sparse point cloud but the steps can be followed all the same with the .nvm file generated by COLMAP).

Cordless Drill Images

https://drive.google.com/file/d/1lGYj5yPYpYdZUFDlQ8HiWUEHe1q5vL9n/view?usp=sharing

FB post

https://www.facebook.com/groups/398759490316633/permalink/899635686895675/

Presentation

edit

Current Progress

41 images of a Black and Decker Cordless Drill have been taken for use with COLMAP

A 'Sparse' point-cloud reconstruction was achieved using only a CPU since the GPUs wouldn't work

However, only a 'Dense' point-cloud can be exported as a mesh for use in MeshLab. This requires a CUDA enabled GPU, which I have and can install

An alternative is to attempt to construct a mesh from the 'Sparse' point cloud we have generated using another open source program called OpenMVS.

Next Steps

Continue by installing CUDA API for my Nvidia GPU, and/or attempt mesh reconstruction from sparse point-cloud with OpenMVS pipeline

Past Work

Was unable to get COLMAP to carry out a reconstruction using the GPU on the following machines:

PC: Sony Vaio Laptop

OS: Ubuntu 18.04

CPU: Intel i5-2540M

GPU: Radeon 6630M

Notes: GUI crashes to terminal with SIGABRT error when attempting reconstruction

PC: Custom AMD

OS: Ubuntu 18.04

CPU:AMD FX-6300

GPU: AMD Vega FE

Notes: GUI crashes to terminal with SIGABRT error when attempting reconstruction

PC: Custom AMD

OS: Windows 10

CPU: AMD FX-6300

GPU: AMD Vega FE

Notes: Reconstruction analyzes images but finds no features to match

PC: Custom Intel

OS: Ubuntu 18.04

CPU: Intel i5-3570K

GPU: EVGA GTX 970

Notes: Same crash as others

I have had some success with COLMAP analyzing images and finding features using the CPU only (no GPU) on my custom AMD and Intel builds. This has been running for 8.5 hours today, and I don't know if it will be done in a reasonable amount of time because the CPU is much slower at image analysis than the GPU pipeline. ]]