Open Source Ecology - User contributions [en]

PhotogrammetryWorkflows/VisualSFM CMVS-PMVS Meshlab

2020-04-04T06:29:35Z

Diginess: Layout editing, minor notes added

Note: I (Jacob Johnson, JJ) am currently using Windows. I will try to leave some leeway in these instructions for other OSs by including the overall links where you can find software for those versions, but the most detailed instructions at first will be for Windows versions (installation instructions at least, I think app usage will mostly be similar) until I get my Desktop back online with Ubuntu later and run through the process again for that, then document it.

'''Summary:'''
----
Before going into the installation steps, here is a breakdown of what this set of software is doing / what we can do with it.
VisualSFM takes in photos that you've taken around a 3D object and then runs an algorithm/program against them (SiftGPU, based on David Lowe's work - Chang Chang Wu wrote SiftGPU and VisualSFM) to find "feature points" (types of features in images that are easy to find in other images when the object is viewed from a different angle or distance) and from these, find the relative positions of where your camera was when you took each picture. Up to this point, it has performed "sparse reconstruction'. After this, it will turn to the CMVS-PMVS software (if you've copied it into the root VisualSFM install folder with the VisualSFM executable) to do dense reconstruction which involves finding many more features from the image data to fill out the point cloud. When you click on the dense reconstruction button, it'll ask you to provide a folder path where you would like it to save the results. Later, we will use Meshlab and open that folder/project. We will use Meshlab to construct a colored mesh around the point cloud. At that point, we'll have a 3D object with colors and texture. Once that's done, we can open it with any modelling software (or the main open source one, Blender), as well as import it into a slicer program for conversion to gcode for 3D printing.

After searching among many videos on Youtube, I found a good one that documents the installation process here:
https://www.youtube.com/watch?v=GEAbXYDzUjU
I will include timestamps into this video for each step below. Note that where my instructions differ it has to do with getting Windows binaries rather than Linux versions.

Final note before proceeding to installation steps: You might note that I am linking the same video over and over again. That's because I'm including the time stamps in the video for that piece of the instructions. If you already have the video up and do not want to open another tab, just look at the end. For instance, if I wanted the link to open the video at the 5 minute 30 second mark, I would add &t=5m30s to the end of the Youtube video link (stand for "time equals 5 minutes, 30 seconds"). You can just progress the video to that point in the tab you already have up if you want to move forward.

'''Installation:'''
----
* https://www.youtube.com/watch?v=GEAbXYDzUjU&t=2m41s
Download VisualSFM from this page :
http://ccwu.me/vsfm/ (it has multiple OS versions, Windows, Linux, and MacOS, generally you'll want the 64 bit version - if that doesn't work you may have an old computer and need the 32 bit version)
Installing this only consists of downloading the zip file and extracting it (there's no installer - once extracted, the executable is in the extracted folder and ready to run).

* Next get CMVS-PMVS:
https://www.youtube.com/watch?v=GEAbXYDzUjU&t=2m52s
https://github.com/pmoulon/CMVS-PMVS
Click on the green button to the top right of the repo file list "Clone or Download", then select the option to clone download as a zip file. Extract the zip file and navigate to the subfolder that has the version for your operating system (for instance, for Windows the path from the top level "CMVS-PMVS-Master" extracted repo zip file to the Win64 version is: CMVS-PMVS-master.zip\CMVS-PMVS-master\binariesWin-Linux\Win64-VS2010). Copy everything from that subfolder into the folder where you put VisualSFM (further questions on how / what files to copy, watch the Youtube video starting at the 4 minute mark).

* Get Meshlab:
https://www.youtube.com/watch?v=GEAbXYDzUjU&t=3m15s
https://sourceforge.net/projects/meshlab/
Sourceforge is no longer including adware automatically in all downloads. I installed this and had no issues (JJ, as of Apr 4th, 2020).

* Get Blender:
https://www.blender.org/download/

* Handling pictures:
If you're using your cell phone, you'll want a way to easily get pictures from your phone to your computer so that you can run these apps on them to make the model. I had some initial difficulties, probably due to my phone being encrypted, where when I connected it to my computer, I could not see the phone's internal storage from Windows File Explorer. To overcome that, what I did was upload all the images to Google drive and installed Google Backup and Sync on my computer. Once you've uploaded photos to Google drive, when you sign into Google Backup and sync on the same Google account, it will download all of the files on Google drive to your computer. Its default directory is C:\Users\<YourUserNameHere>\Google Drive.
I didn't do anything else with the files from there, only selected them in VisualSFM from where they were on my computer / where Google Backup and sync put them (see above). You may have to wait a few minutes after uploading for Backup and Sync to download all the pictures to your computer's hard drive. It will happen automatically as long as the app is signed into that Google account.

'''Running the pipeline:'''
----
'''VisualSFM:'''
https://www.youtube.com/watch?v=GEAbXYDzUjU&t=4m33s
Open VisualSFM by going into the folder and double clicking the executable (the one with the green 'V' icon).

* Load the images into VisualSFM:
https://www.youtube.com/watch?v=GEAbXYDzUjU&t=4m45s
Click on the "Open Multiple Images" button (it's third from the left, just under the File tab in VisualSFM).
Navigate to the location you stored your pictures on your computer (if you're using Google Backup and Sync, that's "C:\Users\Your User Name\Google Drive\Whatever you called the image folder"). Select all of the images by left clicking the first one, pressing shift and holding it down while you left click the last one. Finally, click the "open" button on the bottom right of that dialog window (while all the images still highlighted/selected).
Now wait for a bit while it processes all the images. When it is done, you will see mini icons for all of the images in the main view and a summary in the log on the right.

* Calculate missing matches:
https://www.youtube.com/watch?v=GEAbXYDzUjU&t=5m10s
Select the "compute missing matches" icon. It's sixth from the right and looks like four arrows pointed at diagonal corners.
Wait until it finishes.

* Sparse reconstruction:
https://www.youtube.com/watch?v=GEAbXYDzUjU&t=5m23s (through 5m52s)
Within VisualSFM, select "SfM" from the menu at top (2nd one from left). Within that menu, select "Reconstruct sparse".
Wait until it finishes. You should then see the beginnings of a pointcloud with indicators where the cameras were located when the pictures were taken.

* Dense reconstruction (VisualSFM also uses the CMVS-PMVS software here):

https://www.youtube.com/watch?v=GEAbXYDzUjU&t=5m52s
Within VisualSFM, select "SfM" from the menu at top (2nd one from left). Within that menu, select "Reconstruct dense".
It will pop up a dialog asking where it should save the files it creates. Give it a location on your hard drive by navigating, and type a file name you would like the project file to be called (you'll want to remember where you saved it so you can open it in Meshlab later - maybe write the path down).
To see the results, go to View on the menu and select "Dense 3D points" (or just press tab).
That's it, you're done with the VisualSFM and CMVS-PMVS portion of the pipeline. Next we will move on to Meshlab to connect the point cloud surface together with a mesh.

'''Troubleshooting the VSFM process:'''
----
* Note, after you get done with the "Calculate missing matches" step, if you notice that there are two few camera indicators, it is likely that one of them has a red "X" over it and is overall red. This means that while you were taking your pictures, the locations you took them from were likely too far (too many degrees) apart and it could not "connect the dots" between images. Try taking more images in between where there are more distant gaps in positions between the cameras, add the pictures and go back and do the prior step "calculate missing matches" again. You should shoot for 50-60 pictures at least, both rotating around and above (below if possible) the object and perhaps even closeups. More pictures should give a better point cloud, but costs more processing time.

* Make sure that the object you are taking pictures of has good lighting and contrast and there is enough detail on it, matte surfaces work really well. This process might not do well for scenes that don't have enough features/edges/contrast/detail - for instance a blank white wall wouldn't work.
* The process doesn't work at all for moving subjects.
* You'll want to avoid reflective surfaces if at all possible, because when you change your perspective, the reflected light changes as well, messing up the solver, meaning you won't capture any points there and you'll likely have a gap in the mesh.

'''Meshlab:'''
----
* Build the geometry:
https://www.youtube.com/watch?v=GEAbXYDzUjU&t=7m07s
Open Meshlab and go to the File menu option and select "Open project". Navigate to where you saved the project through the dialog window in the "Dense reconstruction" step in VisualSFM above. Select the project file and wait until it finishes loading - you should see the point cloud.
Go to the View menu option and select "Show Layer Dialog" - you should now see a pane to the right representing layers.
Rotate the point cloud around until you have a good idea where your object is (left click and drag) and what points you can cut.
Make sure that you have a good rotation for the cuts, then click the "Select vertexes" icon from the toolbar.
Click and drag to create boxes to select points you don't want in your 3D model. You can hold control to click and drag again to add more boxes of points to your selection. When you're done, click on the first red X icon (it's an X over connected vertexes) to delete the selected points.
Next, go to the File menu and select "Import mesh". Select the file (the dialog should already be in the correct folder since you loaded the project from there, and there will probably only be one file showing - it will end in the .ply extension) and click "open".
Now you should have a much better view and be able to see which of the remaining points you want to delete. Repeat the select vertexes / delete vertexes step again until you have deleted those.
Once you are happy that you have deleted all the points you want gone, go to the Filters menu, select "Point Set", then select "Surface Reconstructions Poisson". In the dialog window, set the octree depth to 12 and solver divide to 7, leaving the other two settings as is. Click "Apply".
To get rid of the weird bubble that it creates while keeping the modifications to the model/mesh, on layers click on the model and .ply layers to make them invisible, then select the poisson layer (click on the name, not on the icon) highlighting it, then use the "select vertexes" and "delete vertexes" to get rid of the top part of the bubble.
Extra notes: At this point, I went to File, "Export Mesh" and exported the mesh as a .obj file. It was 410KB, not too bad. When I loaded it into a slicer program, nothing came up. I think the origin of the 3D coordinate system for the model may have been too far from the model itself.
Note: I was looking for a way to recenter the origin in Meshlab, but found an easy way to do it in Blender. For now, I'll leave that to a later step in Blender (just in case you need it).

* Simplify the geometry:
https://www.youtube.com/watch?v=GEAbXYDzUjU&t=10m11s
Look at the number of faces you have in your layers. If you have more than a million, you should probably do this step the quadric decimation:
Go to the Filters menu, select "Remeshing, Simplification and Reconstruction", then select "Quadric Edge Collapse Decimation" (sounds intimidating and final). Give it a target number of faces of maybe 700,000 and apply.
Next, go to the Filters menu, select "Cleaning and Repairing", and select "Remove Faces from Non Manifold Edges" (this prevents crazy things from happening in the next step where we do texturing).

* Generate the Texture:
https://www.youtube.com/watch?v=GEAbXYDzUjU&t=11m06s
Go to the Filters menu, Texture, and select "Parameterization + texturing from registered rasters" (not the one that is named almost exactly the same but without the '+').
In the dialog window, set the Texture size to 8192, change the texture name to what you want, and click apply.
Wait for it to finish, may take a bit. You should be able to see a progress bar in the lower right.

* Save the model:
In Meshlab, go to the File menu, select "Export Mesh As", and in the dialog, change the file type to "Alias Wavefront Object (*.obj)" and give it a descriptive name, then click save. In the next dialog box, uncheck "normals" and click OK. Now to make things easier for the later step, copy the texture (.png file) and your latest .obj file into a new folder (name that something descriptive).

'''Blender:'''
----
* Create the scene in Blender:
https://www.youtube.com/watch?v=GEAbXYDzUjU&t=13m04s
Open Blender. Go to File, select new, general.
Import your model - go to File, import, wavefront (.obj). In the dialog, navigate to your model you saved in the "Save the model" step above.
Once it has imported, take a look around the scene by rotating (in Blender, that is by default holding down the middle mouse button while you move the mouse).

* Recentering the model (if necessary):
Press shift-ctrl-alt-c and select "geometry to origin". The model should move to the center.
If when doing that, the model suddenly disappears, it's probably just inside the gray box that appears by default in Blender at the origin of the scene, which means you succeeded (you can scroll to zoom inside the box and check). You can delete the cube from the scene collection pane at the right by selecting it and selecting "delete".

TODO: At the end of this process, I had a model in Blender that still looked somewhat point-cloudish. I need to find more tutorials on Meshlab to include tips/tricks. The texture also didn't come through the process, so I need to go back over that and find a fix. Most of the tutorials tell you not to expect to have super nice looking models when you first start. I'll add the most helpful tips and tricks as I find them.

Open Source Photogrammetry

2020-04-04T06:08:55Z

Diginess: Added link to VisualSFM pipeline notes

=2020 Update=
It is now 2020. Is there an effective phone app today that is capable of high quality photogrammetry?

Meshroom appears to work for Prusa - https://blog.prusaprinters.org/photogrammetry-2-3d-scanning-simpler-better-than-ever/

On my computer, it looks like it just froze on Ubuntu 18.04. - MJ

=3D Scanning a Cordless Drill=

See https://www.facebook.com/marcin.jakubowski.378/posts/10214266043662164

=Intro=

*Wikipedia article on Photogrammetry - [https://en.wikipedia.org/wiki/Photogrammetry]
*International Society for Photogrammetry and Remote Sensing (ISPRS)
*http://opensourcephotogrammetry.blogspot.com/
*[http://www.cs.cornell.edu/~snavely/bundler/ Bundler]
* VisualSFM/CMVS-PMVS/Meshlab/Blender pipeline: [[PhotogrammetryWorkflows/VisualSFM_CMVS-PMVS_Meshlab]]
*[https://colmap.github.io/ Colmap] -
*[http://www.efoto.eng.uerj.br/en Efoto]. PDF on the topic - Brazilian - [http://www.efoto.eng.uerj.br/images/Documentos/fotogrametria_digital_revisado.pdf]
*[https://sourceforge.net/projects/insight3dng/ Insight3dng].
*[https://alicevision.github.io AliceVision/Meshroom] allows you to run the whole photogrammetric pipeline. Put your images and it will generate a textured mesh automatically.
*[https://opendronemap.github.io/odm/pages/about.html Open Drone Map]. [https://demo.webodm.org/dashboard/ WebODM]
*[https://www.gcc.tu-darmstadt.de/home/proj/mve/ MVE]. Paper on Floating Scale Surface Reconstruction - [https://www.gcc.tu-darmstadt.de/media/gcc/papers/Fuhrmann-2014-FSS.pdf]
*[http://imagine.enpc.fr/~moulonp/openMVG/ OpenMVG]
*[[Regard3D]].

=Colmap=
See Joseph Prusa's video. This one uses Colmap:

<html><iframe width="560" height="315" src="https://www.youtube.com/embed/ye-C-OOFsX8" frameborder="0" allow="autoplay; encrypted-media" allowfullscreen></iframe></html>

=With VisualSFM=
Available, but no license.

<html><iframe width="560" height="315" src="https://www.youtube.com/embed/D6eqW6yk50k" frameborder="0" allow="autoplay; encrypted-media" allowfullscreen></iframe></html>

=AliceVision/Meshroom=

<html><iframe width="560" height="315" src="https://www.youtube.com/embed/v_O6tYKQEBA" frameborder="0" allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe></html>

<html><iframe width="560" height="315" src="https://www.youtube.com/embed/1D0EhSi-vvc" frameborder="0" allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe></html>

[https://sketchfab.com/blogs/community/tutorial-meshroom-for-beginners Tutorial: Meshroom for Beginners on Sketchfab Blog.]

=See Also=

*[[Open Source 3D Scanning/Motion Capture Construction Set]]

=Useful Links=

* [https://www.youtube.com/watch?v=1won7AYden0 This video has an intresting (albeit expensive) Stil Photogrammetry Setup]

* [https://www.youtube.com/watch?v=Ryawcs6OZmI This video has an intresting (and impossibly expensive) video Photogrammetry Setup]

* https://www.youtube.com/watch?v=ye-C-OOFsX8

*OpenScan - https://www.youtube.com/watch?v=m7ee4jLk_As&feature=push-sd&attr_tag=REkFiylbiOEAE-v7%3A6 It sounds like they may be confusing freeware & opensource? Interesting project and good scanning ideas.

PhotogrammetryWorkflows/VisualSFM CMVS-PMVS Meshlab

2020-04-04T06:05:14Z

Diginess: Bolding headers

Note: I (Jacob Johnson, JJ) am currently using Windows. I will try to leave some leeway in these instructions for other OSs by including the overall links where you can find software for those versions, but the most detailed instructions at first will be for Windows versions (installation instructions at least, I think app usage will mostly be similar) until I get my Desktop back online with Ubuntu later and run through the process again for that, then document it.

'''Summary:'''
Before going into the installation steps, here is a breakdown of what this set of software is doing / what we can do with it. This is what I have gathered so far, and it is still loose / rough around the edges. Please feel free to correct any mistakes I've made, or I will adjust as I learn more.
VisualSFM takes in photos that you've taken around a 3D object and then runs an algorithm/program against them (SiftGPU, based on David Lowe's work - Chang Chang Wu wrote SiftGPU and VisualSFM) to find "feature points" (types of features in images that are easy to find in other images when the object is viewed from a different angle or distance) and from these, find the relative positions of where your camera was when you took each picture. Up to this point, it has performed "sparse reconstruction'. After this, it will turn to the CMVS-PMVS software (if you've copied it into the root VisualSFM install folder with the VisualSFM executable) to do dense reconstruction which involves finding many more features from the image data to fill out the point cloud. When you click on the dense reconstruction button, it'll ask you to provide a folder path where you would like it to save the results. Later, we will use Meshlab and open that folder/project. We will use Meshlab to construct a colored mesh around the point cloud. At that point, we'll have a 3D object with colors and texture. Once that's done, we can open it with any modelling software (or the main open source one, Blender), as well as import it into a slicer program for conversion to gcode for 3D printing.

After searching among many videos on Youtube, I found a good one that documents the installation process here:
https://www.youtube.com/watch?v=GEAbXYDzUjU
I will include timestamps into this video for each step below. Note that where my instructions differ it has to do with getting Windows binaries rather than Linux versions.

Final note before proceeding to installation steps: You might note that I am linking the same video over and over again. That's because I'm including the time stamps in the video for that piece of the instructions. If you already have the video up and do not want to open another tab, just look at the end. For instance, if I wanted the link to open the video at the 5 minute 30 second mark, I would add &t=5m30s to the end of the Youtube video link (stand for "time equals 5 minutes, 30 seconds"). You can just progress the video to that point in the tab you already have up if you want to move forward.

'''Installation:'''
1. https://www.youtube.com/watch?v=GEAbXYDzUjU&t=2m41s
Download VisualSFM from this page :
http://ccwu.me/vsfm/ (it has multiple OS versions, Windows, Linux, and MacOS, generally you'll want the 64 bit version - if that doesn't work you may have an old computer and need the 32 bit version)
Installing this only consists of downloading the zip file and extracting it (there's no installer - once extracted, the executable is in the extracted folder and ready to run).

Next get CMVS-PMVS:
https://www.youtube.com/watch?v=GEAbXYDzUjU&t=2m52s
https://github.com/pmoulon/CMVS-PMVS
Click on the green button to the top right of the repo file list "Clone or Download", then select the option to clone download as a zip file. Extract the zip file and navigate to the subfolder that has the version for your operating system (for instance, for Windows the path from the top level "CMVS-PMVS-Master" extracted repo zip file to the Win64 version is: CMVS-PMVS-master.zip\CMVS-PMVS-master\binariesWin-Linux\Win64-VS2010). Copy everything from that subfolder into the folder where you put VisualSFM (further questions on how / what files to copy, watch the Youtube video starting at the 4 minute mark).

Next get Meshlab:
https://www.youtube.com/watch?v=GEAbXYDzUjU&t=3m15s
https://sourceforge.net/projects/meshlab/
Sourceforge is no longer including adware automatically in all downloads. I installed this and had no issues (JJ, as of Apr 4th, 2020).

Finally, download and install Blender:
https://www.blender.org/download/

'''Handling pictures:'''
If you're using your cell phone, you'll want a way to easily get pictures from your phone to your computer so that you can run these apps on them to make the model. I had some initial difficulties, probably due to my phone being encrypted, where when I connected it to my computer, I could not see the phone's internal storage from Windows File Explorer. To overcome that, what I did was upload all the images to Google drive and installed Google Backup and Sync on my computer. Once you've uploaded photos to Google drive, when you sign into Google Backup and sync on the same Google account, it will download all of the files on Google drive to your computer. Its default directory is C:\Users\<YourUserNameHere>\Google Drive.
I didn't do anything else with the files from there, only selected them in VisualSFM from where they were on my computer / where Google Backup and sync put them (see above). You may have to wait a few minutes after uploading for Backup and Sync to download all the pictures to your computer's hard drive. It will happen automatically as long as the app is signed into that Google account.

'''Running the pipeline:'''

'''VisualSFM:'''
https://www.youtube.com/watch?v=GEAbXYDzUjU&t=4m33s
Open VisualSFM by going into the folder and double clicking the executable (the one with the green 'V' icon).

'''Load the images into VisualSFM:'''
https://www.youtube.com/watch?v=GEAbXYDzUjU&t=4m45s
Click on the "Open Multiple Images" button (it's third from the left, just under the File tab in VisualSFM).
Navigate to the location you stored your pictures on your computer (if you're using Google Backup and Sync, that's "C:\Users\Your User Name\Google Drive\Whatever you called the image folder"). Select all of the images by left clicking the first one, pressing shift and holding it down while you left click the last one. Finally, click the "open" button on the bottom right of that dialog window (while all the images still highlighted/selected).
Now wait for a bit while it processes all the images. When it is done, you will see mini icons for all of the images in the main view and a summary in the log on the right.

'''Calculate missing matches:'''
https://www.youtube.com/watch?v=GEAbXYDzUjU&t=5m10s
Select the "compute missing matches" icon. It's sixth from the right and looks like four arrows pointed at diagonal corners.
Wait until it finishes.

'''Sparse reconstruction:'''
https://www.youtube.com/watch?v=GEAbXYDzUjU&t=5m23s (through 5m52s)
Within VisualSFM, select "SfM" from the menu at top (2nd one from left). Within that menu, select "Reconstruct sparse".
Wait until it finishes. You should then see the beginnings of a pointcloud with indicators where the cameras were located when the pictures were taken.

'''Troubleshooting:'''
Note, at this point when it has finished, if you notice that there are two few camera indicators, it is likely that one of them has a red "X" over it and is overall red. This means that while you were taking your pictures, the locations you took them from were likely too far (too many degrees) apart and it could not "connect the dots" between images. Try taking more images in between where there are more distant gaps in positions between the cameras, add the pictures and go back and do the prior step "calculate missing matches" again. This was happening to me and I got confused before really starting to think about it. I had taken about 20 images around a book on my couch. It wasn't nearly enough. Note that the video author took 70 pictures to capture his Millenium Falcon model. So I went back and took around 40 more (I had 64 when I was done, 5 were duplicates - this process doesn't care about those), being careful to move around the book no more than 15 degrees between images. Note that the time each step of this process takes will increase as the number of pictures goes up. After doing the missing matches step again, this time I had 2080 features instead of just over 100 and 434 seconds versus the 43 seconds it had taken prior (the last time it had stopped after getting confused after 3 pictures - if that step stops after less than a minute, you might have too few pictures). If the "compute missing matches" took enough time like this / you have plenty of pictures and features, you're probably going to be OK with the sparse reconstruction step that comes next. The next dense reconstruction for me after I took the extra pictures took 10 minutes, 55 seconds - note that unless you properly copied CMVS-PMVS into the VisualSFM folder as documented in the CMVS-PMVS installation step above, dense reconstruction will not work.
Other: Make sure that the object you are taking pictures of has good lighting and contrast and there is detail on it. This process might not do well for scenes that don't have enough features/edges/contrast/detail - for instance a blank white wall wouldn't be captured well at all.
For something as plain as a big piece of steel where the surface is very smooth, it might have to be enough to capture all the edges and for it to be dimensionally correct. Otherwise those surfaces will need detail somehow.
(if you want a real challenge, try to do this to your cat when it happens to lay down - the process doesn't work at all for moving subjects, so move quickly - just kidding, probably a bad idea that will frustrate you to no end)
Another thing I noticed was that the top cover of the book I was capturing was partially reflective. You'll want to avoid reflective surfaces if at all possible, because when you change your perspective, the reflected light changes as well, messing up the solver. It is very likely nothing will get captured at all from the reflective spots because matching of features can't occur between images there, causing gaps in the point cloud and resulting mesh.

'''Dense reconstruction (VisualSFM also uses the CMVS-PMVS software here):'''
https://www.youtube.com/watch?v=GEAbXYDzUjU&t=5m52s
Within VisualSFM, select "SfM" from the menu at top (2nd one from left). Within that menu, select "Reconstruct dense".
It will pop up a dialog asking where it should save the files it creates. Give it a location on your hard drive by navigating, and type a file name you would like the project file to be called (you'll want to remember where you saved it so you can open it in Meshlab later - maybe write the path down).
To see the results, go to View on the menu and select "Dense 3D points" (or just press tab).
That's it, you're done with the VisualSFM and CMVS-PMVS portion of the pipeline. Next we will move on to Meshlab to connect the point cloud surface together with a mesh.

'''Meshlab:'''

'''Build the geometry:'''
https://www.youtube.com/watch?v=GEAbXYDzUjU&t=7m07s
Open Meshlab and go to the File menu option and select "Open project". Navigate to where you saved the project through the dialog window in the "Dense reconstruction" step in VisualSFM above. Select the project file and wait until it finishes loading - you should see the point cloud.
Go to the View menu option and select "Show Layer Dialog" - you should now see a pane to the right representing layers.
Rotate the point cloud around until you have a good idea where your object is (left click and drag) and what points you can cut.
Make sure that you have a good rotation for the cuts, then click the "Select vertexes" icon from the toolbar.
Click and drag to create boxes to select points you don't want in your 3D model. You can hold control to click and drag again to add more boxes of points to your selection. When you're done, click on the first red X icon (it's an X over connected vertexes) to delete the selected points.
Next, go to the File menu and select "Import mesh". Select the file (the dialog should already be in the correct folder since you loaded the project from there, and there will probably only be one file showing - it will end in the .ply extension) and click "open".
Now you should have a much better view and be able to see which of the remaining points you want to delete. Repeat the select vertexes / delete vertexes step again until you have deleted those.
Once you are happy that you have deleted all the points you want gone, go to the Filters menu, select "Point Set", then select "Surface Reconstructions Poisson". In the dialog window, set the octree depth to 12 and solver divide to 7, leaving the other two settings as is. Click "Apply".
To get rid of the weird bubble that it creates while keeping the modifications to the model/mesh, on layers click on the model and .ply layers to make them invisible, then select the poisson layer (click on the name, not on the icon) highlighting it, then use the "select vertexes" and "delete vertexes" to get rid of the top part of the bubble.
Extra notes: At this point, I went to File, "Export Mesh" and exported the mesh as a .obj file. It was 410KB, not too bad. When I loaded it into a slicer program, nothing came up. I think the origin of the 3D coordinate system for the model may have been too far from the model itself.
Note: I was looking for a way to recenter the origin in Meshlab, but found an easy way to do it in Blender. For now, I'll leave that to a later step in Blender (just in case you need it).

'''Simplify the geometry:'''
https://www.youtube.com/watch?v=GEAbXYDzUjU&t=10m11s
Look at the number of faces you have in your layers. If you have more than a million, you should probably do this step the quadric decimation:
Go to the Filters menu, select "Remeshing, Simplification and Reconstruction", then select "Quadric Edge Collapse Decimation" (sounds intimidating and final). Give it a target number of faces of maybe 700,000 and apply.
Next, go to the Filters menu, select "Cleaning and Repairing", and select "Remove Faces from Non Manifold Edges" (this prevents crazy things from happening in the next step where we do texturing).

'''Generate the Texture:'''
https://www.youtube.com/watch?v=GEAbXYDzUjU&t=11m06s
Go to the Filters menu, Texture, and select "Parameterization + texturing from registered rasters" (not the one that is named almost exactly the same but without the '+').
In the dialog window, set the Texture size to 8192, change the texture name to what you want, and click apply.
Wait for it to finish, may take a bit. You should be able to see a progress bar in the lower right.

'''Save the model:'''
In Meshlab, go to the File menu, select "Export Mesh As", and in the dialog, change the file type to "Alias Wavefront Object (*.obj)" and give it a descriptive name, then click save. In the next dialog box, uncheck "normals" and click OK. Now to make things easier for the later step, copy the texture (.png file) and your latest .obj file into a new folder (name that something descriptive).

'''Blender:'''
'''Create the scene in Blender:'''
https://www.youtube.com/watch?v=GEAbXYDzUjU&t=13m04s
Open Blender. Go to File, select new, general.
Import your model - go to File, import, wavefront (.obj). In the dialog, navigate to your model you saved in the "Save the model" step above.
Once it has imported, take a look around the scene by rotating (in Blender, that is by default holding down the middle mouse button while you move the mouse).

If you need to recenter the model in the scene, select the model in the scene then press shift-ctrl-alt-c and select "geometry to origin". The model should move to the center.
If when doing that, the model suddenly disappears, it's probably just inside the gray box that appears by default in Blender at the origin of the scene, which means you succeeded (you can scroll to zoom inside the box and check). You can delete the cube from the scene collection pane at the right by selecting it and selecting "delete".

TODO: At the end of this process, I had a model in Blender that still looked somewhat point-cloudish. I need to find more tutorials on Meshlab to include tips/tricks. The texture also didn't come through the process, so I need to go back over that and find a fix.

PhotogrammetryWorkflows/VisualSFM CMVS-PMVS Meshlab

2020-04-04T06:01:34Z

Diginess: Tip about surface reflection being undesirable

Note: I (Jacob Johnson, JJ) am currently using Windows. I will try to leave some leeway in these instructions for other OSs by including the overall links where you can find software for those versions, but the most detailed instructions at first will be for Windows versions (installation instructions at least, I think app usage will mostly be similar) until I get my Desktop back online with Ubuntu later and run through the process again for that, then document it.

Before going into the installation steps, here is a breakdown of what this set of software is doing / what we can do with it. This is what I have gathered so far, and it is still loose / rough around the edges. Please feel free to correct any mistakes I've made, or I will adjust as I learn more.
VisualSFM takes in photos that you've taken around a 3D object and then runs an algorithm/program against them (SiftGPU, based on David Lowe's work - Chang Chang Wu wrote SiftGPU and VisualSFM) to find "feature points" (types of features in images that are easy to find in other images when the object is viewed from a different angle or distance) and from these, find the relative positions of where your camera was when you took each picture. Up to this point, it has performed "sparse reconstruction'. After this, it will turn to the CMVS-PMVS software (if you've copied it into the root VisualSFM install folder with the VisualSFM executable) to do dense reconstruction which involves finding many more features from the image data to fill out the point cloud. When you click on the dense reconstruction button, it'll ask you to provide a folder path where you would like it to save the results. Later, we will use Meshlab and open that folder/project. We will use Meshlab to construct a colored mesh around the point cloud. At that point, we'll have a 3D object with colors and texture. Once that's done, we can open it with any modelling software (or the main open source one, Blender), as well as import it into a slicer program for conversion to gcode for 3D printing.

After searching among many videos on Youtube, I found a good one that documents the installation process here:
https://www.youtube.com/watch?v=GEAbXYDzUjU
I will include timestamps into this video for each step below. Note that where my instructions differ it has to do with getting Windows binaries rather than Linux versions.

Final note before proceeding to installation steps: You might note that I am linking the same video over and over again. That's because I'm including the time stamps in the video for that piece of the instructions. If you already have the video up and do not want to open another tab, just look at the end. For instance, if I wanted the link to open the video at the 5 minute 30 second mark, I would add &t=5m30s to the end of the Youtube video link (stand for "time equals 5 minutes, 30 seconds"). You can just progress the video to that point in the tab you already have up if you want to move forward.

Installation:
1. https://www.youtube.com/watch?v=GEAbXYDzUjU&t=2m41s
Download VisualSFM from this page :
http://ccwu.me/vsfm/ (it has multiple OS versions, Windows, Linux, and MacOS, generally you'll want the 64 bit version - if that doesn't work you may have an old computer and need the 32 bit version)
Installing this only consists of downloading the zip file and extracting it (there's no installer - once extracted, the executable is in the extracted folder and ready to run).

Next get CMVS-PMVS:
https://www.youtube.com/watch?v=GEAbXYDzUjU&t=2m52s
https://github.com/pmoulon/CMVS-PMVS
Click on the green button to the top right of the repo file list "Clone or Download", then select the option to clone download as a zip file. Extract the zip file and navigate to the subfolder that has the version for your operating system (for instance, for Windows the path from the top level "CMVS-PMVS-Master" extracted repo zip file to the Win64 version is: CMVS-PMVS-master.zip\CMVS-PMVS-master\binariesWin-Linux\Win64-VS2010). Copy everything from that subfolder into the folder where you put VisualSFM (further questions on how / what files to copy, watch the Youtube video starting at the 4 minute mark).

Next get Meshlab:
https://www.youtube.com/watch?v=GEAbXYDzUjU&t=3m15s
https://sourceforge.net/projects/meshlab/
Sourceforge is no longer including adware automatically in all downloads. I installed this and had no issues (JJ, as of Apr 4th, 2020).

Finally, download and install Blender:
https://www.blender.org/download/

Handling pictures:
If you're using your cell phone, you'll want a way to easily get pictures from your phone to your computer so that you can run these apps on them to make the model. I had some initial difficulties, probably due to my phone being encrypted, where when I connected it to my computer, I could not see the phone's internal storage from Windows File Explorer. To overcome that, what I did was upload all the images to Google drive and installed Google Backup and Sync on my computer. Once you've uploaded photos to Google drive, when you sign into Google Backup and sync on the same Google account, it will download all of the files on Google drive to your computer. Its default directory is C:\Users\<YourUserNameHere>\Google Drive.
I didn't do anything else with the files from there, only selected them in VisualSFM from where they were on my computer / where Google Backup and sync put them (see above). You may have to wait a few minutes after uploading for Backup and Sync to download all the pictures to your computer's hard drive. It will happen automatically as long as the app is signed into that Google account.

Running the pipeline:

VisualSFM steps:
https://www.youtube.com/watch?v=GEAbXYDzUjU&t=4m33s
Open VisualSFM by going into the folder and double clicking the executable (the one with the green 'V' icon).

Open the images in VisualSFM:
https://www.youtube.com/watch?v=GEAbXYDzUjU&t=4m45s
Click on the "Open Multiple Images" button (it's third from the left, just under the File tab in VisualSFM).
Navigate to the location you stored your pictures on your computer (if you're using Google Backup and Sync, that's "C:\Users\Your User Name\Google Drive\Whatever you called the image folder"). Select all of the images by left clicking the first one, pressing shift and holding it down while you left click the last one. Finally, click the "open" button on the bottom right of that dialog window (while all the images still highlighted/selected).
Now wait for a bit while it processes all the images. When it is done, you will see mini icons for all of the images in the main view and a summary in the log on the right.

Calculate missing matches:
https://www.youtube.com/watch?v=GEAbXYDzUjU&t=5m10s
Select the "compute missing matches" icon. It's sixth from the right and looks like four arrows pointed at diagonal corners.
Wait until it finishes.

Sparse reconstruction:
https://www.youtube.com/watch?v=GEAbXYDzUjU&t=5m23s (through 5m52s)
Within VisualSFM, select "SfM" from the menu at top (2nd one from left). Within that menu, select "Reconstruct sparse".
Wait until it finishes. You should then see the beginnings of a pointcloud with indicators where the cameras were located when the pictures were taken.
Troubleshooting: Note, at this point when it has finished, if you notice that there are two few camera indicators, it is likely that one of them has a red "X" over it and is overall red. This means that while you were taking your pictures, the locations you took them from were likely too far (too many degrees) apart and it could not "connect the dots" between images. Try taking more images in between where there are more distant gaps in positions between the cameras, add the pictures and go back and do the prior step "calculate missing matches" again. This was happening to me and I got confused before really starting to think about it. I had taken about 20 images around a book on my couch. It wasn't nearly enough. Note that the video author took 70 pictures to capture his Millenium Falcon model. So I went back and took around 40 more (I had 64 when I was done, 5 were duplicates - this process doesn't care about those), being careful to move around the book no more than 15 degrees between images. Note that the time each step of this process takes will increase as the number of pictures goes up. After doing the missing matches step again, this time I had 2080 features instead of just over 100 and 434 seconds versus the 43 seconds it had taken prior (the last time it had stopped after getting confused after 3 pictures - if that step stops after less than a minute, you might have too few pictures). If the "compute missing matches" took enough time like this / you have plenty of pictures and features, you're probably going to be OK with the sparse reconstruction step that comes next. The next dense reconstruction for me after I took the extra pictures took 10 minutes, 55 seconds - note that unless you properly copied CMVS-PMVS into the VisualSFM folder as documented in the CMVS-PMVS installation step above, dense reconstruction will not work.
Other: Make sure that the object you are taking pictures of has good lighting and contrast and there is detail on it. This process might not do well for scenes that don't have enough features/edges/contrast/detail - for instance a blank white wall wouldn't be captured well at all.
For something as plain as a big piece of steel where the surface is very smooth, it might have to be enough to capture all the edges and for it to be dimensionally correct. Otherwise those surfaces will need detail somehow.
(if you want a real challenge, try to do this to your cat when it happens to lay down - the process doesn't work at all for moving subjects, so move quickly - just kidding, probably a bad idea that will frustrate you to no end)
Another thing I noticed was that the top cover of the book I was capturing was partially reflective. You'll want to avoid reflective surfaces if at all possible, because when you change your perspective, the reflected light changes as well, messing up the solver. It is very likely nothing will get captured at all from the reflective spots because matching of features can't occur between images there, causing gaps in the point cloud and resulting mesh.

Dense reconstruction (VisualSFM also uses the CMVS-PMVS software here):
https://www.youtube.com/watch?v=GEAbXYDzUjU&t=5m52s
Within VisualSFM, select "SfM" from the menu at top (2nd one from left). Within that menu, select "Reconstruct dense".
It will pop up a dialog asking where it should save the files it creates. Give it a location on your hard drive by navigating, and type a file name you would like the project file to be called (you'll want to remember where you saved it so you can open it in Meshlab later - maybe write the path down).
To see the results, go to View on the menu and select "Dense 3D points" (or just press tab).
That's it, you're done with the VisualSFM and CMVS-PMVS portion of the pipeline. Next we will move on to Meshlab to connect the point cloud surface together with a mesh.

Meshlab:

Build the geometry:
https://www.youtube.com/watch?v=GEAbXYDzUjU&t=7m07s
Open Meshlab and go to the File menu option and select "Open project". Navigate to where you saved the project through the dialog window in the "Dense reconstruction" step in VisualSFM above. Select the project file and wait until it finishes loading - you should see the point cloud.
Go to the View menu option and select "Show Layer Dialog" - you should now see a pane to the right representing layers.
Rotate the point cloud around until you have a good idea where your object is (left click and drag) and what points you can cut.
Make sure that you have a good rotation for the cuts, then click the "Select vertexes" icon from the toolbar.
Click and drag to create boxes to select points you don't want in your 3D model. You can hold control to click and drag again to add more boxes of points to your selection. When you're done, click on the first red X icon (it's an X over connected vertexes) to delete the selected points.
Next, go to the File menu and select "Import mesh". Select the file (the dialog should already be in the correct folder since you loaded the project from there, and there will probably only be one file showing - it will end in the .ply extension) and click "open".
Now you should have a much better view and be able to see which of the remaining points you want to delete. Repeat the select vertexes / delete vertexes step again until you have deleted those.
Once you are happy that you have deleted all the points you want gone, go to the Filters menu, select "Point Set", then select "Surface Reconstructions Poisson". In the dialog window, set the octree depth to 12 and solver divide to 7, leaving the other two settings as is. Click "Apply".
To get rid of the weird bubble that it creates while keeping the modifications to the model/mesh, on layers click on the model and .ply layers to make them invisible, then select the poisson layer (click on the name, not on the icon) highlighting it, then use the "select vertexes" and "delete vertexes" to get rid of the top part of the bubble.
Extra notes: At this point, I went to File, "Export Mesh" and exported the mesh as a .obj file. It was 410KB, not too bad. When I loaded it into a slicer program, nothing came up. I think the origin of the 3D coordinate system for the model may have been too far from the model itself.
Note: I was looking for a way to recenter the origin in Meshlab, but found an easy way to do it in Blender. For now, I'll leave that to a later step in Blender (just in case you need it).

Simplify the geometry:
https://www.youtube.com/watch?v=GEAbXYDzUjU&t=10m11s
Look at the number of faces you have in your layers. If you have more than a million, you should probably do this step the quadric decimation:
Go to the Filters menu, select "Remeshing, Simplification and Reconstruction", then select "Quadric Edge Collapse Decimation" (sounds intimidating and final). Give it a target number of faces of maybe 700,000 and apply.
Next, go to the Filters menu, select "Cleaning and Repairing", and select "Remove Faces from Non Manifold Edges" (this prevents crazy things from happening in the next step where we do texturing).

Generate the Texture:
https://www.youtube.com/watch?v=GEAbXYDzUjU&t=11m06s
Go to the Filters menu, Texture, and select "Parameterization + texturing from registered rasters" (not the one that is named almost exactly the same but without the '+').
In the dialog window, set the Texture size to 8192, change the texture name to what you want, and click apply.
Wait for it to finish, may take a bit. You should be able to see a progress bar in the lower right.

Save the model:
In Meshlab, go to the File menu, select "Export Mesh As", and in the dialog, change the file type to "Alias Wavefront Object (*.obj)" and give it a descriptive name, then click save. In the next dialog box, uncheck "normals" and click OK. Now to make things easier for the later step, copy the texture (.png file) and your latest .obj file into a new folder (name that something descriptive).

Create the scene in Blender:
https://www.youtube.com/watch?v=GEAbXYDzUjU&t=13m04s
Open Blender. Go to File, select new, general.
Import your model - go to File, import, wavefront (.obj). In the dialog, navigate to your model you saved in the "Save the model" step above.
Once it has imported, take a look around the scene by rotating (in Blender, that is by default holding down the middle mouse button while you move the mouse).

If you need to recenter the model in the scene, select the model in the scene then press shift-ctrl-alt-c and select "geometry to origin". The model should move to the center.
If when doing that, the model suddenly disappears, it's probably just inside the gray box that appears by default in Blender at the origin of the scene, which means you succeeded (you can scroll to zoom inside the box and check). You can delete the cube from the scene collection pane at the right by selecting it and selecting "delete".

TODO: At the end of this process, I had a model in Blender that still looked somewhat point-cloudish. I need to find more tutorials on Meshlab to include tips/tricks. The texture also didn't come through the process, so I need to go back over that and find a fix.

PhotogrammetryWorkflows/VisualSFM CMVS-PMVS Meshlab

2020-04-04T04:32:41Z

Diginess: Added TODO to end

Note: I (Jacob Johnson, JJ) am currently using Windows. I will try to leave some leeway in these instructions for other OSs by including the overall links where you can find software for those versions, but the most detailed instructions at first will be for Windows versions (installation instructions at least, I think app usage will mostly be similar) until I get my Desktop back online with Ubuntu later and run through the process again for that, then document it.

Before going into the installation steps, here is a breakdown of what this set of software is doing / what we can do with it. This is what I have gathered so far, and it is still loose / rough around the edges. Please feel free to correct any mistakes I've made, or I will adjust as I learn more.
VisualSFM takes in photos that you've taken around a 3D object and then runs an algorithm/program against them (SiftGPU, based on David Lowe's work - Chang Chang Wu wrote SiftGPU and VisualSFM) to find "feature points" (types of features in images that are easy to find in other images when the object is viewed from a different angle or distance) and from these, find the relative positions of where your camera was when you took each picture. Up to this point, it has performed "sparse reconstruction'. After this, it will turn to the CMVS-PMVS software (if you've copied it into the root VisualSFM install folder with the VisualSFM executable) to do dense reconstruction which involves finding many more features from the image data to fill out the point cloud. When you click on the dense reconstruction button, it'll ask you to provide a folder path where you would like it to save the results. Later, we will use Meshlab and open that folder/project. We will use Meshlab to construct a colored mesh around the point cloud. At that point, we'll have a 3D object with colors and texture. Once that's done, we can open it with any modelling software (or the main open source one, Blender), as well as import it into a slicer program for conversion to gcode for 3D printing.

After searching among many videos on Youtube, I found a good one that documents the installation process here:
https://www.youtube.com/watch?v=GEAbXYDzUjU
I will include timestamps into this video for each step below. Note that where my instructions differ it has to do with getting Windows binaries rather than Linux versions.

Final note before proceeding to installation steps: You might note that I am linking the same video over and over again. That's because I'm including the time stamps in the video for that piece of the instructions. If you already have the video up and do not want to open another tab, just look at the end. For instance, if I wanted the link to open the video at the 5 minute 30 second mark, I would add &t=5m30s to the end of the Youtube video link (stand for "time equals 5 minutes, 30 seconds"). You can just progress the video to that point in the tab you already have up if you want to move forward.

Installation:
1. https://www.youtube.com/watch?v=GEAbXYDzUjU&t=2m41s
Download VisualSFM from this page :
http://ccwu.me/vsfm/ (it has multiple OS versions, Windows, Linux, and MacOS, generally you'll want the 64 bit version - if that doesn't work you may have an old computer and need the 32 bit version)
Installing this only consists of downloading the zip file and extracting it (there's no installer - once extracted, the executable is in the extracted folder and ready to run).

Next get CMVS-PMVS:
https://www.youtube.com/watch?v=GEAbXYDzUjU&t=2m52s
https://github.com/pmoulon/CMVS-PMVS
Click on the green button to the top right of the repo file list "Clone or Download", then select the option to clone download as a zip file. Extract the zip file and navigate to the subfolder that has the version for your operating system (for instance, for Windows the path from the top level "CMVS-PMVS-Master" extracted repo zip file to the Win64 version is: CMVS-PMVS-master.zip\CMVS-PMVS-master\binariesWin-Linux\Win64-VS2010). Copy everything from that subfolder into the folder where you put VisualSFM (further questions on how / what files to copy, watch the Youtube video starting at the 4 minute mark).

Next get Meshlab:
https://www.youtube.com/watch?v=GEAbXYDzUjU&t=3m15s
Yes, it's at SourceForge. They are no longer including adware automatically in all downloads. Grab the installer and install it.

Finally, download and install Blender:
https://www.blender.org/download/

Handling pictures:
If you're using your cell phone, you'll want a way to easily get pictures from your phone to your computer so that you can run these apps on them to make the model. I had some initial difficulties, probably due to my phone being encrypted, where when I connected it to my computer, I could not see the phone's internal storage from Windows File Explorer. To overcome that, what I did was upload all the images to Google drive and installed Google Backup and Sync on my computer. Once you've uploaded photos to Google drive, when you sign into Google Backup and sync on the same Google account, it will download all of the files on Google drive to your computer. Its default directory is C:\Users\<YourUserNameHere>\Google Drive.
I didn't do anything else with the files from there, only selected them in VisualSFM from where they were on my computer / where Google Backup and sync put them (see above). You may have to wait a few minutes after uploading for Backup and Sync to download all the pictures to your computer's hard drive. It will happen automatically as long as the app is signed into that Google account.

Running the pipeline:

VisualSFM steps:
https://www.youtube.com/watch?v=GEAbXYDzUjU&t=4m33s
Open VisualSFM by going into the folder and double clicking the executable (the one with the green 'V' icon).

Open the images in VisualSFM:
https://www.youtube.com/watch?v=GEAbXYDzUjU&t=4m45s
Click on the "Open Multiple Images" button (it's third from the left, just under the File tab in VisualSFM).
Navigate to the location you stored your pictures on your computer (if you're using Google Backup and Sync, that's "C:\Users\Your User Name\Google Drive\Whatever you called the image folder"). Select all of the images by left clicking the first one, pressing shift and holding it down while you left click the last one. Finally, click the "open" button on the bottom right of that dialog window (while all the images still highlighted/selected).
Now wait for a bit while it processes all the images. When it is done, you will see mini icons for all of the images in the main view and a summary in the log on the right.

Calculate missing matches:
https://www.youtube.com/watch?v=GEAbXYDzUjU&t=5m10s
Select the "compute missing matches" icon. It's sixth from the right and looks like four arrows pointed at diagonal corners.
Wait until it finishes.

Sparse reconstruction:
https://www.youtube.com/watch?v=GEAbXYDzUjU&t=5m23s (through 5m52s)
Within VisualSFM, select "SfM" from the menu at top (2nd one from left). Within that menu, select "Reconstruct sparse".
Wait until it finishes. You should then see the beginnings of a pointcloud with indicators where the cameras were located when the pictures were taken.
Troubleshooting: Note, at this point when it has finished, if you notice that there are two few camera indicators, it is likely that one of them has a red "X" over it and is overall red. This means that while you were taking your pictures, the locations you took them from were likely too far (too many degrees) apart and it could not "connect the dots" between images. Try taking more images in between where there are more distant gaps in positions between the cameras, add the pictures and go back and do the prior step "calculate missing matches" again. This was happening to me and I got confused before really starting to think about it. I had taken about 20 images around a book on my couch. It wasn't nearly enough. Note that the video author took 70 pictures to capture his Millenium Falcon model. So I went back and took around 40 more (I had 64 when I was done, 5 were duplicates - this process doesn't care about those), being careful to move around the book no more than 15 degrees between images. Note that the time each step of this process takes will increase as the number of pictures goes up. After doing the missing matches step again, this time I had 2080 features instead of just over 100 and 434 seconds versus the 43 seconds it had taken prior (the last time it had stopped after getting confused after 3 pictures - if that step stops after less than a minute, you might have too few pictures). If the "compute missing matches" took enough time like this / you have plenty of pictures and features, you're probably going to be OK with the sparse reconstruction step that comes next. The next dense reconstruction for me after I took the extra pictures took 10 minutes, 55 seconds - note that unless you properly copied CMVS-PMVS into the VisualSFM folder as documented in the CMVS-PMVS installation step above, dense reconstruction will not work.
Other: Make sure that the object you are taking pictures of has good lighting and contrast and there is detail on it. This process might not do well for scenes that don't have enough features/edges/contrast/detail - for instance a blank white wall wouldn't be captured well at all.
For something as plain as a big piece of steel where the surface is very smooth, it might have to be enough to capture all the edges and for it to be dimensionally correct. Otherwise those surfaces will need detail somehow.
(if you want a real challenge, try to do this to your cat when it happens to lay down - the process doesn't work at all for moving subjects, so move quickly - just kidding, probably a bad idea that will frustrate you to no end)

Dense reconstruction (VisualSFM also uses the CMVS-PMVS software here):
https://www.youtube.com/watch?v=GEAbXYDzUjU&t=5m52s
Within VisualSFM, select "SfM" from the menu at top (2nd one from left). Within that menu, select "Reconstruct dense".
It will pop up a dialog asking where it should save the files it creates. Give it a location on your hard drive by navigating, and type a file name you would like the project file to be called (you'll want to remember where you saved it so you can open it in Meshlab later - maybe write the path down).
To see the results, go to View on the menu and select "Dense 3D points" (or just press tab).
That's it, you're done with the VisualSFM and CMVS-PMVS portion of the pipeline. Next we will move on to Meshlab to connect the point cloud surface together with a mesh.

Meshlab:

Build the geometry:
https://www.youtube.com/watch?v=GEAbXYDzUjU&t=7m07s
Open Meshlab and go to the File menu option and select "Open project". Navigate to where you saved the project through the dialog window in the "Dense reconstruction" step in VisualSFM above. Select the project file and wait until it finishes loading - you should see the point cloud.
Go to the View menu option and select "Show Layer Dialog" - you should now see a pane to the right representing layers.
Rotate the point cloud around until you have a good idea where your object is (left click and drag) and what points you can cut.
Make sure that you have a good rotation for the cuts, then click the "Select vertexes" icon from the toolbar.
Click and drag to create boxes to select points you don't want in your 3D model. You can hold control to click and drag again to add more boxes of points to your selection. When you're done, click on the first red X icon (it's an X over connected vertexes) to delete the selected points.
Next, go to the File menu and select "Import mesh". Select the file (the dialog should already be in the correct folder since you loaded the project from there, and there will probably only be one file showing - it will end in the .ply extension) and click "open".
Now you should have a much better view and be able to see which of the remaining points you want to delete. Repeat the select vertexes / delete vertexes step again until you have deleted those.
Once you are happy that you have deleted all the points you want gone, go to the Filters menu, select "Point Set", then select "Surface Reconstructions Poisson". In the dialog window, set the octree depth to 12 and solver divide to 7, leaving the other two settings as is. Click "Apply".
To get rid of the weird bubble that it creates while keeping the modifications to the model/mesh, on layers click on the model and .ply layers to make them invisible, then select the poisson layer (click on the name, not on the icon) highlighting it, then use the "select vertexes" and "delete vertexes" to get rid of the top part of the bubble.
Extra notes: At this point, I went to File, "Export Mesh" and exported the mesh as a .obj file. It was 410KB, not too bad. When I loaded it into a slicer program, nothing came up. I think the origin of the 3D coordinate system for the model may have been too far from the model itself.
Note: I was looking for a way to recenter the origin in Meshlab, but found an easy way to do it in Blender. For now, I'll leave that to a later step in Blender (just in case you need it).

Simplify the geometry:
https://www.youtube.com/watch?v=GEAbXYDzUjU&t=10m11s
Look at the number of faces you have in your layers. If you have more than a million, you should probably do this step the quadric decimation:
Go to the Filters menu, select "Remeshing, Simplification and Reconstruction", then select "Quadric Edge Collapse Decimation" (sounds intimidating and final). Give it a target number of faces of maybe 700,000 and apply.
Next, go to the Filters menu, select "Cleaning and Repairing", and select "Remove Faces from Non Manifold Edges" (this prevents crazy things from happening in the next step where we do texturing).

Generate the Texture:
https://www.youtube.com/watch?v=GEAbXYDzUjU&t=11m06s
Go to the Filters menu, Texture, and select "Parameterization + texturing from registered rasters" (not the one that is named almost exactly the same but without the '+').
In the dialog window, set the Texture size to 8192, change the texture name to what you want, and click apply.
Wait for it to finish, may take a bit. You should be able to see a progress bar in the lower right.

Save the model:
In Meshlab, go to the File menu, select "Export Mesh As", and in the dialog, change the file type to "Alias Wavefront Object (*.obj)" and give it a descriptive name, then click save. In the next dialog box, uncheck "normals" and click OK. Now to make things easier for the later step, copy the texture (.png file) and your latest .obj file into a new folder (name that something descriptive).

Create the scene in Blender:
https://www.youtube.com/watch?v=GEAbXYDzUjU&t=13m04s
Open Blender. Go to File, select new, general.
Import your model - go to File, import, wavefront (.obj). In the dialog, navigate to your model you saved in the "Save the model" step above.
Once it has imported, take a look around the scene by rotating (in Blender, that is by default holding down the middle mouse button while you move the mouse).

If you need to recenter the model in the scene, select the model in the scene then press shift-ctrl-alt-c and select "geometry to origin". The model should move to the center.
If when doing that, the model suddenly disappears, it's probably just inside the gray box that appears by default in Blender at the origin of the scene, which means you succeeded (you can scroll to zoom inside the box and check). You can delete the cube from the scene collection pane at the right by selecting it and selecting "delete".

TODO: At the end of this process, I had a model in Blender that still looked somewhat point-cloudish. I need to find more tutorials on Meshlab to include tips/tricks. The texture also didn't come through the process, so I need to go back over that and find a fix.

PhotogrammetryWorkflows/VisualSFM CMVS-PMVS Meshlab

2020-04-04T04:26:51Z

Diginess: Change to model centering instructions to simplify.

Note: I (Jacob Johnson, JJ) am currently using Windows. I will try to leave some leeway in these instructions for other OSs by including the overall links where you can find software for those versions, but the most detailed instructions at first will be for Windows versions (installation instructions at least, I think app usage will mostly be similar) until I get my Desktop back online with Ubuntu later and run through the process again for that, then document it.

Before going into the installation steps, here is a breakdown of what this set of software is doing / what we can do with it. This is what I have gathered so far, and it is still loose / rough around the edges. Please feel free to correct any mistakes I've made, or I will adjust as I learn more.
VisualSFM takes in photos that you've taken around a 3D object and then runs an algorithm/program against them (SiftGPU, based on David Lowe's work - Chang Chang Wu wrote SiftGPU and VisualSFM) to find "feature points" (types of features in images that are easy to find in other images when the object is viewed from a different angle or distance) and from these, find the relative positions of where your camera was when you took each picture. Up to this point, it has performed "sparse reconstruction'. After this, it will turn to the CMVS-PMVS software (if you've copied it into the root VisualSFM install folder with the VisualSFM executable) to do dense reconstruction which involves finding many more features from the image data to fill out the point cloud. When you click on the dense reconstruction button, it'll ask you to provide a folder path where you would like it to save the results. Later, we will use Meshlab and open that folder/project. We will use Meshlab to construct a colored mesh around the point cloud. At that point, we'll have a 3D object with colors and texture. Once that's done, we can open it with any modelling software (or the main open source one, Blender), as well as import it into a slicer program for conversion to gcode for 3D printing.

After searching among many videos on Youtube, I found a good one that documents the installation process here:
https://www.youtube.com/watch?v=GEAbXYDzUjU
I will include timestamps into this video for each step below. Note that where my instructions differ it has to do with getting Windows binaries rather than Linux versions.

Final note before proceeding to installation steps: You might note that I am linking the same video over and over again. That's because I'm including the time stamps in the video for that piece of the instructions. If you already have the video up and do not want to open another tab, just look at the end. For instance, if I wanted the link to open the video at the 5 minute 30 second mark, I would add &t=5m30s to the end of the Youtube video link (stand for "time equals 5 minutes, 30 seconds"). You can just progress the video to that point in the tab you already have up if you want to move forward.

Installation:
1. https://www.youtube.com/watch?v=GEAbXYDzUjU&t=2m41s
Download VisualSFM from this page :
http://ccwu.me/vsfm/ (it has multiple OS versions, Windows, Linux, and MacOS, generally you'll want the 64 bit version - if that doesn't work you may have an old computer and need the 32 bit version)
Installing this only consists of downloading the zip file and extracting it (there's no installer - once extracted, the executable is in the extracted folder and ready to run).

Next get CMVS-PMVS:
https://www.youtube.com/watch?v=GEAbXYDzUjU&t=2m52s
https://github.com/pmoulon/CMVS-PMVS
Click on the green button to the top right of the repo file list "Clone or Download", then select the option to clone download as a zip file. Extract the zip file and navigate to the subfolder that has the version for your operating system (for instance, for Windows the path from the top level "CMVS-PMVS-Master" extracted repo zip file to the Win64 version is: CMVS-PMVS-master.zip\CMVS-PMVS-master\binariesWin-Linux\Win64-VS2010). Copy everything from that subfolder into the folder where you put VisualSFM (further questions on how / what files to copy, watch the Youtube video starting at the 4 minute mark).

Next get Meshlab:
https://www.youtube.com/watch?v=GEAbXYDzUjU&t=3m15s
Yes, it's at SourceForge. They are no longer including adware automatically in all downloads. Grab the installer and install it.

Finally, download and install Blender:
https://www.blender.org/download/

Handling pictures:
If you're using your cell phone, you'll want a way to easily get pictures from your phone to your computer so that you can run these apps on them to make the model. I had some initial difficulties, probably due to my phone being encrypted, where when I connected it to my computer, I could not see the phone's internal storage from Windows File Explorer. To overcome that, what I did was upload all the images to Google drive and installed Google Backup and Sync on my computer. Once you've uploaded photos to Google drive, when you sign into Google Backup and sync on the same Google account, it will download all of the files on Google drive to your computer. Its default directory is C:\Users\<YourUserNameHere>\Google Drive.
I didn't do anything else with the files from there, only selected them in VisualSFM from where they were on my computer / where Google Backup and sync put them (see above). You may have to wait a few minutes after uploading for Backup and Sync to download all the pictures to your computer's hard drive. It will happen automatically as long as the app is signed into that Google account.

Running the pipeline:

VisualSFM steps:
https://www.youtube.com/watch?v=GEAbXYDzUjU&t=4m33s
Open VisualSFM by going into the folder and double clicking the executable (the one with the green 'V' icon).

Open the images in VisualSFM:
https://www.youtube.com/watch?v=GEAbXYDzUjU&t=4m45s
Click on the "Open Multiple Images" button (it's third from the left, just under the File tab in VisualSFM).
Navigate to the location you stored your pictures on your computer (if you're using Google Backup and Sync, that's "C:\Users\Your User Name\Google Drive\Whatever you called the image folder"). Select all of the images by left clicking the first one, pressing shift and holding it down while you left click the last one. Finally, click the "open" button on the bottom right of that dialog window (while all the images still highlighted/selected).
Now wait for a bit while it processes all the images. When it is done, you will see mini icons for all of the images in the main view and a summary in the log on the right.

Calculate missing matches:
https://www.youtube.com/watch?v=GEAbXYDzUjU&t=5m10s
Select the "compute missing matches" icon. It's sixth from the right and looks like four arrows pointed at diagonal corners.
Wait until it finishes.

Sparse reconstruction:
https://www.youtube.com/watch?v=GEAbXYDzUjU&t=5m23s (through 5m52s)
Within VisualSFM, select "SfM" from the menu at top (2nd one from left). Within that menu, select "Reconstruct sparse".
Wait until it finishes. You should then see the beginnings of a pointcloud with indicators where the cameras were located when the pictures were taken.
Troubleshooting: Note, at this point when it has finished, if you notice that there are two few camera indicators, it is likely that one of them has a red "X" over it and is overall red. This means that while you were taking your pictures, the locations you took them from were likely too far (too many degrees) apart and it could not "connect the dots" between images. Try taking more images in between where there are more distant gaps in positions between the cameras, add the pictures and go back and do the prior step "calculate missing matches" again. This was happening to me and I got confused before really starting to think about it. I had taken about 20 images around a book on my couch. It wasn't nearly enough. Note that the video author took 70 pictures to capture his Millenium Falcon model. So I went back and took around 40 more (I had 64 when I was done, 5 were duplicates - this process doesn't care about those), being careful to move around the book no more than 15 degrees between images. Note that the time each step of this process takes will increase as the number of pictures goes up. After doing the missing matches step again, this time I had 2080 features instead of just over 100 and 434 seconds versus the 43 seconds it had taken prior (the last time it had stopped after getting confused after 3 pictures - if that step stops after less than a minute, you might have too few pictures). If the "compute missing matches" took enough time like this / you have plenty of pictures and features, you're probably going to be OK with the sparse reconstruction step that comes next. The next dense reconstruction for me after I took the extra pictures took 10 minutes, 55 seconds - note that unless you properly copied CMVS-PMVS into the VisualSFM folder as documented in the CMVS-PMVS installation step above, dense reconstruction will not work.
Other: Make sure that the object you are taking pictures of has good lighting and contrast and there is detail on it. This process might not do well for scenes that don't have enough features/edges/contrast/detail - for instance a blank white wall wouldn't be captured well at all.
For something as plain as a big piece of steel where the surface is very smooth, it might have to be enough to capture all the edges and for it to be dimensionally correct. Otherwise those surfaces will need detail somehow.
(if you want a real challenge, try to do this to your cat when it happens to lay down - the process doesn't work at all for moving subjects, so move quickly - just kidding, probably a bad idea that will frustrate you to no end)

Dense reconstruction (VisualSFM also uses the CMVS-PMVS software here):
https://www.youtube.com/watch?v=GEAbXYDzUjU&t=5m52s
Within VisualSFM, select "SfM" from the menu at top (2nd one from left). Within that menu, select "Reconstruct dense".
It will pop up a dialog asking where it should save the files it creates. Give it a location on your hard drive by navigating, and type a file name you would like the project file to be called (you'll want to remember where you saved it so you can open it in Meshlab later - maybe write the path down).
To see the results, go to View on the menu and select "Dense 3D points" (or just press tab).
That's it, you're done with the VisualSFM and CMVS-PMVS portion of the pipeline. Next we will move on to Meshlab to connect the point cloud surface together with a mesh.

Meshlab:

Build the geometry:
https://www.youtube.com/watch?v=GEAbXYDzUjU&t=7m07s
Open Meshlab and go to the File menu option and select "Open project". Navigate to where you saved the project through the dialog window in the "Dense reconstruction" step in VisualSFM above. Select the project file and wait until it finishes loading - you should see the point cloud.
Go to the View menu option and select "Show Layer Dialog" - you should now see a pane to the right representing layers.
Rotate the point cloud around until you have a good idea where your object is (left click and drag) and what points you can cut.
Make sure that you have a good rotation for the cuts, then click the "Select vertexes" icon from the toolbar.
Click and drag to create boxes to select points you don't want in your 3D model. You can hold control to click and drag again to add more boxes of points to your selection. When you're done, click on the first red X icon (it's an X over connected vertexes) to delete the selected points.
Next, go to the File menu and select "Import mesh". Select the file (the dialog should already be in the correct folder since you loaded the project from there, and there will probably only be one file showing - it will end in the .ply extension) and click "open".
Now you should have a much better view and be able to see which of the remaining points you want to delete. Repeat the select vertexes / delete vertexes step again until you have deleted those.
Once you are happy that you have deleted all the points you want gone, go to the Filters menu, select "Point Set", then select "Surface Reconstructions Poisson". In the dialog window, set the octree depth to 12 and solver divide to 7, leaving the other two settings as is. Click "Apply".
To get rid of the weird bubble that it creates while keeping the modifications to the model/mesh, on layers click on the model and .ply layers to make them invisible, then select the poisson layer (click on the name, not on the icon) highlighting it, then use the "select vertexes" and "delete vertexes" to get rid of the top part of the bubble.
Extra notes: At this point, I went to File, "Export Mesh" and exported the mesh as a .obj file. It was 410KB, not too bad. When I loaded it into a slicer program, nothing came up. I think the origin of the 3D coordinate system for the model may have been too far from the model itself.
Note: I was looking for a way to recenter the origin in Meshlab, but found an easy way to do it in Blender. For now, I'll leave that to a later step in Blender (just in case you need it).

Simplify the geometry:
https://www.youtube.com/watch?v=GEAbXYDzUjU&t=10m11s
Look at the number of faces you have in your layers. If you have more than a million, you should probably do this step the quadric decimation:
Go to the Filters menu, select "Remeshing, Simplification and Reconstruction", then select "Quadric Edge Collapse Decimation" (sounds intimidating and final). Give it a target number of faces of maybe 700,000 and apply.
Next, go to the Filters menu, select "Cleaning and Repairing", and select "Remove Faces from Non Manifold Edges" (this prevents crazy things from happening in the next step where we do texturing).

Generate the Texture:
https://www.youtube.com/watch?v=GEAbXYDzUjU&t=11m06s
Go to the Filters menu, Texture, and select "Parameterization + texturing from registered rasters" (not the one that is named almost exactly the same but without the '+').
In the dialog window, set the Texture size to 8192, change the texture name to what you want, and click apply.
Wait for it to finish, may take a bit. You should be able to see a progress bar in the lower right.

Save the model:
In Meshlab, go to the File menu, select "Export Mesh As", and in the dialog, change the file type to "Alias Wavefront Object (*.obj)" and give it a descriptive name, then click save. In the next dialog box, uncheck "normals" and click OK. Now to make things easier for the later step, copy the texture (.png file) and your latest .obj file into a new folder (name that something descriptive).

Create the scene in Blender:
https://www.youtube.com/watch?v=GEAbXYDzUjU&t=13m04s
Open Blender. Go to File, select new, general.
Import your model - go to File, import, wavefront (.obj). In the dialog, navigate to your model you saved in the "Save the model" step above.
Once it has imported, take a look around the scene by rotating (in Blender, that is by default holding down the middle mouse button while you move the mouse).

If you need to recenter the model in the scene, select the model in the scene then press shift-ctrl-alt-c and select "geometry to origin". The model should move to the center.
If when doing that, the model suddenly disappears, it's probably just inside the gray box that appears by default in Blender at the origin of the scene, which means you succeeded (you can scroll to zoom inside the box and check). You can delete the cube from the scene collection pane at the right by selecting it and selecting "delete".

PhotogrammetryWorkflows/VisualSFM CMVS-PMVS Meshlab

2020-04-04T03:45:24Z

Diginess: Finishing up first draft - I'm still not happy with the result, but I need more expertise in Meshlab to really finalize this product

Note: I (Jacob Johnson, JJ) am currently using Windows. I will try to leave some leeway in these instructions for other OSs by including the overall links where you can find software for those versions, but the most detailed instructions at first will be for Windows versions (installation instructions at least, I think app usage will mostly be similar) until I get my Desktop back online with Ubuntu later and run through the process again for that, then document it.

Before going into the installation steps, here is a breakdown of what this set of software is doing / what we can do with it. This is what I have gathered so far, and it is still loose / rough around the edges. Please feel free to correct any mistakes I've made, or I will adjust as I learn more.
VisualSFM takes in photos that you've taken around a 3D object and then runs an algorithm/program against them (SiftGPU, based on David Lowe's work - Chang Chang Wu wrote SiftGPU and VisualSFM) to find "feature points" (types of features in images that are easy to find in other images when the object is viewed from a different angle or distance) and from these, find the relative positions of where your camera was when you took each picture. Up to this point, it has performed "sparse reconstruction'. After this, it will turn to the CMVS-PMVS software (if you've copied it into the root VisualSFM install folder with the VisualSFM executable) to do dense reconstruction which involves finding many more features from the image data to fill out the point cloud. When you click on the dense reconstruction button, it'll ask you to provide a folder path where you would like it to save the results. Later, we will use Meshlab and open that folder/project. We will use Meshlab to construct a colored mesh around the point cloud. At that point, we'll have a 3D object with colors and texture. Once that's done, we can open it with any modelling software (or the main open source one, Blender), as well as import it into a slicer program for conversion to gcode for 3D printing.

After searching among many videos on Youtube, I found a good one that documents the installation process here:
https://www.youtube.com/watch?v=GEAbXYDzUjU
I will include timestamps into this video for each step below. Note that where my instructions differ it has to do with getting Windows binaries rather than Linux versions.

Final note before proceeding to installation steps: You might note that I am linking the same video over and over again. That's because I'm including the time stamps in the video for that piece of the instructions. If you already have the video up and do not want to open another tab, just look at the end. For instance, if I wanted the link to open the video at the 5 minute 30 second mark, I would add &t=5m30s to the end of the Youtube video link (stand for "time equals 5 minutes, 30 seconds"). You can just progress the video to that point in the tab you already have up if you want to move forward.

Installation:
1. https://www.youtube.com/watch?v=GEAbXYDzUjU&t=2m41s
Download VisualSFM from this page :
http://ccwu.me/vsfm/ (it has multiple OS versions, Windows, Linux, and MacOS, generally you'll want the 64 bit version - if that doesn't work you may have an old computer and need the 32 bit version)
Installing this only consists of downloading the zip file and extracting it (there's no installer - once extracted, the executable is in the extracted folder and ready to run).

Next get CMVS-PMVS:
https://www.youtube.com/watch?v=GEAbXYDzUjU&t=2m52s
https://github.com/pmoulon/CMVS-PMVS
Click on the green button to the top right of the repo file list "Clone or Download", then select the option to clone download as a zip file. Extract the zip file and navigate to the subfolder that has the version for your operating system (for instance, for Windows the path from the top level "CMVS-PMVS-Master" extracted repo zip file to the Win64 version is: CMVS-PMVS-master.zip\CMVS-PMVS-master\binariesWin-Linux\Win64-VS2010). Copy everything from that subfolder into the folder where you put VisualSFM (further questions on how / what files to copy, watch the Youtube video starting at the 4 minute mark).

Next get Meshlab:
https://www.youtube.com/watch?v=GEAbXYDzUjU&t=3m15s
Yes, it's at SourceForge. They are no longer including adware automatically in all downloads. Grab the installer and install it.

Finally, download and install Blender:
https://www.blender.org/download/

Handling pictures:
If you're using your cell phone, you'll want a way to easily get pictures from your phone to your computer so that you can run these apps on them to make the model. I had some initial difficulties, probably due to my phone being encrypted, where when I connected it to my computer, I could not see the phone's internal storage from Windows File Explorer. To overcome that, what I did was upload all the images to Google drive and installed Google Backup and Sync on my computer. Once you've uploaded photos to Google drive, when you sign into Google Backup and sync on the same Google account, it will download all of the files on Google drive to your computer. Its default directory is C:\Users\<YourUserNameHere>\Google Drive.
I didn't do anything else with the files from there, only selected them in VisualSFM from where they were on my computer / where Google Backup and sync put them (see above). You may have to wait a few minutes after uploading for Backup and Sync to download all the pictures to your computer's hard drive. It will happen automatically as long as the app is signed into that Google account.

Running the pipeline:

VisualSFM steps:
https://www.youtube.com/watch?v=GEAbXYDzUjU&t=4m33s
Open VisualSFM by going into the folder and double clicking the executable (the one with the green 'V' icon).

Open the images in VisualSFM:
https://www.youtube.com/watch?v=GEAbXYDzUjU&t=4m45s
Click on the "Open Multiple Images" button (it's third from the left, just under the File tab in VisualSFM).
Navigate to the location you stored your pictures on your computer (if you're using Google Backup and Sync, that's "C:\Users\Your User Name\Google Drive\Whatever you called the image folder"). Select all of the images by left clicking the first one, pressing shift and holding it down while you left click the last one. Finally, click the "open" button on the bottom right of that dialog window (while all the images still highlighted/selected).
Now wait for a bit while it processes all the images. When it is done, you will see mini icons for all of the images in the main view and a summary in the log on the right.

Calculate missing matches:
https://www.youtube.com/watch?v=GEAbXYDzUjU&t=5m10s
Select the "compute missing matches" icon. It's sixth from the right and looks like four arrows pointed at diagonal corners.
Wait until it finishes.

Sparse reconstruction:
https://www.youtube.com/watch?v=GEAbXYDzUjU&t=5m23s (through 5m52s)
Within VisualSFM, select "SfM" from the menu at top (2nd one from left). Within that menu, select "Reconstruct sparse".
Wait until it finishes. You should then see the beginnings of a pointcloud with indicators where the cameras were located when the pictures were taken.
Troubleshooting: Note, at this point when it has finished, if you notice that there are two few camera indicators, it is likely that one of them has a red "X" over it and is overall red. This means that while you were taking your pictures, the locations you took them from were likely too far (too many degrees) apart and it could not "connect the dots" between images. Try taking more images in between where there are more distant gaps in positions between the cameras, add the pictures and go back and do the prior step "calculate missing matches" again. This was happening to me and I got confused before really starting to think about it. I had taken about 20 images around a book on my couch. It wasn't nearly enough. Note that the video author took 70 pictures to capture his Millenium Falcon model. So I went back and took around 40 more (I had 64 when I was done, 5 were duplicates - this process doesn't care about those), being careful to move around the book no more than 15 degrees between images. Note that the time each step of this process takes will increase as the number of pictures goes up. After doing the missing matches step again, this time I had 2080 features instead of just over 100 and 434 seconds versus the 43 seconds it had taken prior (the last time it had stopped after getting confused after 3 pictures - if that step stops after less than a minute, you might have too few pictures). If the "compute missing matches" took enough time like this / you have plenty of pictures and features, you're probably going to be OK with the sparse reconstruction step that comes next. The next dense reconstruction for me after I took the extra pictures took 10 minutes, 55 seconds - note that unless you properly copied CMVS-PMVS into the VisualSFM folder as documented in the CMVS-PMVS installation step above, dense reconstruction will not work.
Other: Make sure that the object you are taking pictures of has good lighting and contrast and there is detail on it. This process might not do well for scenes that don't have enough features/edges/contrast/detail - for instance a blank white wall wouldn't be captured well at all.
For something as plain as a big piece of steel where the surface is very smooth, it might have to be enough to capture all the edges and for it to be dimensionally correct. Otherwise those surfaces will need detail somehow.
(if you want a real challenge, try to do this to your cat when it happens to lay down - the process doesn't work at all for moving subjects, so move quickly - just kidding, probably a bad idea that will frustrate you to no end)

Dense reconstruction (VisualSFM also uses the CMVS-PMVS software here):
https://www.youtube.com/watch?v=GEAbXYDzUjU&t=5m52s
Within VisualSFM, select "SfM" from the menu at top (2nd one from left). Within that menu, select "Reconstruct dense".
It will pop up a dialog asking where it should save the files it creates. Give it a location on your hard drive by navigating, and type a file name you would like the project file to be called (you'll want to remember where you saved it so you can open it in Meshlab later - maybe write the path down).
To see the results, go to View on the menu and select "Dense 3D points" (or just press tab).
That's it, you're done with the VisualSFM and CMVS-PMVS portion of the pipeline. Next we will move on to Meshlab to connect the point cloud surface together with a mesh.

Meshlab:

Build the geometry:
https://www.youtube.com/watch?v=GEAbXYDzUjU&t=7m07s
Open Meshlab and go to the File menu option and select "Open project". Navigate to where you saved the project through the dialog window in the "Dense reconstruction" step in VisualSFM above. Select the project file and wait until it finishes loading - you should see the point cloud.
Go to the View menu option and select "Show Layer Dialog" - you should now see a pane to the right representing layers.
Rotate the point cloud around until you have a good idea where your object is (left click and drag) and what points you can cut.
Make sure that you have a good rotation for the cuts, then click the "Select vertexes" icon from the toolbar.
Click and drag to create boxes to select points you don't want in your 3D model. You can hold control to click and drag again to add more boxes of points to your selection. When you're done, click on the first red X icon (it's an X over connected vertexes) to delete the selected points.
Next, go to the File menu and select "Import mesh". Select the file (the dialog should already be in the correct folder since you loaded the project from there, and there will probably only be one file showing - it will end in the .ply extension) and click "open".
Now you should have a much better view and be able to see which of the remaining points you want to delete. Repeat the select vertexes / delete vertexes step again until you have deleted those.
Once you are happy that you have deleted all the points you want gone, go to the Filters menu, select "Point Set", then select "Surface Reconstructions Poisson". In the dialog window, set the octree depth to 12 and solver divide to 7, leaving the other two settings as is. Click "Apply".
To get rid of the weird bubble that it creates while keeping the modifications to the model/mesh, on layers click on the model and .ply layers to make them invisible, then select the poisson layer (click on the name, not on the icon) highlighting it, then use the "select vertexes" and "delete vertexes" to get rid of the top part of the bubble.
Extra notes: At this point, I went to File, "Export Mesh" and exported the mesh as a .obj file. It was 410KB, not too bad. When I loaded it into a slicer program, nothing came up. I think the origin of the 3D coordinate system for the model may have been too far from the model itself.
Note: I was looking for a way to recenter the origin in Meshlab, but found an easy way to do it in Blender. For now, I'll leave that to a later step in Blender (just in case you need it).

Simplify the geometry:
https://www.youtube.com/watch?v=GEAbXYDzUjU&t=10m11s
Look at the number of faces you have in your layers. If you have more than a million, you should probably do this step the quadric decimation:
Go to the Filters menu, select "Remeshing, Simplification and Reconstruction", then select "Quadric Edge Collapse Decimation" (sounds intimidating and final). Give it a target number of faces of maybe 700,000 and apply.
Next, go to the Filters menu, select "Cleaning and Repairing", and select "Remove Faces from Non Manifold Edges" (this prevents crazy things from happening in the next step where we do texturing).

Generate the Texture:
https://www.youtube.com/watch?v=GEAbXYDzUjU&t=11m06s
Go to the Filters menu, Texture, and select "Parameterization + texturing from registered rasters" (not the one that is named almost exactly the same but without the '+').
In the dialog window, set the Texture size to 8192, change the texture name to what you want, and click apply.
Wait for it to finish, may take a bit. You should be able to see a progress bar in the lower right.

Save the model:
In Meshlab, go to the File menu, select "Export Mesh As", and in the dialog, change the file type to "Alias Wavefront Object (*.obj)" and give it a descriptive name, then click save. In the next dialog box, uncheck "normals" and click OK. Now to make things easier for the later step, copy the texture (.png file) and your latest .obj file into a new folder (name that something descriptive).

Opening the object in Blender:
Open Blender. Go to File, select new, general.
Import your model - go to File, import, wavefront (.obj). In the dialog, navigate to your model you saved in the "Save the model" step above.
Once it has imported, take a look around the scene by rotating (in Blender, that is by default holding down the middle mouse button while you move the mouse). If you need to recenter the model in the scene, press the following all at once, see this:
https://blender.stackexchange.com/questions/14294/how-to-recenter-an-objects-origin
If when doing that, the model suddenly disappears, it's probably just inside the gray box that appears by default in Blender at the origin of the scene, which means you succeeded (you can scroll to zoom inside the box and check). You can delete the cube from the scene collection pane at the right by selecting it and selecting "delete".

PhotogrammetryWorkflows/VisualSFM CMVS-PMVS Meshlab

2020-04-04T02:14:00Z

Diginess: Continuing on this pipeline, steps for Meshlab. Saving so I don't lose the work.

Note: I (Jacob Johnson, JJ) am currently using Windows. I will try to leave some leeway in these instructions for other OSs by including the overall links where you can find software for those versions, but the most detailed instructions at first will be for Windows versions (installation instructions at least, I think app usage will mostly be similar) until I get my Desktop back online with Ubuntu later and run through the process again for that, then document it.

Before going into the installation steps, here is a breakdown of what this set of software is doing / what we can do with it. This is what I have gathered so far, and it is still loose / rough around the edges. Please feel free to correct any mistakes I've made, or I will adjust as I learn more.
VisualSFM takes in photos that you've taken around a 3D object and then runs an algorithm/program against them (SiftGPU, based on David Lowe's work - Chang Chang Wu wrote SiftGPU and VisualSFM) to find "feature points" (types of features in images that are easy to find in other images when the object is viewed from a different angle or distance) and from these, find the relative positions of where your camera was when you took each picture. Up to this point, it has performed "sparse reconstruction'. After this, it will turn to the CMVS-PMVS software (if you've copied it into the root VisualSFM install folder with the VisualSFM executable) to do dense reconstruction which involves finding many more features from the image data to fill out the point cloud. When you click on the dense reconstruction button, it'll ask you to provide a folder path where you would like it to save the results. Later, we will use Meshlab and open that folder/project. We will use Meshlab to construct a colored mesh around the point cloud. At that point, we'll have a 3D object with colors, but no texture (if you want fancy / any textures at all, that's not part of this process yet). Once that's done, we can open it with any modelling software (or the main open source one, Blender), as well as import it into a splicer program for conversion to gcode for 3D printing.

After searching among many videos on Youtube, I found a good one that documents the installation process here:
https://www.youtube.com/watch?v=GEAbXYDzUjU
I will include timestamps into this video for each step below. Note that where my instructions differ it has to do with getting Windows binaries rather than Linux versions.

Final note before proceeding to installation steps: You might note that I am linking the same video over and over again. That's because I'm including the time stamps in the video for that piece of the instructions. If you already have the video up and do not want to open another tab, just look at the end. For instance, if I wanted the link to open the video at the 5 minute 30 second mark, I would add &t=5m30s to the end of the Youtube video link (stand for "time equals 5 minutes, 30 seconds"). You can just progress the video to that point in the tab you already have up if you want to move forward.

Installation:
1. https://www.youtube.com/watch?v=GEAbXYDzUjU&t=2m41s
Download VisualSFM from this page :
http://ccwu.me/vsfm/ (it has multiple OS versions, Windows, Linux, and MacOS, generally you'll want the 64 bit version - if that doesn't work you may have an old computer and need the 32 bit version)
Installing this only consists of downloading the zip file and extracting it (there's no installer - once extracted, the executable is in the extracted folder and ready to run).

Next get CMVS-PMVS:
https://www.youtube.com/watch?v=GEAbXYDzUjU&t=2m52s
https://github.com/pmoulon/CMVS-PMVS
Click on the green button to the top right of the repo file list "Clone or Download", then select the option to clone download as a zip file. Extract the zip file and navigate to the subfolder that has the version for your operating system (for instance, for Windows the path from the top level "CMVS-PMVS-Master" extracted repo zip file to the Win64 version is: CMVS-PMVS-master.zip\CMVS-PMVS-master\binariesWin-Linux\Win64-VS2010). Copy everything from that subfolder into the folder where you put VisualSFM (further questions on how / what files to copy, watch the Youtube video starting at the 4 minute mark).

Next get Meshlab:
https://www.youtube.com/watch?v=GEAbXYDzUjU&t=3m15s
Yes, it's at SourceForge. They are no longer including adware automatically in all downloads. Grab the installer and install it.

Handling pictures:
If you're using your cell phone, you'll want a way to easily get pictures from your phone to your computer so that you can run these apps on them to make the model. I had some initial difficulties, probably due to my phone being encrypted, where when I connected it to my computer, I could not see the phone's internal storage from Windows File Explorer. To overcome that, what I did was upload all the images to Google drive and installed Google Backup and Sync on my computer. Once you've uploaded photos to Google drive, when you sign into Google Backup and sync on the same Google account, it will download all of the files on Google drive to your computer. Its default directory is C:\Users\<YourUserNameHere>\Google Drive.
I didn't do anything else with the files from there, only selected them in VisualSFM from where they were on my computer / where Google Backup and sync put them (see above). You may have to wait a few minutes after uploading for Backup and Sync to download all the pictures to your computer's hard drive. It will happen automatically as long as the app is signed into that Google account.

Running the pipeline:

VisualSFM steps:
https://www.youtube.com/watch?v=GEAbXYDzUjU&t=4m33s
Open VisualSFM by going into the folder and double clicking the executable (the one with the green 'V' icon).

Open the images in VisualSFM:
https://www.youtube.com/watch?v=GEAbXYDzUjU&t=4m45s
Click on the "Open Multiple Images" button (it's third from the left, just under the File tab in VisualSFM).
Navigate to the location you stored your pictures on your computer (if you're using Google Backup and Sync, that's "C:\Users\Your User Name\Google Drive\Whatever you called the image folder"). Select all of the images by left clicking the first one, pressing shift and holding it down while you left click the last one. Finally, click the "open" button on the bottom right of that dialog window (while all the images still highlighted/selected).
Now wait for a bit while it processes all the images. When it is done, you will see mini icons for all of the images in the main view and a summary in the log on the right.

Calculate missing matches:
https://www.youtube.com/watch?v=GEAbXYDzUjU&t=5m10s
Select the "compute missing matches" icon. It's sixth from the right and looks like four arrows pointed at diagonal corners.
Wait until it finishes.

Sparse reconstruction:
https://www.youtube.com/watch?v=GEAbXYDzUjU&t=5m23s (through 5m52s)
Within VisualSFM, select "SfM" from the menu at top (2nd one from left). Within that menu, select "Reconstruct sparse".
Wait until it finishes. You should then see the beginnings of a pointcloud with indicators where the cameras were located when the pictures were taken.
Troubleshooting: Note, at this point when it has finished, if you notice that there are two few camera indicators, it is likely that one of them has a red "X" over it and is overall red. This means that while you were taking your pictures, the locations you took them from were likely too far (too many degrees) apart and it could not "connect the dots" between images. Try taking more images in between where there are more distant gaps in positions between the cameras, add the pictures and go back and do the prior step "calculate missing matches" again. This was happening to me and I got confused before really starting to think about it. I had taken about 20 images around a book on my couch. It wasn't nearly enough. Note that the video author took 70 pictures to capture his Millenium Falcon model. So I went back and took around 40 more (I had 64 when I was done, 5 were duplicates - this process doesn't care about those), being careful to move around the book no more than 15 degrees between images. Note that the time each step of this process takes will increase as the number of pictures goes up. After doing the missing matches step again, this time I had 2080 features instead of just over 100 and 434 seconds versus the 43 seconds it had taken prior (the last time it had stopped after getting confused after 3 pictures - if that step stops after less than a minute, you might have too few pictures). If the "compute missing matches" took enough time like this / you have plenty of pictures and features, you're probably going to be OK with the sparse reconstruction step that comes next. The next dense reconstruction for me after I took the extra pictures took 10 minutes, 55 seconds - note that unless you properly copied CMVS-PMVS into the VisualSFM folder as documented in the CMVS-PMVS installation step above, dense reconstruction will not work.
Other: Make sure that the object you are taking pictures of has good lighting and contrast and there is detail on it. This process might not do well for scenes that don't have enough features/edges/contrast/detail - for instance a blank white wall wouldn't be captured well at all.
For something as plain as a big piece of steel where the surface is very smooth, it might have to be enough to capture all the edges and for it to be dimensionally correct. Otherwise those surfaces will need detail somehow.
(if you want a real challenge, try to do this to your cat when it happens to lay down - the process doesn't work at all for moving subjects, so move quickly - just kidding, probably a bad idea that will frustrate you to no end)

Dense reconstruction (VisualSFM also uses the CMVS-PMVS software here):
https://www.youtube.com/watch?v=GEAbXYDzUjU&t=5m52s
Within VisualSFM, select "SfM" from the menu at top (2nd one from left). Within that menu, select "Reconstruct dense".
It will pop up a dialog asking where it should save the files it creates. Give it a location on your hard drive by navigating, and type a file name you would like the project file to be called (you'll want to remember where you saved it so you can open it in Meshlab later - maybe write the path down).
To see the results, go to View on the menu and select "Dense 3D points" (or just press tab).
That's it, you're done with the VisualSFM and CMVS-PMVS portion of the pipeline. Next we will move on to Meshlab to connect the point cloud surface together with a mesh.

Meshlab:

Build the geometry:
https://www.youtube.com/watch?v=GEAbXYDzUjU&t=7m07s
Open Meshlab and go to the File menu option and select "Open project". Navigate to where you saved the project through the dialog window in the "Dense reconstruction" step in VisualSFM above. Select the project file and wait until it finishes loading - you should see the point cloud.
Go to the View menu option and select "Show Layer Dialog" - you should now see a pane to the right representing layers.
Rotate the point cloud around until you have a good idea where your object is (left click and drag) and what points you can cut.
Make sure that you have a good rotation for the cuts, then click the "Select vertexes" icon from the toolbar.
Click and drag to create boxes to select points you don't want in your 3D model. You can hold control to click and drag again to add more boxes of points to your selection. When you're done, click on the first red X icon (it's an X over connected vertexes) to delete the selected points.
Next, go to the File menu and select "Import mesh". Select the file (the dialog should already be in the correct folder since you loaded the project from there, and there will probably only be one file showing - it will end in the .ply extension) and click "open".
Now you should have a much better view and be able to see which of the remaining points you want to delete. Repeat the select vertexes / delete vertexes step again until you have deleted those.
Once you are happy that you have deleted all the points you want gone, go to the Filters menu, select "Point Set", then select "Surface Reconstructions Poisson". In the dialog window, set the octree depth to 12 and solver divide to 7, leaving the other two settings as is. Click "Apply".
To get rid of the weird bubble that it creates while keeping the modifications to the model/mesh, on layers click on the model and .ply layers to make them invisible, then select the poisson layer (click on the name, not on the icon) highlighting it, then use the "select vertexes" and "delete vertexes" to get rid of the top part of the bubble.

Simplify the geometry:
https://www.youtube.com/watch?v=GEAbXYDzUjU&t=10m11s

PhotogrammetryWorkflows/VisualSFM CMVS-PMVS Meshlab

2020-04-04T00:26:49Z

Diginess: Troubleshooting detail in case of too many gaps in between pictures - how to tell early in the process

PhotogrammetryWorkflows/VisualSFM CMVS-PMVS Meshlab

2020-04-03T23:27:44Z

Diginess: Lots of min edits, adding steps in the VisualSFM portion of the pipeline.

PhotogrammetryWorkflows/VisualSFM CMVS-PMVS Meshlab

2020-04-03T23:10:37Z

Diginess: Summary and installation instructions for open source software in the VisualSFM, CMVS-PMVS, Meshlab, Blender photogrammetry pipeline - saving so I don't lose progress

Note: I (Jacob Johnson, JJ) am currently working on Windows. I will try to leave some leeway in these instructions for other OSs by including the overall links where you can find software for those versions, but the most detailed instructions at first will be Windows until I get my Desktop back online with Ubuntu later and run through the process again for that, then document it.

Before going into the installation steps, here is a breakdown of what this set of software is doing / what we can do with it. This is what I have gathered so far, and it is still loose / rough around the edges. Please feel free to correct any mistakes I've made, or I will adjust as I learn more.
VisualSFM takes in photos that you've taken around a 3D object and then runs an algorithm/program against them (SiftGPU, based on David Lowe's work - Chang Chang Wu wrote SiftGPU and VisualSFM) to find "feature points" (types of features in images that are easy to find in other images when the object is viewed from a different angle or distance) and from these, find the relative positions of where your camera was when you took each picture. Up to this point, it has performed "sparse reconstruction'. After this, it will turn to the CMVS-PMVS software (if you've copied it into the root VisualSFM install folder with the VisualSFM executable) to do dense reconstruction which involves finding many more features from the image data to fill out the point cloud. When you click on the dense reconstruction button, it'll ask you to provide a folder path where you would like it to save the results. Later, we will use Meshlab and open that folder/project. We will use Meshlab to construct a colored mesh around the point cloud. At that point, we'll have a 3D object with colors, but no texture (if you want fancy / any textures at all, that's not part of this process yet). Once that's done, we can open it with any modelling software (or the main open source one, Blender), as well as import it into a splicer program for conversion to gcode for 3D printing.

After searching among many videos on Youtube, I found a good one that documents the installation process here:
https://www.youtube.com/watch?v=GEAbXYDzUjU
I will include timestamps into this video for each step below. Note that where my instructions differ it has to do with getting Windows binaries rather than Linux versions.

Installation:
1. https://www.youtube.com/watch?v=GEAbXYDzUjU&t=2m41s
Download VisualSFM from this page :
http://ccwu.me/vsfm/ (it has multiple OS versions, Windows, Linux, and MacOS, generally you'll want the 64 bit version - if that doesn't work you may have an old computer and need the 32 bit version)
Installing this only consists of downloading the zip file and extracting it (there's no installer - once extracted, the executable is in the extracted folder and ready to run).

Next get CMVS-PMVS:
https://www.youtube.com/watch?v=GEAbXYDzUjU&t=2m52s
https://github.com/pmoulon/CMVS-PMVS
Click on the green button to the top right of the repo file list "Clone or Download", then select the option to clone download as a zip file. Extract the zip file and navigate to the subfolder that has the version for your operating system (for instance, for Windows the path from the top level "CMVS-PMVS-Master" extracted repo zip file to the Win64 version is: CMVS-PMVS-master.zip\CMVS-PMVS-master\binariesWin-Linux\Win64-VS2010). Copy everything from that subfolder into the folder where you put VisualSFM (further questions on how / what files to copy, watch the Youtube video starting at the 4 minute mark).

Next get Meshlab:
https://www.youtube.com/watch?v=GEAbXYDzUjU&t=3m15s
Yes, it's at SourceForge. They are no longer including adware automatically in all downloads. Grab the installer and install it.

Handling pictures:
If you're using your cell phone, you'll want a way to easily get pictures from your phone to your computer so that you can run these apps on them to make the model. I had some initial difficulties, probably due to my phone being encrypted, where when I connected it to my computer, I could not see the phone's internal storage from Windows File Explorer. To overcome that, what I did was upload all the images to Google drive and installed Google Backup and Sync on my computer. Once you've uploaded photos to Google drive, when you sign into Google Backup and sync on the same Google account, it will download all of the files on Google drive to your computer. Its default directory is C:\Users\<YourUserNameHere>\Google Drive.
I didn't do anything else with the files from there, only selected them in VisualSFM from where they were on my computer / where Google Backup and sync put them (see above). You may have to wait a few minutes after uploading for Backup and Sync to download all the pictures to your computer's hard drive. It will happen automatically as long as the app is signed into that Google account.

Running the pipeline:

VisualSFM steps:
https://www.youtube.com/watch?v=GEAbXYDzUjU&t=4m33s
Open VisualSFM by going into the folder and double clicking the executable (the one with the green 'V' icon).

Open the images in VisualSFM:
Click on the "Open Multiple Images" button (it's third from the left, just under the File tab in VisualSFM).
Navigate to the location you stored your pictures on your computer (if you're using Google Backup and Sync, that's "C:\Users\Your User Name\Google Drive\Whatever you called the image folder"). Select all of the images by left clicking the first one, pressing shift and holding it down while you left click the last one. Finally, click the "open" button on the bottom right of that dialog window (while all the images still highlighted/selected).

Ultrasonic Distance Sensor

2017-04-20T16:51:55Z

Diginess: /* R&D */

=Requirements=
*1mm accuracy optimal, 3 mm is absolute max
*Open source version of http://www.cnccontrols.com/thcontrol.asp - 20-250 mm range for pointing to metal, with no obstructions

=Tech Support=
*Maxbotics - [http://www.maxbotix.com/contact.htm]. 2' has under 1% error.
*Sparkfun has them - [https://www.sparkfun.com/categories/84]. Online chat at https://www.sparkfun.com/technical_assistance
*Banner Engineering - $121 - [http://www.plccenter.com/Shop/BANNER%20ENGINEERING/S18UBAQ/USPP?source=GoogleShopping&IgnoreRedirect=true&gclid=CO2y_4is3s4CFQuNaQod8TsBEw&kwid=productads-adid^113089519769-device^c-plaid^93272993125-sku^15000003399-adType^PLA]
*Xenix - [https://senix.com/ultrasonic-sensor-toughsonic-3/]

=YouTube=
*Ultrasonic sensor on plasma table - [https://www.youtube.com/watch?v=pZJozdF_OaE]

=R&D=
*From EMC list [http://info.bannersalesforce.com/cs/groups/public/documents/literature/110738.pdf] - Banner Engineering sensor PDF - 1-11" range - [http://info.bannersalesforce.com/cs/groups/public/documents/literature/110738.pdf]
*Making a better HC-SR04 distance sensor - Emil - [http://uglyduck.ath.cx/ep/archive/2014/01/Making_a_better_HC_SR04_Echo_Locator.html]

Here is a good starting template Arduino sketch (code measuring distance) for the HC-SR04: https://www.arduino.cc/en/tutorial/ping
Also present on the top of that page is an accurate visual circuit diagram of how the wires should be connected to work with the sketch.
Connect the USB / serial cable from your computer to the Arduino, go to the Tools menu, select the port that you will use. I am using the Genuino Arduino IDE - the official IDE developed by the Arduino team.
Select the arrow under the File menu option to upload the sketch to the Arduino. If you have an error message, it is likely you selected the wrong port (also verify you have a light on the Arduino, an LED should light up if the cable is connected) or you may not have run the IDE with administrator rights (on Linux you will likely have to use "sudo"). If you are using Ubuntu, here is the page I used to look up the error I had: https://askubuntu.com/questions/646254/arduino-serial-port-com1-problem

Once you are able to upload the sketch to the Arduino without an error, it's actually running right away. In order to see the output of the program, go to the Tools menu and select "serial monitor". A window will come up where you can see the instantaneous distance measurements scrolling.

The code templates available are not actually giving very accurate readings - typically they print out the distance in centimeters, or even worse, inches. We need millimeters. In order to get millimeters, we have to modify the code and add another division to go from cm to mm (divide by 100). However, this is not enough if it'll only ever return the same thing it did but listed as in millimeters.

Let's first review some physics: http://learn.parallax.com/support/reference/speed-sound-air-vs-temperature
Short summary: sound leaves the ultrasonic wavelength speaker on the sensor, bounces off whatever surface it hits, and hopefully comes back toward the microphone. The Arduino measures the time between when the sound leaves the speaker and when it is detected by the microphone. It uses the following equation to convert from the delay time to the calculated distance. Note that the speed of sound in air is dependent on temperature more than anything else. A difference of 10 degrees Fahrenheit will change the speed of sound by approx 4.64 meters per second, which will affect the calculated distance as well. In other words, we are going to have to include temperature in the equation, and we should enter the temperature or get it using another sensor before depending on the readings for precise measurements.

Here is the equation:
Let C be speed of sound in air
Let Tc be temperature in degrees Celsius
Let Tf be temperature in degrees Fahrenheit

The starting equation is in Celsius (from the link above):
C = 331.5 + (0.6 * Tc) m/s

To convert to Tf, we use this equation: Tc = (Tf - 32) / 1.8
and substitute that for Tc in the first equation:
C = 331.5 + (0.6 * (Tf - 32) / 1.8) m/s

Simplified:
C = 331.5 + ((0.6/1.8) * (Tf - 32) = 331.5 + (1/3)*Tf - 32/3
C = 320.83333 + 1/3Tf

-----OK, now for the code to go along with the above - open a new Arduino sketch and paste this code in:--------------

/* Ping))) Sensor

This sketch reads a PING))) ultrasonic rangefinder and returns the
distance to the closest object in range. To do this, it sends a pulse
to the sensor to initiate a reading, then listens for a pulse
to return. The length of the returning pulse is proportional to
the distance of the object from the sensor.

The circuit:
* +V connection of the PING))) attached to +5V
* GND connection of the PING))) attached to ground
* SIG connection of the PING))) attached to digital pin 7

http://www.arduino.cc/en/Tutorial/Ping

created 3 Nov 2008
by David A. Mellis
modified 30 Aug 2011
by Tom Igoe
modified 20 Apr 2017
by Jacob Johnson

This example code is in the public domain.

*/

// this constant won't change. It's the pin number
// of the sensor's output:
const int pingPin = 7;

long duration = 0;

//Change this to the current temp before running the sensor.
//We will get this from a temp sensor later.
const int tempF = 78; //In degres Fahrenheit

void setup() {
// initialize serial communication:
Serial.begin(9600);
}

void loop() {
//convert the time into a distance
//inches = microsecondsToInches(duration);
//cm = microsecondsToCentimeters(duration);

float sum = 0.0000;
int counter = 0;
while (counter < 20){
duration = pingAndGetDurationFromEcho();
float mm = microsecondsToMillimeters(duration, tempF);
sum += mm;
counter++;
}

float averageMillimeters = sum / 20;

Serial.print("duration: ");
Serial.print(duration);
Serial.print(", millimeters: ");
Serial.print(averageMillimeters);
Serial.print("\n");

delay(100);
}

long pingAndGetDurationFromEcho(){
// The PING))) is triggered by a HIGH pulse of 2 or more microseconds.
// Give a short LOW pulse beforehand to ensure a clean HIGH pulse:
pinMode(pingPin, OUTPUT);
digitalWrite(pingPin, LOW);
delayMicroseconds(2);
digitalWrite(pingPin, HIGH);
delayMicroseconds(5);
digitalWrite(pingPin, LOW);

// The same pin is used to read the signal from the PING))): a HIGH
// pulse whose duration is the time (in microseconds) from the sending
// of the ping to the reception of its echo off of an object.
pinMode(pingPin, INPUT);
duration = pulseIn(pingPin, HIGH);

return duration;
}

long microsecondsToInches(long microseconds) {
// According to Parallax's datasheet for the PING))), there are
// 73.746 microseconds per inch (i.e. sound travels at 1130 feet per
// second). This gives the distance travelled by the ping, outbound
// and return, so we divide by 2 to get the distance of the obstacle.
// See: http://www.parallax.com/dl/docs/prod/acc/28015-PING-v1.3.pdf
return microseconds / 74 / 2;
}

long microsecondsToCentimeters(long microseconds) {
// The speed of sound is 340 m/s or 29 microseconds per centimeter.
// The ping travels out and back, so to find the distance of the
// object we take half of the distance travelled.
return microseconds / 29 / 2;
}

//All the math for calculating distance in millimeters, starting
//from the equations at:
//http://learn.parallax.com/support/reference/speed-sound-air-vs-temperature
float microsecondsToMillimeters(long microseconds, int temperature) {
//The above calculations do not take temperature into account.
//A 10F difference causes a 4.64 m/s difference in sound velocity.
//long metersPerSecond = 331.5 + (0.6 * (temperature - 32)/1.8);
long metersPerSecond = 320.8334 + (0.333 * temperature);
float millimetersPerSecond = metersPerSecond * 1000.0000;
float millimetersPerMicroSecond = millimetersPerSecond / 1000000.00000;
float millimeters = millimetersPerMicroSecond * microseconds;
return millimeters;
}

=Selection=
*Maxbotics - 1 mm resolution options, not water protected - 12" minimum distance - $40 - [http://www.maxbotix.com/Ultrasonic_Sensors/MB1013.htm]. Adafruit has it.
*Microptik - 3 mm resolution - http://www.micropik.com/PDF/HCSR04.pdf - 2-400 cm range - services@elecfreaks.com. Sample Arduino sketch - [http://www.elecfreaks.com/244.html]
*HC-SR04 - appears to get 3 mm accuracy - [https://www.youtube.com/watch?v=-1SDVVepaP8]. Sparkfun has the sensor + code - [https://codebender.cc/sketch:356078#HC-SR04%20Ultrasonic%20Sensor%20Example.ino]
:*Video shows 15 cm measurement, with oscillation +- 1mm - and text says well within 2% accuracy. 2% of 15 cm is 3 mm - so this would do for measurements of torch height for 1/2" steel easily (12 mm thickness).

=Comparisons=
*http://www.robotshop.com/en/compare-infrared-ultrasonic-distance-sensors.html

=Explanation=
*Nice video - [https://www.youtube.com/watch?v=aLkkAsrSibo]

=Sourcing=
*VexIQ - sensor for robots - [http://www.robotmesh.com/vexiq-distance-sensor?gclid=CMKUrNml3s4CFYGHaQoduNsEEw]
*4-30 cm - $99 - [http://www.automationdirect.com/adc/Shopping/Catalog/Sensors_-z-_Encoders/Ultrasonic_Proximity_Sensors/18mm_Round,_400mm_Maximum_Sensing_Distance_(UK_Series)/UK6A-D1-0A?utm_source=google&utm_medium=product-search&gclid=CNHMntmD3s4CFQuLaQod-MkCMw]
*1-100 inch - $25 - [http://www.robotmesh.com/vexiq-distance-sensor?gclid=COfEnNiD3s4CFQGsaQodUHsC1A]
*$4 Sparkfun - 6"-10' - [https://www.sparkfun.com/products/13959]
*$6 - .5 cm accuracy - [https://www.amazon.com/Arrela%C2%AE-Hc-sr04-Ultrasonic-Distance-Measuring/dp/B00KKKT7YK]

=Links=
*[[Ultrasonic Sensor Log]]

CNC

2016-09-25T02:48:42Z

Diginess: /* Controlling ultrasonic sensor from Arduino */

=Summary=
Computer numerical control, a computer controller that reads instructions and drives a machine tool.

In modern CNC systems, end-to-end component design is highly automated using computer-aided design ([[CAD]]) and computer-aided manufacturing ([[CAM]]) programs. The programs produce a computer file that is interpreted to extract the commands needed to operate a particular machine via a postprocessor, and then loaded into the CNC machines for production. Since any particular component might require the use of a number of different tools-drills, saws, etc., modern machines often combine multiple tools into a single "cell". In other cases, a number of different machines are used with an external controller and human or robotic operators that move the component from machine to machine. In either case, the complex series of steps needed to produce any part is highly automated and produces a part that closely matches the original CAD design.

[[File: cnctoolchain.svg|500px]]

Use the [[Processing]] [[IDE]] to open the [[Gctrl]] [[GUI]]

Then you can send [[CAM]] [[Electrical Information]] from your [[Computer]] via its [[USB]] port to the [[Arduino]] [[Microcontroller]]

The [[Electrical Information]] will be interpreted by the [[GRBL]] software and then you will get processed [[Electrical Information]] as the output

You can configure the [[GRBL]] settings using the [[Arduino]] [[IDE]]'s [[Serial Monitor]]

=Modules=

[[Processing]]

[[Gctrl]]

[http://www.amazon.com/Arduino-Rev-3-Uno-R3/dp/B006H06TVG/ref=sr_1_1?ie=UTF8&qid=1339352565&sr=8-1 Arduino Uno R3 from Amazon]

[[Arduino]]

[[RXTX]]

[[GRBL]]

[[GRBL Config]]

=Notes=

sudo apt-get install librxtx-java

arduino IDE Serial Monitor: carriage return

sudo apt-get install openjdk-6-jre

----

sudo ./arduino

sudo ./processing

=Fedora Notes (for install of the Arduino development environment)=
Type this command (without quotations) to install the development environment for the Arduino:

"sudo dnf install arduino"

To start the environment:

"arduino"

=Controlling ultrasonic sensor from Arduino=
Diginess's notes: I bought this: https://www.radioshack.com/products/radioshack-ultrasonic-range-sensor from Radio Shack ($30) - have some on order from Amazon, but ran out of time.

The wiring is very simple - I used a solderless prototyping board and followed the instructions on the manual (page 4): https://github.com/RadioShackCorp/2760342-Range-Sensor/blob/master/2760342_PM_EN.pdf

Note that the picture shows the view from the sensor's front - I initially wired it backwards, but there are only 4 wires. If you get 0s showing for distance measurements, you probably reversed the signal wire and ground.

Here's the code to paste in the development environment: https://github.com/RadioShackCorp/2760342-Range-Sensor/blob/master/UltrasonicRangeSensor.ino

Plug the Arduino into your computer with a USB A-B cable. Wait for initialization on the Arduino.

From the environment (after code has been pasted), go to the File menu and select Upload. Wait for the LEDs on the Arduino to stop flashing, indicating code upload is complete.

Next, go to the Tools menu option, and select "Serial Monitor". The code will immediately be ran on the Arduino, and you will see a scrolling log of the results (distance to object in front of the sensor).

==Research==
*[[CNC Software]]
*[[Stepper Motor]]

==Product Ecology==
{{3D Printer}}
{{Circuit Mill}}
{{Multimachine}}
{{Torch Table}}

==See Also==
*[[Digital Fabrication]]
*[[Vinyl Cutter]]
*[http://en.wikipedia.org/wiki/Cnc Wikipedia: CNC]

[[Category: Links]]

CNC

2016-09-25T02:47:43Z

Diginess: /* Fedora Notes (for install of the Arduino development environment) */

=Summary=
Computer numerical control, a computer controller that reads instructions and drives a machine tool.

In modern CNC systems, end-to-end component design is highly automated using computer-aided design ([[CAD]]) and computer-aided manufacturing ([[CAM]]) programs. The programs produce a computer file that is interpreted to extract the commands needed to operate a particular machine via a postprocessor, and then loaded into the CNC machines for production. Since any particular component might require the use of a number of different tools-drills, saws, etc., modern machines often combine multiple tools into a single "cell". In other cases, a number of different machines are used with an external controller and human or robotic operators that move the component from machine to machine. In either case, the complex series of steps needed to produce any part is highly automated and produces a part that closely matches the original CAD design.

[[File: cnctoolchain.svg|500px]]

Use the [[Processing]] [[IDE]] to open the [[Gctrl]] [[GUI]]

Then you can send [[CAM]] [[Electrical Information]] from your [[Computer]] via its [[USB]] port to the [[Arduino]] [[Microcontroller]]

The [[Electrical Information]] will be interpreted by the [[GRBL]] software and then you will get processed [[Electrical Information]] as the output

You can configure the [[GRBL]] settings using the [[Arduino]] [[IDE]]'s [[Serial Monitor]]

=Modules=

[[Processing]]

[[Gctrl]]

[http://www.amazon.com/Arduino-Rev-3-Uno-R3/dp/B006H06TVG/ref=sr_1_1?ie=UTF8&qid=1339352565&sr=8-1 Arduino Uno R3 from Amazon]

[[Arduino]]

[[RXTX]]

[[GRBL]]

[[GRBL Config]]

=Notes=

sudo apt-get install librxtx-java

arduino IDE Serial Monitor: carriage return

sudo apt-get install openjdk-6-jre

----

sudo ./arduino

sudo ./processing

=Fedora Notes (for install of the Arduino development environment)=
Type this command (without quotations) to install the development environment for the Arduino:

"sudo dnf install arduino"

To start the environment:

"arduino"

=Controlling ultrasonic sensor from Arduino=
Diginess's notes: I bought this: https://www.radioshack.com/products/radioshack-ultrasonic-range-sensor from Radio Shack ($30) - have some on order from Amazon, but ran out of time.
The wiring is very simple - I used a solderless prototyping board and followed the instructions on the manual (page 4): https://github.com/RadioShackCorp/2760342-Range-Sensor/blob/master/2760342_PM_EN.pdf
Note that the picture shows the view from the sensor's front - I initially wired it backwards, but there are only 4 wires. If you get 0s showing for distance measurements, you probably reversed the signal wire and ground.
Here's the code to paste in the development environment: https://github.com/RadioShackCorp/2760342-Range-Sensor/blob/master/UltrasonicRangeSensor.ino
Plug the Arduino into your computer with a USB A-B cable. Wait for initialization on the Arduino.
From the environment (after code has been pasted), go to the File menu and select Upload. Wait for the LEDs on the Arduino to stop flashing, indicating code upload is complete. Next, go to the Tools menu option, and select "Serial Monitor". The code will immediately be ran on the Arduino, and you will see a scrolling log of the results (distance to object in front of the sensor).

==Research==
*[[CNC Software]]
*[[Stepper Motor]]

==Product Ecology==
{{3D Printer}}
{{Circuit Mill}}
{{Multimachine}}
{{Torch Table}}

==See Also==
*[[Digital Fabrication]]
*[[Vinyl Cutter]]
*[http://en.wikipedia.org/wiki/Cnc Wikipedia: CNC]

[[Category: Links]]

CNC

2016-09-25T02:46:39Z

Diginess: /* Notes */

=Summary=
Computer numerical control, a computer controller that reads instructions and drives a machine tool.

In modern CNC systems, end-to-end component design is highly automated using computer-aided design ([[CAD]]) and computer-aided manufacturing ([[CAM]]) programs. The programs produce a computer file that is interpreted to extract the commands needed to operate a particular machine via a postprocessor, and then loaded into the CNC machines for production. Since any particular component might require the use of a number of different tools-drills, saws, etc., modern machines often combine multiple tools into a single "cell". In other cases, a number of different machines are used with an external controller and human or robotic operators that move the component from machine to machine. In either case, the complex series of steps needed to produce any part is highly automated and produces a part that closely matches the original CAD design.

[[File: cnctoolchain.svg|500px]]

Use the [[Processing]] [[IDE]] to open the [[Gctrl]] [[GUI]]

Then you can send [[CAM]] [[Electrical Information]] from your [[Computer]] via its [[USB]] port to the [[Arduino]] [[Microcontroller]]

The [[Electrical Information]] will be interpreted by the [[GRBL]] software and then you will get processed [[Electrical Information]] as the output

You can configure the [[GRBL]] settings using the [[Arduino]] [[IDE]]'s [[Serial Monitor]]

=Modules=

[[Processing]]

[[Gctrl]]

[http://www.amazon.com/Arduino-Rev-3-Uno-R3/dp/B006H06TVG/ref=sr_1_1?ie=UTF8&qid=1339352565&sr=8-1 Arduino Uno R3 from Amazon]

[[Arduino]]

[[RXTX]]

[[GRBL]]

[[GRBL Config]]

=Notes=

sudo apt-get install librxtx-java

arduino IDE Serial Monitor: carriage return

sudo apt-get install openjdk-6-jre

----

sudo ./arduino

sudo ./processing

=Fedora Notes (for install of the Arduino development environment)=
Type this command (without quotations) to install the development environment for the Arduino:
"sudo dnf install arduino"
To start the environment:
"arduino"

=Controlling ultrasonic sensor from Arduino=
Diginess's notes: I bought this: https://www.radioshack.com/products/radioshack-ultrasonic-range-sensor from Radio Shack ($30) - have some on order from Amazon, but ran out of time.
The wiring is very simple - I used a solderless prototyping board and followed the instructions on the manual (page 4): https://github.com/RadioShackCorp/2760342-Range-Sensor/blob/master/2760342_PM_EN.pdf
Note that the picture shows the view from the sensor's front - I initially wired it backwards, but there are only 4 wires. If you get 0s showing for distance measurements, you probably reversed the signal wire and ground.
Here's the code to paste in the development environment: https://github.com/RadioShackCorp/2760342-Range-Sensor/blob/master/UltrasonicRangeSensor.ino
Plug the Arduino into your computer with a USB A-B cable. Wait for initialization on the Arduino.
From the environment (after code has been pasted), go to the File menu and select Upload. Wait for the LEDs on the Arduino to stop flashing, indicating code upload is complete. Next, go to the Tools menu option, and select "Serial Monitor". The code will immediately be ran on the Arduino, and you will see a scrolling log of the results (distance to object in front of the sensor).

==Research==
*[[CNC Software]]
*[[Stepper Motor]]

==Product Ecology==
{{3D Printer}}
{{Circuit Mill}}
{{Multimachine}}
{{Torch Table}}

==See Also==
*[[Digital Fabrication]]
*[[Vinyl Cutter]]
*[http://en.wikipedia.org/wiki/Cnc Wikipedia: CNC]

[[Category: Links]]