PhotogrammetryWorkflows/VisualSFM CMVS-PMVS Meshlab: Difference between revisions
(Lots of min edits, adding steps in the VisualSFM portion of the pipeline.) |
(Troubleshooting detail in case of too many gaps in between pictures - how to tell early in the process) |
||
| Line 47: | Line 47: | ||
Sparse reconstruction: | Sparse reconstruction: | ||
https://www.youtube.com/watch?v=GEAbXYDzUjU&t= | https://www.youtube.com/watch?v=GEAbXYDzUjU&t=5m23s (through 5m52s) | ||
Within VisualSFM, select "SfM" from the menu at top (2nd one from left). Within that menu, select "Reconstruct sparse". | |||
Wait until it finishes. You should then see the beginnings of a pointcloud with indicators where the cameras were located when the pictures were taken. | |||
Note, at this point when it has finished, if you notice that there are two few camera indicators, it is likely that one of them has a red "X" over it and is overall red. This means that while you were taking your pictures, the locations you took them from were likely too far (too many degrees) apart and it could not "connect the dots" between images. Try taking more images in between where there are more distant gaps in positions between the cameras. This was happening to me and I got confused before really starting to think about it. I had taken about 20 images around a book on my couch. It wasn't nearly enough. So I went back and took around 40 more (I had 64 when I was done, 5 were duplicates - this process doesn't care about those), being careful to move around the book no more than 15 degrees between images. Note that the time each step of this process takes will increase as the number of pictures goes up. After doing the missing matches step again, this time I had 2080 features instead of just over 100 and 434 seconds versus the 43 seconds it had taken prior (the last time it had stopped after getting confused after 3 pictures - if that step stops after less than a minute, you might have too few pictures). | |||
Other: Make sure that the object you are taking pictures of has good lighting and contrast and there is detail on it. This process might not do well for scenes that don't have enough features/edges/contrast/detail - for instance a blank white wall wouldn't be captured well at all. | |||
For something as plain as a big piece of steel where the surface is very smooth, it might have to be enough to capture all the edges and for it to be dimensionally correct. Otherwise those surfaces will need detail somehow. | |||
(if you want a real challenge, try to do this to your cat when it happens to lay down - the process doesn't work at all for moving subjects, so move quickly - just kidding, probably a bad idea that will frustrate you to no end) | |||
Dense reconstruction: | |||
https://www.youtube.com/watch?v=GEAbXYDzUjU&t=5m52s | |||
Within VisualSFM, select "SfM" from the menu at top (2nd one from left). Within that menu, select "Reconstruct dense". | |||
It will pop up a dialog asking where it should save the files it creates. Give it a location on your hard drive by navigating, and type a file name you would like the project file to be called (you'll want to remember where you saved it so you can open it in Meshlab later - maybe write the path down). | |||
Revision as of 00:26, 4 April 2020
Note: I (Jacob Johnson, JJ) am currently using Windows. I will try to leave some leeway in these instructions for other OSs by including the overall links where you can find software for those versions, but the most detailed instructions at first will be for Windows versions (installation instructions at least, I think app usage will mostly be similar) until I get my Desktop back online with Ubuntu later and run through the process again for that, then document it.
Before going into the installation steps, here is a breakdown of what this set of software is doing / what we can do with it. This is what I have gathered so far, and it is still loose / rough around the edges. Please feel free to correct any mistakes I've made, or I will adjust as I learn more. VisualSFM takes in photos that you've taken around a 3D object and then runs an algorithm/program against them (SiftGPU, based on David Lowe's work - Chang Chang Wu wrote SiftGPU and VisualSFM) to find "feature points" (types of features in images that are easy to find in other images when the object is viewed from a different angle or distance) and from these, find the relative positions of where your camera was when you took each picture. Up to this point, it has performed "sparse reconstruction'. After this, it will turn to the CMVS-PMVS software (if you've copied it into the root VisualSFM install folder with the VisualSFM executable) to do dense reconstruction which involves finding many more features from the image data to fill out the point cloud. When you click on the dense reconstruction button, it'll ask you to provide a folder path where you would like it to save the results. Later, we will use Meshlab and open that folder/project. We will use Meshlab to construct a colored mesh around the point cloud. At that point, we'll have a 3D object with colors, but no texture (if you want fancy / any textures at all, that's not part of this process yet). Once that's done, we can open it with any modelling software (or the main open source one, Blender), as well as import it into a splicer program for conversion to gcode for 3D printing.
After searching among many videos on Youtube, I found a good one that documents the installation process here: https://www.youtube.com/watch?v=GEAbXYDzUjU I will include timestamps into this video for each step below. Note that where my instructions differ it has to do with getting Windows binaries rather than Linux versions.
Final note before proceeding to installation steps: You might note that I am linking the same video over and over again. That's because I'm including the time stamps in the video for that piece of the instructions. If you already have the video up and do not want to open another tab, just look at the end. For instance, if I wanted the link to open the video at the 5 minute 30 second mark, I would add &t=5m30s to the end of the Youtube video link (stand for "time equals 5 minutes, 30 seconds"). You can just progress the video to that point in the tab you already have up if you want to move forward.
Installation: 1. https://www.youtube.com/watch?v=GEAbXYDzUjU&t=2m41s Download VisualSFM from this page : http://ccwu.me/vsfm/ (it has multiple OS versions, Windows, Linux, and MacOS, generally you'll want the 64 bit version - if that doesn't work you may have an old computer and need the 32 bit version) Installing this only consists of downloading the zip file and extracting it (there's no installer - once extracted, the executable is in the extracted folder and ready to run).
Next get CMVS-PMVS: https://www.youtube.com/watch?v=GEAbXYDzUjU&t=2m52s https://github.com/pmoulon/CMVS-PMVS Click on the green button to the top right of the repo file list "Clone or Download", then select the option to clone download as a zip file. Extract the zip file and navigate to the subfolder that has the version for your operating system (for instance, for Windows the path from the top level "CMVS-PMVS-Master" extracted repo zip file to the Win64 version is: CMVS-PMVS-master.zip\CMVS-PMVS-master\binariesWin-Linux\Win64-VS2010). Copy everything from that subfolder into the folder where you put VisualSFM (further questions on how / what files to copy, watch the Youtube video starting at the 4 minute mark).
Next get Meshlab: https://www.youtube.com/watch?v=GEAbXYDzUjU&t=3m15s Yes, it's at SourceForge. They are no longer including adware automatically in all downloads. Grab the installer and install it.
Handling pictures: If you're using your cell phone, you'll want a way to easily get pictures from your phone to your computer so that you can run these apps on them to make the model. I had some initial difficulties, probably due to my phone being encrypted, where when I connected it to my computer, I could not see the phone's internal storage from Windows File Explorer. To overcome that, what I did was upload all the images to Google drive and installed Google Backup and Sync on my computer. Once you've uploaded photos to Google drive, when you sign into Google Backup and sync on the same Google account, it will download all of the files on Google drive to your computer. Its default directory is C:\Users\<YourUserNameHere>\Google Drive. I didn't do anything else with the files from there, only selected them in VisualSFM from where they were on my computer / where Google Backup and sync put them (see above). You may have to wait a few minutes after uploading for Backup and Sync to download all the pictures to your computer's hard drive. It will happen automatically as long as the app is signed into that Google account.
Running the pipeline:
VisualSFM steps: https://www.youtube.com/watch?v=GEAbXYDzUjU&t=4m33s Open VisualSFM by going into the folder and double clicking the executable (the one with the green 'V' icon).
Open the images in VisualSFM: https://www.youtube.com/watch?v=GEAbXYDzUjU&t=4m45s Click on the "Open Multiple Images" button (it's third from the left, just under the File tab in VisualSFM). Navigate to the location you stored your pictures on your computer (if you're using Google Backup and Sync, that's "C:\Users\Your User Name\Google Drive\Whatever you called the image folder"). Select all of the images by left clicking the first one, pressing shift and holding it down while you left click the last one. Finally, click the "open" button on the bottom right of that dialog window (while all the images still highlighted/selected). Now wait for a bit while it processes all the images. When it is done, you will see mini icons for all of the images in the main view and a summary in the log on the right.
Do initial matching of images: https://www.youtube.com/watch?v=GEAbXYDzUjU&t=5m10s Select the "compute missing matches" icon. It's sixth from the right and looks like four arrows pointed at diagonal corners. Wait until it finishes.
Sparse reconstruction: https://www.youtube.com/watch?v=GEAbXYDzUjU&t=5m23s (through 5m52s) Within VisualSFM, select "SfM" from the menu at top (2nd one from left). Within that menu, select "Reconstruct sparse". Wait until it finishes. You should then see the beginnings of a pointcloud with indicators where the cameras were located when the pictures were taken. Note, at this point when it has finished, if you notice that there are two few camera indicators, it is likely that one of them has a red "X" over it and is overall red. This means that while you were taking your pictures, the locations you took them from were likely too far (too many degrees) apart and it could not "connect the dots" between images. Try taking more images in between where there are more distant gaps in positions between the cameras. This was happening to me and I got confused before really starting to think about it. I had taken about 20 images around a book on my couch. It wasn't nearly enough. So I went back and took around 40 more (I had 64 when I was done, 5 were duplicates - this process doesn't care about those), being careful to move around the book no more than 15 degrees between images. Note that the time each step of this process takes will increase as the number of pictures goes up. After doing the missing matches step again, this time I had 2080 features instead of just over 100 and 434 seconds versus the 43 seconds it had taken prior (the last time it had stopped after getting confused after 3 pictures - if that step stops after less than a minute, you might have too few pictures). Other: Make sure that the object you are taking pictures of has good lighting and contrast and there is detail on it. This process might not do well for scenes that don't have enough features/edges/contrast/detail - for instance a blank white wall wouldn't be captured well at all. For something as plain as a big piece of steel where the surface is very smooth, it might have to be enough to capture all the edges and for it to be dimensionally correct. Otherwise those surfaces will need detail somehow. (if you want a real challenge, try to do this to your cat when it happens to lay down - the process doesn't work at all for moving subjects, so move quickly - just kidding, probably a bad idea that will frustrate you to no end)
Dense reconstruction: https://www.youtube.com/watch?v=GEAbXYDzUjU&t=5m52s Within VisualSFM, select "SfM" from the menu at top (2nd one from left). Within that menu, select "Reconstruct dense". It will pop up a dialog asking where it should save the files it creates. Give it a location on your hard drive by navigating, and type a file name you would like the project file to be called (you'll want to remember where you saved it so you can open it in Meshlab later - maybe write the path down).