PhotogrammetryWorkflows/VisualSFM CMVS-PMVS Meshlab - Revision history

Marcin at 07:01, 4 April 2020

2020-04-04T07:01:48Z

← Older revision		Revision as of 07:01, 4 April 2020
Line 118:		Line 118:

	TODO: At the end of this process, I had a model in Blender that still looked somewhat point-cloudish. I need to find more tutorials on Meshlab to include tips/tricks. The texture also didn't come through the process, so I need to go back over that and find a fix. Most of the tutorials tell you not to expect to have super nice looking models when you first start. I'll add the most helpful tips and tricks as I find them.		TODO: At the end of this process, I had a model in Blender that still looked somewhat point-cloudish. I need to find more tutorials on Meshlab to include tips/tricks. The texture also didn't come through the process, so I need to go back over that and find a fix. Most of the tutorials tell you not to expect to have super nice looking models when you first start. I'll add the most helpful tips and tricks as I find them.

			=Links=
			*[[Photogrammetry]]

Diginess: Layout editing, minor notes added

2020-04-04T06:29:35Z

Layout editing, minor notes added

Show changes

Diginess: Bolding headers

2020-04-04T06:05:14Z

Bolding headers

Show changes

Diginess: Tip about surface reflection being undesirable

2020-04-04T06:01:34Z

Tip about surface reflection being undesirable

← Older revision		Revision as of 06:01, 4 April 2020
Line 23:		Line 23:
	Next get Meshlab:		Next get Meshlab:
	https://www.youtube.com/watch?v=GEAbXYDzUjU&t=3m15s		https://www.youtube.com/watch?v=GEAbXYDzUjU&t=3m15s
	~~Yes, it's at SourceForge~~. ~~They are~~ no longer including adware automatically in all downloads. ~~Grab the installer~~ and ~~install it~~.		https://sourceforge.net/projects/meshlab/
			Sourceforge is no longer including adware automatically in all downloads. I installed this and had no issues (JJ, as of Apr 4th, 2020).

	Finally, download and install Blender:		Finally, download and install Blender:
Line 57:		Line 58:
	For something as plain as a big piece of steel where the surface is very smooth, it might have to be enough to capture all the edges and for it to be dimensionally correct. Otherwise those surfaces will need detail somehow.		For something as plain as a big piece of steel where the surface is very smooth, it might have to be enough to capture all the edges and for it to be dimensionally correct. Otherwise those surfaces will need detail somehow.
	(if you want a real challenge, try to do this to your cat when it happens to lay down - the process doesn't work at all for moving subjects, so move quickly - just kidding, probably a bad idea that will frustrate you to no end)		(if you want a real challenge, try to do this to your cat when it happens to lay down - the process doesn't work at all for moving subjects, so move quickly - just kidding, probably a bad idea that will frustrate you to no end)
			Another thing I noticed was that the top cover of the book I was capturing was partially reflective. You'll want to avoid reflective surfaces if at all possible, because when you change your perspective, the reflected light changes as well, messing up the solver. It is very likely nothing will get captured at all from the reflective spots because matching of features can't occur between images there, causing gaps in the point cloud and resulting mesh.

	Dense reconstruction (VisualSFM also uses the CMVS-PMVS software here):		Dense reconstruction (VisualSFM also uses the CMVS-PMVS software here):

Diginess: Added TODO to end

2020-04-04T04:32:41Z

Added TODO to end

← Older revision		Revision as of 04:32, 4 April 2020
Line 104:		Line 104:
	If you need to recenter the model in the scene, select the model in the scene then press shift-ctrl-alt-c and select "geometry to origin". The model should move to the center.		If you need to recenter the model in the scene, select the model in the scene then press shift-ctrl-alt-c and select "geometry to origin". The model should move to the center.
	If when doing that, the model suddenly disappears, it's probably just inside the gray box that appears by default in Blender at the origin of the scene, which means you succeeded (you can scroll to zoom inside the box and check). You can delete the cube from the scene collection pane at the right by selecting it and selecting "delete".		If when doing that, the model suddenly disappears, it's probably just inside the gray box that appears by default in Blender at the origin of the scene, which means you succeeded (you can scroll to zoom inside the box and check). You can delete the cube from the scene collection pane at the right by selecting it and selecting "delete".

			TODO: At the end of this process, I had a model in Blender that still looked somewhat point-cloudish. I need to find more tutorials on Meshlab to include tips/tricks. The texture also didn't come through the process, so I need to go back over that and find a fix.

Diginess: Change to model centering instructions to simplify.

2020-04-04T04:26:51Z

Change to model centering instructions to simplify.

← Older revision		Revision as of 04:26, 4 April 2020
Line 96:		Line 96:
	In Meshlab, go to the File menu, select "Export Mesh As", and in the dialog, change the file type to "Alias Wavefront Object (*.obj)" and give it a descriptive name, then click save. In the next dialog box, uncheck "normals" and click OK. Now to make things easier for the later step, copy the texture (.png file) and your latest .obj file into a new folder (name that something descriptive).		In Meshlab, go to the File menu, select "Export Mesh As", and in the dialog, change the file type to "Alias Wavefront Object (*.obj)" and give it a descriptive name, then click save. In the next dialog box, uncheck "normals" and click OK. Now to make things easier for the later step, copy the texture (.png file) and your latest .obj file into a new folder (name that something descriptive).

	~~Opening~~ the ~~object~~ in Blender:		Create the scene in Blender:
			https://www.youtube.com/watch?v=GEAbXYDzUjU&t=13m04s
	Open Blender. Go to File, select new, general.		Open Blender. Go to File, select new, general.
	Import your model - go to File, import, wavefront (.obj). In the dialog, navigate to your model you saved in the "Save the model" step above.		Import your model - go to File, import, wavefront (.obj). In the dialog, navigate to your model you saved in the "Save the model" step above.
	Once it has imported, take a look around the scene by rotating (in Blender, that is by default holding down the middle mouse button while you move the mouse). If you need to recenter the model in the scene, press ~~the following all at once, see this:~~		Once it has imported, take a look around the scene by rotating (in Blender, that is by default holding down the middle mouse button while you move the mouse).
	~~https://blender.stackexchange.com/questions/14294/how-to~~-~~recenter~~-~~an-objects~~-origin
			If you need to recenter the model in the scene, select the model in the scene then press shift-ctrl-alt-c and select "geometry to origin". The model should move to the center.
	If when doing that, the model suddenly disappears, it's probably just inside the gray box that appears by default in Blender at the origin of the scene, which means you succeeded (you can scroll to zoom inside the box and check). You can delete the cube from the scene collection pane at the right by selecting it and selecting "delete".		If when doing that, the model suddenly disappears, it's probably just inside the gray box that appears by default in Blender at the origin of the scene, which means you succeeded (you can scroll to zoom inside the box and check). You can delete the cube from the scene collection pane at the right by selecting it and selecting "delete".

Diginess: Finishing up first draft - I'm still not happy with the result, but I need more expertise in Meshlab to really finalize this product

2020-04-04T03:45:24Z

Finishing up first draft - I'm still not happy with the result, but I need more expertise in Meshlab to really finalize this product

@@ Line 2: / Line 2: @@
 Before going into the installation steps, here is a breakdown of what this set of software is doing / what we can do with it.  This is what I have gathered so far, and it is still loose / rough around the edges.  Please feel free to correct any mistakes I've made, or I will adjust as I learn more.
-VisualSFM takes in photos that you've taken around a 3D object and then runs an algorithm/program against them (SiftGPU, based on David Lowe's work - Chang Chang Wu wrote SiftGPU and VisualSFM) to find "feature points" (types of features in images that are easy to find in other images when the object is viewed from a different angle or distance) and from these, find the relative positions of where your camera was when you took each picture.  Up to this point, it has performed "sparse reconstruction'.  After this, it will turn to the CMVS-PMVS software (if you've copied it into the root VisualSFM install folder with the VisualSFM executable) to do dense reconstruction which involves finding many more features from the image data to fill out the point cloud.  When you click on the dense reconstruction button, it'll ask you to provide a folder path where you would like it to save the results.  Later, we will use Meshlab and open that folder/project.  We will use Meshlab to construct a colored mesh around the point cloud.  At that point, we'll have a 3D object with colors, but no texture (if you want fancy / any textures at all, that's not part of this process yet).  Once that's done, we can open it with any modelling software (or the main open source one, Blender), as well as import it into a splicer program for conversion to gcode for 3D printing.
+VisualSFM takes in photos that you've taken around a 3D object and then runs an algorithm/program against them (SiftGPU, based on David Lowe's work - Chang Chang Wu wrote SiftGPU and VisualSFM) to find "feature points" (types of features in images that are easy to find in other images when the object is viewed from a different angle or distance) and from these, find the relative positions of where your camera was when you took each picture.  Up to this point, it has performed "sparse reconstruction'.  After this, it will turn to the CMVS-PMVS software (if you've copied it into the root VisualSFM install folder with the VisualSFM executable) to do dense reconstruction which involves finding many more features from the image data to fill out the point cloud.  When you click on the dense reconstruction button, it'll ask you to provide a folder path where you would like it to save the results.  Later, we will use Meshlab and open that folder/project.  We will use Meshlab to construct a colored mesh around the point cloud.  At that point, we'll have a 3D object with colors and texture.  Once that's done, we can open it with any modelling software (or the main open source one, Blender), as well as import it into a slicer program for conversion to gcode for 3D printing.
 After searching among many videos on Youtube, I found a good one that documents the installation process here:
@@ Line 24: / Line 24: @@
 https://www.youtube.com/watch?v=GEAbXYDzUjU&t=3m15s
 Yes, it's at SourceForge.  They are no longer including adware automatically in all downloads.  Grab the installer and install it.
 Handling pictures:
@@ Line 75: / Line 78: @@
 Once you are happy that you have deleted all the points you want gone, go to the Filters menu, select "Point Set", then select "Surface Reconstructions Poisson".  In the dialog window, set the octree depth to 12 and solver divide to 7, leaving the other two settings as is.  Click "Apply".
 To get rid of the weird bubble that it creates while keeping the modifications to the model/mesh, on layers click on the model and .ply layers to make them invisible, then select the poisson layer (click on the name, not on the icon) highlighting it, then use the "select vertexes" and "delete vertexes" to get rid of the top part of the bubble.
 Simplify the geometry:
 https://www.youtube.com/watch?v=GEAbXYDzUjU&t=10m11s

Diginess: Continuing on this pipeline, steps for Meshlab. Saving so I don't lose the work.

2020-04-04T02:14:00Z

Continuing on this pipeline, steps for Meshlab. Saving so I don't lose the work.

← Older revision		Revision as of 02:14, 4 April 2020
Line 41:		Line 41:
	Now wait for a bit while it processes all the images. When it is done, you will see mini icons for all of the images in the main view and a summary in the log on the right.		Now wait for a bit while it processes all the images. When it is done, you will see mini icons for all of the images in the main view and a summary in the log on the right.

	~~Do initial matching of images~~:		Calculate missing matches:
	https://www.youtube.com/watch?v=GEAbXYDzUjU&t=5m10s		https://www.youtube.com/watch?v=GEAbXYDzUjU&t=5m10s
	Select the "compute missing matches" icon. It's sixth from the right and looks like four arrows pointed at diagonal corners.		Select the "compute missing matches" icon. It's sixth from the right and looks like four arrows pointed at diagonal corners.
Line 50:		Line 50:
	Within VisualSFM, select "SfM" from the menu at top (2nd one from left). Within that menu, select "Reconstruct sparse".		Within VisualSFM, select "SfM" from the menu at top (2nd one from left). Within that menu, select "Reconstruct sparse".
	Wait until it finishes. You should then see the beginnings of a pointcloud with indicators where the cameras were located when the pictures were taken.		Wait until it finishes. You should then see the beginnings of a pointcloud with indicators where the cameras were located when the pictures were taken.
	Note, at this point when it has finished, if you notice that there are two few camera indicators, it is likely that one of them has a red "X" over it and is overall red. This means that while you were taking your pictures, the locations you took them from were likely too far (too many degrees) apart and it could not "connect the dots" between images. Try taking more images in between where there are more distant gaps in positions between the cameras. This was happening to me and I got confused before really starting to think about it. I had taken about 20 images around a book on my couch. It wasn't nearly enough. So I went back and took around 40 more (I had 64 when I was done, 5 were duplicates - this process doesn't care about those), being careful to move around the book no more than 15 degrees between images. Note that the time each step of this process takes will increase as the number of pictures goes up. After doing the missing matches step again, this time I had 2080 features instead of just over 100 and 434 seconds versus the 43 seconds it had taken prior (the last time it had stopped after getting confused after 3 pictures - if that step stops after less than a minute, you might have too few pictures).		Troubleshooting: Note, at this point when it has finished, if you notice that there are two few camera indicators, it is likely that one of them has a red "X" over it and is overall red. This means that while you were taking your pictures, the locations you took them from were likely too far (too many degrees) apart and it could not "connect the dots" between images. Try taking more images in between where there are more distant gaps in positions between the cameras, add the pictures and go back and do the prior step "calculate missing matches" again. This was happening to me and I got confused before really starting to think about it. I had taken about 20 images around a book on my couch. It wasn't nearly enough. Note that the video author took 70 pictures to capture his Millenium Falcon model. So I went back and took around 40 more (I had 64 when I was done, 5 were duplicates - this process doesn't care about those), being careful to move around the book no more than 15 degrees between images. Note that the time each step of this process takes will increase as the number of pictures goes up. After doing the missing matches step again, this time I had 2080 features instead of just over 100 and 434 seconds versus the 43 seconds it had taken prior (the last time it had stopped after getting confused after 3 pictures - if that step stops after less than a minute, you might have too few pictures). If the "compute missing matches" took enough time like this / you have plenty of pictures and features, you're probably going to be OK with the sparse reconstruction step that comes next. The next dense reconstruction for me after I took the extra pictures took 10 minutes, 55 seconds - note that unless you properly copied CMVS-PMVS into the VisualSFM folder as documented in the CMVS-PMVS installation step above, dense reconstruction will not work.
	Other: Make sure that the object you are taking pictures of has good lighting and contrast and there is detail on it. This process might not do well for scenes that don't have enough features/edges/contrast/detail - for instance a blank white wall wouldn't be captured well at all.		Other: Make sure that the object you are taking pictures of has good lighting and contrast and there is detail on it. This process might not do well for scenes that don't have enough features/edges/contrast/detail - for instance a blank white wall wouldn't be captured well at all.
	For something as plain as a big piece of steel where the surface is very smooth, it might have to be enough to capture all the edges and for it to be dimensionally correct. Otherwise those surfaces will need detail somehow.		For something as plain as a big piece of steel where the surface is very smooth, it might have to be enough to capture all the edges and for it to be dimensionally correct. Otherwise those surfaces will need detail somehow.
	(if you want a real challenge, try to do this to your cat when it happens to lay down - the process doesn't work at all for moving subjects, so move quickly - just kidding, probably a bad idea that will frustrate you to no end)		(if you want a real challenge, try to do this to your cat when it happens to lay down - the process doesn't work at all for moving subjects, so move quickly - just kidding, probably a bad idea that will frustrate you to no end)

	Dense reconstruction:		Dense reconstruction (VisualSFM also uses the CMVS-PMVS software here):
	https://www.youtube.com/watch?v=GEAbXYDzUjU&t=5m52s		https://www.youtube.com/watch?v=GEAbXYDzUjU&t=5m52s
	Within VisualSFM, select "SfM" from the menu at top (2nd one from left). Within that menu, select "Reconstruct dense".		Within VisualSFM, select "SfM" from the menu at top (2nd one from left). Within that menu, select "Reconstruct dense".
	It will pop up a dialog asking where it should save the files it creates. Give it a location on your hard drive by navigating, and type a file name you would like the project file to be called (you'll want to remember where you saved it so you can open it in Meshlab later - maybe write the path down).		It will pop up a dialog asking where it should save the files it creates. Give it a location on your hard drive by navigating, and type a file name you would like the project file to be called (you'll want to remember where you saved it so you can open it in Meshlab later - maybe write the path down).
			To see the results, go to View on the menu and select "Dense 3D points" (or just press tab).
			That's it, you're done with the VisualSFM and CMVS-PMVS portion of the pipeline. Next we will move on to Meshlab to connect the point cloud surface together with a mesh.

			Meshlab:

			Build the geometry:
			https://www.youtube.com/watch?v=GEAbXYDzUjU&t=7m07s
			Open Meshlab and go to the File menu option and select "Open project". Navigate to where you saved the project through the dialog window in the "Dense reconstruction" step in VisualSFM above. Select the project file and wait until it finishes loading - you should see the point cloud.
			Go to the View menu option and select "Show Layer Dialog" - you should now see a pane to the right representing layers.
			Rotate the point cloud around until you have a good idea where your object is (left click and drag) and what points you can cut.
			Make sure that you have a good rotation for the cuts, then click the "Select vertexes" icon from the toolbar.
			Click and drag to create boxes to select points you don't want in your 3D model. You can hold control to click and drag again to add more boxes of points to your selection. When you're done, click on the first red X icon (it's an X over connected vertexes) to delete the selected points.
			Next, go to the File menu and select "Import mesh". Select the file (the dialog should already be in the correct folder since you loaded the project from there, and there will probably only be one file showing - it will end in the .ply extension) and click "open".
			Now you should have a much better view and be able to see which of the remaining points you want to delete. Repeat the select vertexes / delete vertexes step again until you have deleted those.
			Once you are happy that you have deleted all the points you want gone, go to the Filters menu, select "Point Set", then select "Surface Reconstructions Poisson". In the dialog window, set the octree depth to 12 and solver divide to 7, leaving the other two settings as is. Click "Apply".
			To get rid of the weird bubble that it creates while keeping the modifications to the model/mesh, on layers click on the model and .ply layers to make them invisible, then select the poisson layer (click on the name, not on the icon) highlighting it, then use the "select vertexes" and "delete vertexes" to get rid of the top part of the bubble.

			Simplify the geometry:
			https://www.youtube.com/watch?v=GEAbXYDzUjU&t=10m11s

Diginess: Troubleshooting detail in case of too many gaps in between pictures - how to tell early in the process

2020-04-04T00:26:49Z

Troubleshooting detail in case of too many gaps in between pictures - how to tell early in the process

← Older revision		Revision as of 00:26, 4 April 2020
Line 47:		Line 47:

	Sparse reconstruction:		Sparse reconstruction:
	https://www.youtube.com/watch?v=GEAbXYDzUjU&t=		https://www.youtube.com/watch?v=GEAbXYDzUjU&t=5m23s (through 5m52s)
			Within VisualSFM, select "SfM" from the menu at top (2nd one from left). Within that menu, select "Reconstruct sparse".
			Wait until it finishes. You should then see the beginnings of a pointcloud with indicators where the cameras were located when the pictures were taken.
			Note, at this point when it has finished, if you notice that there are two few camera indicators, it is likely that one of them has a red "X" over it and is overall red. This means that while you were taking your pictures, the locations you took them from were likely too far (too many degrees) apart and it could not "connect the dots" between images. Try taking more images in between where there are more distant gaps in positions between the cameras. This was happening to me and I got confused before really starting to think about it. I had taken about 20 images around a book on my couch. It wasn't nearly enough. So I went back and took around 40 more (I had 64 when I was done, 5 were duplicates - this process doesn't care about those), being careful to move around the book no more than 15 degrees between images. Note that the time each step of this process takes will increase as the number of pictures goes up. After doing the missing matches step again, this time I had 2080 features instead of just over 100 and 434 seconds versus the 43 seconds it had taken prior (the last time it had stopped after getting confused after 3 pictures - if that step stops after less than a minute, you might have too few pictures).
			Other: Make sure that the object you are taking pictures of has good lighting and contrast and there is detail on it. This process might not do well for scenes that don't have enough features/edges/contrast/detail - for instance a blank white wall wouldn't be captured well at all.
			For something as plain as a big piece of steel where the surface is very smooth, it might have to be enough to capture all the edges and for it to be dimensionally correct. Otherwise those surfaces will need detail somehow.
			(if you want a real challenge, try to do this to your cat when it happens to lay down - the process doesn't work at all for moving subjects, so move quickly - just kidding, probably a bad idea that will frustrate you to no end)

			Dense reconstruction:
			https://www.youtube.com/watch?v=GEAbXYDzUjU&t=5m52s
			Within VisualSFM, select "SfM" from the menu at top (2nd one from left). Within that menu, select "Reconstruct dense".
			It will pop up a dialog asking where it should save the files it creates. Give it a location on your hard drive by navigating, and type a file name you would like the project file to be called (you'll want to remember where you saved it so you can open it in Meshlab later - maybe write the path down).

Diginess: Lots of min edits, adding steps in the VisualSFM portion of the pipeline.

2020-04-03T23:27:44Z

Lots of min edits, adding steps in the VisualSFM portion of the pipeline.

← Older revision		Revision as of 23:27, 3 April 2020
Line 1:		Line 1:
	Note: I (Jacob Johnson, JJ) am currently ~~working on~~ Windows. I will try to leave some leeway in these instructions for other OSs by including the overall links where you can find software for those versions, but the most detailed instructions at first will be Windows until I get my Desktop back online with Ubuntu later and run through the process again for that, then document it.		Note: I (Jacob Johnson, JJ) am currently using Windows. I will try to leave some leeway in these instructions for other OSs by including the overall links where you can find software for those versions, but the most detailed instructions at first will be for Windows versions (installation instructions at least, I think app usage will mostly be similar) until I get my Desktop back online with Ubuntu later and run through the process again for that, then document it.

	Before going into the installation steps, here is a breakdown of what this set of software is doing / what we can do with it. This is what I have gathered so far, and it is still loose / rough around the edges. Please feel free to correct any mistakes I've made, or I will adjust as I learn more.		Before going into the installation steps, here is a breakdown of what this set of software is doing / what we can do with it. This is what I have gathered so far, and it is still loose / rough around the edges. Please feel free to correct any mistakes I've made, or I will adjust as I learn more.
Line 7:		Line 7:
	https://www.youtube.com/watch?v=GEAbXYDzUjU		https://www.youtube.com/watch?v=GEAbXYDzUjU
	I will include timestamps into this video for each step below. Note that where my instructions differ it has to do with getting Windows binaries rather than Linux versions.		I will include timestamps into this video for each step below. Note that where my instructions differ it has to do with getting Windows binaries rather than Linux versions.

			Final note before proceeding to installation steps: You might note that I am linking the same video over and over again. That's because I'm including the time stamps in the video for that piece of the instructions. If you already have the video up and do not want to open another tab, just look at the end. For instance, if I wanted the link to open the video at the 5 minute 30 second mark, I would add &t=5m30s to the end of the Youtube video link (stand for "time equals 5 minutes, 30 seconds"). You can just progress the video to that point in the tab you already have up if you want to move forward.

	Installation:		Installation:
Line 34:		Line 36:

	Open the images in VisualSFM:		Open the images in VisualSFM:
			https://www.youtube.com/watch?v=GEAbXYDzUjU&t=4m45s
	Click on the "Open Multiple Images" button (it's third from the left, just under the File tab in VisualSFM).		Click on the "Open Multiple Images" button (it's third from the left, just under the File tab in VisualSFM).
	Navigate to the location you stored your pictures on your computer (if you're using Google Backup and Sync, that's "C:\Users\Your User Name\Google Drive\Whatever you called the image folder"). Select all of the images by left clicking the first one, pressing shift and holding it down while you left click the last one. Finally, click the "open" button on the bottom right of that dialog window (while all the images still highlighted/selected).		Navigate to the location you stored your pictures on your computer (if you're using Google Backup and Sync, that's "C:\Users\Your User Name\Google Drive\Whatever you called the image folder"). Select all of the images by left clicking the first one, pressing shift and holding it down while you left click the last one. Finally, click the "open" button on the bottom right of that dialog window (while all the images still highlighted/selected).
			Now wait for a bit while it processes all the images. When it is done, you will see mini icons for all of the images in the main view and a summary in the log on the right.

			Do initial matching of images:
			https://www.youtube.com/watch?v=GEAbXYDzUjU&t=5m10s
			Select the "compute missing matches" icon. It's sixth from the right and looks like four arrows pointed at diagonal corners.
			Wait until it finishes.

			Sparse reconstruction:
			https://www.youtube.com/watch?v=GEAbXYDzUjU&t=