Note: Photogrammetry support is a beta feature in v8.2.0 and is only available in Cinema and Games editions of the Modeler. The documentation for this feature is a work in progress.
This page describes how to prepare and use a photogrammetry generated mesh that represents branches or twigs. The first section explains why you'll likely need to prepare the mesh before using it (either in SpeedTree or an external application) and the second section covers how you might use the branch mesh once you have.
Note: The description of the process that follows assumes you are familiar with the basics of SpeedTree and is more of a guide to the general approach than it is a step-by-step tutorial.
Scans of trunks are often usable as is. You just put them in the scene and start building off of them. Branch scans are typically not ready without some preparation. First, the pivot point is typically not in the proper location. Second, It is often desirable to procedurally combine branch meshes into more complex branch structures. You might want to grow leaves off the ends, as well. In order to do either of these things, you must identify valid locations for which to grow things off of the mesh.
The general approach is to create a SpeedTree model specifically for preparing the mesh and then using it to export a new, doctored mesh. While it is possible to do the preparation in an external application, SpeedTree provides several tools to make this process easier. Follow the procedure detailed below to prepare a branch mesh.
In order to use a 3d scan, you must first import the mesh as a mesh asset. Next, create a material asset that uses the textures associated with the scan. Be sure to associate the mesh asset with the material.
Meshes must be a single model that can have a single material on it. Photogrammetry meshes are typically saved as single meshes with unwrapped UV data that correspond to a texture atlas.
Once you have assets in place you'll use a “Mesh” generator to make it part of your model. Add a “Mesh” generator and then assign the material asset to it as pictured below. It will automatically pick up the mesh asset via its association with the material asset.
Switch to node editing mode and select the mesh in the Tree Window. Use the gizmo to position the mesh such that the base of the branch is located exactly on the scene origin. The scene origin will be the pivot point when the fully prepared mesh is exported. Rotate the mesh such that the branch is pointing down the “Y” axis.
Now, we're going to mark spots on the mesh where branches might grow. Create a blank material, name it “Branches”, and set the color to orange. Import “Helper.fbx” from the “<installation location>/samples/Photogrammetry/All_Mesh_Tree_Example/Meshes” folder and assign it to the material. We'll use this material/mesh combo to visualize the branch anchor points.
Select the Mesh generator and add the “Place 3D Anchors” geometry template. Assign the helper material to the Leaf Mesh generator. Even though we're marking branch locations, all anchors are exported via the leaf reference system. We'll identify them as branches based on the ID assigned to them later. In the Tree Window, hold the space bar and click on the mesh to create a new anchor point. You should see the helper mesh stuck to the branch and pointing towards the camera. Continue adding anchors for every point you might want a branch to grow.
Create a blank material, name it “Leaves”, and set the color to green. Assign the helper mesh asset to the new material. We'll use this material/mesh combo to visualize the leaf anchor points.
Select the Mesh generator and add a second copy of the “Place 3D Anchors” geometry template. Assign the “Leaves” material to the new Leaf Mesh generator. In the Tree Window, hold the space bar and click on the mesh to create a new set of anchor points. You should now see an orange and a green set of anchor indicators on your mesh.
Select “File→Export mesh…”. Select “SpeedTree XML” as the file type and then set the export options as shown below.
Notice that “Leaf references” is selected. The helper objects will be replaced by 3D anchors in the exported mesh. These anchors will act as growth locations when the mesh is used on a model. Using the “by material” option means each unique material will be given an integer id you can use to distinguish branch anchors from leaf anchors.
The newly exported mesh is ready to be used to construct a more complex model. Import it into any model and you should see anchors represented as green dots in the Mesh Asset Bar.
Note: It's a good idea to save the file you used to prepare the mesh. You may decide to delete, add, or move anchor points in the future.
Mesh generators can be added as the child of Branch, Leaf Mesh, and other Mesh generators. Mesh Generators can spawn Branch, Mesh, and Leaf Mesh generators. Use any of these generators in combination to create arbitrarily complex models.
Meshes that are generated as the child of a branch will be deformed by the branch. Gravity, roll, start angle, and all other branch shaping tools will now control the shape of the mesh. In the pic below, the “Placement” branch (set to “Spine only”) uses gravity to pull down meshes attached to the trunk mesh.
Note: The “Y” axis of the mesh asset will be mapped to branch. Orient the mesh asset (either in the mesh asset bar or in an external editor) so that the mesh “points” along the “Y” axis.
Note: You can set the branch's skin type to “Spine only” to ensure the branch geometry is not part of the model. In this way, the branch is simply used to shape the mesh.
When you add a generator as a child of a Mesh generator one of two things will happen. Children will be spawned at the 3D anchor points if the mesh asset has been prepared as discussed above; or, children will be randomly placed on the surface of the mesh (good for flooding the geometry but hard to control).
If you're using 3D anchors, you can select which types of anchors each child uses by editing the “Generation:Anchors” property.
The following list outlines a few things to keep in mind when using photogrammetry branch meshes.