week 3&4 - Tree growth 1
/// Goal ///
Before developing tree growth, I need to decide on the workflow. Should I use procedurally generated trees or trees created by scanning? How should the trees grow? After conducting numerous experiments, I ultimately concluded that I would use procedurally generated trees and create realistic growth animations rather than simple erase-and-grow animations.
/// Mesh to Splat - camera array rendering method ///
In Week 2, I mentioned a method for converting meshes to GS, but that method has many limitations and is prone to bugs—the number of faces, face distribution, whether the model is closed, UVs, texture formats, and so on can all cause the conversion to fail. Therefore, here is a second method: using the Camera Array Tool to convert the model. This Blender add-on generates a camera array to capture render images, allowing you to bake lighting and material information into the GS file. It can also export a sequence of frames, which can be used with Postshot to create GS animations. However, unlike the free 3DGS, it costs $45.
more details please check：Camera Array Tool - Superhive
A tip:
When using HDRI, rendering the HDRI background better results than using a transparent background. GS models trained on images rendered with a transparent background will have black borders.
/// SpeedTree modeling and conversion ///
Since I need to create procedural trees, I taught myself SpeedTree. This software allows for rapid creation of realistic trees, and it’s easier to control their shapes and lighting compared to Gaussian trees created by stitching together photogrammetry trees. Then I can convert the polygon tree to GS tree. However it’s not as easy as I thought...
We all know that SpeedTree can create growth animations directly, but the animated model is exported as an ABC file which lack topology and materials, so it naturally cannot be converted to a PLY sequence using the 3DGS render workflow.
I also encountered many issues while using the 3DGS Render.
- If the UVs of models exported from SpeedTree are not confined to a 1x1 grid, it can cause errors;
- An insufficient polygon count may result in poor splat effects, while an excessively high polygon count may lead to unknown errors or crashes;
-
Models exported from SpeedTree apply multiple textures to a single model, causing the resulting splat model to bake only the bottom-most texture.
Even if the conversion is successful, the result may still be incorrect.
So I changed the plan. Using camera array rendering method for GS conversion could be a better solution.
Using USD export will set the material more corrctly than using obj or fbx. At first, I wasn't very confident about such a complex structure, but the results completely exceeded my expectations. Almost every leaf was generated quite well.
/// Procedural plants growth ///
It’s time to figure out the growth animation now. Rigging can be a good method to chontrol each branch’s action but I’d like to find a easier way to control the growth. Indpired by intargama’s tutorial, I noticed that the point deform node can manipulate point cloud changes using a polygon model. That gave me an idea: to create a growth animation using the tree polygon model, and then use that animation to control GS particle growth. GSOPs provides a similar node called “Gaussian Deform,” which functions in a similar way but works more accurately with GS.
Might be more suitable for herbaceous plants.
However, when I tried to use this method to create a growth animation, I ran into a problem — obviously, the reason was that I used the sweep node to generate the base mesh that influenced the particles(see figure 3). To achieve precise growth, I needed to use an entirely consistent model and able to control it. So I'm stuck at this step.
/// Window photogrammetry ///
I wasn't very satisfied with the wall. So I merged the window into the other wall. But I still need to decide which wall to use.
/// Next Plan ///
1. Find a suitable way to create the plant growth animation
2. Start GS building destruction method research