How to generate 3D assets with more number of faces?

[supersyz]

Hi, Really appreciate for you great open-source work!
I notice the output objects have small number of faces, how to generate 3D assets with more number of faces?

[kitcheng]

In the example.py and app.py files, search for the term "simplify", and set it to a fixed value of 0.

This way, it will not reduce the number of faces.

[JackDainzh]

Better just edit the Simplify gradio slider to reach to the desired minimum in app.py

[visualbruno]

I changed the app.py, added the number of vertices, increased the sampling steps to 100 and changed the slider for mesh simplification from 0 to 0.98 and increased the max texture size to 4096.

Download my app.py file here

[realisticdreamer114514]

@visualbruno what guidance strengths are the best for details/fidelity in the 2 stages? Do we really need 100 steps for this level of detail?

[visualbruno]

@visualbruno what guidance strengths are the best for details/fidelity in the 2 stages? Do we really need 100 steps for this level of detail?

Hi. I'm not sure if 100 steps is better than 50. Sometimes with 20, it is not good enough.
When the guidance strength is at 10 in the 2 stages, it respects more the image.

Check the screenshots:
1st screenshot : Guidance of 0
2nd screenshot : Guidance of 10

The 2nd screenshot, the fidelity is very good

[cjjkoko]

@visualbruno what guidance strengths are the best for details/fidelity in the 2 stages? Do we really need 100 steps for this level of detail?

Hi. I'm not sure if 100 steps is better than 50. Sometimes with 20, it is not good enough. When the guidance strength is at 10 in the 2 stages, it respects more the image.

Check the screenshots: 1st screenshot : Guidance of 0 2nd screenshot : Guidance of 10

The 2nd screenshot, the fidelity is very good

My params:
simply:0.7
texture_size:2048

seed=20,
sparse_structure_sampler_params={
"steps": 100,
"cfg_strength": 7.5,
},
slat_sampler_params={
"steps": 100,
"cfg_strength": 3.5,
},
It works fine, but I'm still working on better parameters, including modifying some that aren't exposed

[visualbruno]

I think, the biggest issue is about the input picture resolution that is resized to 518x518 in trellis_image_to_3d.py
I tested with a higher resolution like 2058x2058, but the result was horrible. Probably they trained the model with this low resolution.

[cjjkoko]

I think, the biggest issue is about the input picture resolution that is resized to 518x518 in trellis_image_to_3d.py I tested with a higher resolution like 2058x2058, but the result was horrible. Probably they trained the model with this low resolution.

Maybe postprocessing will get a great result

[realisticdreamer114514]

@cjjkoko What kind of postprocessing do you use?

[cjjkoko]

@cjjkoko What kind of postprocessing do you use?

trimesh and open3d. use laplacian

[realisticdreamer114514]

trimesh and open3d

These are trying to smooth the output meshes without trying to improve the quality during generation.
You can see that even with a high-resolution input image and high texture size e.g. 4096, much detail of the final meshes' texture map is lost when they should be preserved (I tested with some character cosplay photos and found out about this). Might be better if someone has the GPU power to train/finetune the I23D model for it to work in input image resolution of say 770^2 or 1036^2, since as visualbruno points out the pipeline is set at the resolution the official model was trained on (518^2) and this kind of downsizing might explain detail loss.

[cjjkoko]

trimesh and open3d

These are trying to smooth the output meshes without trying to improve the quality during generation. You can see that even with a high-resolution input image and high texture size e.g. 4096, much detail of the final meshes' texture map is lost when they should be preserved (I tested with some character cosplay photos and found out about this). Might be better if someone has the GPU power to train/finetune the I23D model for it to work in input image resolution of say 770^2 or 1036^2, since as visualbruno points out the pipeline is set at the resolution the official model was trained on (518^2) and this kind of downsizing might explain detail loss.

Yes, but there is no specific date for the training.

[realisticdreamer114514]

there is no specific date for the training

Even at the current default resolution, the official I23D checkpoint seems quite undertrained (not sure if this is the right way to put it) so it doesn't adhere to the input image closely enough and tends to distort details that are still clear when downscaled. Finetuning on this framework can't come sooner...

[cjjkoko]

there is no specific date for the training

Even at the current default resolution, the official I23D checkpoint seems quite undertrained (not sure if this is the right way to put it) so it doesn't adhere to the input image closely enough and tends to distort details that are still clear when downscaled. Finetuning on this framework can't come sooner...

emmm, Expect great breaking updates in the next release. At this point, you can only adjust the seed to fit each image.I am currently using this very painful

[visualbruno]

I played with many parameters like the input image resizing, the number of sampling steps, texture resolution and the "number of views" used in the postprocessing. So I modified all main scripts and the app.py to play with these parameters.

The best result I got is with:

• Input image resized to 770, instead of 518.
• Number of Sampling Steps : 500
• Texture resolution : 2048
• Postprocessing "Number of Views": 120, instead of 100 (it removes a bit the artifacts on the texture)

For sure, with these values, it takes much more time to generate the model.

I tried to increase the input image resolution to 1036 and above, but the results were worse.
I tried with a number of sampling steps of 800 and 1000, but it did not improve the result too.
A texture resolution of 4096 is not better than 2048.
I tried with 200 for the "number of views" in post processing, it did not improve a lot the texture and the rendering time was multiplied by 10.

I tested with 2d anime pictures and it never renders very well, probably because this kind of pictures is flat and lacks relief and depth.

With Marlin from Seven Deadly Sins, Input Picture:

Result:

With Cleopatra, Input Picture:

Result:

With Knight, Input Picture:

Result:

[QuantumLight0]

I played with many parameters like the input image resizing, the number of sampling steps, texture resolution and the "number of views" used in the postprocessing. So I modified all main scripts and the app.py to play with these parameters.

The best result I got is with:

• Input image resized to 770, instead of 518.
• Number of Sampling Steps : 500
• Texture resolution : 2048
• Postprocessing "Number of Views": 120, instead of 100 (it removes a bit the artifacts on the texture)

For sure, with these values, it takes much more time to generate the model.

I tried to increase the input image resolution to 1036 and above, but the results were worse. I tried with a number of sampling steps of 800 and 1000, but it did not improve the result too. A texture resolution of 4096 is not better than 2048. I tried with 200 for the "number of views" in post processing, it did not improve a lot the texture and the rendering time was multiplied by 10.

I tested with 2d anime pictures and it never renders very well, probably because this kind of pictures is flat and lacks relief and depth.

With Marlin from Seven Deadly Sins, Input Picture: Result:

With Cleopatra, Input Picture: Result:

With Knight, Input Picture: Result:

I believe the model is under trained for anime models. anime models have flat normals so no depth, so I believe that a model needs to be train on anime model' faces in order for it to understand the faces. I do belive however the multidiffusion has potential in this area and I will provide a sample of why I believe so.
https://github.com/user-attachments/assets/bd2bf57b-0c55-42ad-b28c-2f8b2e5d84f9

[visualbruno]

@QuantumLight0 What parameters did you use to generate this model ?

[QuantumLight0]

@QuantumLight0 What parameters did you use to generate this model ?
I set everything to max

[visualbruno]

@QuantumLight0 I did not see they updated the repository with multi images algorithm. I will play with it.

[cjjkoko]

Any new breakthroughs ?