Releases: maximeraafat/BlenderNeRF
BlenderNeRF v6
This release introduces two major updates:
- Transition from the deprecated Blender add-ons to the new Blender extension format convention introduced in Blender 4.2,
- Support for Gaussian Splatting dataset creation.
Although the bl_info
dictionary in the __init__.py
file is no longer necessary, I have retained it for extended compatibility.
Gaussian Splatting Dataset Creation Support
The Gaussian Splatting repository (referred to as 3DGS) natively supports the NeRF file format convention, but benefits from an additional point cloud initialisation. This release enables the optional creation of a points3d.ply
file when the Gaussian Points (PLY file)
option is selected.
The points3d.ply
file is generated using Blender's built-in bpy.ops.wm.ply_export
function, introduced in Blender 4.0. Therefore Blender 4.0 is the lowest supported version of this release. While previous versions of BlenderNeRF (compatible with Blender versions >= 3.0.0
) remain available, using Blender 4.0 or later is recommended for this functionality.
The points3d.ply
file contains vertices of all visible meshes at render time, serving as the initial points for 3DGS. Only mesh objects visible in the render will be included. Meshes hidden in render or within hidden collections will be excluded.
Known issues
- The functionality may break in specific scenarios, such as scenes with rendered child objects but hidden parent objects (e.g., a particle system where the emitter is hidden). Despite potential mismatches, the
ply
file is merely a starting point for 3DGS and will be optimised further. - Modifiers are applied to meshes during vertex storage, affecting the
ply
file's total vertex count. If more or fewer points are desired for 3DGS initialisation, consider applying subdivision or decimate modifiers accordingly.
Additional notes
- Vertex colours: if vertex colours are available, they will be included in the
ply
file. If not, vertex colours will default to zero, as their existence is required by 3DGS. - Export time: generating the
ply
file can be time-consuming, naturally depending on the scene’s vertex count.
When using NeRF datasets, 3DGS currently requires both transforms_train.json
and transforms_test.json
camera files. For users solely interested in creating a splat file, the new dummy
option creates an empty transforms_test.json
file, bypassing the need for a complete test dataset. This is a temporary measure until the 3DGS repository potentially supports optional NeRF test camera poses.
File path adjustments
When exporting with the Gaussian Points
option, file extensions are removed from paths in the transforms.json
files, as 3DGS automatically appends a PNG extension. This feature remains until the 3DGS repository offers an argument for specifying file extensions or automatically detects them.
BlenderNeRF v5
This release mainly focuses on fixing a bug pointed out in issue #18, in which a miscomputation of the camera intrinsics sometimes leads to mismatching fields of view between ground truth Blender renders and the corresponding NeRF renders. The mismatch is due to the wrong computation of focal lengths, when the camera sensor fit is not Horizontal
.
To this end, 27 datasets have been captured for debugging purposes, in which different aspect ratios, pixel ratios and camera sensor fits have been evaluated. The dataset and respective Blender file are made available here and a thorough description of the data and capturing process is described in the contained README.txt
file.
The mismatch should now have been alleviated. Below are depicted a few results in which NeRF renders (left column of each image) are compared to their respective ground truth with pixel ratio 1:1 rendered in Blender (right column). Camera parameters for each set of renders are described in the table underneath the image. The NGP renders are automatically stretched or squeezed to undo the non uniform pixel ratio in training frames. Feel free to inspect the data for further details.
Left | Middle | Right | |
---|---|---|---|
Sensor Fit | Auto |
Horizontal |
Vertical |
Aspect Ratio | 1 : 1 | 16 : 9 | 2 : 3 |
Resolution | 300 * 300 | 576 * 324 | 300 * 450 |
Pixel Ratio | 1 : 1.5 | 1.5 : 1 | 1 : 1 |
Below are a two relevant observations and take aways.
- For the middle image, the NeRF volume is partially cropped at the top and bottom of the donut. This is because the training images are stretched out (pixel ratio 1.5:1), and therefore a smaller region of the donut is only visible in these frames (see the corresponding frames in the data). As validated here, the NeRF render reshapes the scene to a uniform pixel ratio, thereby undoing the stretching effect.
- The
Vertical
sensor fit often results in somewhat distorted NeRF volumes. I am however unsure of the cause, and consequently recommend avoiding it if possible.
This release additionally comprises a new functionality and a few warning/error fixes.
- The number of training frames used from the training camera with the TTC method can now be changed independently of the number of testing frames. The latter (number of testing frames) remains the frame range of the Blender scene.
- The issues highlighted in the pull request #19 have been resolved.
BlenderNeRF v4
New features and updated functionalities
- Starting frame with the COS training method is now the default scene frame start, instead of frame 1
- New option for outwards facing cameras with the COS training method
- Revised file and folder output structure: the output convention follows the original NeRF synthetic dataset structure. The output folder contains the
transforms.json
files directly, alongside atrain
folder with images - Enabled support for the original NeRF
transforms.json
file format convention (alongside the Instant NGP file convention) - Enabled option to save a log file containing all the necessary information for reproducibility on a BlenderNeRF run
BlenderNeRF v3
Third stable version of BlenderNeRF add-on : support for all methods (SOF, TTC and COS)
BlenderNeRF v2
Second stable version of BlenderNeRF add-on : support for SOF and TTC methods + simplified UI redesign
BlenderNeRF v1
First public release with stable version of BlenderNeRF add-on : currently only supports SOF with Instant NGP