Skip to content

Releases: maximeraafat/BlenderNeRF

BlenderNeRF v6

06 Aug 23:38
b13b39e
Compare
Choose a tag to compare

This release introduces two major updates:

  • Transition from the deprecated Blender add-ons to the new Blender extension format convention introduced in Blender 4.2,
  • Support for Gaussian Splatting dataset creation.

Although the bl_info dictionary in the __init__.py file is no longer necessary, I have retained it for extended compatibility.

Gaussian Splatting Dataset Creation Support

The Gaussian Splatting repository (referred to as 3DGS) natively supports the NeRF file format convention, but benefits from an additional point cloud initialisation. This release enables the optional creation of a points3d.ply file when the Gaussian Points (PLY file) option is selected.

The points3d.ply file is generated using Blender's built-in bpy.ops.wm.ply_export function, introduced in Blender 4.0. Therefore Blender 4.0 is the lowest supported version of this release. While previous versions of BlenderNeRF (compatible with Blender versions >= 3.0.0) remain available, using Blender 4.0 or later is recommended for this functionality.

The points3d.ply file contains vertices of all visible meshes at render time, serving as the initial points for 3DGS. Only mesh objects visible in the render will be included. Meshes hidden in render or within hidden collections will be excluded.

Known issues

  • The functionality may break in specific scenarios, such as scenes with rendered child objects but hidden parent objects (e.g., a particle system where the emitter is hidden). Despite potential mismatches, the ply file is merely a starting point for 3DGS and will be optimised further.
  • Modifiers are applied to meshes during vertex storage, affecting the ply file's total vertex count. If more or fewer points are desired for 3DGS initialisation, consider applying subdivision or decimate modifiers accordingly.

Additional notes

  • Vertex colours: if vertex colours are available, they will be included in the ply file. If not, vertex colours will default to zero, as their existence is required by 3DGS.
  • Export time: generating the ply file can be time-consuming, naturally depending on the scene’s vertex count.

When using NeRF datasets, 3DGS currently requires both transforms_train.json and transforms_test.json camera files. For users solely interested in creating a splat file, the new dummy option creates an empty transforms_test.json file, bypassing the need for a complete test dataset. This is a temporary measure until the 3DGS repository potentially supports optional NeRF test camera poses.

File path adjustments

When exporting with the Gaussian Points option, file extensions are removed from paths in the transforms.json files, as 3DGS automatically appends a PNG extension. This feature remains until the 3DGS repository offers an argument for specifying file extensions or automatically detects them.

BlenderNeRF v5

11 May 17:54
6da9096
Compare
Choose a tag to compare

This release mainly focuses on fixing a bug pointed out in issue #18, in which a miscomputation of the camera intrinsics sometimes leads to mismatching fields of view between ground truth Blender renders and the corresponding NeRF renders. The mismatch is due to the wrong computation of focal lengths, when the camera sensor fit is not Horizontal.

To this end, 27 datasets have been captured for debugging purposes, in which different aspect ratios, pixel ratios and camera sensor fits have been evaluated. The dataset and respective Blender file are made available here and a thorough description of the data and capturing process is described in the contained README.txt file.

The mismatch should now have been alleviated. Below are depicted a few results in which NeRF renders (left column of each image) are compared to their respective ground truth with pixel ratio 1:1 rendered in Blender (right column). Camera parameters for each set of renders are described in the table underneath the image. The NGP renders are automatically stretched or squeezed to undo the non uniform pixel ratio in training frames. Feel free to inspect the data for further details.

debug

Left Middle Right
Sensor Fit Auto Horizontal Vertical
Aspect Ratio 1 : 1 16 : 9 2 : 3
Resolution 300 * 300 576 * 324 300 * 450
Pixel Ratio 1 : 1.5 1.5 : 1 1 : 1

Below are a two relevant observations and take aways.

  • For the middle image, the NeRF volume is partially cropped at the top and bottom of the donut. This is because the training images are stretched out (pixel ratio 1.5:1), and therefore a smaller region of the donut is only visible in these frames (see the corresponding frames in the data). As validated here, the NeRF render reshapes the scene to a uniform pixel ratio, thereby undoing the stretching effect.
  • The Vertical sensor fit often results in somewhat distorted NeRF volumes. I am however unsure of the cause, and consequently recommend avoiding it if possible.

This release additionally comprises a new functionality and a few warning/error fixes.

  1. The number of training frames used from the training camera with the TTC method can now be changed independently of the number of testing frames. The latter (number of testing frames) remains the frame range of the Blender scene.
  2. The issues highlighted in the pull request #19 have been resolved.

BlenderNeRF v4

11 Feb 23:19
Compare
Choose a tag to compare

New features and updated functionalities

  1. Starting frame with the COS training method is now the default scene frame start, instead of frame 1
  2. New option for outwards facing cameras with the COS training method
  3. Revised file and folder output structure: the output convention follows the original NeRF synthetic dataset structure. The output folder contains the transforms.json files directly, alongside a train folder with images
  4. Enabled support for the original NeRF transforms.json file format convention (alongside the Instant NGP file convention)
  5. Enabled option to save a log file containing all the necessary information for reproducibility on a BlenderNeRF run

BlenderNeRF v3

07 Jan 00:35
Compare
Choose a tag to compare

Third stable version of BlenderNeRF add-on : support for all methods (SOF, TTC and COS)

BlenderNeRF v2

19 Jul 15:50
Compare
Choose a tag to compare

Second stable version of BlenderNeRF add-on : support for SOF and TTC methods + simplified UI redesign

BlenderNeRF v1

13 Jul 22:39
Compare
Choose a tag to compare

First public release with stable version of BlenderNeRF add-on : currently only supports SOF with Instant NGP