Point cloud shading

[j9liu]

Intro

Building off of #197, we'll want to add something akin to PointCloudShading in CesiumJS to Cesium for Unity. This will improve the visual quality of point clouds in Unity, should users choose to add it to their point clouds.

The main benefits of point cloud shading are:

• attenuation, which allows more control over point size (points are currently drawn as a single pixel)
• eye dome lighting (EDL), which shades points so that they appear to have more depth. This will be important when we allow users to increase point size, since constant-color point clouds will appear like big globs of color without it.

This Sandcastle demonstrates PointCloudShading's effects.

No shading	Attenuation only	Attenuation + EDL

In CesiumJS, attenuation accounts for various factors and adjusts the gl_pointSize accordingly. EDL is accomplished by rendering the depth values of the point cloud to a separate texture, then blending them with the point cloud colors. (CesiumJS implementation here) So to summarize, the following is necessary to implement point cloud shading:

• a way to visualize points at larger sizes
• the ability to write to two outputs, depth + color
• the ability to blend the two outputs together

Unity implementation

Points are currently rendered just like other tilesets. The only difference is that the mesh is constructed with MeshTopology.Points instead of MeshTopology.Triangles. This is a great start, but there's no way to control the size of the points that are rendered. I also don't know how to get the depth values of only the points that are rendered

Here, I'm chronicling all of the methods I've considered / tried.

• ❌ Use the PSIZE attribute in a Unity shader
  • This involves forcing Unity to use OpenGL, which is pretty unideal.
• ❌ Write a geometry shader to expand the points to quads
  • Geometry shaders are not supported by Metal.
• ❔ Draw instanced quad meshes wherever the point positions are
  • This only works with DrawMeshInstanced. I was not able to get DrawMeshInstancedProcedural or DrawMeshInstancedIndirect to work because apparently the instance ID attribute is not available in those functions, so it's 0 for every instance 🤔
  • This method involves some duplication of data, since one of the arguments to DrawMeshInstanced is a Matrix4x4[] array.
  • You'd have to convert the desired point size to an actual world scale for each quad -- something like the way minimumPixelSize affects a model's scale in CesiumJS.
  • This is compatible with Shader Graph, so it might be easier to integrate with what we do for Feature styling with metadata #200.
  • I haven't looked into how I'd extract the depth values with this method yet, if it's possible.
• ❔ Use a compute shader to compute point size, then expand the points to quads.
  • This should be supported by all the platforms Cesium for Unity supports, according to the documentation.
  • This can be used as an alternative to a geometry shader, and can also help with rendering to multiple outputs.
  • This isn't compatible with Shader Graph, since it only expects one position output. This could have implications for Feature styling with metadata #200.

I'll update this issue as I go so I can document my various approaches / struggles / woes with the Unity implementation.

[kring]

One more implementation idea: Turn every point into four vertex (all at the same location, the location of the point) and two triangles (connecting the four triangles into a quad) on the CPU. Add a vertex attribute that indicates which "corner" each vertex corresponds to (i.e. top-left, top-right, bottom-left, bottom-right). Render as a single mesh. In the vertex shader, displace the vertices according to the corner and desired point size. Cozzi describes the technique in chapter 9 of the virtual globe book, and I think this is what CesiumJS does due to the lack of geometry shaders in WebGL. The only part that might be challenging in Unity is that extra vertex attribute for the corner indicator.

[j9liu]

For the record: attenuation was added in #218, but we still don't support eye dome lighting.

[j9liu]

I'm concerned with how users will want to apply a material to a point cloud AND apply attenuation. Attenuation is hardcoded in a single material that doesn't flex for other user inputs. I wonder if it's possible to convert attenuation to a shader subgraph (building off of #265), and attaching it to the default tileset shader, as a branch that is traversed when attenuation is enabled. (Of course, it will only be functional if a tileset actually contains points.)

So maybe it's worth looking into #265 again and seeing how it interacts with the default shader. If it can be made into a subgraph then hopefully a user can plug it into their own materials, should they need a custom shader effect with attenuation.