Rendering Sequence
Unigine has a combination of full deferred renderer with forward rendering techniques:
- All opaque (non-transparent) geometry is rendered in the deferred pass
- Transparent geometry is rendered in the forward pass.
Before the deferred rendering pass, the engine performs all the necessary calculations. It specifies what nodes should be rendered, the order of rendering, how many lights on the scene and so forth.
This article introduces the detailed Unigine rendering pipeline including all the auxiliary stages which can be skipped by user.
Rendering Pipeline Overview
A frame is a complex structure including a lot of different calculations and textures rendering.
Here is a rendering pipeline, all stages are performed during one frame composition. Some stages can be switched off if it is necessary: decals rendering stage, auxiliary pass, post-materials, etc.
During the rendering, the engine initializes necessary texture buffers and most of them not cleared completely (the engine cuts out the sky areas). Some passes just re-use these already created buffers. Such optimizations help to increase the rendering performance.
The schematic representation of the rendering pipeline helps you to understand the major stages of the rendering order.
During the frame rendering, we will observe the simple test scene with a variety of materials (transparent and opaque), environment probes, decals, planar reflections, different light sources.
Common Pass
A Common pass is the very first step of the rendering sequence. In this pass environment LUTs, Environment cubemap are rendered. These textures are rendered only once before all the geometry on the scene and will be used further for the final screen composition: for scattering rendering and non-dynamic reflections.
|
Environment cubemap is composed from 6 pieces (each 256x256 px). |
Schematic representation of the 1x256 LUT textures.
|
LUT textures are represented as 1x256 px textures taken from the current state of the loaded LUT textures for scattering. |
Calculations
In this step, nothing will be rendered at all. This stage prepares the scene before geometry rendering: the engine analyzes the visible part of the scene to know which objects should be drawn and which shouldn't. The engine does the following:
- Checks the intersections of objects with frustum (frustum culling) and finds those which present in the frame.
- Finds all the light sources on the scene.
- Finds all the surfaces that should be rendered (taking into account occluder objects and sectors).
- Sorts Lights and Environment Probes by size (from large to small).
- Sorts Decal objects by their rendering order.
- Sets the Occlusion Query flags to objects that shouldn't be rendered.
- Surfaces Batching. All the opaque surfaces are grouped and rendered in batches according to materials assigned , decreasing the number of DIP calls, therefore, increasing the performance.
Occlusion Query works with a delay: it is a very performance-hit operation that requires much time. The optimization is the following: the engine checks the scene asynchronously and sets the "should not be rendered" flag during the Occlusion Query stage. That's why occlusion query runs at least 1 frame late. But the object will be hidden without "hard switching" when the FPS value is high.
Shadow Maps
After calculations, shadow maps are rendered. The way of rendering these maps depends on a type of the light source. Generated shadow maps will be used in the further rendering stages.
World Light
For the world lights, the engine creates cascaded shadow maps. World Light's shadow map is rendered into the orthographic projection.
The shadow map of the world light source is rendered into texture divided into four parts (since the 4 is the maximum quantity of shadow cascades).
|
Shadow map of the world light source. |
Shadows near the camera have higher resolutions, while shadows that far away from the camera have fewer details.
|
Rectangular shadow cascades on the final scene. |
Omni Light
Omni light sources use perspective projection for shadow maps. It uses 6 cameras that generate shadow maps.
Keep in mind that using a lot of Omni lights can sufficiently decrease the performance.
Proj Light
Shadow maps for Proj light sources are rendered only once because only 1 camera is used.
Dynamic Reflections
On this stage, the engine renders necessary textures for dynamic (that changed each frame) reflections: cubemaps for environment probes and 2D textures for planar reflections. To render these textures, the engine goes through all rendering pipeline (but misses some passes described below).
Cubemaps are generated by using 6 cameras, planar reflections use only 1 camera. Both render the final image (one more rendering cycle) with some hooks:
- Textures use shadow maps, that were already generated in the previous shadow maps rendering pass.
- All post effects are ignored.
- Final image for dynamic reflections is rendered missing TAA.
|
The final texture of the dynamic reflections. |
Deferred Pass for Opaque Objects
This one of the key passes of the rendering: all the opaque geometry is rendered during this pass one-by-one.
In the end of this pass, the engine has the texture (color texture) with all opaque geometry and scattering.
Native Depth Pre-Pass
In this pre-pass, the GPU performs a depth-test for surface culling. The pre-pass is performed only for alpha test and complex materials (like Terrain materials with a lot of layers).
The depth buffer stores native depth values z/w.
|
The depth data stored in a depth buffer texture format D32F |
During this pass, the GPU can discard a pixel (the shading for this pixel won't be calculated).
Filling the GBuffer
During this step of rendering, the engine fills the Gbuffer (Geometry buffer) for shading.
Depth
Depth buffer contains scene objects in the current field of view (found between the near and far clipping planes). Objects are rendered as pure geometric models and stored into this buffer.
The engine also uses native depth.
|
|
The depth data stored in a depth buffer texture format D32F |
Albedo
Albedo colors buffer stores pure albedo colors of all material textures:
|
The format of the texture is RGBA8 (RT0):
|
Shading
Shading buffer stores the shading information of objects on the scene.
|
The format of the texture is RGBA8 (RT1):
|
Translucency is used to simulate light shine through objects (leaves, for example).
Normal
Normal buffer stores normal vectors for each vertex of geometry which will be necessary to calculate the proper lighting.
|
The format of the texture is RGBA8 (RT2):
|
Velocity
Velocity buffer stores information about the displacement of pixels per frame. When the image is still the buffer is filled with zero (black) values. These values are necessary to make the temporal anti-aliasing (TAA) and motion blur work correctly.
|
The format of the texture is RG16F (RT3):
|
You can disable it if you don't need velocity-related effects: motion blur, TAA, etc.
Material Mask
Material mask texture. The first 8 bits are reserved for post effects, other bits can be used for materials.
|
The format of the texture is R32U (RT4):
|
Lightmap
Lightmap buffer is used to add baked light to the scene.
|
The format of the texture is RG11B10F (RT5):
|
Occlusion Query
During the Occlusion query step, the engine excludes objects (geometry, lights and decals) which have the Culled by occlusion query flag enabled from rendering sequence.
Occlusion query is an operation with a lot of calculations (i.e. has a huge influence on performance) and it is executed asynchronously. The engine checks object bounds and sets the flag the to renderer if it is necessary. In the next frame, during the Calculation stage, the renderer excludes the object from rendering sequence.
Decals
During this step of the deferred pass for opaque objects, the engine renders decals. Decals have been already sorted on the Calculations stage and during this stage the engine performs rendering by using this order.
Decals are rendered by using alpha blending, but since normal and shading textures' A channels are occupied by microfiber and roughness values, the engine copies necessary values into new textures and performs alpha blending.
|
New Normal texture (uncompressed) that is used for alpha blending |
|
New Roughness texture to store Roughness value in R channel, and Microfiber value in G channel. |
Shoreline Wetness
This rendering stage is responsible for shoreline wetness effect of the Global Water object. The behavior of this step is like in a previous Decals rendering step. So we can call this step Post-Decal Rendering.
The effect is performed by using Compute Shaders and Unordered Access Textures techniques.
Linear Depth, Color Old Reprojection and Unpacked Normals
To perform screen-space effects, the engine renders Linear Depth, Color Old (previous frame) Reprojection and Normal Unpack (screen-space normals) textures.
These textures will be used in Screen-Space Reflections and Screen-Space Global Illumination stages.
Linear depth texture is generated by using G-buffer depth. Mips of linear depth, Color old and normal unpack textures are also created.
SSRTGI
This stage performs the Screen-Space Ray-Traced Global Illumination post-effect during which the engine fills such textures using the Ray Tracing:
- SSAO, if the values of render_ssao, render_ssao_ray_tracing console variables are set to 1, defining whether should the SSAO and SSAO Ray Tracing perform.
- SSGI, if the values of render_ssgi, render_ssgi_ray_tracing console variables are set to 1, defining whether should the SSGI and SSGI Ray Tracing perform.
- Bent Normal, if the value of render_bent_normal_ray_tracing console variable is set to 1, which defines necessity of the Bent Normal performing.
Bent Normal is rendered in RGBA8 texture.
Bent Normal also may have own TAA depending on :
- If the noise is enabled and SSRTGI is rendered in full resolution, there is no TAA in this stage.
- If the noise is enabled and SSRTGI is rendered in half resolution or quarter resolution, TAA is applied.
|
New Ray-Traced SSAO texture |
|
New Bent Normal texture |
Deferred Light Pass
In the deferred light pass the engine creates a buffer (Deferred Light Map) which is a 2D array of RG11B10F textures. The first layer of the array is for diffuse light, the second is for specular (which can be switched off for VR).
Light sources use already generated shadow maps.
All lights sources are rendered one-by-one in the single 2D array to be applied during the Deferred Composite.
If you have 2 or more world light sources, 1 world light source will always be rendered during the deferred composite, other will be rendered during this step.
|
Light sources diffuse layer |
|
Light sources specular layer |
The engine uses tile rendering technique for Omni lights without shadows. With this optimization, the omni light sources are grouped and rendered in batches, decreasing the number of DIP calls, therefore, increasing the performance.
Deferred Reflections
During this stage of rendering, the engine renders all the environment probes that had been already sorted on the calculations stage.
Environment probes are rendered into the 2D array of RGBA16F textures: the first layer contains reflections, the second layer contains ambient light.
The renderer doesn't clear the deferred reflection textures if at least one environment probe has infinite size.
|
Reflection texture of Environment Probe |
|
Ambient texture of Environment Probe |
Reflection Pass
In the reflection pass, planar dynamic reflections (that had been already rendered) are applied. Dynamic reflections are rendered in the same texture that was used for Deferred Reflections.
|
Planar dynamic reflection is applied |
Auxiliary Pass
Auxiliary pass is a type of custom pass. Materials with Auxiliary pass enabled will be rendered in the auxiliary RGBA8 texture.
Auxiliary texture is often used for different post-effects. It has his own TAA stage.
|
Auxiliary buffer texture |
Refractions
On this stage of the pipeline, the refractions are rendered. They are rendered in the Refraction texture and will be applied in the transparent object stage.
Refraction is RGBA8 texture. The texture contains distortion values.
|
Refractions buffer texture |
Transparent Blur Surfaces
On this stage are rendered the transparent surfaces which are used in Transparent Blur effect. They are rendered in the Transparent Blur texture and will be applied in the post effects stage.
Transparent Blur is R16F texture, it contains blurriness values.
|
Transparent Blur buffer texture |
SSR
Screen-space Reflections are rendered in RGBA16F textures. It will be applied to the deferred composite stage.
|
SSR buffer texture |
There are two different types of SSR: with importance sampling and without:
Importance Sampling is On
If the importance sampling is enabled, the rendered doesn't use linear depth texture.
In this case, SSR color texture and SSR velocity texture are rendered. Velocity is used for avoiding artifacts whilst the camera or objects are moving.
SSR has its own TAA stage.
Importance Sampling is Off
In this case, three textures are rendered: SSR velocity, SSR color texture, and ray-length texture. After that, the renderer applies TAA for the SSR.
Three textures mentioned above are used to perform blurring to provide realistic reflection behavior.
SSAO
Screen-space Ambient Occlusion is rendered in its own R8 texture. It will be applied in the deferred composite stage.
SSAO can be rendered with or without noise. This option has influence on TAA:
- If the noise is enabled and SSAO is rendered in full resolution, there is no TAA in this stage.
- If the noise is enabled and SSAO is rendered in half resolution, this stage has its own TAA.
|
SSAO texture |
SSGI
Screen-space Global Illumination is rendered in 1 RG11B10F texture.
SSGI stage also has own TAA.
Underwater Fog
In this step, the Underwater Fog texture is initialized and cleared.
The renderer does it here to perform the underwater world lighting correctly.
Deferred Composite
During this stage, the engine uses all the necessary textures from the previous stages and passes to create the final texture with opaque geometry. The engine combines buffers and calculates shading for the final texture of this pass.
In this stage, the renderer calculates the light of the 1 World light and applies it. Environment reflections, ambient, haze (scattering) are also rendered here.
|
The final image of deferred pass for opaque geometry (excluding the Emission pass and SSS) |
Emission
Emission pass goes after the deferred composite image created. It applies emission effect over the deferred composite image.
|
Emission applied |
SSS
Subsurface Scattering changes the final deferred texture: it applies the exponential blur.
|
SSS applied |
Transparent Objects Rendering
Almost all transparent objects are rendered in the forward rendering pass.
During the forward pass, the renderer fills the deferred buffers to let forward transparent object participate in post-effects.
Refraction
During this stage of rendering sequence, the refraction texture (which had been already rendered) is applied
|
Refraction applied |
Volumetric Clouds
On this step all the layers of procedural Volumetric Clouds are rendered into the RGBA16F texture using parameters of the clouds_base material.
|
An example of the Clouds texture content |
Due to optimization, the Volumetric Clouds will be rendered in the certain order depending on a couple of states:
- Before Water and Transparent Objects - if the value of the render_clouds_transparent_order console command is set to 0 and the current camera is below the water highest point or under the surface.
- Between Water and Transparent Objects - if the value of the render_clouds_transparent_order console command is set to 0 and the current camera is placed over the water highest point.
- After Water and Transparent Objects - if the value of the render_clouds_transparent_order console command is set to 1.
Water
Water rendering in Unigine is really complex thing: it has its own deferred buffer (called WBuffer) with light and environment probes passes.
FieldHeigth and FieldShoreline
First, FieldHeight and Field Shoreline textures are rendered.
Height
Field Height textures are rendered in a 2D array of R16F or R32F textures (depends on settings). All Field Height textures are packed into 2D array to pass the data to the shader.
Shoreline
All Field Shoreline textures are also are packed into 2D array (of RGBA8) to pass the data to the shader.
WBuffer
The renderer initializes the WBuffer.
Diffuse
The diffuse color of the water is black, and diffuse texture is necessary for decals, that will be displayed over the water surface.
|
The format of the texture is RGBA8:
|
Normal
The normal texture stores normal for lighting, and alpha channel stores mesh transparency values (it can be used for soft intersections with water geometry).
|
The format of the texture is RGBA8:
|
Water
The Water texture is used to create the procedural foam mask. The mask shows where the foam will be depicted.
|
The format of the texture is RG8:
|
Constants
We use a Deferred Constant Transferring approach for water meshes. This channel stores the ID value of the water mesh which is used to load the corresponding textures and parameters for it.
|
The format of the texture is R32U:
|
Underwater Mask
Underwater mask is used only for Global water, since water mesh doesn't have an underwater mode.
|
The format of the texture is RGB8:
|
Underwater Fog
|
The format of the texture is RGBA16:
|
Copy Opacity Screen
During this step, the renderer copies the opacity screen that has been rendered before into new texture.
Clear Buffers Textures
In this stage, the renderer clears buffer texture to prepare for water rendering. It clears reflection texture (both specular and diffuse), Water buffer texture, and, if there is water mesh on the scene, Constants and Underwater Fog textures.
Select the Water Mode
After, the renderer checks the camera position to know, what part of the water should be rendered. There are 3 modes:
- UNDERWATER. Only underwater will be rendered.
- OVERWATER. Only the upper surface of the water will be rendered.
- BOTH. Both underwater and overwater will be rendered including separating waterline.
Filling the WBuffer
In this step, the WBuffer textures are filled with the corresponding data.
In case of water mesh, WBuffer is rendered like GBuffer because it doesn't have an underwater mode.
Global water is rendered differently: most buffers are rendered as deferred buffers but caustics of the water, fog, and underwater mask are rendered as post effects.
Water Decals
In this stage, water decals are rendered. Decals are rendered (normal and diffuse) by using alpha-blending.
Water Lights and Environment Probes
After decals DLM (Deferred Light Map) and Environment Probes textures are rendered.
Underwater Shafts
Here underwater shafts are rendered by using underwater mask shaft samples values (stored in B channel). They are rendered only if the camera has UNDERWATER mode or BOTH.
Water Composite
After all of the water stages, the renderer writes two composite for water pass:
- For underwater
- For overwater
After water composite the waterline (a black separating line) is rendered.
Water DOF
After the composite texture is rendered, the depth of field effect for water is rendered. It performs blurring over the whole texture.
If the Global DOF is enabled, it is used the depth of the water (water will be rendered to depth texture where opaque object are already rendered) to perform the DoF blurring correctly.
Sorting Transparent Objects
Transparent objects are sorted from the farthest to the nearest and will be rendered one by another.
Transparent Objects Rendering
Transparent objects rendering has two modes: Multiple Environment Probes is on and off depending on the value of the Multiple Environment Probes parameter of the mesh_base material.
Multiple Environment Probes is On
First, the Depth Buffer is rendered and Environment Probes texture is cleaned.
Environment Probe Rendering
During this stage, environment probe ambient and reflection are rendered into Environment Probes texture.
Ambient Pass
By using Environment Probe texture, the renderer lerps sky ambient and reflection with Environment Probe texture.
Light Passes
The renderer adds all the lights to the texture.
Multiple Environment Probes is Off
Ambient Pass
During the ambient pass the environment, lightmaps, emission for transparent objects are calculated.
Light Passes
During the light passes for transparent objects, the light is calculated. Render goes only through passes that were specified in the material states.
Ultimately, all the lights for transparent objects are added one after another.
|
After the transparent objects rendering. |
Filling Deferred Buffers
Transparent objects also write information into deferred buffers to let them participate into post-effects.
The following buffers are modified on this stage:
- Albedo
- Normal
- Shading
- Opacity Depth
- Material Mask
Color Texture Copy
Current screen (color) texture is copied in this stage. The texture will be used for TAA and also for SSR and SSGI screen-space effects.
SRGB Correction and Static Exposure
In this pass, the engine performs SRBG correction and static exposure (if it is enabled). If the static exposure is off, the engine goes to the Adaptive Exposure stage.
|
After the SRGB correction |
Adaptive Exposure
During this step, the engine renders luminance texture and applies it to the screen by using corresponding exposure: logarithmic or quadratic.
|
After the Adaptive Exposure |
Temporal Anti-Aliasing (TAA)
This pass is used when TAA is enabled. Otherwise, the engine skips this pass and goes to the next one.
|
After the TAA |
TAA uses previous frames to improve the current frame by using linear interpolation.
Transparent Blur
On this stage, the blurinness behind transparent objects is applied. The renderer uses the texture from the Transparent Blur Surfaces stage.
|
The frame with the Transparent Blurriness applied |
Render Post Materials
In this pass, the engine generates procedural textures for Render Post Materials that should affected by the Camera Effects.
Camera Effects
Almost all camera effects are rendered in their own textures to be used in the final screen composition stage to create the final image.
Motion Blur
Motion blur uses Velocity buffer for texture blurring. It hasn't its own buffer and changes the screen.
Depth of Field (DoF)
For the Depth of Field effect, the engine generates a DOF RG8 mask.
- R channel stores the farthest blurring values (that behind the in-focus objects).
- G channel stores the nearest blurring values (that in front of the in-focus objects).
DOF works with RG11B10F texture for fast work, therefore, if the screen texture has RGB16F it will be converted into RG11B10F in this step. Also, renderer performs some accuracy operations to improve the texture in the boundary areas of objects that in focus.
Chromatic aberrations are also rendered here.
Bright Texture
The renderer generates new texture which pixels are brighter than the threshold that had been set. This texture determines the areas that will be illuminated with other camera effects that mentioned below:
- Sun Shafts
- Bloom
- Cross
- Lens
Sun Shafts
Sun shafts are rendered into their texture and will be applied in the final screen composition stage.
Bloom
If the bloom is enabled, the engine generates up to 8 bloom textures: each texture has lower resolution (original size, original size /2, original size /4, so forth) with bloom effect. After that, all these bloom textures with the different resolution are composed for the final bloom texture.
Cross and Lens textures use the bloom texture (not final one) which has 1/4 screen resolution even if the bloom is not rendered.
Cross
Cross texture uses the original size /4 bloom texture to create the cross camera effect even if the bloom is disabled.
Lens
Lens texture also uses the original size /4 bloom texture to create the bloom effect even if the bloom is disabled. Dirt on the lens is also applied here.
Shadow Shafts
In this step, the engine renders the texture with shadow shafts. That texture will be used in the final screen composition stage.
Shadow shafts have their own mask (specifies where shadow shafts should be applied) and TAA for that mask.
Final Screen Composition
On this stage, the engine assembles a final texture by using all the textures that were rendering in the previous steps. Filmic tone mapping, dithering, and the LUT texture is applied here during the final screen composition.
|
Final screen composition. |
Post Materials
After the final screen composition, all the post-effects applied.
If the procedural texture is necessary for a post-material, it will be calculated in this step and then the post-effect will be applied to the final composed screen.
Post Engine Materials
Border
In this engine post effect, the engine cuts out the edges of the screen, which are outside the main window.
FXAA
If the FXAA is enabled, the engine doesn't perform TAA pass and performs the anti-aliasing here by using FXAA algorithm.
Sharpen
If the Sharpen effect is enabled, the engine applied it here to the final screen.
Post Debug Materials
During this step, the Debug Materials are applied to the composed image.
Transparent Overlap
On this step are rendered the transparent surfaces with the Overlapping enabled. It is the very last rendering stage of surfaces that lets to create nodes which are not affected by all the post-effects, e.g. tooltips.
Visualizer
On this stage the engine visualizer is rendered: mesh wireframes, bounding boxes, nodes culled with occlusion query, etc.
Fade Material
Fade material is used to create a smooth screen blacking. It usually used for the beginnings and endings of tracks for Tracker.
GUI
GUI is rendered last after all the stages and passes are rendered.
|
After GUI rendering. |
Finally, the engine rendered a single frame!