R.

Login & Accounts are under-development

R.

How to Preserve VR Performance when Anti-Aliasing



File(s): TSCMAA-CodeSample (76.6 MB)
License: BSD 3-clause






Optimized with..  
OS: Home windows® 10
{Hardware}: Intel® HD Graphics 630
Software program: Microsoft Visible Studio* 2017,

Common Home windows* Platform improvement,

Home windows 10 SDK 10.0.15063
Stipulations: An understanding of DirectX* 11 is useful however not important.

Seshupriya Alluru, Rebecca David, Matthew Goyder, Anupreet Karla, Yaz Khabiri, Sungye Kim, Pavan Lanka, Filip Strugar, and Kai Xiao contributed to TSCMAA and samples supplied on this doc.

This text describes Temporally Steady Conservative Morphological Anti-Aliasing (TSCMAA), which was designed and developed as a multisample anti-aliasing (MSAA) alternate method to offer higher temporal stability and efficiency workaround for Digital Actuality (VR) functions.

Introduction

Anti-aliasing (AA) in pc graphics refers to a set of methods used to beat aliasing artifacts within the rendered pictures, that are a byproduct when representing a high-resolution picture at a decrease decision (for instance, rasterization or sampling). MSAA is probably the most extensively used spatial AA algorithm in varied functions however at the price of efficiency. Publish-processing AA algorithms like Conservative Morphological Anti-Aliasing (CMAA), Quick Approximate Anti-Aliasing (FXAA), and Subpixel Morphological Anti-Aliasing (SMAA) can present fixed efficiency however undergo from temporal instability. For VR, AA is extra essential for the reason that show is nearer to the eyes, thus artifacts are extra noticeable, and temporal stability turns into a key facet in offering a superb consumer expertise.

This text discusses TSCMAA, which employs an optimized CMAA and integrates temporal accumulation with a view to create a temporally steady post-processing AA resolution as a MSAA alternate method with out compromising on picture high quality. TSCMAA can also be designed to run effectively on low- and medium-end graphics processing models (GPUs), comparable to built-in GPUs, and to be minimally invasive. This makes it acceptable as an MSAA alternate with higher temporal stability in a variety of functions, which embrace aliasing-prune geometry comparable to textual content, patterns, strains, and foliage.

Determine 1 exhibits a TSCMAA move that mixes a CMAA go (high) and a TAA go (backside). In TSCMAA, CMAA and TAA passes run just for edge candidates recognized by edge detection, leading to quick oblique dispatch of shaders.



Determine 1. TSCMAA move.

Conservative Morphological Anti-Aliasing

CMAA takes a rendered colour body as an enter and updates the body with spatially anti-aliased edge pixels by processing edge candidates (any form and Z-shape edges). To determine such edge candidates, edge detection makes use of the luminance distinction in colour with a given edge threshold. In TSCMAA, a default edge threshold worth is 0.045f (1.f/22.f) based mostly on experiments with our pattern scene, however it may be adjusted for different scenes as a high quality versus efficiency knob. The small print of CMAA are discovered within the authentic article.

Temporal Anti-Aliasing

To scale back temporal aliasing comparable to shimmering or crawling brought on by the movement between frames, TAA blends a present pixel with a historical past pixel. Then the output of TAA turns into a historical past body for the following TAA body in a suggestions loop. In VR, TAA is extra essential on account of fairly a little bit of jitter from head pose modifications over time, leading to rendered frames at all times containing movement in a head-mounted show (HMD). TSCMAA takes care of each spatial and temporal aliasing by combining CMAA and TAA, and main advantages come up as a result of TSCMAA runs just for edge pixels, leading to sooner pixels which are much less blurred general, in comparison with the outcomes of different TAA algorithms.

TAA Edge Candidates

Solely a portion of edge candidates from edge detection are thought of as TAA edge candidates. In TSCMAA, we use 50 p.c of CMAA edge candidates for TAA. This default worth is predicated on experiments with our pattern scene however might be adjusted for different scenes as a high quality versus efficiency knob. Determine 2 exhibits edge detection outcomes with the default edge threshold worth for CMAA edge candidates on the left and TAA edge candidates on the best. The TAA go is utilized solely to the pixels in TAA edge candidates, leading to quick TAA and fewer blurring on non-edge pixels.



Determine 2. Edge candidates for CMAA (left) and TAA (proper). TAA edge candidates are 50 p.c of the sCMAA edge candidates.

Historical past Pixel Sampling

The historical past body maintains the earlier TSCMAA body so {that a} CMAA pixel is mixed with a historical past pixel to generate the present TSCMAA body (see Determine 1). To discover a appropriate historical past pixel, we reproject a texture coordinate with view and projection of the present rendered body based mostly on depth. Then a historical past pixel is sampled with the reprojected texture coordinate utilizing bicubic sampling. TSCMAA employs a five-tap approximation for a Hermite/Catmull-Rom bicubic filter for sharpness-preserving sampling in a historical past body however with sooner approximation.

Variance Clipping

When there are transferring objects in a scene, reusing stale pixels within the historical past body creates a ghosting artifact. A neighborhood colour clipping utilizing an axis-aligned bounding field (AABB) has been used in its place for costly colour clipping utilizing a convex hull. Nevertheless colour clipping utilizing AABB leads to poor high quality when a brand new colour clipped by AABB is just too removed from the unique pixel. Many articles focus on the advantages of variance clipping to offer much less ghosting artifacts. TSCMAA makes use of variance clipping in YCoCg house to attenuate such a ghosting artifact on transferring objects.

Mixing with the CMAA Pixel and TSCMAA Historical past Pixel for TAA Edge Candidates

For the reason that TAA go is processed just for TAA edge candidates, mixing between CMAA pixels and historical past pixels can also be achieved just for TAA edge candidates with a mix weight of 0.8f and different non TAA edge pixels are instantly taken from CMAA pixels which is identical to utilizing a mix weight of 0.f. Then the ultimate blended output generates a TSCMAA body and turns into a historical past body for the following TSCMAA body as proven in Determine 1.

Pattern Purposes

Within the TSCMAA pattern bundle attachment, we offer two VR pattern functions utilizing the identical take a look at scene as proven in Determine 3. One (AASample_WMR) is for Microsoft’s Home windows Combined Actuality (WMR) HMD and the opposite (AASample) makes use of OpenVR* so that it’ll work on all OpenVR-compatible HMDs. AASample may also run as a desktop software with out the “_OPENVR_” preprocessor in undertaking configuration properties.



Determine 3. TSCMAA pattern scene with strains, foliage, and a clear object.

High quality and Efficiency

Determine Four exhibits a high quality comparability amongst MSAA2x, MSAA4x, TSCMAA, and No AA. TSCMAA high quality is reasonably equal to MSAA4x when considered with an HMD and general higher than MSAA2x with default edge threshold values. For the reason that TAA go in TSCMAA is predicated on temporal accumulation, someway blurred pixels are inevitable within the last TSCMAA body. Nevertheless, we decrease the blurring by accumulating solely TAA edge pixels in a historical past body and supply higher temporal stability. Since we can’t present TAA advantages with static screenshots, this text additionally offers an connected pattern code bundle with TSCMAA shaders .



Determine 4. High quality comparability amongst MSAA2x (high left), MSAA4x (high proper), TSCMAA (backside left), and No AA (backside proper).

TSCMAA efficiency will depend on a number of features, comparable to edge depend within the scene and render goal decision. For our take a look at scene, TSCMAA exhibits MSAA4x high quality with round a 40 p.c value discount and is equal to MSAA2x efficiency with default edge threshold values. Right here we additionally present how TSCMAA efficiency scales as the sting depend within the scene and render goal decision change. These experiments had been carried out with 1280×1280/eye on Intel® HD Graphics 630 system at 1150 MHz.

Since TSCMAA runs solely on edge candidates after edge detection, and efficiency will depend on the sting pixel depend within the present view. Desk 1 exhibits TSCMAA efficiency scaling for various edge counts from 53Okay to 100Okay pixels. The baseline accommodates 73Okay edge pixels, which is from a view proven in Determine 3. Desk 1 exhibits TSCMAA efficiency modifications of 10‒15 p.c for a 30 p.c distinction in edge depend.

Desk 1. TSCMAA efficiency scaling ratio for various edge depend. Baseline (1.0) has 73Okay edges and we evaluate with completely different scene (or views) the place edge depend is 30 p.c much less or better than baseline.




Scale Issue over Base 53Okay Edges 73Okay Edges (Base = 1.0) 100Okay Edges
Edge depend 0.72 1.0 1.36
TSCMAA time 0.86 1.0 1.08

Desk 2 exhibits TSCMAA efficiency scaling for 2Kx2K render goal decision whereas sustaining the identical edge depend. In comparison with a baseline with 1280×1280 decision, a 2Kx2K render goal accommodates a 2.56x bigger variety of pixels, contributing to a TSCMAA efficiency enhance of as much as 1.6x. The rise is generally from an edge detection step which is decision dependent.

Desk 2. TSCMAA performance-scaling ratio for various render goal decision. Baseline (1.0) is 1280×1280 per eye, and we evaluate with 2Kx2K per eye whereas preserving edge depend.





Scale Issue over Base 1280×1280/Eye (Base = 1.0) 2Kx2K/Eye
Pixel depend 1.0 2.56
Edge depend 1.0 1.0
TSCMAA time 1.0 1.59

Desk Three exhibits the mixed contribution from bigger render goal decision and thereby the elevated variety of edge pixels.

Desk 3. TSCMAA efficiency scaling ratio for various render goal decision.





Scale Issue over Base 1280×1280/Eye (Base = 1.0) 2Kx2K/Eye
Pixel depend 1.0 2.56
Edge depend 1.0 1.91
TSCMAA time 1.0 2.12

Supported Useful resource Codecs

To use TSCMAA, functions are required to make use of one in every of TSCMAA supported useful resource codecs for render goal and depth sources. For the render goal texture, TSCMAA helps 32- bit RGBA and BGRA typeless codecs, comparable to

  • DXGI_FORMAT_R8G8B8A8_TYPELESS
  • DXGI_FORMAT_B8G8R8A8_TYPELESS and creates inner view sources with corresponding UNORM and UNORM_SRGB codecs.
  • DXGI_FORMAT_R8G8B8A8_UNORM
  • DXGI_FORMAT_R8G8B8A8_UNORM_SRGB
  • DXGI_FORMAT_B8G8R8A8_UNORM
  • DXGI_FORMAT_B8G8R8A8_UNORM_SRGB

The 32-bit BGRA format is the one useful resource format that Microsoft’s WMR software makes use of on the time of this writing.

For depth, TSCMAA helps 24- and 32-bit useful resource codecs with or with no stencil, comparable to:

  • DXGI_FORMAT_D32_FLOAT
  • DXGI_FORMAT_D24_UNORM_S8_UINT
  • DXGI_FORMAT_D32_FLOAT_S8X24_UINT
  • DXGI_FORMAT_R32_TYPELESS
  • DXGI_FORMAT_R24G8_TYPELESS
  • DXGI_FORMAT_R32G8X24_TYPELESS

TSCMAA Interface

TSCMAA offers a easy interface to help each customary desktop and VR functions.

TSCMAA::Create(…) initializes TSCMAA.


1 HRESULT TSCMAA::Create(ID3D11System * pDevice,


2                        DXGI_FORMAT format,


3                        int width,


4                        int peak,


5                        unsigned int numEyes = 1);
  • pDevice: D3D11 gadget pointer from software
  • format: Render goal useful resource format
  • width: Render goal width
  • peak: Render goal peak
  • numEyes: The variety of eyes. Default is 1 for traditional desktop software. For VR functions, numEyes needs to be 2.

TSCMAA::Resize(…) is optionally referred to as to resize sources in TSCMAA when the appliance render goal is resized.


1 HRESULT TSCMAA::Resize(ID3D11System * pDevice,


2                        DXGI_FOARMAT format,


3                        int width,


4                        int peak);
  • pDevice: D3D11 gadget pointer from software
  • format: Render goal useful resource format
  • width,/code>: Render goal width
  • peak: Render goal peak

TSCMAA::Draw(…)applies TSCMAA-given colour and depth sources and returns the ultimate TSCMAA useful resource to ppOutTex. To supply enter colour and depth sources from the appliance to TSCMAA, the appliance ought to put together ColorDepthIn by calling ColorDepthIn::Create(…).


1 HRESULT TSCMAA::Draw(ID3D11DeviceContext * pContext,


2                     ColorDepthIn * pColorDepthIn,


3                     DirectX::XMFLOAT4x4 &projection,


4                      DirectX::XMFLOAT4x4 &view,


5                     ID3D11Texture2D ** ppOutTex,


6                     unsigned int eye = 0);
  • pContext: D3D11 gadget context pointer from the appliance
  • pColorDepthIn: A pointer of ColorDepthIn for TSCMAA that’s created within the software aspect by calling ColorDepthIn::Create(…)
  • projection: Projection matrix
  • view: View matrix
  • ppOutTex: A pointer of TSCMAA output texture useful resource, which resides within the TSCMAA aspect
  • eye: eye indexm the place the left eye is Zero and the best eye is 1

To destroy TSCMAA, name TSCMAA::Destroy() which will even launch all sources in TSCMAA. void TSCMAA::Destroy();

TSCMAA has a number of management knobs to regulate the variety of edges. Since CMAA and TAA passes run solely on edge candidates, the variety of edges decides efficiency and high quality. To regulate the variety of edge pixels detected, TSCMAA offers SetEdgeThresholds(…) which is able to set an edge threshold and a non-dominant edge elimination quantity for edge detection.


1 void TSCMAABase::SetEdgeThresholds(float edgeDetectionThreshold,


2                                    float nonDominantEdgeRemovalAmount,


3                                    float bluriness);
  • edgeDetectionThreshold: The advisable worth ranges [(1.f/32.f), (1.f/1.f)] and default worth is (1.f/22.f).
  • nonDominantEdgeRemovalAmount: The advisable worth ranges [0.f, 3.f] and default worth is 0.5f.
  • bluriness: The advisable worth ranges [0.5f, 2.f] and default worth is 0.7f.

Word that bluriness doesn’t have an effect on edge detection however does have an effect on processing edge candidates. Therefore edgeDetectionThreshold and nonDominantEdgeRemovalAmount are knobs to manage the variety of edges in edge candidates.

To get the present edge thresholds, use TSCMAA::GetEdgeThresholds(…).


1 void TSCMAABase::GetEdgeThresholds(float &edgeDetectionThreshold,


2                                   float &nonDominantEdgeRemovalAmount,


3                                   float &bluriness);

The best way to Combine TSCMAA into Different DirectX 11* Purposes

To combine the TSCMAA library into the appliance:

  1. Construct the TSCMAA library with “Intel/TSCMAA.sln” in the event you shouldn’t have TSCMAA.lib.
  2. Hyperlink “Intel/lib/TSCMAA.lib” to your software undertaking.
  3. Embody “Intel/TSCMAA/TSCMAA.h” in your software.
  4. Create a TSCMAA occasion and ColorDepthIn occasion.
  5. Create useful resource views from software’s colour and depth textures for TSCMAA by calling ColorDepthIn::Create(…). To create colour and depth textures, use one of many supported codecs in TSCMAA.
  6. Initialize TSCMAA by calling TSCMAA::Create().
  7. Render colour and depth textures in an software render loop.
  8. Apply TSCMAA by calling TSCMAA::Draw() after software render.
  9. Submit TSCMAA output to HMD (or backbuffer).
  10. Repeat steps 7‒9 for each body.
  11. Destroy TSCMAA sources by calling TSCMAA::Destroy() and ColorDepthIn::Destroy().

The pattern code is proven under.


01 #embrace “Intel/TSCMAA/TSCMAA.h”


02 TSCMAA::TSCMAA _tscmaa;


03 TSCMAA::ColorDepthIn _tscmaaColorDepthIn;


05 // In software init


06 _tscmaaColorDepthIn.Create(pDevice, pEyeTex, pEyeDepthTex);


07 _tscmaa.Create(pDevice, DXGI_FORMAT_B8G8R8A8_TYPELESS, 1080, 1200, 2);


11     ID3D11Texture2D * pEyeTSCMAAOutTex[2] = { nullptr, };


12     for (every eye) {


13         // App renders into pEyeTex and pEyeDepthTex


14         // …


15         _tscmaa.Draw(pContext,


16                 projectionMat,


17                 viewMat,


18                 _tscmaaColorDepthIn,


19                 &pEyeTSCMAAOutTex[eye],


20                 eye);


22     // Submit pEyeTSCMAAOutTex[2] to your HMD again buffer


26 // In software destroy


27 _tscmaaColorDepthIn.Destroy();

Abstract

This text described TSCMAA and the way simply VR software builders can combine TSCMAA into their functions as a MSAA alternate with aggressive spatial high quality and higher temporal stability. TSCMAA high quality and efficiency can also be adjustable with the variety of edges since CMAA and TAA passes are processed solely on the sting candidates, leading to sooner, much less blurred pixels. The connected TSCMAA pattern bundle has been optimized on Intel HD Graphics 630, nevertheless it runs on any platforms.

Click on right here to Be part of the Intel® Recreation Dev program totally free instruments, sources, and alternatives that can assist you carry the most effective sport expertise to the largest worldwide viewers

Supply: https://software program.intel.com/en-us/articles/temporally-stable-conservative-morphological-anti-aliasing-tscmaa

[fb-reactions]

Leave a Reply

Login To Comment.

Login to follow creators & categories, to create posts, to comment on posts.

#random-yet-related.....

Share on facebook
Share on twitter
Share on linkedin
Share on email