When I was younger, I played a popular MMORPG which combined detailed 2D isometric pixel art with 3D environments, called Ragnarok Online.

This was a game that had massive environments and hundreds of thousands of players, PvP and PvE elements, rare items, and a bustling economy of players buying and selling items.

Despite this being a virtual world, the individual interests and incentive structures of the game aligned themselves in such a way that players collaborated to reduce the cost of travel and improving themselves. Thus, like the real world, a large amount of wealth had concentrated in people that had very valuable items and services, with little stopping the accumulation of wealth and inflation.

The amount of money that could be made from mercantile business was astonishing, where simply performing PvE item sales could net you a few hundred or thousand of the currency, merchandise regularly sold for 100-1000 times that amount or even more thanks to the economy's standing wealth, reminiscent of our own where the multiplicative nature of the finance market leads to more standing wealth.

Similar stories exist in other massively multiplayer games, such as Runescape, Tibia, Maple Story, and of course, World of Warcraft. The value of person to person trade is astonishing and serves as a window to how valuable it is in the real world and how societies progress Maslow's hierarchy of needs.

Windfall

On the close of a typical school day in 5th grade, I ran to my father's HP Pavilion a618x, turned on my 1280x1024 pixel CRT display, and logged onto Ragnarok Online. While exploring the innermost parts of the map alone, a borderland between the Sograt Desert and Payon Forest, I stumbled across another player that proposed something interesting. They claimed their friend was logging off for good and wanted to transfer in game currency to another player, with the promise that I might receive half.

Looking back on it now, it sounded like they were setting me up as a cash mule. Perhaps the player quitting was actually part of some cash-to-currency exchange or sold duplicated items. Maybe they did something socially terrible to their online friends group, or perhaps their parents just wanted them to quit playing so much. I can't say what their motivations were, just that mine were immediately blinded by greed and my 10 year old naivete couldn't see the possible red flags.

In hindsight, they were most likely just quitting, as the amount was a paltry sum in the context of the game economy, but very noticeable to me.

I was to take 3 million zeny, split in 2, so 1.5M to me, 1.5M to the other player. The amount of access this windfall afforded me was astonishing. At the time the most I ever saved was 60,000 zeny (0.04%), so suddenly having access to 1.5M meant the world was open to me. I could buy items sold by players in the capital city of Prontera, build a community via a guild, or travel and explore the greater map quickly through air travel and actually afford the high fare to use airships.

Conclusion

Eventually I'd find myself exploring the entire map, becoming a Crusader, then Paladin, then finally a Royal Guard, and without that windfall I wouldn't have maintained the motivation or desire to keep going since the calculus of time-to-value was growing too high. Thus it became clear how meta-gameplay of economics and social structures also factored into time savings. For the marginalized, the poor, young and lacking, these moments can resonate with them as they discover their real-life analogues later in life. It makes you wonder how that story might play out in the lives of others, and how they might react if they knew they were born to inherit a silver spoon or not.

Games are meant to teach lessons, and socioeconomic lessons are among the most difficult lessons of all to teach and gamify.

Resources

• How to create 2D characters in a 3D World like Ragnarok Online by Fofuxo.

• Rate My Server - Ragnarok Database

The art of making the perfect espresso drink is an engaging and interesting problem to solve. From engineering the perfect espresso machine, gathering data such as bean grind size/weight/distribution, water type/volume, and managing the pressure in the group head at different stages in the extraction process; your 1-2 ounces of espresso requires careful consideration to nail the perfect shot. In addition, the dexterous art of pouring the perfect drink is challenging and satisfying when successful. One must make sure to position of the milk pitcher/cup correctly, manage flow velocity, properly homogenize your liquids, and use specific techniques that make pouring feel more like working with a canvas and brush.

It wouldn't be an exaggeration to say I love espresso drinks. They're delicious and everything about them is so pretty, the rich dark crema, the art, the stainless steel machinery and ceramic cups. I've been roasting beans, buying roasts directly from local roasters, making roast blends, brewing espresso, and making hearts, leaves, and rosetta art on single/double shot lattes/cortados multiple times a day, every day for the past few years. In addition, I've been dabbling lightly in homesteading dairy cows and growing coffee plants, though not enough to get a drink yet!

Do you folks like coffee? Real coffee, from the hills of Colombia? ~ Duncan Hills Coffee Jingle (Deathklok Season 1 Episode 1)

The following are a summary of my notes on crafting espresso drinks, from the engineering to the dexterous art that goes on top. Crafting drinks and engineering the tools to do so is a passion a sizable group of people share, so please take a look at the references and additional resources for more contacts, modifications, and tools.

Drinks

A sizable amount of people don't know the names of the drinks they want, and cafes are aware of this. Thus when you order a latte, cortado, flat white, cappuccino, etc. you'll often get the same exact drink. There are definitions for these drinks such as the Instituto Espresso Italiano (IET) Italian Espresso , however, I'll provide a definition based on what most baristas and roasters local to the United States agree on.

Espresso

Espresso is normally a single ounce for a single shot (or 2 ounces in the case of a double shot) of concentrated coffee brewed from an espresso machine. That's a rough definition however, as it's more accurate to craft an espresso based on the weight of the initial dose of grinds. That weight can vary depending on your goals, are you aiming for a stronger more concentrated shot, or one that's lighter?

There's a wide range of flavors that an espresso can have, from sweet and fruity to sour and bitter, and depend heavily on how how the shot is pulled extracted and the beans you choose to use. Generally a shot should be a ratio of 1:2, so for every gram of espresso grinds, you extract 2 grams of espresso. This also tells you how big your portafilter basket should be, enough to accommodate your intended amount of espresso.

With coffee beans, the fresher the better, and light to medium roasts tends to have the most floral flavors, and dark roasts tend to have the more caramel and bitter flavors.

Extraction refers to the process of making a shot of espresso. When pulling a shot of espresso, your compacted grinds go through the following brewing steps:

1. Pre-infusion is when you soak your espresso puck with low pressure water to start your extraction at generally 1-3 bar of pressure. Once the puck is fully immersed in water, we can begin extracting at high pressure.

2. Infusion is when you're pushing high pressure water through the puck. 9 bars of pressure is the standard for espresso, but lever action machines can have a curve of pressure.

Clive coffee goes into more detail here on pre-infusion here.

As for types of espresso, these include:

• Ristretto - typically 40 grams or 1.4 oz. A short shot of espresso, made of only the first portion of a shot, and is thus more flavorful.

• Turboshot - a fast espresso drink that's shorter than the typical 25-35 second extraction.

• Lungo - A longer shot where more water passes through a shot.

Latte

A Latte can vary greatly depending on what you're looking for. The cup its served in generally determines what you get to drink, with 12 oz cups being the standard, and 8 oz cups being an alternative size. As much as the cup is advertized as a given size, it's real capacity is often more than its size. Some 6 oz cups hold 8.5 oz max. Some 12 oz cups hold exactly 12 oz max. Usually the standard is 12 oz exactly.

• A Flat White is a drink with very little to no foam, so little to no stretching of the milk, and only frothing/heating it. In spanish communities, a Cafe con Leche is very similar to a flat white, but uses 2-3 oz of coffee instead of an ounce of espresso.

• A Cortado is A 6 oz drink with 1 oz of espresso, and 5 oz of steamed milk. It's a concentrated beverage, and essentially half of a 12 oz latte.

• Cappuccinos are often interchangeable with a latte, but have significantly more foamy milk, and often have a layer of white foam on top, garnished with powdered chocolate and even decorated.

Latte Art

It's possible to draw anything on top of your espresso drink, even going so far as to print art on top of a given drink. You could do use a stencil with a chocolate powder dispenser, or use a stainless steel latte art pen to adjust your art, but for the most part that's very easy. The challenge with latte art is only using steamed milk to do it.

Texturing

Getting the perfect milk texture is paramount to successfully doing latte art. Generally you want to let the milk naturally disable the "ripping" sound, by forming a layer of foam at a medium density for lattes, with lower/higher amounts of stretching depending on if you're aiming for a flat white or cappuccino. This can vary from machine to machine, and is ultimately driven by your steam wand pressure.

When pouring, maintaining a constant flow and reasonable velocity is very important, that is, approximately 3 fluid oz per second.

• Stir your espresso and swirl your steamed milk well to make each more homogenous. You'll find the best parts of your espresso at the bottom of the liquid.

• Pour from above to establish your base. You may want to add some chocolate powder prior to this for some delicious contrast to the white milk coming later.

• Try resting the spout on the edge of your cup, once you've poured your base. This can offer more control over the pour.

• Pour closely to establish your white art on top, and form a heart by pouring forward, do pulsing waves for a rosetta, and/or restart the process for tulip layers.

Espresso Machines

Espresso machines come at a wide range of prices, from very low to very high. The following is a summary of machines at a variety of price ranges under $6000 USD. Realistically, a good price target that have the majority of important features a barista is looking for is around $1500 USD, and entry level is about $500 USD. Espresso machines can be very expensive, but what matters are specs and performance. A few things to look out for are:

1. What's the heating system? The cheapest machines use thermoblocks, while more expensive machines use heat exchangers and dedicated boiler(s). [Hoffmann 2022]

2. Is it lever based or feature a flow control system? There's 2 schools of thought with espresso, constant or variable pressure, and generally the variable pressure of a lever machine can result in uniquely flavored espresso.

3. How computerized is the machine? Does it feature a PID controller or multiple? Does it feature flow controls, timers, scales, etc?

4. What's the group head size? Usually you want a standard E61 group head, as this is the size with the most accessories.

5. What are the components made of? Plastic, stainless steel, brass, chrome, wood?

6. Is it modable? Anything can be modded to add sensors, replace parts with wood, but some products have more resources and tutorials than others. The Gaggia Classic Evo Pro arduino mod can improve shot profiles tremendously.

There's other machines at higher price points for the prosumer/commercial market, while there's a very large number of these machines, here's a few:

• Tone Swiss - Arguably the most interesting technologically, Tone Swiss has been working on highly efficient espresso machines.

• La Marzocco - The Gibson of the espresso machine world, La Marzzoco makes incredible espresso consistently.

• Victoria Arduino - This company has been designing espresso machines since 1905!

• Sanremo Coffee Machines - Designing high quality commercial espresso machines with smart features.

There's also other styles of machines such as manual lever press machines.

Components

• Proportional-Integral-Derivative (PID) Controller - Manages the machine target temperature.

• Over Pressurization Valve (OPV) - Manages extraction pressure. Generally low end espresso machines have this set too high, you should aim to have this configured for 9 bars, but this could even be managed by a microcontroller for pressure profiling.

• Grouphead - the mechanism your portafilter attaches to, that holds the spout where pressurized water comes out. The E61 grouphead is one of the more common designs, modeled after the Faema E61.

• Portafilter - The insert that holds your basket and grinds. Most come in 58mm in diameter, while other like the Breville barista express use 53mm.

• Portafilter Basket - The stainless steel basket inside the portafilter. The size and shape depend on the dose you intend to put in it, with double shot baskets resulting in the best extraction.

• Steam Wand - The tool to steam milk.

• Drip Tray - Where excess water gathers for disposal later.

Grinders

Some espresso machines such as the Breville come with a built in grinder, but most do not, so you're required to purchase a grinder separately.

For grinders, what's important is:

1. How fine they can it grind? This is often measured in average particle size. When brewing espresso, the common meme phrase is: "Did you try grinding finer?", and it's true!

2. How quickly can it grind? as a rule, the bigger the burrs the faster it can grind.

3. What's the retention rate of the unit? Does it feature a plasma generator? Bellow to blow air?

4. How computerized is it? Does it have a built in scale? Variable grind size profiles?

Roasters

Roasting beans yourself can be an extermely effective way of producing amazing coffee. Here's a few high tech roasters:

• Kaleido

• Stronghold

Accessories

For espresso machine accessories, it's important to bear in mind sizes. Check the size of your group head and portafilter prior to getting any accessory. Generally machines have 58mm group heads, but cheaper Breville/Sage machines uses 53mm.

• Portafilter - Can be bottomless, or with different spouts.

• Dosing Ring - A ring used to help keep your portafilter clean.

• Dosing Cup - A cup that can be used to ensure doing weights when serving espresso and keep your grinds clean.

• Filter Paper - a paper that goes on the bottom of your espresso basket.

• Puck Screen - A screen that goes on top of your tamped coffee. Helps keep the group head clean.

• Refractometer - Measure your extraction yield. Generally you should aim for a 20-25% yield.

A few vendors of accessories include:

• Normcore

• MHW-3BOMBER

• notNeutral

• Coffeezone

• Subminimal

• Prima Coffee

• Timemore

Roasters

There's a variety of commercial roasters that ship quality roasted coffee to your doorstep, among which are:

• Onyx Coffee Lab

• Black White Roasters

• The Boy and the Bear Roasters

• Sey Coffee

• HS Coffee Roasters

• Sweet Bloom Coffee

• Presta

• Bigface

• Kurasu

There's large and medium sized roasters such as:

• Counter Culture

• Stumptown

• City Girl Coffee

And depending on your area, you may find great local roasters. Here in Florida there's:

• Buddy Brew

• Foxtail Coffee

• Blazing Bean Roasters

There's certainly more roasters out there, so look for local roasters in your area.

Additional Resources

• Espresso Aficionados

  • Discord

• Barista Hustle

• The Milan Coffee Festival

• Specialty Coffee Expo

• Espresso Parts

• Cropster

• Mtpack

• Roest

• Espresso Setup Builder

Video

• Lance Hedrick

• Emilee Bryant

• James Hoffmann

While tone might seem impossible to quantify, there's no such thing as "magic diodes", tone is a signal, and what you hear is primarily influenced by how it's compressed, filtered, and combined, both prior to the speaker, and especially after as arguably the biggest impact on your sound is your cabinet and the speaker cones that make it up.

Humbuckers

EMG 81s use copper coils that connect to different parts of the preamp stage built into them to produce their sound, and use a smaller humbucker internally, about the size of a firebird pickup, with a solid magnet piece across. This results in a pickup that is less sensitive to interference and has a lower noise floor. This article goes into the circuit design in more detail.

Fishmans use stacked PCBs for their pickups, which lets them control which parts of the pickup they can turn on/off to get a single coil/alnico sound with the different voice options. While this is a departure from traditional wound copper coils, the result is consistency and ease of variability. Tosin Abasi for instance detailed how he designed his signature Fishmans in this video.

Currently the state of the art in commercial pickups looks to be Fishmans as they're consistent and have high variability.

Pedal Components

Pedals aren't magic, they're intentional designs meant to solve specific musical problems, such as introducing clipping, adding fuzz, or denoising a signal. To understand these effects, we can examine the fundamental building blocks used in their design. While this list isn't exhaustive, it covers the most common components found in pedal motherboards. For a detailed look at how these components interact, you can refer to a this Tube Screamer Circuit Analysis.

Resistors and Capacitors

Resistors and capacitors form an RC (resistor-capacitor) filter, where the capacitor acts as a frequency-dependent gate and the resistor controls how strongly the capacitor can divert signal to ground, setting the cutoff frequency.

• Low Pass Filters: the capacitor shunts high frequencies to ground as signal frequency rises, darkening or "warming" the tone and cutting fizz from a distorted signal.

• High Pass Filters: the capacitor blocks low frequencies while passing high ones, brightening the tone and removing muddy lows.

A guitar's tone pot uses an RC circuit. As you roll it down, the resistance to ground drops, letting the tone capacitor bleed more high frequencies away and gradually darkening the sound. Larger capacitors bleed more highs for a warmer result, while smaller ones preserve the natural treble of the instrument.

Transistors

Transistors are the primary way to generate a classic fuzz sound, often by providing hard clipping to the input signal. The type of transistor used significantly impacts your sound:

• BJT (Bipolar Junction Transistor): Classic fuzz and distortion, like for the Big Muff Pi. They are known for high gain and the ability to produce aggressive, saturated clipping. The Big Muff Pi, for instance, uses 4 transistors in cascaded stages to achieve its signature, thick, "synth-like" sound.

• JFET (Junction Field-Effect Transistor): Often used in boutique pedals to emulate the "warmth" of vacuum tubes. They typically offer much higher input impedance, making them gentler on guitar pickups and ideal for cleaner, more expressive overdrive sounds.

• MOSFET (Metal-Oxide-Semiconductor Field-Effect Transistor): Modern gain. They are often used in high-gain pedals or clean boosts where maintaining signal clarity is important.

Diodes

Diodes limit the signal at various peaks, which serve to help soft-clip your signal, perfect for overdrive and metal circuits.

The number of diodes heavily influences the sound, and how they're laid out.

• Asymmetric clipping - Used by the BOSS OD-1, leads to an interesting sound where only one side of a frequency is clipped.

• Symmetric clipping - Typical overdrive sound, where both sides are clipped.

The Axial Schottky diode (DO-41) is one of the more famous diodes, and is ideal for soft clipping and achieving "warmer" tones.

Operational Amplifiers (Op Amps)

These are the most common ICs used to shape gain and clipping characteristics in pedals.

• TL072 - Dual JFET-input op amp (Datasheet) - Used in the Tube Screamer.

• TL074 - Quad JFET Operational Amplifier - Used in Amptweaker Fat Metal.

• TL082 - Dual JFET Op Amp - Used in Earthquaker Plumes.

• OP275 - Dual Bipolar/JFET Audio Amplifier - Used in Mad Professor Sweet Honey Pedal.

• JRC4580D - Dual op-amp, DIP8 (Nisshinbo Micro Devices) - A common alternative in the King of Tone.

Voltage Regulators

Often used to increase or stabilize the voltage of a given circuit, which can affect the headroom and output tone.

• LM108 - LDO Voltage regulator IC - Used in the RAT.

• TC1044SCPA - Charge-pump Switching Voltage Regulator. Often used in fuzz and bass overdrive pedals.

Interesting Pedals and Brands

• Eldritch Blast

• Black Fountain

• Acorn Amplifiers ADHD Synth Fuzz

• Kernom Ridge - the addition of controllable clipping, pre/post EQ, and mixing means this pedal can be a Klon, a Tube Screamer, and everything in between.

• JHS Notaklon - A DIY kit that offers a hands-on way to recreate the legendary Klon Centaur sound.

• Lichtlaerm Gehenna

• Amptweaker Tight Metal

Conclusion

Ultimately, understanding the circuitry just the first step, it will take actually building a circuit to really understand where tone resides. Once you understand the role of each part, you move from simply buying a sound to designing one.

More Resources

• PCB design communities such as:

  • Pedal PCB - A large catalog of PCB layouts for clones of a variety of pedals, from Klons to Tube Screamers, Delays, and everything in between.

  • Aion FX - Similar catalog

  • PCB Guitar Mania

  • Deadend FX

  • KiKad

• Wampler Pedals - How To Design An Overdrive Pedal Circuit

• Sweetwater Boost, Overdrive, Distortion & Fuzz Pedals

• JHS Pedals How to Find the Fuzz Pedal You Need

• Understand Guitar Overdrive Pedals

• BasicPitch Audio to MIDI converter

• Spectre Sound Studios is a YouTube channel that compares expensive gear and reasonably priced gear.

General Purpose Graphics Programming Unit (GPGPU) computing is an effective way of using all of the computational resources your computer has to offer by offloading work to the GPU, making it an easy way to access heterogeneous parallel computing. Even though the "G" in GPU stands for graphics, it's more accurate to call modern GPUs highly parallel general processors capable of processing large quantities of data quickly thanks to having a large number of simple microprocessor cores, called Stream Processors (AMD) or Compute Unified Device Architecture (CUDA) cores (NVIDIA), or just GPU cores in other architectures like the Apple M series or Qualcomm Snapdragon. These microprocessor cores are designed to not perform well with latency or branching, but instead focus on throughput [Hwu et al. 2022]. This architecture is designed around processing large amounts of data with a single instruction, otherwise known as a Single Instruction Multiple Data (SIMD) or more accurately Single Instruction Multiple Thread (SIMT) architecture, where a group of threads process a single instruction in lock-step.

GPUs can be asked to perform local arithmetic operations, transcendental operations that cost 4 cycles such as square roots, cosine/sine, load/store operations, hardware accelerated ray tracing traversal or matrix multiplications for executing machine learning (ML) models, making them both general and application specific depending on the workload.

Compute shaders (or kernels) are programs that execute computations on the GPU, and are useful for a variety of tasks within and outside of computer graphics:

• Executing and Training ML Models - From Leaky ReLUs to convolutions, machine learning models can be written as shader code and execute quickly and easily.

• Geometric Processing - every aspect of ray tracing, from the routines to trace rays, build bounding volume hierarchies (BVHs), and updating them, can be done by the GPU, which can process large quantities of triangle data in real time. Physics simulations such as rigid bodies can be simulated by processing vertices.

• Encoding/Compression - Video encoding, compressing mesh data, compressing frame buffer attachments such as 3D normals to 2D view space normals.

• Postprocessing - traditional fragment based post-processing effects can be easily rewritten as compute shaders and executed more efficiently and even at the same time.

• Materials and Lighting - Some games opt to use compute shaders in tandem with lookup structured buffers to add a level of abstraction from rasterization, with Nanite using software rendering in combination with meshlets; Mortal Kombat 11 indexed material IDs from a given fragment for its lighting passes.

When processing a compute shader, every scheduled microprocessor (96 Compute Units (CUs) with 64 shading units each for a maximum of 6144 stream processors on a 7900 XTX) receives the same instructions, with the only differences being what the current thread ID is for that a single thread, which waves are on which instruction, and what data is accessible between threads of the same threadgroup:

; API shader hash:                  0x00000000000001009731533EAB7158CE
; API PSO hash:                     0xB4116117903F261D
; Driver internal pipeline hash:    0x9337B876E9519DBAD84A08F55EB136FC
;
; Vector registers:                 56 (64 allocated)
; Scalar registers:                 52 (128 allocated)

_amdgpu_cs_main:
 s_version 0x4004                 ; 000000000000: B0804004
 s_inst_prefetch 0x3              ; 000000000004: BFA00003
 s_getpc_b64 s[0:1]               ; 000000000008: BE801F80
 s_mov_b32 s4, s2                 ; 00000000000C: BE840302
 s_mov_b32 s5, s1                 ; 000000000010: BE850301
 s_mov_b32 s7, s1                 ; 000000000014: BE870301
 s_movk_i32 s10, 0x1000           ; 000000000018: B00A1000
    ; ...

Groups of threads are further grouped together in hardware in what's called a Wave (DirectX), Subgroup (Vulkan), Wavefront (AMD), SIMD Group (Apple), or Warp (NVIDIA CUDA). On both recent AMD and NVIDIA hardware, waves have 32 lanes, with each lane executing one thread. Older AMD hardware traditionally uses 64 threads per wave.

Since these terms are different for every hardware vendor and graphics API, we'll stick with the word threadgroup for all general mentions of groups threads, and wave for mentions of hardware groups of 32-64 threads, unless working in a specific graphics API.

Let's review GPGPU computing, the terminology, and techniques involved in writing GPGPU programs in different graphics and compute APIs, with a focus on WebGPU and DirectX 12. We'll cover how to write compute kernels to do the following:

1. Convolution - a basic operation that is useful for blurring, sharpening, and executing a machine learning model.

2. Histogram - Form a summary of your input data that shows how many times a given value appears within a range of values.

3. Sorting - Sort a list of values from smallest to greatest with radix and merge sort kernels.

Dispatch

// 🚄 Dispatch Compute Work
let width = 1920;
let height = 1080;
let workgroupSize = { x: 8, y: 4 };
let workgroupCount = {
  x: Math.ciel(width / workgroupSize.x),
  y: Math.ciel(height / workgroupSize.y),
  z: 1
};
passEncoder.dispatchWorkgroups(
  workgroupCount.x,
  workgroupCount.y,
  workgroupCount.z
);

// 🚄 Dispatch Compute Work
unsigned width = 1920;
unsigned height = 1080;
unsigned numThreadsX = 8;
unsigned numThreadsY = 4;
unsigned threadgroupX = static_cast<unsigned>(ceil(static_cast<float>(width) / static_cast<float>(numThreadsX)));
unsigned threadgroupY = static_cast<unsigned>(ceil(static_cast<float>(height) / static_cast<float>(numThreadsY)));
unsigned threadgroupZ = 1;
commandList->Dispatch(threadgroupX, threadgroupY, threadgroupZ);

A dispatch call in a given graphics API can process groups of work (aptly called workgroups or threadgroups), normally a set of threads that can number anywhere between 8-256, though generally you'll want to stick with groups of 32 or 64 threads, the size of a wave on most hardware, unless your workload would benefit from more data locality over large groups such as a radix sort. In most graphics APIs, the shader itself describes the thread counts for each of these thread groups, Metal is the only API that doesn't have this requirement and lets you set the number of threads per group at dispatch time. Other APIs could do the same however with preprocessor definitions.

A typical application will dispatch a given task across a large workload, then another dispatch that takes the results of that previous pass. For instance, you may need to first sort a list of numbers such as morton codes, then use that sorted list in a separate pass. This is important to note, if a pass requires another to be finished, you must either execute a separate dispatch, or use clever task synchronization methods across an entire dispatch such as using an atomic task counter.

Compute Shader

A compute shader consists of a dispatch to a shader kernel that executes groups of threads, each of which has the following basic intrinsics:

// 🫂 Local Group Thread ID
@builtin(local_invocation_id) localInvocationID: vec3<u32>

// 👶 Group ID
@builtin(workgroup_id) workgroupID: vec3

// 🖼️ Dispatch Thread ID
@builtin(global_invocation_id) globalInvocationID: vec3<u32>

// 🗂️ Group Index
@builtin(local_invocation_index) localInvocationIndex: u32

// 🔢 Number of Thread Groups
@builtin(num_workgroups) numWorkgroups: vec3<u32>

// 🫂 Local Group Thread ID
uint3 groupThreadID : SV_GroupThreadID

// 👶 Group ID
uint3 groupID : SV_GroupID

// 🖼️ Dispatch Thread ID
uint3 dispatchThreadID: SV_DispatchThreadID

// 🗂️ Group Index
uint groupIndex : SV_GroupIndex

// 🔢 Number of Thread Groups
// HLSL Has no builtin for the number of workgroups.

// 🫂 Local Group Thread ID
gl_LocalInvocationID

// 👶 Group ID
gl_WorkGroupID

// 🖼️ Dispatch Thread ID
gl_GlobalInvocationID

// 🗂️ Group Index
gl_LocalInvocationIndex

// 🔢 Number of Thread Groups
gl_NumWorkGroups

// 🫂 Local Group Thread ID
uint3 threadPositionInThreadgroup [[thread_position_in_threadgroup]]

// 👶 Group ID
uint3 threadPositionInGrid [[threadgroup_position_in_grid]]

// 🖼️ Dispatch Thread ID
uint3 dispatchThreadID [[thread_position_in_grid]]

// 🗂️ Group Index
uint threadIndex [[thread_index_in_threadgroup]]

// 🔢 Number of Thread Groups
uint3 tpg [[threads_per_grid]]

// 🫂 Local Group Thread ID
uint3 threadIdx;

// 👶 Group ID
uint3 blockIdx;

// 🖼️ Dispatch Thread ID (You must compute this yourself):
uint3 dispatchThreadIdx = blockIdx * blockDim + threadIdx;

// 🗂️ Group Index (You must compute this yourself):

// 🔢 Number of Thread Groups

// 🎡 Group Size
uint3 blockDim;

• Group Thread ID - What thread you're currently working on within a workgroup.

• Group ID - What workgroup you're currently exectuing in, useful for designating certain workgroups for the final parts of an algorithm such as prefix sum.

• Dispatch Thread ID - The actual thread that's currently being executed by the hardware, the "pixel" in the case of a screen space dispatch.

• Group Index - a linear representation of the current thread index.

• Number of Thread Groups - What's passed as arguments to your dispatch, the number of thread groups currently executing for this shader kernel.

In addition, you can take advantage of shared memory across a given workgroup

Group Local Data

var<workgroup> local_data: array<i32,32*16>;

groupshared uint localData[32*16];

Each workgroup can allocate memory within that group that can be shared between threads of that group. This can be used as a local cache across the whole threadgroup to do privatization of the final resultant data structure, enable scan algorithms like Kogge-Stone and Brent-Kung, prefix sum of a given value in every thread, radix binning, sweeps, scatter, and compaction. These primitives can then be used to do more advanced workloads like sorting algorithms, building hierarchical structures like BVHs, neural network model execution, and much more.

In the RDNA3 hardware this looks nearly identical to the shader itself, with that data shared within a given compute unit's L1 cache, known as the local data share (LDS). Level cache rules apply to GPUs as well, the more local your data the faster your algorithm can execute, so try to keep data as close to a thread as possible. This means using local variables (which become scalar or vector registers in hardware) more often than global memory such as your strctured buffers, textures, etc.

Atomic Operations

Atomic operations allow for threads to read/write to the same location without worrying about race conditions with data. These operations are significantly slower than standard read/writes, as an atomic operation that is performed by a trailing thread cannot be started until the atomic operation of a leading thread completes. This effectively serializes the atomic operations that are being performed on a memory location, slowing execution down by a factor of 10x [Hwu et al. 2022].

void InterlockedAdd(in R dest, in T value, out T original_value);

// https://www.w3.org/TR/WGSL/#atomic-builtin-functions
fn atomicLoad(atomic_ptr: ptr<AS, atomic<T>, read_write>) -> T
fn atomicStore(atomic_ptr: ptr<AS, atomic<T>, read_write>, v: T)

fn atomicAdd(atomic_ptr: ptr<AS, atomic<T>, read_write>, v: T) -> T
fn atomicSub(atomic_ptr: ptr<AS, atomic<T>, read_write>, v: T) -> T
fn atomicMax(atomic_ptr: ptr<AS, atomic<T>, read_write>, v: T) -> T
fn atomicMin(atomic_ptr: ptr<AS, atomic<T>, read_write>, v: T) -> T
fn atomicAnd(atomic_ptr: ptr<AS, atomic<T>, read_write>, v: T) -> T
fn atomicOr(atomic_ptr: ptr<AS, atomic<T>, read_write>, v: T) -> T
fn atomicXor(atomic_ptr: ptr<AS, atomic<T>, read_write>, v: T) -> T

fn atomicExchange(atomic_ptr: ptr<AS, atomic<T>, read_write>, v: T) -> T
fn atomicCompareExchangeWeak(atomic_ptr: ptr<AS, atomic<T>, read_write>, cmp: T, v: T) -> __atomic_compare_exchange_result<T>
struct __atomic_compare_exchange_result<T> {
  old_value : T; // old value stored in the atomic
  exchanged : bool; // true if the exchange was done
}

Atomic operations allow for threads to perform logic on a large dataset and write the result to either group shared or global memory. This is useful for performing a variety of tasks such as atomicAdds for prefix sums, atomic min/max for finding the range of points, or atomicXors for compression.

WebGPU is unique in that atomics are strictly typed, whereas with HLSL atomic operations can work on regular memory locations.

Shader Memory Barriers

Barriers allow for a you to guarantee that group shared memory or device memory accesses are finished within a threadgroup.

void GroupMemoryBarrier(void);
void GroupMemoryBarrierWithGroupSync(void);

Wave Intrinsics

Shader Model 6.0 introduces wave intrinsics to HLSL, though this is exclusive to both HLSL and GLSL as subgroup intrinsics at the moment. There is a proposal to bring this to WGSL as well.

if (localId == 0)
{
 // Your code block...
}

// Vs.

bool WaveIsFirstLane(void);
if (WaveIsFirstLane())
{
    // Your code block...
}

Conclusion

GPGPU computing is an effective way to solve problems with large datasets, which is why you find it most often used in computer graphics, machine learning, and simulations. The SIMT architecture of the GPU requires that you conceptualize the solution to a problem in a different way, mapping the problem to a large number of groups of threads.

More Resources

• RDNA Whitepaper

• RDNA3 Instruction Set Architecture

• NVIDIA Volta Whitepaper

• NVIDIA CUDA C++ Programming Guide

• GPU Gems 2 - GPGPU Primer

• GPU Programming Primitives for Computer Graphics - Siggraph Asia 2024

• Lou Kramer (Technology Developer Engineer @ AMD) released a talk titled Compute Shaders.

• Intel Implicit SPMD Program Compiler

• HLSL Wave Intrinsics

• HLSL Shader Model 6.5

• HLSL Shader Model 6.6

• HLSL Shader Model 6.8

Acceleration structures as the name implies are spatial data structures that speed up searching for triangles, distance fields, etc. in a given scene. Ray intersection tests for certain shapes like axis aligned bounding boxes (AABBs) are extremely cheap and easy to implement, so by traversing a hierarchical structure of AABBs, you can determine if a triangle is intersected, and use a triangle-ray intersection test to get the necessary data such as barycentrics, interpolated attributes such as normals [McGuire 2018] to perform any number of ray traced effects such as accurate shadows, ambient occlusion, global illumination, reflections, or even a full path tracer [Kajiya 1986] [Pharr et al. 2016].

// ✅ Boolean tests for AABB and Triangle-Ray intersection.
// You may need more information such as baricentric coordinates,
// Ray distance, etc.
//
// Adapted from David Eberly's Geometric Tools
// Copyright (c) 1998-2022 | Boost License, v1.0.
// https://www.geometrictools.com/GTE/Mathematics/IntrRay3AlignedBox3.h
// https://www.geometrictools.com/GTE/Mathematics/IntrRay3Triangle3.h
 
struct AABB
{
    float3 center;
    float3 extent;
};
 
bool intersectAABB(float3 origin, float3 direction, AABB box)
{
    float3 boxCenter = box.center;
    float3 boxExtent = box.extent;
 
    // Transform the ray to the aligned-box coordinate system.
    float3 rayOrigin = origin - boxCenter;
 
    if (any(abs(rayOrigin) > boxExtent && rayOrigin * direction >= 0.0))
    {
        return false;
    }
 
    // Perform AABB-line test
    float3 WxD = cross(direction, rayOrigin);
    float3 absWdU = abs(direction);
    if (any(WxD > float3(dot(boxExtent.yz, absWdU.zy),
                         dot(boxExtent.xz, absWdU.zx),
                         dot(boxExtent.xy, absWdU.yx))))
    {
        return false;
    }
 
    return true;
}
 
 
struct Triangle
{
    float3 v[3];
};
 
bool intersectTriangle(float3 origin, float3 direction, Triangle triangle)
{
    float3 diff = origin - triangle.v[0];
    float3 edge1 = triangle.v[1] - triangle.v[0];
    float3 edge2 = triangle.v[2] - triangle.v[0];
    float3 normal = cross(edge1, edge2);
 
    float DdN = dot(direction, normal);
    float sign = 0.0;
    if (DdN > 0.0)
    {
        sign = 1.0;
    }
    else if (DdN < 0)
    {
        sign = -1.0;
        DdN = -DdN;
    }
    else
    {
        // Ray and triangle are parallel.
        return false;
    }
 
    float DdQxE2 = sign * dot(direction, cross(diff, edge2));
    if (DdQxE2 >= 0.0)
    {
        float DdE1xQ = sign * dot(direction, cross(edge1, diff));
        if (DdE1xQ >= 0.0)
        {
            if (DdQxE2 + DdE1xQ <= DdN)
            {
                // Line intersects triangle, check whether ray does.
                float QdN = -sign * dot(diff, normal);
                if (QdN >= 0.0)
                {
                    // Ray intersects triangle.
                    return true;
                }
            }
        }
    }
 
    return false;
}

A ray intersection test for AABBs or triangles can easily be represented in only a few DirectX Intermediate Language (DXIL) instructions alone, and manufacturers can take that concept even further by converting those instructions to application specific integrated curcuits (ASICs), or even converting other aspects of a ray tracer such as acceleration structure builds [J. Doyle et al. 2012] or traversal [Liktor et al. 2016] [Vaidyanathan et al. 2016] to ASIC hardware.

Hardware accelerated intersection tests is what AMD did with the RDNA2 Architecture. When you executed your DirectX 12 Ray Tracing or Vulkan Ray Tracing shaders:

template<typename payload_t>
void TraceRay(RaytracingAccelerationStructure AccelerationStructure,
              uint RayFlags,
              uint InstanceInclusionMask,
              uint RayContributionToHitGroupIndex,
              uint MultiplierForGeometryContributionToHitGroupIndex,
              uint MissShaderIndex,
              RayDesc Ray,
              inout payload_t Payload);

All those calls to TraceRay become DXIL which eventually lead to this AMDIL instruction:

These instructions receive ray data from the Vector General Purpose Registers (VGPRs) and fetch Bounding Volume Hierarchy (BVH) data from memory. [AMD 2020]

• Box BVH nodes perform 4x Ray/Box intersection, sorts the 4 children based on intersection distance and returns the child pointers and hit status.
• Triangle nodes perform 1 Ray/Triangle intersection test and returns the intersection point and triangle ID.

; AMDIL for RDNA2
image_bvh_intersect_ray vgpr_d[4], vgpr_a[11], sgpr_r[4]
image_bvh_intersect_ray vgpr_d[4], vgpr_a[8], sgpr_r[4] A16=1
image_bvh64_intersect_ray vgpr_d[4], vgpr_a[12], sgpr_r[4]
image_bvh64_intersect_ray vgpr_d[4], vgpr_a[9], sgpr_r[4] A16=1

Despite the fact that in the application layer this is handled by a given graphics API, it's worth understanding how acceleration structures work, either for CPU based collision tests, or to better futureproof one's understanding of the subject. Let's review acceleration structures, how they can be represented, how they can be built/updated, and how they can be traversed to query for intersections.

Terminology

A Bounding Volume Hierarchy (BVH) is a hierarchical spatial structure of nodes, and can be represented in a variety of ways. You could use axis aligned bounding boxes (AABBs), maintain a tree of 2 leafs per node, a variable number of leafs per node (an example of a wide BVH [Vaidyanathan et al. 2019]), attempt to split the tree into smaller boxes to improve performance [Benthin et al. 2017], refit the bounds of the BVH nodes according to changes to the mesh, reorient the tree to avoid paths with many nodes or to prioritize nodes that are visited often, and much more. There's a vast swath of research in the area of tree based data structures that you can take advantage of as well. [Karras 2020]

Bounding Boxes can be easily represented with lightweight nodes, for example, this struct can represent a node in as little as 64 bytes (and smaller if you take on compression schemes, use spare bits for cached information like the splitPlane), and you can encode extra data to that payload such as the AABB itself and an address to a triangle, parent, etc. [Meister et al. 2021]

struct BVHNode
{
    // 📤 Left AABB
    float3 leftMin;
    float3 leftMax;
    // 📥 Right AABB
    float3 rightMin;
    float3 rightMax;
    // *️ Pointers
    uint leftPointer;
    uint rightPointer;
};

Building a BVH can vary depending on your requirements, with the rule generally being a higher quality BVH is slower to build but can be traversed faster. BVH builders have been categorized as top-down, bottom-up and insertion-based. In general real time builders use fast techniques such as bottom up builders [Benthin et al. 2022] do well due to the highly parallel nature of the GPU.

Contrast a BVH with a KD-Tree, a simpler tree of data which is faster to construct and contains less information per node, but is slower to traverse. [Vinkler et al. 2014]

struct KDNode
{
    uint parent;
    uint flags;
    uint leftPointer;
    uint rightPointer;
};

Notice that both of these examples include pointers, this could be to the triangle itself, but ideally should be to the triangle index stored in a separate list, that way the triangles can be rearanged in case the BVH needs to change due to animations, transformations, procedural geometry, etc. [Bikker 2022]

The Surface Area Heuristic (SAH) function can be used to estimate the efficiency of a BVH. In recent real time builders such as PLOC, this Heuristic is used to better perform splits when building a BVH.

Where $SA$ is the surface area function, and $N$ represents a node. [Pharr et al. 2016]

Thanks to the SAH, we can better estimate the probability that we hit the sub-nodes of a given BVH node through the ratio surface area of their bounding boxes and their parent, which can be useful for optimization passes to the BVH, selecting split planes for building BVHs, optimzing the BVH much more.

// XYZ XYZ XYZ
uint morton = 0b001002003;

Morton codes are a small hash value that can encode a 3D position in 32 bits, so in BVH implementations one can use this to easily index a given location (or morton curve space) in the structure when building the hierarchy. Each bit can represent a location in the hierarchy and groups of bits can be used for locations in XYZ space.

A builder can encode the position of box nodes of a BVH as a morton code normalized by the scale of the scene, and easily store and query for that value when building a BVH. This is done by PLOC.

• C++ Library: Libmorton (MIT)

Ray Tracing APIs

DirectX 12 and Vulkan both support hardware accelerated raytracing and the interface for these APIs is the same. Hardware accelerated ray tracing uses a 2-level BVH, one level for the scene and one for individual meshes.

Building acceleration structures with graphics APIs involves allocating scratch buffers for API and vendor specific data, providing the API with Bottom Level Acceleration Structure (BLAS) data in the form of your index and vertex position buffer, and top level acceleration structures (TLAS) built from your scene hierarchy with their transforms, and the instance of the BLAS the node correspond to.

// 👋 Declare Handles
ID3D12Device5* device;
ID3D12Resource* vertexBuffer;
ID3D12Resource* indexBuffer;
ID3D12GraphicsCommandList4* commandList;
 
// Declare Outputs
ID3D12Resource* asBuffer;
ID3D12Resource* asScratchBuffer;
ID3D12Resource* instanceDescs;
HRESULT hr;
 
// 🌳 Top Level Acceleration Structure
D3D12_RAYTRACING_GEOMETRY_DESC geomDescs[1];
geomDescs[0].Type = D3D12_RAYTRACING_GEOMETRY_TYPE_TRIANGLES;
geomDescs[0].Triangles.IndexBuffer = indexBuffer->GetGPUVirtualAddress();
geomDescs[0].Triangles.IndexCount =
    static_cast<UINT>(indexBuffer->GetDesc().Width) / sizeof(UINT16);
geomDescs[0].Triangles.IndexFormat = DXGI_FORMAT_R16_UINT;
geomDescs[0].Triangles.Transform3x4 = 0;
geomDescs[0].Triangles.VertexFormat = DXGI_FORMAT_R32G32B32_FLOAT;
geomDescs[0].Triangles.VertexCount =
    static_cast<UINT>(vertexBuffer->GetDesc().Width) / sizeof(Vertex);
geomDescs[0].Triangles.VertexBuffer.StartAddress =
    vertexBuffer->GetGPUVirtualAddress();
geomDescs[0].Triangles.VertexBuffer.StrideInBytes = sizeof(Vertex);
 
// 🧱 Get prebuild info
D3D12_BUILD_RAYTRACING_ACCELERATION_STRUCTURE_INPUTS topLevelInputs[1];
topLevelInputs[0].DescsLayout = D3D12_ELEMENTS_LAYOUT_ARRAY;
topLevelInputs[0].Flags =
    D3D12_RAYTRACING_ACCELERATION_STRUCTURE_BUILD_FLAG_PREFER_FAST_TRACE;
topLevelInputs[0].NumDescs = 1;
topLevelInputs[0].Type = D3D12_RAYTRACING_ACCELERATION_STRUCTURE_TYPE_TOP_LEVEL;
D3D12_RAYTRACING_ACCELERATION_STRUCTURE_PREBUILD_INFO topLevelPrebuildInfo = {};
device->GetRaytracingAccelerationStructurePrebuildInfo(topLevelInputs,
                                                       topLevelPrebuildInfo);
 
D3D12_BUILD_RAYTRACING_ACCELERATION_STRUCTURE_INPUTS bottomLevelInputs[1];
bottomLevelInputs[0] = topLevelInputs[0];
bottomLevelInputs.Type =
    D3D12_RAYTRACING_ACCELERATION_STRUCTURE_TYPE_BOTTOM_LEVEL;
bottomLevelInputs.pGeometryDescs = geomDescs;
D3D12_RAYTRACING_ACCELERATION_STRUCTURE_PREBUILD_INFO bottomLevelPrebuildInfo =
    {};
device->GetRaytracingAccelerationStructurePrebuildInfo(
    &bottomLevelInputs, &bottomLevelPrebuildInfo);
 
// 🍱 Create Scratch Buffer
D3D12_RESOURCE_DESC asResourceDesc;
asResourceDesc.Dimension = D3D12_RESOURCE_DIMENSION_BUFFER;
asResourceDesc.Alignment = 0;
asResourceDesc.Width = max(topLevelPrebuildInfo.ScratchDataSizeInBytes,
                           bottomLevelPrebuildInfo.ScratchDataSizeInBytes);
asResourceDesc.Height = 1;
asResourceDesc.DepthOrArraySize = 1;
asResourceDesc.MipLevels = 1;
asResourceDesc.Format = DXGI_FORMAT_UNKNOWN;
asResourceDesc.SampleDesc.Count = 1;
asResourceDesc.SampleDesc.Quality = 0;
asResourceDesc.Layout = D3D12_TEXTURE_LAYOUT_ROW_MAJOR;
asResourceDesc.Flags = D3D12_RESOURCE_FLAG_ALLOW_UNORDERED_ACCESS;
 
// 🍱 Create AS Scratch Buffer
hr = device->CreateCommittedResource(
    &heapProps, D3D12_HEAP_FLAG_NONE, &asResourceDesc,
    D3D12_RESOURCE_STATE_RAYTRACING_ACCELERATION_STRUCTURE, nullptr,
    IID_PPV_ARGS(&asScratchBuffer));
 
// 🍱 Create TLAS Buffer
asResourceDesc.width = topLevelPrebuildInfo.ResultDataMaxSizeInBytes;
hr = device->CreateCommittedResource(
    &heapProps, D3D12_HEAP_FLAG_NONE, &asResourceDesc,
    D3D12_RESOURCE_STATE_RAYTRACING_ACCELERATION_STRUCTURE, nullptr,
    IID_PPV_ARGS(&tlasBuffer));
 
// 🍱 Create BLAS Buffer
asResourceDesc.width = bottomLevelPrebuildInfo.ResultDataMaxSizeInBytes;
hr = device->CreateCommittedResource(
    &heapProps, D3D12_HEAP_FLAG_NONE, &asResourceDesc,
    D3D12_RESOURCE_STATE_RAYTRACING_ACCELERATION_STRUCTURE, nullptr,
    IID_PPV_ARGS(&blasBuffer));
 
D3D12_RAYTRACING_INSTANCE_DESC instanceDesc = {};
instanceDesc.Transform[0][0] = instanceDesc.Transform[1][1] =
    instanceDesc.Transform[2][2] = 1;
instanceDesc.InstanceMask = 1;
instanceDesc.AccelerationStructure = blasBuffer->GetGPUVirtualAddress();
D3D12_HEAP_PROPERTIES uploadHeapProperties = {};
uploadHeapProperties.Type = D3D12_HEAP_TYPE_UPLOAD;
uploadHeapProperties.CPUPageProperty = D3D12_CPU_PAGE_PROPERTY_UNKNOWN;
uploadHeapProperties.MemoryPoolPreference = D3D12_MEMORY_POOL_UNKNOWN;
uploadHeapProperties.CreationNodeMask = 1;
uploadHeapProperties.VisibleNodeMask = 1;
 
D3D12_RESOURCE_DESC bufferDesc = {};
bufferDesc.Dimension = D3D12_RESOURCE_DIMENSION_BUFFER;
bufferDesc.Alignment = 0;
bufferDesc.Width = sizeof(instanceDesc);
bufferDesc.Height = 1;
bufferDesc.DepthOrArraySize = 1;
bufferDesc.MipLevels = 1;
bufferDesc.Format = DXGI_FORMAT_UNKNOWN;
bufferDesc.SampleDesc.Count = 1;
bufferDesc.SampleDesc.Quality = 0;
bufferDesc.Layout = D3D12_TEXTURE_LAYOUT_ROW_MAJOR;
bufferDesc.Flags = D3D12_RESOURCE_FLAG_NONE;
hr = device->CreateCommittedResource(
    &uploadHeapProperties, D3D12_HEAP_FLAG_NONE, &bufferDesc,
    D3D12_RESOURCE_STATE_GENERIC_READ, nullptr, IID_PPV_ARGS(instanceDescs));
 
void* pMappedData;
(*instanceDescs)->Map(0, nullptr, &pMappedData);
memcpy(pMappedData, &instanceDesc, sizeof(instanceDesc));
(*instanceDescs)->Unmap(0, nullptr);
 
// 🏗️ BLAS build command
D3D12_BUILD_RAYTRACING_ACCELERATION_STRUCTURE_DESC bottomLevelBuildDesc = {};
 
bottomLevelBuildDesc.Inputs = bottomLevelInputs;
bottomLevelBuildDesc.ScratchAccelerationStructureData =
    scratchResource->GetGPUVirtualAddress();
bottomLevelBuildDesc.DestAccelerationStructureData =
    blasBuffer->GetGPUVirtualAddress();
 
// 🏗️ TLAS build command
D3D12_BUILD_RAYTRACING_ACCELERATION_STRUCTURE_DESC topLevelBuildDesc = {};
 
topLevelInputs.InstanceDescs = instanceDescs->GetGPUVirtualAddress();
topLevelBuildDesc.Inputs = topLevelInputs;
topLevelBuildDesc.DestAccelerationStructureData =
    tlasBuffer->GetGPUVirtualAddress();
topLevelBuildDesc.ScratchAccelerationStructureData =
    scratchResource->GetGPUVirtualAddress();
 
// 🏢 Execute build
commandList->BuildRaytracingAccelerationStructure(&bottomLevelBuildDesc, 0,
                                                  nullptr);
 
D3D12_RESOURCE_BARRIER barrier = {};
barrier.Type = D3D12_RESOURCE_BARRIER_TYPE_UAV;
barrier.UAV.pResource = blasBuffer;
commandList->ResourceBarrier(1, &barrier);
commandList->BuildRaytracingAccelerationStructure(&topLevelBuildDesc, 0,
                                                  nullptr);

Traversal occurs by executing a ray tracing pipeline with a given graphics API, and further in ray tracing programmable shader steps when calling the builtin TraceRay function.

// 🌎 Global Resources
RaytracingAccelerationStructure Scene : register(t0, space0);
RWTexture2D<float4> tOutput : register(u0);
cbuffer cb : register(b0)
{
    row_major float4x4 projectionMatrix : packoffset(c0);
    row_major float4x4 viewMatrix : packoffset(c4);
    float3 origin : packoffset(c8);
    float near : packoffset(c9.x);
    float far : packoffset(c9.y);
};
 
[shader("raygeneration")]
void raygen()
{
    float2 screenCoords =
        (float2)DispatchRaysIndex() / (float2)DispatchRaysDimensions();
    float2 ndc = screenCoords * 2.0 - float2(1.0, 1.0);
 
    float near = .1;
    float far = 300.0;
 
    float3 rayDir =
        normalize(mul(mul(viewMatrix, projectionMatrix),
                      float4(ndc * (far - near), far + near, far - near)))
            .xyz;
 
    RayDesc ray;
    ray.Origin = origin.xyz;
    ray.Direction = rayDir;
    ray.TMin = 0.1;
    ray.TMax = 300.0;
    RayPayload payload = {float4(0, 0, 0.5, 0)};
    TraceRay(Scene, RAY_FLAG_NONE, ~0, 0, 1, 0, ray, payload);
 
    // Write the raytraced color to the output texture.
    tOutput[DispatchRaysIndex().xy] = payload.color;
}

More Resources

• Tero Karras (NVIDIA) released a 3 part series describing collision detection, tree traversal, and tree construction on the GPU.

• Jacco Bikker (@j_bikker) wrote a series of blog posts on BVH construction here.

• My Raw DirectX 12 Book - Chapter 4 focuses on ray tracing with DirectX 12 and provides more details on how to execute ray tracing in DirectX 12.

• Geometric Tools for Computer Graphics by Philip Schneider and David H. Eberly has a resources page that provides C++ header files for intersection tests for axis aligned bounding boxes as well as other volumes.

• There's a video presentation on a Survey on Bounding Volume Hierarchies for Ray Tracing provided by High Performance Graphics.

• Computer Graphics at TU Wien gave a lecture on spatial acceleration structures here.

• Ray Tracing: The Next Week reviews these concepts as well.

[McGuire 2018] The Graphics Codex Morgan McGuire (@CasualEffects) Casual Effects 2018 graphicscodex.courses.nvidia.com

[Kajiya 1986] The Rendering Equation James Kajiya ACM, SIGGRAPH Computer Graphics 1986 doi.acm.org

[Pharr et al. 2016] Physically based rendering: From theory to implementation Matt Pharr, Wenzel Jakob and Greg Humphreys Morgan Kaufmann 2016 pbr-book.org

[J. Doyle et al. 2012] Hardware Accelerated Construction of SAH-Based Bounding Volume Hierarchies for Interactive Ray Tracing Michael J. Doyle, Colin Fowler and Michael Manzke ACM 2012 doi.org

[Liktor et al. 2016] Bandwidth-Efficient BVH Layout for Incremental Hardware Traversal Gabor Liktor and Karthik Vaidyanathan Eurographics 2016 intel.com

[Vaidyanathan et al. 2016] Watertight Ray Traversal with Reduced Precision Karthik Vaidyanathan, Tomas Akenine-Möller (@inversepixel) and Marco Salvi (@marcosalvi) Eurographics 2016 intel.com

[AMD 2020] RDNA 2 Instruction Set Architecture AMD 2020 developer.amd.com

[Vaidyanathan et al. 2019] Wide BVH Traversal with a Short Stack Karthik Vaidyanathan, Sven Woop and Carsten Benthin Eurographics 2019 intel.com

[Benthin et al. 2017] Improved Two-Level BVHs Using Partial Re-Braiding Carsten Benthin, Sven Woop, Ingo Wald (@IngoWald) and Attila T. Afra ACM 2017 sven-woop.de

[Karras 2020] Thinking parallel, part III: Tree construction on the GPU Tero Karras NVIDIA 2020 developer.nvidia.com

[Meister et al. 2021] A Survey on Bounding Volume Hierarchies for Ray Tracing Daniel Meister, Shinji Ogaki (@ShinjiOgaki), Carsten Benthin, Michael J. Doyle (@mjdoyle0), Michael Guthe and Jiří Bittner Computer Graphics Forum 2021 meistdan.github.io

[Benthin et al. 2022] PLOC++: Parallel Locally-Ordered Clustering for Bounding Volume Hierarchy Construction Revisited Carsten Benthin, Radoslaw Drabinski, Lorenzo Tessari and Addis Dittebrandt ACM 2022 intel.com

[Vinkler et al. 2014] Bounding Volume Hierarchies versus Kd-Trees on Contemporary Many-Core Architectures Marek Vinkler, Vlastimil Havran and Jiří Bittner ACM 2014 doi.org

[Bikker 2022] How to build a BVH – Part 1: Basics Jacco Bikker (@j_bikker) 2022 jacco.ompf2.com

Shaders are a set of instructions (so a program or kernel) that execute on programmable steps of a given GPU pipeline, be it a raster pipeline, compute, ray tracing, etc.

They could be used to transform vertices, shade pixels in raster based rendering, handle intersections of primitives in ray tracing, machine learning, perform general computations, and much more. Shaders execute on a Single-Instruction-Multiple-Data (SIMD) architecture, where groups of processing cores execute the same instruction across large swaths of data.

Depending on the vendor this can mean different things. For instance, on AMD hardware a kernel executes across groups of 32 or 64 work-items (or dispatch threads) called wavefronts (or thread groups). These work-items execute on a small processor that handles scalar logic, vector logic, and scalar/vector memory stored in the local cache of the group or in GPU memory. [AMD 2020]

There are a number of shader languages used in real-time rendering:

With the addition of other platform specific languages such as PlayStation Shading Language (PSL), experimental languages such as Rust to SPIR-V, and the older C for Graphics (Cg) used in older Unity 3D code [Doppioslash 2018]. Depending on which platform/API you're targeting, you'll be stuck with one of these languages.

The shader language world is fragmented by design given there's so many vendors and APIs, but every shading language is similar enough to C that one can easily switch between them. Depending on the language, there could be an API specific intermediary representation (IR) of the shader code, and each vendor (AMD, NVIDIA, Intel, Qualcomm, etc.) converts that IR to their machine code at the driver level.

Let's review some of the similarities and differences between shader languages, their features, how they execute, and how to approach writing shaders for your applications.

We'll review a vertex shader in HLSL, GLSL, MSL, and WGSL where we:

• Access data received from the Input Assembly step of the graphics pipeline.

• Pass data to the next step of the graphics pipeline.

• Receive data through uniforms or constant buffer views.

In addition, we'll review a compute shader in each of those languages where we:

• Read and write to buffers.

• Perform atomic operations.

• Share data across thread groups.

From there we'll go over how to approach compiling those shaders and include them in your applications.

Language Features

Nearly all shading languages are very similar to C, and have all the usual features associated with C.

In addition, shading languages feature a standard library of intrinsic functions available in the global namespace for performing common math operations. These standard functions include:

• Vector Math (matrix multiplication, vector addition, dot products). Some languages overload operator*, others use a function mul.

• Transcendentals (Sin, Cos, Log, etc.) Some languages distinguish tan(v) and tan2(a, b).

• Neighbor Operations (derivatives) with minor differences in function names.

• Logic (checking all members of a boolean vector with all(), etc.)

• GPGPU memory synchronization such as atomicAdd.

Most of these functions are even named the same, but some are slightly different (HLSL's frac vs GLSL's fract) or so simple that they're not included in other languages like HLSL's saturate() vs clamp(val, 0.0, 1.0).

• DirectX Shader Compiler (DXC) features a page in their documentation that showcases HLSL vs. GLSL intrinsic functions.

HLSL

High-Level Shading Language (HLSL) is arguably the best language for writing shaders, as it's very well supported, separates samplers and textures (unlike GLSL), and has common language features like:

• namespaces
• template generics
• overloadable operators
• #include

and much more as of HLSL 2021.

For more details on the HLSL language, review the following:

• Built-in Types

• Semantic Decorators

• Intrinsic Functions

// ❎ HLSL Vertex Shader

struct Constants
{
  row_major float4x4 modelViewProjection : packoffset(c0);
  float4x4 inverseTransposeModel : packoffset(c4);
  float4x4 view : packoffset(c8);
};

ConstantBuffer<Constants> constants : register(b0);

struct VertexInput
{
    float3 position : POSITION;
    float3 normal : NORMAL;
    float2 texCoord : TEXCOORD;
};

struct VertexOutput
{
    float4 position : SV_Position;
    float3 normal : NORMAL;
    float2 texCoord : TEXCOORD;

};

VertexOutput main(VertexInput vertexInput)
{
    float4 position = mul(float4(vertexInput.position, 1.0f), constants.modelViewProjection);
    float3 outNormal = mul(float4(vertexInput.normal, 1.0), constants.inverseTransposeModel).xyz;
    float2 outTexCoord = vertexInput.texCoord;

    VertexOutput output;
    output.position = position;
    output.normal = outNormal;
    output.texCoord = outTexCoord;

    return output;
}

If you want globally accessible shader constants (like how uniforms worked in older APIs), you can use the keyword cbuffer.

cbuffer cb : register(b0)
{
    row_major float4x4 modelViewProjection : packoffset(c0);
    float3 origin : packoffset(c4);
    float time : packoffset(c5);
};

// Later...

float4 position = mul(float4(vertexInput.position, 1.0f), modelViewProjection);

HLSL is unique in that it describes attribute inputs with semantics (for example, float3 position : POSITION), though you can and should opt to avoid these and instead use the more generic ATTRIB. You can also use this syntax to describe other things about your variable similar to a bitfield, such as alignment or what buffer/texture/sampler registers correspond to a given variable.

HLSL shaders are very intuitive, they have an input, and output. They can have global constants or constants contained in a constant buffer.

Now let's look at a compute shader:

// ❎ HLSL Compute Shader
struct Constants
{
  float time : packoffset(c0);
};

ConstantBuffer<Constants> constants : register(b0);
Texture2D<float4> albedoTex : register(t1);
Texture2D<float4> pbrLutTex : register(t2);
SamplerState gSampler : register(s0);

RWTexture2D<float4> tOutput : register(u0);

groupshared float groupData[4];

namespace random
{
    float foo()
    {
        return 16.0;
    }
}

[numthreads(8, 4, 1)]
void main(uint3 groupThreadID : SV_GroupThreadID,
         uint3 groupID : SV_GroupID,
         uint  groupIndex : SV_GroupIndex,
         uint3 dispatchThreadID: SV_DispatchThreadID)
{
    tOutput[dispatchThreadID.xy] = float4( float(groupThreadID.x) / random::foo(), float(groupThreadID.y) / 16.0, dispatchThreadID.x / 1280.0, 1.0);
}

You can easily define compute shader intrinsic values as arguments of the main function and expect those to be passed in, and name them whatever you might like. Otherwise this is practically the same as a vertex shader and as we'll see, the same across languages.

GLSL

OpenGL Shading Language (GLSL) is the standard shader programming language for Khronos APIs such as Vulkan, OpenGL 4.x through optional extensions for different versions of GLSL, and WebGL. There's a few fundamental differences between GLSL and other languages:

• Vector types use the prefix vec (for example, vec4 instead of float4 and mat4 instead of float4x4)

• Samplers and texture bindings are tied together. This is a big difference and area of contention, as this isn't necessary and other languages separate these two data structures.

Feature wise however GLSL is similar to HLSL, with support for #include through optional extensions and similar intrinsic functions.

The GLSL code begins with layout declarations that define exactly how each struct and object is laid out in memory. Each location corresponds to an address pointing to a maximum of 4 single precision floats. The keyword in denotes that this data is coming from the Input Assembly step of the graphics pipeline, and the keyword out denotes that this is data being passed to the next step of the graphics pipeline (in could be described as passing a reference).

The uniforms in this shader are struct based, and are bound to sets and binding indexes. Uniforms can be Uniform Buffer Objects (UBOs) where the last parameter can be an n-dimensional matrix if needed, or the could be Samplers. Note you don't need to denote the namespace of the parameters of the UBOs, each name is unique and accessible in the global namespace (model.transform == transform).

It's worth noting that there's a wide range of supported versions of GLSL and OpenGL, with GLSL ES 1.0 being the version used for WebGL, and GLSL ES 3.0 used for WebGL 2.

In the main function we perform our shader logic and write our output position and varying variables. Note that the use of the keyword main can be set to anything you would like.

// 🌋 GLSL Vertex Shader
#version 450

struct VertexInput
{
    vec3 position;
    vec3 normal;
    vec2 texCoord;
};

struct VertexOutput
{
    vec4 position;
    vec3 normal;
    vec2 texCoord;
};

struct Constants
{
    mat4 modelViewProjection;
    mat4 inverseTransposeModel;
    mat4 view;
};

layout(set = 0) uniform Constants constants;

layout (location = 0) in vec3 position;
layout (location = 1) in vec3 normal;
layout (location = 2) in vec2 texCoord;

layout (location = 0) out vec3 outNormal;
layout (location = 1) out vec2 outTexCoord;

void main()
{
    vec4 position =  constants.modelViewProjection * vec4(position, 1.0f);
    outNormal = (constants.inverseTransposeModel * vec4(normal, 1.0)).xyz;
    outTexCoord = texCoord;
    gl_Position = position;
}

Compute shaders are generally the same as HLSL, though each built-in value passed in HLSL is just a global in GLSL. Writing data to an image is done through a function instead of the array syntax. There's also no need to annotate the main function to specify what kind of shader it is, since that's written out at compile time to SPIR-V.

// 🌋 GLSL Compute Shader
#version 450
layout(local_size_x = 8, local_size_y = 4, local_size_z = 1) in;

layout(binding = 0, std140) uniform Constants
{
    float time;
} constants;

layout(binding = 0, rgba32f) uniform readonly writeonly image2D tOutput;

shared float groupData[4];

float foo()
{
    return 16.0;
}

void main()
{
    imageStore(tOutput, gl_GlobalInvocationID.xy,
               vec4(float(gl_SubgroupInvocationID.x) / foo(),
                    float(gl_SubgroupInvocationID.y) / 16.0,
                    gl_GlobalInvocationID.x / 1280.0, 1.0));
}

MSL

In Metal Shading Language (MSL) there is no uniform keyword, everything that is a buffer could be a uniform. Similar to HLSL, attributes are passed as arguments in the main function, with directives to denote where that data is coming from. Unlike HLSL or GLSL, data to be passed to later stages of the pipeline is returned by the main function.

Metal vector types are named the same as HLSL, and the intrinsic functions are nearly all the same as well.

// 🤖 Metal Vertex Shader
#include <metal_stdlib>
#include <simd/simd.h>

using namespace metal;

struct Constants
{
    float4x4 modelViewProjection;
    float4x4 inverseTransposeModel;
    float4x4 view;
};

struct VertexOutput
{
    float3 normal [[user(locn0)]];
    float2 texCoord [[user(locn1)]];
    float4 position [[position]];
};

struct VertexInput
{
    float3 position [[attribute(0)]];
    float3 normal [[attribute(1)]];
    float2 texCoord [[attribute(2)]];
};

vertex VertexOutput main(VertexInput in [[stage_in]],
                         constant Constants& constants [[buffer(0)]],
                         uint gl_VertexID [[vertex_id]],
                         uint gl_InstanceID [[instance_id]])
{
    VertexOutput out = {};
    float4 position = constants.modelViewProjection * float4(in.position, 1.0);
    out.normal = (constants.inverseTransposeModel * float4(in.normal, 1.0)).xyz;
    out.texCoord = in.texCoord;
    out.position = position;
    return out;
}

Metal's input/output model is much more clear than in other languages, where the main function shows those bindings, and this applies to their compute shaders as well (denoted with the keyword kernel).

// 🤖 Metal Compute Shader
#include <metal_stdlib>
#include <simd/simd.h>

using namespace metal;

struct Constants
{
    float time;
};

float foo()
{
    return 16.0;
}

kernel void main(
    uint3 groupThreadID [[ thread_position_in_threadgroup ]],
    uint3 groupID [[threadgroup_position_in_grid]],
    uint groupIndex [[thread_index_in_threadgroup]],
    uint3 dispatchThreadID [[thread_position_in_grid]],
    constant Constants& constants [[buffer(0)]],
    texture2d<float, access::read_write> tOutput [[texture(0)]])
{
    threadgroup float groupData[4];
    tOutput[dispatchThreadID.xy] = float4( float(groupThreadID.x) / foo(), float(groupThreadID.y) / 16.0, dispatchThreadID.x / 1280.0, 1.0);
}

WGSL

WebGPU Shading Language (WGSL) is pretty unique, with a syntax that's a bit like JavaScript, Rust, C, and Apple Metal. It's evolved a lot over its development, it used to be much more verbose [Maxfield 2018] but has since become a blend of Metal Shader Language and HLSL.

Vector types are unique in that they are similar to both HLSL and GLSL, but have a more clearly defined size for each type.

// 🌐 WGSL Vertex Shader
struct Constants
{
  modelViewProjection: mat4x4<f32>,
  inverseTransposeModel: mat4x4<f32>,
  view: mat4x4<f32>
};

struct VertexOutput
{
    @builtin(position) position: vec4<f32>,
    @location(0) normal: vec3<f32>,
    @location(1) uv: vec2<f32>
};

@group(0) @binding(0) var<uniform> constants : Constants;

@vertex
fn main(@location(0) position: vec4<f32>,
        @location(1) uv: vec2<f32>) -> VertexOutput {
    var vsOut: VertexOutput;
    vsOut.position = constants.modelViewProjection * vec4<f32>(position, 1.0);
    vsOut.position = constants.inverseTransposeModel * vec4<f32>(normal, 1.0);
    vsOut.uv = uv;
    return vsOut;
}

Compute shaders are similar to GLSL, with similar names for built-in values such as the global invocation ID, etc:

// 🌐 WGSL Compute Shader
struct Constants
{
    time: f32
};

@group(0) @binding(0) var<uniform> constants: Constants;
@group(0) @binding(1) var tOutput: texture_storage_2d<rgba8unorm,write>;

fn foo() -> f32
{
    return 4.0;
}

@compute @workgroup_size(8, 4, 1)
fn main(@builtin(local_invocation_id) localInvocationID: vec3<u32>,
        @builtin(workgroup_id) workgroupID: vec3<u32>,
        @builtin(local_invocation_index) localInvocationIndex: u32,
        @builtin(global_invocation_id) globalInvocationID: vec3<u32>) {

    var color = vec4<f32>(
        (f32(localInvocationID.x) / 8.0) + sin((f32(globalInvocationID.x) / 160.0) + (constants.time * 0.01)),
        f32(localInvocationID.y) / foo(),
        f32(globalInvocationID.y) / 360.0,
        255
    );

    textureStore(tOutput, vec2<i32>(globalInvocationID.xy), color);
}

OpenCL

OpenCL looks incredibly similar to both Apple Metal and OpenGL. While the idea of a graphics pipeline doesn't exist in OpenCL, a compute kernel in other APIs is essentially the same thing.

struct Constants
{
    float time;
    uint width;
    uint height;
};

float foo()
{
    return 16.0;
}

__kernel void main
(

    __constant Constants constants,
    __global float4* tOutput
)
{
    uint localId = get_local_id(0);
    uint workgroupId = get_group_id(0);
    uint globalId = get_global_id(0);

    int x = globalId % constants.width;
    int y = globalId / constants.width;

    if (x >= width || y >= constants.height)
    {
        return;
    }
    tOutput[pixel_idx] = (float4)((float)(x) / foo(), (float)(y) / 16.0, (float)(x) / 1280.0, 1.0);
}

Shader Compilation

Offline

Yak-shave: When I press "F5" now in this side project, cmake actually detects if I've changed shaders, compiles them to SPIR-V, converts that to hex in a .c file, compiles those c files into a lib, and then links it to the exe so I can reference the bytecode without file reads. ~ Jeremy Ong (@m_ninepoints)

You should always aim to compile your shaders in advance to your target graphics API's IR. This will result in a noticeable improvement in performance, since you no longer have to wait for shaders to compile before submitting commands. Some languages however encourage you not to compile shaders to an IR such as GLSL with older versions of OpenGL (though you can now use SPIR-V thanks to extensions) or WebGPU Shading Language (WGSL), and for those you can minify/obfuscate them similar to JavaScript for production.

HLSL

# ❎ DirectX
dxc.exe -T lib_6_3 -Fo assets/triangle.vert.dxil assets/triangle.vert.hlsl

# 🌋 Vulkan / OpenGL
dxc.exe -spirv -Fo assets/triangle.vert.spv assets/triangle.vert.hlsl

HLSL Shaders can be compiled offline to DXIL with DirectX Shader Compiler.

If you want to output SPIR-V, you will need to pass the -spirv flag as well as annotate your HLSL with SPIR-V decorators.

GLSL

# 🌋 Vulkan / OpenGL
glslangValidator -V triangle.vert -o triangle.vert.spv

GLSLang Validator is Khronos' reference front end for GLSL, and can compile it to SPIR-V.

Metal

# 🤖 Metal
metallib triangle.vert.msl triangle.vert.mbc

The metallib compiler is available in the command line in any recent MacOS installation.

Online

// Vulkan (glslang.h)

SH_IMPORT_EXPORT int ShCompile(
    const void* shHandle,
    const char* const shaderStrings[],
    const int numStrings,
    const int* lengths,
    const EShOptimizationLevel,
    const TBuiltInResource *resources,
    int debugOptions,
    int defaultVersion = 110,            // use 100 for ES environment, overridden by #version in shader
    bool forwardCompatible = false,      // give errors for use of deprecated features
    EShMessages messages = EShMsgDefault // warnings and errors
    );

Vulkan introduces offline shader compilation, so to compile you'll need to either write a compiler to SPIR-V or use the official compiler. GLSLang even supports compiling HLSL to SPIR-V, though this feature is still experimental.

// OpenGL

GLuint shader = glCreateShader(VETEX_SHADER);

glShaderSource(shader, srcStr);

glCompileShader(shader);

GLint isCompiled = 0;
glGetShaderiv(shader, GL_COMPILE_STATUS, &isCompiled);
if(isCompiled == GL_FALSE)
{
  GLint maxLength = 0;
  glGetShaderiv(shader, GL_INFO_LOG_LENGTH, &maxLength);
  std::vector<GLchar> errorLog(maxLength);
  glGetShaderInfoLog(shader, maxLength, &maxLength, &errorLog[0]);
}

OpenGL is pretty straight forward, you send it the string and check if it's valid. Alternatively, newer versions of OpenGL let you load SPIR-V directly.

// DirectX

HRESULT WINAPI D3DCompile(
  in      LPCVOID pSrcData,
  in      SIZE_T SrcDataSize,
  in_opt  LPCSTR pSourceName,
  in_opt  const D3D_SHADER_MACRO pDefines,
  in_opt  ID3DInclude pInclude,
  in_opt  LPCSTR pEntrypoint,
  in      LPCSTR pTarget,
  in      UINT Flags1,
  in      UINT Flags2,
  out     ID3DBlob ppCode,
  out_opt ID3DBlob ppErrorMsgs
);

DirectX like OpenGL supports compiling shaders at runtime, however both now support loading both bytecode or shader strings.

id<MTLLibrary> library = [self.device newDefaultLibrary];
id<MTLFunction> vertexFunc = [library newFunctionWithName:@"vertex_main"];
id<MTLFunction> fragmentFunc = [library newFunctionWithName:@"fragment_main"];

Rather than managing shader modules yourself, in Metal, you can create libraries of shader functions and load those as needed. If you need to grab the main of a given library's function list, you can use the newFunctionWithName method.

Transpiling

So long as a shader is written in valid GLSL or HLSL, transpiling it to either GLSL, HLSL, or MSL is possible with SPIRV-Cross. You may need to annotate some parts of your shader though with API specific information.

# 🌋 Compile shaders to SPIR-V binary
glslangValidator -V triangle.vert -o triangle.vert.spv
glslangValidator -V triangle.frag -o triangle.frag.spv

# ❎ HLSL
spirv-cross triangle.vert.spv --hlsl --shader-model 50 --set-hlsl-vertex-input-semantic 0 POSITION --set-hlsl-vertex-input-semantic 1 COLOR --output triangle.vert.hlsl
spirv-cross triangle.frag.spv --hlsl --shader-model 50 --set-hlsl-vertex-input-semantic 0 COLOR --output triangle.frag.hlsl

# ⚪ OpenGL ES 3.1
spirv-cross triangle.vert.spv --version 310 --es --output triangle.vert.glsl
spirv-cross triangle.frag.spv --version 310 --es --output triangle.frag.glsl

# 🤖 Metal
spirv-cross triangle.vert.spv --msl --output triangle.vert.msl
spirv-cross triangle.frag.spv --msl --output triangle.frag.msl

In addition, WGSL compilers can also transpile directly:

# 🦊 Mozilla Naga
# https://github.com/gfx-rs/naga

# ⚪ Convert the WGSL to GLSL vertex stage under ES 3.20 profile
cargo run my_shader.wgsl my_shader.vert --profile es310

# 🤖 Convert the SPV to Metal
cargo run my_shader.spv my_shader.metal

Conclusion

Shading languages are similar to each other, with minor differences in keywords, but different underlying designs philosophies such as the union of samplers and textures in GLSL, or the buffer attribute model of Metal. It's not too difficult to work around these differences though, and bridge the gap between HLSL, GLSL, MSL, and WGSL.

There's a few areas that we didn't go over that are worth reviewing such as:

• Ray Tracing Shaders - DirectX 12 and Vulkan share the same type of ray tracing shaders, but Metal takes a more unique approach.

• Meshlet Shaders - These can be reviewed in their own posts, but are currently limited to DirectX 12 and Vulkan.

Resources

Each shader language has documentation pages that go over their nuances in more detail:

• The HLSL Reference can be reviewed in from their documentation page.

• The GLSL Reference is combined with OpenGL's reference, it's a bit tough to parse through, but anything with gl_ or without a glCamelCase is part of GLSL.

• The MSL Reference details their intrinsic functions, keywords, etc.

• The WGSL Reference features everything about the language from intrinsic functions to decorators.

• The OpenCL C Reference showcases the kernel language used for OpenCL. Even though the language is for compute only, it's extremely similar to GLSL and worth a look.

It's also worth reviewing vendor ISAs to better understand what instructions your shader will eventually become:

• The AMD RDNA2 ISA reference guide will help you better architect your shaders by showing what instructions DXIL and SPIR-V ultimately become in AMD hardware.

• NVIDIA's Parallel Thread Execution (PTX) ISA shows what your shader IR will eventually become on NVIDIA hardware, and describes the instructions CUDA cores can execute.

It's also worth reviewing the test suite for the various compilers for these languages:

• DirectX Shader Compiler can also output SPIR-V code, and would serve as a better tool from translating HLSL to GLSL than say, SPIR-V Cross.

• GLSLang is Khronos' official GLSL compiler, and besides compile SPRIR-V it can also reflect GLSL and even compile some HLSL.

• Mozilla Naga is the Firefox WGSL compiler written in Rust.

• Google Tint is Chromium's WGSL compiler.

• SPIRV-Cross can convert SPIR-V to GLSL, HLSL, and MSL easily.

• Tim Jones (@tim_jones) released a VS Code Plugin called HLSL Tools that lets you lint your shaders more easily.

There's also a number of videos, blog posts, and projects that focus on shaders:

• The State of Shader Tooling in Vulkan by David Neto (@dneto1969) contrasts different shader IRs and their compilers. You can find the slides here. (Thanks for sharing this and the WGSL compilers on Twitter David!)

• Slang is an new shading language that compiles to GLSL, HLSL and SPIR-V, a spiritual successor of Cg.

• AMD released a talk titled Trip Down the Shader Compiler Pipeline, which reviews how shaders go from source code to their vendor specific IR.

• Matthäus G. Chajdas (@NIV_Anteru) released a blog post detailing how HLSL and GLSL map to each other.

• Dzmitry Malyshau benchmarked the Rust WGSL shader compiler Naga with different compilers.

• Shader Model 6.0 adds new features to HLSL that are worth reviewing here, with Shader Model 6.7 providing new Gather operations.

• The Shader Playground by Tim Jones (@tim_jones) provides an easy interface to see what your shaders compile to in various ISAs for different vendors/APIs.

• Adam Sawicki (@Reg__) released an article detailing the difference between the two shader compilers of DirectX 12 . (Thanks @raphlinus from Hacker News for this and all the feedback!)

I wrote a variety of blog posts that may be worth reviewing as well:

• If you're curious about shader debugging, visit this article comparing different graphics API debugging tools.

• Visit my blog post comparing different modern graphics APIs to get more insights into how shaders and their host APIs differ in design.

[AMD 2020] RDNA 2 Instruction Set Architecture AMD 2020 developer.amd.com

[Doppioslash 2018] Physically based Shader Development for unity 2017: Develop Custom Lighting Systems Claudia Doppioslash (@doppioslash) Apress 2018 link.springer.com

[Maxfield 2018] Web High Level Shading Language Myles Maxfield (@Litherum) 2018 webkit.org

DirectX 12 is the latest iteration of Microsoft's proprietary computer graphics API used for Windows and Xbox platforms. DirectX 12, like newer graphics APIs such as Vulkan, Apple Metal, or WebGPU, was designed to map closer to the GPU's underlying architecture, resulting in a much less complex and faster driver. This means the application developer is responsible for the creation of pipeline states directly, specifying which commands get executed with queues, and now has the option to build these data structures with multiple threads.

DirectX focuses on real time rendering, thus is designed for Game Developers and Computer Aided Design (CAD) Software Engineers. As the industry standard in Computer Graphics APIs, you can expect nearly all compliant hardware to have robust support for the API, making it the standard for commercial projects.

From 3D/image authoring software like Marmoset Toolbag, Adobe PhotoShop, Unreal Engine, to commercial games such as Activision Blizzard's OverWatch, Call of Duty, Epic's Fortnite, the vast majority of titles on Valve's Steam and much more, this graphics API is the most popular and ubiquitous one of them all, despite its platform exclusivity.

That's not to say there aren't uses for it outside of 3D rendering. Google Chrome renders HTML and SVG elements with DirectX, Tensorflow features a DirectX 12 backend for machine learning execution via DirectML, and GPGPU computations with the compute pipeline can execute all manner of workloads, from physics simulations, BVH construction, and much more.

DirectX 12 is currently supported on:

• 🗔 Windows 7 - 11

• ✖️ Xbox One - Xbox Series X/S

With Windows 7 having partial support for it via Microsoft's D3D12 On 7.

While the official languages might be C and C++, many languages support DirectX 12:

• C

• C++

• C#

• Rust (through dx-sys)

I've prepared a Github Repo with everything we need to get started. We're going to walk through a Hello Triangle app in C++, a program that creates a triangle with the raster graphics pipeline and renders it onto the screen.

Setup

First install:

• Git

• CMake

• An IDE such as Visual Studio, XCode, or a compiler such as GCC.

Then type the following in any terminal such as VS Code's Integrated Terminal.

# 🐑 Clone the repo
git clone https://github.com/alaingalvan/directx12-seed --recurse-submodules

# 💿 go inside the folder
cd directx12-seed

# 👯 If you forget to `recurse-submodules` you can always run:
git submodule update --init

# 👷 Make a ./build/ folder and place your project files inside:

# 🖼️ To build your Visual Studio solution on Windows x64
cmake . -B build -A x64

# 🔨 Build on any platform:
cmake --build build

Overview

DirectX 12's documentation recommends the use of ComPtr<T> as an alternative to std::shared_ptr<T>, with the benefit of better debugging and easier initialization of DirectX 12 data structures.

Regardless of whether or not you choose to use ComPtr<T>, the steps to rendering raster graphics with DirectX 12 are pretty similar to other modern graphics APIs:

1. Initialize the API - Create your IDXGIFactory, IDXGIAdapter, ID3D12Device, ID3D12CommandQueue, ID3D12CommandAllocator, ID3D12GraphicsCommandList.

2. Setup Frame Backings - Create your IDXGISwapChain,ID3D12DescriptorHeap for your back buffers, your back buffer ID3D12Resource Render Target Views, a ID3D12Fence to detect when a frame is finished rendering.

3. Initialize Resources - Create your Triangle Data such as your ID3D12Resource Vertex Buffers, ID3D12Resource Index Buffer, ID3D12Fences to detect when uploads to GPU memory are complete. Load your shaders ID3DBlobs, your constant buffer ID3D12Resources and their ID3D12DescriptorHeap, describe what resources will be accessible with a ID3D12RootSignature, and build your ID3D12PipelineState.

4. Encode Commands - Write commands to your ID3D12GraphicsCommandList with what pipeline commands you intend to execute, making sure to put ResourceBarriers where appropriate.

5. Render - Update your GPU constant buffer data (Uniforms), submit commands to the ID3D12CommandQueue with ExecuteCommandLists, Present your swapchain, and await the next frame.

6. Destroy - Destroy any data structures you're done using with Release() or rely on ComPtr<T> deallocating for you.

The following will explain snippets that can be found in the Github repo, with certain parts omitted, and member variables (mMemberVariable) declared inline without the m prefix so their type is easier to see and the examples here can work on their own.

Window Creation

We're using CrossWindow to handle cross platform window creation, so creating a Win32 window and updating it is very easy:

#include "CrossWindow/CrossWindow.h"
#include "Renderer.h"

#include <iostream>

void xmain(int argc, const char** argv)
{
  // 🖼️ Create Window
  xwin::WindowDesc wdesc;
  wdesc.title = "DirectX 12 Seed";
  wdesc.name = "MainWindow";
  wdesc.visible = true;
  wdesc.width = 640;
  wdesc.height = 640;
  wdesc.fullscreen = false;

  xwin::Window window;
  xwin::EventQueue eventQueue;

  if (!window.create(wdesc, eventQueue))
  { return; };

  // 🎨 Create a renderer
  Renderer renderer(window);

  // 🏁 Engine loop
  bool isRunning = true;
  while (isRunning)
  {
    bool shouldRender = true;

    // ♻️ Update the event queue
    eventQueue.update();

    // 🎈 Iterate through that queue:
    while (!eventQueue.empty())
    {
      //Update Events
      const xwin::Event& event = eventQueue.front();

      // 💗 On Resize:
      if (event.type == xwin::EventType::Resize)
      {
        const xwin::ResizeData data = event.data.resize;
        renderer.resize(data.width, data.height);
        shouldRender = false;
      }

      // ❌ On Close:
      if (event.type == xwin::EventType::Close)
      {
        window.close();
        shouldRender = false;
        isRunning = false;
      }

      eventQueue.pop();
    }

    // ✨ Update Visuals
    if (shouldRender)
    {
      renderer.render();
    }
  }
}

As an alternative to CrossWindow, you could use another library like GLFW, SFML, SDL, QT, or just interface directly with the Win32 or UWP APIs.

Initialize API

Factory

Factories are the entry point to the DirectX 12 API, and will allow you to find adapters that you can use to execute DirectX 12 commands.

You can also create a Debug Controller alongside the factory which can enable API usage validation, though this should only be used in debug builds.

• Reference: IDXGIFactory7

// 👋 Declare DirectX 12 Handles
IDXGIFactory4* factory;
ID3D12Debug1* debugController;

// 🏭 Create Factory
UINT dxgiFactoryFlags = 0;

#if defined(_DEBUG)
// 🐛 Create a Debug Controller to track errors
ID3D12Debug* dc;
ThrowIfFailed(D3D12GetDebugInterface(IID_PPV_ARGS(&dc)));
ThrowIfFailed(dc->QueryInterface(IID_PPV_ARGS(&debugController)));
debugController->EnableDebugLayer();
debugController->SetEnableGPUBasedValidation(true);

dxgiFactoryFlags |= DXGI_CREATE_FACTORY_DEBUG;

dc->Release();
dc = nullptr;
#endif

HRESULT result = CreateDXGIFactory2(dxgiFactoryFlags, IID_PPV_ARGS(&factory));

Adapter

An Adapter provides information on the physical properties of a given DirectX device. You can query your current GPU's name, manufacturer, how much memory it has, and much more.

There's 2 categories for adapters, software and hardware adapters. Microsoft Windows always includes a software based DirectX implementation that can be used in the event of there being no dedicated hardware such as a discrete or integrated GPU.

• Reference: IDXGIAdapter4

// 👋 Declare Handles
IDXGIAdapter1* adapter;

// 🔌 Create Adapter
for (UINT adapterIndex = 0;
     DXGI_ERROR_NOT_FOUND != factory->EnumAdapters1(adapterIndex, &adapter);
     ++adapterIndex)
{
    DXGI_ADAPTER_DESC1 desc;
    adapter->GetDesc1(&desc);

    // ❌ Don't select the Basic Render Driver adapter.
    if (desc.Flags & DXGI_ADAPTER_FLAG_SOFTWARE)
    {
        continue;
    }

    // ✔️ Check if the adapter supports Direct3D 12, and use that for the rest
    // of the application
    if (SUCCEEDED(D3D12CreateDevice(adapter, D3D_FEATURE_LEVEL_12_0,
                                    _uuidof(ID3D12Device), nullptr)))
    {
        break;
    }

    // ❌ Else we won't use this iteration's adapter, so release it
    adapter->Release();
}

Device

A Device is your primary entry point to the DirectX 12 API, giving you access to the inner parts of the API. This is key to accessing important data structures and functions such as pipelines, shader blobs, render state, resource barriers, etc.

• Reference: IDXGIDevice4

// 👋 Declare Handles
ID3D12Device* device;

// 💻 Create Device
ThrowIfFailed(D3D12CreateDevice(adapter, D3D_FEATURE_LEVEL_12_0,
                                IID_PPV_ARGS(&device)));

A Debug Device allows you to rely on DirectX 12's debug mode. It can be difficult to keep track of data structures created with DirectX. With this you'll be able to prevent data leaks or verify if you're creating or using the API correctly.

• Reference: ID3D12DebugDevice

// 👋 Declare Handles
ID3D12DebugDevice* debugDevice;

#if defined(_DEBUG)
// 💻 Get debug device
ThrowIfFailed(device->QueryInterface(&debugDevice));
#endif

Command Queue

A Command Queue allows you to submit groups of draw calls, known as command lists, together to execute in order, thus allowing a GPU to stay busy and optimize its execution speed.

• Reference: ID3D12CommandQueue

// 👋 Declare Handles
ID3D12CommandQueue* commandQueue;

// 📦 Create Command Queue
D3D12_COMMAND_QUEUE_DESC queueDesc = {};
queueDesc.Flags = D3D12_COMMAND_QUEUE_FLAG_NONE;
queueDesc.Type = D3D12_COMMAND_LIST_TYPE_DIRECT;

ThrowIfFailed(device->CreateCommandQueue(&queueDesc,
                                         IID_PPV_ARGS(&commandQueue)));

Command Allocator

A Command Allocator allows you to create command lists where you can define the functions you want the GPU to execute.

• Reference: ID3D12CommandAllocator

// 👋 Declare Handles
ID3D12CommandAllocator* commandAllocator;

// 🎅 Create Command Allocator
ThrowIfFailed(device->CreateCommandAllocator(D3D12_COMMAND_LIST_TYPE_DIRECT,
                                             IID_PPV_ARGS(&commandAllocator)));

Synchronization

DirectX 12 features a number of synchronization primitives that can help the driver know how resources will be used in the future, know when tasks have been completed by the GPU, etc.

A Fence lets your program know when certain tasks have been executed by the GPU, be it uploads to GPU exclusive memory, or when you've finished presenting to the screen.

• Reference: ID3D12Fence1

// 👋 Declare handles
UINT frameIndex;
HANDLE fenceEvent;
ID3D12Fence* fence;
UINT64 fenceValue;

// 🚧 Create fence
ThrowIfFailed(device->CreateFence(0, D3D12_FENCE_FLAG_NONE,
                                  IID_PPV_ARGS(&fence)));

A Barrier lets the driver know how a resource should be used in upcoming commands. This can be useful if say, you're writing to a texture, and you want to copy that texture to another texture (such as the swapchain's render attachment).

// 👋 Declare handles
ID3D12GraphicsCommandList* commandList;

// 🔮 Create Barrier
D3D12_RESOURCE_BARRIER barrier = {};
barrier.Type = D3D12_RESOURCE_BARRIER_TYPE_TRANSITION;
barrier.Flags = D3D12_RESOURCE_BARRIER_FLAG_NONE;
barrier.Transition.pResource = texResource;
barrier.Transition.StateBefore = D3D12_RESOURCE_STATE_COPY_SOURCE;
barrier.Transition.StateAfter = D3D12_RESOURCE_STATE_UNORDERED_ACCESS;
barrier.Transition.Subresource = D3D12_RESOURCE_BARRIER_ALL_SUBRESOURCES;

commandList->ResourceBarrier(1, &barrier);

Swapchain

Swapchains handle swapping and allocating back buffers to display what you're rendering to a given window.

• Reference: IDXGISwapChain4

// 💾 Declare Data
unsigned width = 640;
unsigned height = 640;

// 👋 Declare Handles
static const UINT backbufferCount = 2;
UINT currentBuffer;
ID3D12DescriptorHeap* renderTargetViewHeap;
ID3D12Resource* renderTargets[backbufferCount];
UINT rtvDescriptorSize;

// ⛓️ Swapchain
IDXGISwapChain3* swapchain;
D3D12_VIEWPORT viewport;
D3D12_RECT surfaceSize;

surfaceSize.left = 0;
surfaceSize.top = 0;
surfaceSize.right = static_cast<LONG>(width);
surfaceSize.bottom = static_cast<LONG>(height);

viewport.TopLeftX = 0.0f;
viewport.TopLeftY = 0.0f;
viewport.Width = static_cast<float>(width);
viewport.Height = static_cast<float>(height);
viewport.MinDepth = .1f;
viewport.MaxDepth = 1000.f;

if (swapchain != nullptr)
{
    // Create Render Target Attachments from swapchain
    swapchain->ResizeBuffers(backbufferCount, width, height,
                             DXGI_FORMAT_R8G8B8A8_UNORM, 0);
}
else
{
    // ⛓️ Create swapchain
    DXGI_SWAP_CHAIN_DESC1 swapchainDesc = {};
    swapchainDesc.BufferCount = backbufferCount;
    swapchainDesc.Width = width;
    swapchainDesc.Height = height;
    swapchainDesc.Format = DXGI_FORMAT_R8G8B8A8_UNORM;
    swapchainDesc.BufferUsage = DXGI_USAGE_RENDER_TARGET_OUTPUT;
    swapchainDesc.SwapEffect = DXGI_SWAP_EFFECT_FLIP_DISCARD;
    swapchainDesc.SampleDesc.Count = 1;

    IDXGISwapChain1* newSwapchain =
        xgfx::createSwapchain(window, factory, commandQueue, &swapchainDesc);
    HRESULT swapchainSupport = swapchain->QueryInterface(
        __uuidof(IDXGISwapChain3), (void**)&newSwapchain);
    if (SUCCEEDED(swapchainSupport))
    {
        swapchain = (IDXGISwapChain3*)newSwapchain;
    }
}

frameIndex = swapchain->GetCurrentBackBufferIndex();

// Describe and create a render target view (RTV) descriptor heap.
D3D12_DESCRIPTOR_HEAP_DESC rtvHeapDesc = {};
rtvHeapDesc.NumDescriptors = backbufferCount;
rtvHeapDesc.Type = D3D12_DESCRIPTOR_HEAP_TYPE_RTV;
rtvHeapDesc.Flags = D3D12_DESCRIPTOR_HEAP_FLAG_NONE;
ThrowIfFailed(device->CreateDescriptorHeap(
    &rtvHeapDesc, IID_PPV_ARGS(&renderTargetViewHeap)));

rtvDescriptorSize =
    device->GetDescriptorHandleIncrementSize(D3D12_DESCRIPTOR_HEAP_TYPE_RTV);

// 🎞️ Create frame resources

D3D12_CPU_DESCRIPTOR_HANDLE
    rtvHandle(renderTargetViewHeap->GetCPUDescriptorHandleForHeapStart());

// Create a RTV for each frame.
for (UINT n = 0; n < backbufferCount; n++)
{
    ThrowIfFailed(swapchain->GetBuffer(n, IID_PPV_ARGS(&renderTargets[n])));
    device->CreateRenderTargetView(renderTargets[n], nullptr, rtvHandle);
    rtvHandle.ptr += (1 * rtvDescriptorSize);
}

Initialize Resources

Descriptor Heaps

Descriptor heaps are objects that handle memory allocation required for storing the descriptions of objects that shaders reference.

• Reference: ID3D12DescriptorHeap

ID3D12DescriptorHeap* renderTargetViewHeap;

D3D12_DESCRIPTOR_HEAP_DESC rtvHeapDesc = {};
rtvHeapDesc.NumDescriptors = backbufferCount;
rtvHeapDesc.Type = D3D12_DESCRIPTOR_HEAP_TYPE_RTV;
rtvHeapDesc.Flags = D3D12_DESCRIPTOR_HEAP_FLAG_NONE;
ThrowIfFailed(device->CreateDescriptorHeap(
    &rtvHeapDesc, IID_PPV_ARGS(&renderTargetViewHeap)));

Root Signature

Root Signatures are objects that define what type of resources are accessible to your shaders, be it constant buffers, structured buffers, samplers, textures, structured buffers, etc.

• Reference: ID3D12RootSignature

// 👋 Declare Handles
ID3D12RootSignature* rootSignature;

// 🔎 Determine if we can get Root Signature Version 1.1:
D3D12_FEATURE_DATA_ROOT_SIGNATURE featureData = {};
featureData.HighestVersion = D3D_ROOT_SIGNATURE_VERSION_1_1;

if (FAILED(device->CheckFeatureSupport(D3D12_FEATURE_ROOT_SIGNATURE,
                                       &featureData, sizeof(featureData))))
{
    featureData.HighestVersion = D3D_ROOT_SIGNATURE_VERSION_1_0;
}

// 📂 Individual GPU Resources
D3D12_DESCRIPTOR_RANGE1 ranges[1];
ranges[0].BaseShaderRegister = 0;
ranges[0].RangeType = D3D12_DESCRIPTOR_RANGE_TYPE_CBV;
ranges[0].NumDescriptors = 1;
ranges[0].RegisterSpace = 0;
ranges[0].OffsetInDescriptorsFromTableStart = 0;
ranges[0].Flags = D3D12_DESCRIPTOR_RANGE_FLAG_NONE;

//🗄️ Groups of GPU Resources
D3D12_ROOT_PARAMETER1 rootParameters[1];
rootParameters[0].ParameterType = D3D12_ROOT_PARAMETER_TYPE_DESCRIPTOR_TABLE;
rootParameters[0].ShaderVisibility = D3D12_SHADER_VISIBILITY_VERTEX;

rootParameters[0].DescriptorTable.NumDescriptorRanges = 1;
rootParameters[0].DescriptorTable.pDescriptorRanges = ranges;

// 🏢  Overall Layout
D3D12_VERSIONED_ROOT_SIGNATURE_DESC rootSignatureDesc;
rootSignatureDesc.Version = D3D_ROOT_SIGNATURE_VERSION_1_1;
rootSignatureDesc.Desc_1_1.Flags =
    D3D12_ROOT_SIGNATURE_FLAG_ALLOW_INPUT_ASSEMBLER_INPUT_LAYOUT;
rootSignatureDesc.Desc_1_1.NumParameters = 1;
rootSignatureDesc.Desc_1_1.pParameters = rootParameters;
rootSignatureDesc.Desc_1_1.NumStaticSamplers = 0;
rootSignatureDesc.Desc_1_1.pStaticSamplers = nullptr;

ID3DBlob* signature;
ID3DBlob* error;
try
{
    // 🌱 Create the root signature
    ThrowIfFailed(D3D12SerializeVersionedRootSignature(&rootSignatureDesc,
                                                       &signature, &error));
    ThrowIfFailed(device->CreateRootSignature(0, signature->GetBufferPointer(),
                                              signature->GetBufferSize(),
                                              IID_PPV_ARGS(&rootSignature)));
    rootSignature->SetName(L"Hello Triangle Root Signature");
}
catch (std::exception e)
{
    const char* errStr = (const char*)error->GetBufferPointer();
    std::cout << errStr;
    error->Release();
    error = nullptr;
}

if (signature)
{
    signature->Release();
    signature = nullptr;
}

While these work well enough, using bindless resources is significantly more easy, Matt Pettineo (@MyNameIsMJP) wrote a blog post and chapter in Ray Tracing Gems 2 about this.

Heaps

Heaps are objects that encompass GPU memory. They can be used to upload resources like vertex buffers or textures to GPU exclusive memory.

• Reference: ID3D12Resource

// 🔼 Upload:

// 👋 Declare Handles
ID3D12Resource* uploadBuffer;
std::vector<unsigned char> sourceData;

D3D12_HEAP_PROPERTIES uploadHeapProps = {D3D12_HEAP_TYPE_UPLOAD,
                                         D3D12_CPU_PAGE_PROPERTY_UNKNOWN,
                                         D3D12_MEMORY_POOL_UNKNOWN, 1u, 1u};
D3D12_RESOURCE_DESC uploadBufferDesc = {D3D12_RESOURCE_DIMENSION_BUFFER,
                                        65536ull,
                                        65536ull,
                                        1u,
                                        1,
                                        1,
                                        DXGI_FORMAT_UNKNOWN,
                                        {1u, 0u},
                                        D3D12_TEXTURE_LAYOUT_ROW_MAJOR,
                                        D3D12_RESOURCE_FLAG_NONE};
result = device->CreateCommittedResource(
    &uploadHeapProps, D3D12_HEAP_FLAG_NONE, &uploadBufferDesc,
    D3D12_RESOURCE_STATE_GENERIC_READ, nullptr, __uuidof(ID3D12Resource),
    ((void**)&uploadBuffer));

uint8_t* data = nullptr;
D3D12_RANGE range{0, SIZE_T(chunkSize)};
auto hr = spStaging -> Map(0, &range, reinterpret_cast<void**>(&data));
if (FAILED(hr))
{
    std::cout << "Could not map resource";
}

// Copy data
if (resourceDesc.Dimension == D3D12_RESOURCE_DIMENSION_BUFFER)
{
    memcpy(data, sourceData.data(), sourceData.size());
}

// 📖 Readback:

// 👋 Declare Handles
ID3D12Resource* readbackBuffer;

D3D12_HEAP_PROPERTIES heapPropsRead = {D3D12_HEAP_TYPE_READBACK,
                                       D3D12_CPU_PAGE_PROPERTY_UNKNOWN,
                                       D3D12_MEMORY_POOL_UNKNOWN, 1u, 1u};
D3D12_RESOURCE_DESC resourceDescDimBuffer = {
    D3D12_RESOURCE_DIMENSION_BUFFER,
    D3D12_DEFAULT_RESOURCE_PLACEMENT_ALIGNMENT,
    2725888ull,
    1u,
    1,
    1,
    DXGI_FORMAT_UNKNOWN,
    {1u, 0u},
    D3D12_TEXTURE_LAYOUT_ROW_MAJOR,
    D3D12_RESOURCE_FLAG_DENY_SHADER_RESOURCE};

result = device->CreateCommittedResource(
    &heapPropsRead, D3D12_HEAP_FLAG_NONE, &resourceDescDimBuffer,
    D3D12_RESOURCE_STATE_COPY_DEST, nullptr, __uuidof(ID3D12Resource),
    ((void**)&readbackBuffer));

You can be even more granular with your memory management by creating your own heaps. It can be a handful to manage heaps though, so you could use a memory allocation library instead.

Vertex Buffer

A Vertex Buffer stores the per vertex information available as attributes in your Vertex Shader. All buffers are ID3D12Resource objects in DirectX 12, be it Vertex Buffers, Index Buffers, Constant Buffers, etc.

• Reference: D3D12_VERTEX_BUFFER_VIEW

// 💾 Declare Data
struct Vertex
{
    float position[3];
    float color[3];
};

Vertex vertexBufferData[3] = {{{1.0f, -1.0f, 0.0f}, {1.0f, 0.0f, 0.0f}},
                              {{-1.0f, -1.0f, 0.0f}, {0.0f, 1.0f, 0.0f}},
                              {{0.0f, 1.0f, 0.0f}, {0.0f, 0.0f, 1.0f}}};

// 👋 Declare Handles
ID3D12Resource* vertexBuffer;
D3D12_VERTEX_BUFFER_VIEW vertexBufferView;

const UINT vertexBufferSize = sizeof(vertexBufferData);

D3D12_HEAP_PROPERTIES heapProps;
heapProps.Type = D3D12_HEAP_TYPE_UPLOAD;
heapProps.CPUPageProperty = D3D12_CPU_PAGE_PROPERTY_UNKNOWN;
heapProps.MemoryPoolPreference = D3D12_MEMORY_POOL_UNKNOWN;
heapProps.CreationNodeMask = 1;
heapProps.VisibleNodeMask = 1;

D3D12_RESOURCE_DESC vertexBufferResourceDesc;
vertexBufferResourceDesc.Dimension = D3D12_RESOURCE_DIMENSION_BUFFER;
vertexBufferResourceDesc.Alignment = 0;
vertexBufferResourceDesc.Width = vertexBufferSize;
vertexBufferResourceDesc.Height = 1;
vertexBufferResourceDesc.DepthOrArraySize = 1;
vertexBufferResourceDesc.MipLevels = 1;
vertexBufferResourceDesc.Format = DXGI_FORMAT_UNKNOWN;
vertexBufferResourceDesc.SampleDesc.Count = 1;
vertexBufferResourceDesc.SampleDesc.Quality = 0;
vertexBufferResourceDesc.Layout = D3D12_TEXTURE_LAYOUT_ROW_MAJOR;
vertexBufferResourceDesc.Flags = D3D12_RESOURCE_FLAG_NONE;

ThrowIfFailed(device->CreateCommittedResource(
    &heapProps, D3D12_HEAP_FLAG_NONE, &vertexBufferResourceDesc,
    D3D12_RESOURCE_STATE_GENERIC_READ, nullptr, IID_PPV_ARGS(&vertexBuffer)));

// 📄 Copy the triangle data to the vertex buffer.
UINT8* pVertexDataBegin;

// 🙈 We do not intend to read from this resource on the CPU.
D3D12_RANGE readRange;
readRange.Begin = 0;
readRange.End = 0;

ThrowIfFailed(vertexBuffer->Map(0, &readRange,
                                reinterpret_cast<void**>(&pVertexDataBegin)));
memcpy(pVertexDataBegin, vertexBufferData, sizeof(vertexBufferData));
vertexBuffer->Unmap(0, nullptr);

// 👀 Initialize the vertex buffer view.
vertexBufferView.BufferLocation = vertexBuffer->GetGPUVirtualAddress();
vertexBufferView.StrideInBytes = sizeof(Vertex);
vertexBufferView.SizeInBytes = vertexBufferSize;

Index Buffer

An Index Buffer contains the individual indices of each triangle/line/point that you intend to draw.

• Reference: D3D12_INDEX_BUFFER_VIEW

// 💾 Declare Data
uint32_t indexBufferData[3] = {0, 1, 2};

// 👋 Declare Handles
ID3D12Resource* indexBuffer;
D3D12_INDEX_BUFFER_VIEW indexBufferView;

const UINT indexBufferSize = sizeof(indexBufferData);

D3D12_HEAP_PROPERTIES heapProps;
heapProps.Type = D3D12_HEAP_TYPE_UPLOAD;
heapProps.CPUPageProperty = D3D12_CPU_PAGE_PROPERTY_UNKNOWN;
heapProps.MemoryPoolPreference = D3D12_MEMORY_POOL_UNKNOWN;
heapProps.CreationNodeMask = 1;
heapProps.VisibleNodeMask = 1;

D3D12_RESOURCE_DESC vertexBufferResourceDesc;
vertexBufferResourceDesc.Dimension = D3D12_RESOURCE_DIMENSION_BUFFER;
vertexBufferResourceDesc.Alignment = 0;
vertexBufferResourceDesc.Width = indexBufferSize;
vertexBufferResourceDesc.Height = 1;
vertexBufferResourceDesc.DepthOrArraySize = 1;
vertexBufferResourceDesc.MipLevels = 1;
vertexBufferResourceDesc.Format = DXGI_FORMAT_UNKNOWN;
vertexBufferResourceDesc.SampleDesc.Count = 1;
vertexBufferResourceDesc.SampleDesc.Quality = 0;
vertexBufferResourceDesc.Layout = D3D12_TEXTURE_LAYOUT_ROW_MAJOR;
vertexBufferResourceDesc.Flags = D3D12_RESOURCE_FLAG_NONE;

ThrowIfFailed(device->CreateCommittedResource(
    &heapProps, D3D12_HEAP_FLAG_NONE, &vertexBufferResourceDesc,
    D3D12_RESOURCE_STATE_GENERIC_READ, nullptr, IID_PPV_ARGS(&indexBuffer)));

// 📄 Copy data to DirectX 12 driver memory:
UINT8* pVertexDataBegin;

// 🙈 We do not intend to read from this resource on the CPU.
D3D12_RANGE readRange;
readRange.Begin = 0;
readRange.End = 0;

ThrowIfFailed(indexBuffer->Map(0, &readRange,
                               reinterpret_cast<void**>(&pVertexDataBegin)));
memcpy(pVertexDataBegin, indexBufferData, sizeof(indexBufferData));
indexBuffer->Unmap(0, nullptr);

// 👀 Initialize the index buffer view.
indexBufferView.BufferLocation = indexBuffer->GetGPUVirtualAddress();
indexBufferView.Format = DXGI_FORMAT_R32_UINT;
indexBufferView.SizeInBytes = indexBufferSize;

Constant Buffer

A Constant Buffer describes data that we'll be sending to shader stages when drawing. Normally you would put Model View Projection Matrices or any specific variable data like colors, sliders, etc. here.

// 💾 Declare Data
struct
{
    glm::mat4 projectionMatrix;
    glm::mat4 modelMatrix;
    glm::mat4 viewMatrix;
} cbVS;

// 👋 Declare Handles
ID3D12Resource* constantBuffer;
ID3D12DescriptorHeap* constantBufferHeap;
UINT8* mappedConstantBuffer;

// 🧊 Create the Constant Buffer

D3D12_HEAP_PROPERTIES heapProps;
heapProps.Type = D3D12_HEAP_TYPE_UPLOAD;
heapProps.CPUPageProperty = D3D12_CPU_PAGE_PROPERTY_UNKNOWN;
heapProps.MemoryPoolPreference = D3D12_MEMORY_POOL_UNKNOWN;
heapProps.CreationNodeMask = 1;
heapProps.VisibleNodeMask = 1;

D3D12_DESCRIPTOR_HEAP_DESC heapDesc = {};
heapDesc.NumDescriptors = 1;
heapDesc.Flags = D3D12_DESCRIPTOR_HEAP_FLAG_SHADER_VISIBLE;
heapDesc.Type = D3D12_DESCRIPTOR_HEAP_TYPE_CBV_SRV_UAV;
ThrowIfFailed(device->CreateDescriptorHeap(&heapDesc,
                                           IID_PPV_ARGS(&constantBufferHeap)));

D3D12_RESOURCE_DESC cbResourceDesc;
cbResourceDesc.Dimension = D3D12_RESOURCE_DIMENSION_BUFFER;
cbResourceDesc.Alignment = 0;
cbResourceDesc.Width = (sizeof(cbVS) + 255) & ~255;
cbResourceDesc.Height = 1;
cbResourceDesc.DepthOrArraySize = 1;
cbResourceDesc.MipLevels = 1;
cbResourceDesc.Format = DXGI_FORMAT_UNKNOWN;
cbResourceDesc.SampleDesc.Count = 1;
cbResourceDesc.SampleDesc.Quality = 0;
cbResourceDesc.Layout = D3D12_TEXTURE_LAYOUT_ROW_MAJOR;
cbResourceDesc.Flags = D3D12_RESOURCE_FLAG_NONE;

ThrowIfFailed(device->CreateCommittedResource(
    &heapProps, D3D12_HEAP_FLAG_NONE, &cbResourceDesc,
    D3D12_RESOURCE_STATE_GENERIC_READ, nullptr, IID_PPV_ARGS(&constantBuffer)));
constantBufferHeap->SetName(L"Constant Buffer Upload Resource Heap");

// 👓 Create our Constant Buffer View
D3D12_CONSTANT_BUFFER_VIEW_DESC cbvDesc = {};
cbvDesc.BufferLocation = constantBuffer->GetGPUVirtualAddress();
cbvDesc.SizeInBytes =
    (sizeof(cbVS) + 255) & ~255; // CB size is required to be 256-byte aligned.

D3D12_CPU_DESCRIPTOR_HANDLE
    cbvHandle(constantBufferHeap->GetCPUDescriptorHandleForHeapStart());
cbvHandle.ptr = cbvHandle.ptr + device->GetDescriptorHandleIncrementSize(
                                    D3D12_DESCRIPTOR_HEAP_TYPE_CBV_SRV_UAV) *
                                    0;

device->CreateConstantBufferView(&cbvDesc, cbvHandle);

// 🙈 We do not intend to read from this resource on the CPU.
D3D12_RANGE readRange;
readRange.Begin = 0;
readRange.End = 0;

ThrowIfFailed(constantBuffer->Map(
    0, &readRange, reinterpret_cast<void**>(&mappedConstantBuffer)));
memcpy(mappedConstantBuffer, &cbVS, sizeof(cbVS));
constantBuffer->Unmap(0, &readRange);

Vertex Shader

Vertex Shaders execute per vertex, and are perfect for transforming a given object, performing per vertex animations with blend shapes, GPU skinning, etc.

cbuffer cb : register(b0)
{
    row_major float4x4 projectionMatrix : packoffset(c0);
    row_major float4x4 modelMatrix : packoffset(c4);
    row_major float4x4 viewMatrix : packoffset(c8);
};

struct VertexInput
{
    float3 inPos : POSITION;
    float3 inColor : COLOR;
};

struct VertexOutput
{
    float3 color : COLOR;
    float4 position : SV_Position;
};

VertexOutput main(VertexInput vertexInput)
{
    float3 inColor = vertexInput.inColor;
    float3 inPos = vertexInput.inPos;
    float3 outColor = inColor;
    float4 position = mul(float4(inPos, 1.0f), mul(modelMatrix, mul(viewMatrix, projectionMatrix)));

    VertexOutput output;
    output.position = position;
    output.color = outColor;
    return output;
}

You could compile shaders using the legacy DirectX Shader Compiler bundled with the DirectX 11/12 API, but it's best to use the newer official compiler.

dxc.exe -T lib_6_3 -Fo assets/triangle.vert.dxil assets/triangle.vert.hlsl

Then you could load the shader as a binary file:

// Helper to read a binary file:
inline std::vector<char> readFile(const std::string& filename)
{
    std::ifstream file(filename, std::ios::ate | std::ios::binary);
    bool exists = (bool)file;

    if (!exists || !file.is_open())
    {
        throw std::runtime_error("failed to open file!");
    }

    size_t fileSize = (size_t)file.tellg();
    std::vector<char> buffer(fileSize);

    file.seekg(0);
    file.read(buffer.data(), fileSize);

    file.close();

    return buffer;
};

// 👋 Declare handles
D3D12_SHADER_BYTECODE vsBytecode;
std::string compiledPath;

std::vector<char> vsBytecodeData = readFile(compCompiledPath);
vsBytecode.pShaderBytecode = vsBytecodeData.data();
vsBytecode.BytecodeLength = vsBytecodeData.size();

Pixel Shader

Pixel Shaders execute per each pixel of your output, including the other attachments that correspond to that pixel coordinate.

struct PixelInput
{
    float3 color : COLOR;
};

struct PixelOutput
{
    float4 attachment0 : SV_Target0;
};

PixelOutput main(PixelInput pixelInput)
{
    float3 inColor = pixelInput.color;
    PixelOutput output;
    output.attachment0 = float4(inColor, 1.0f);
    return output;
}

Pipeline State

The Pipeline State describes everything necessary to execute a given raster based draw call.

• Reference: ID3D12GraphicsCommandList5

// 👋 Declare handles
ID3D12PipelineState* pipelineState;

// ⚗️ Define the Graphics Pipeline
D3D12_GRAPHICS_PIPELINE_STATE_DESC psoDesc = {};

// 🔣 Input Assembly
D3D12_INPUT_ELEMENT_DESC inputElementDescs[] = {
    {"POSITION", 0, DXGI_FORMAT_R32G32B32_FLOAT, 0, 0,
     D3D12_INPUT_CLASSIFICATION_PER_VERTEX_DATA, 0},
    {"COLOR", 0, DXGI_FORMAT_R32G32B32_FLOAT, 0, 12,
     D3D12_INPUT_CLASSIFICATION_PER_VERTEX_DATA, 0}};
psoDesc.InputLayout = {inputElementDescs, _countof(inputElementDescs)};

// 🦄 Resources
psoDesc.pRootSignature = rootSignature;

// 🔺 Vertex Shader
D3D12_SHADER_BYTECODE vsBytecode;
vsBytecode.pShaderBytecode = vertexShaderBlob->GetBufferPointer();
vsBytecode.BytecodeLength = vertexShaderBlob->GetBufferSize();
psoDesc.VS = vsBytecode;

// 🖌️ Pixel Shader
D3D12_SHADER_BYTECODE psBytecode;
psBytecode.pShaderBytecode = pixelShaderBlob->GetBufferPointer();
psBytecode.BytecodeLength = pixelShaderBlob->GetBufferSize();
psoDesc.PS = psBytecode;

// 🟨 Rasterization
D3D12_RASTERIZER_DESC rasterDesc;
rasterDesc.FillMode = D3D12_FILL_MODE_SOLID;
rasterDesc.CullMode = D3D12_CULL_MODE_NONE;
rasterDesc.FrontCounterClockwise = FALSE;
rasterDesc.DepthBias = D3D12_DEFAULT_DEPTH_BIAS;
rasterDesc.DepthBiasClamp = D3D12_DEFAULT_DEPTH_BIAS_CLAMP;
rasterDesc.SlopeScaledDepthBias = D3D12_DEFAULT_SLOPE_SCALED_DEPTH_BIAS;
rasterDesc.DepthClipEnable = TRUE;
rasterDesc.MultisampleEnable = FALSE;
rasterDesc.AntialiasedLineEnable = FALSE;
rasterDesc.ForcedSampleCount = 0;
rasterDesc.ConservativeRaster = D3D12_CONSERVATIVE_RASTERIZATION_MODE_OFF;
psoDesc.RasterizerState = rasterDesc;
psoDesc.PrimitiveTopologyType = D3D12_PRIMITIVE_TOPOLOGY_TYPE_TRIANGLE;

// 🌀 Color/Blend
D3D12_BLEND_DESC blendDesc;
blendDesc.AlphaToCoverageEnable = FALSE;
blendDesc.IndependentBlendEnable = FALSE;
const D3D12_RENDER_TARGET_BLEND_DESC defaultRenderTargetBlendDesc = {
    FALSE,
    FALSE,
    D3D12_BLEND_ONE,
    D3D12_BLEND_ZERO,
    D3D12_BLEND_OP_ADD,
    D3D12_BLEND_ONE,
    D3D12_BLEND_ZERO,
    D3D12_BLEND_OP_ADD,
    D3D12_LOGIC_OP_NOOP,
    D3D12_COLOR_WRITE_ENABLE_ALL,
};
for (UINT i = 0; i < D3D12_SIMULTANEOUS_RENDER_TARGET_COUNT; ++i)
    blendDesc.RenderTarget[i] = defaultRenderTargetBlendDesc;
psoDesc.BlendState = blendDesc;

// 🌑 Depth/Stencil State
psoDesc.DepthStencilState.DepthEnable = FALSE;
psoDesc.DepthStencilState.StencilEnable = FALSE;
psoDesc.SampleMask = UINT_MAX;

// 🖼️ Output
psoDesc.NumRenderTargets = 1;
psoDesc.RTVFormats[0] = DXGI_FORMAT_R8G8B8A8_UNORM;
psoDesc.SampleDesc.Count = 1;

// 🌟 Create the raster pipeline state
try
{
    ThrowIfFailed(device->CreateGraphicsPipelineState(
        &psoDesc, IID_PPV_ARGS(&pipelineState)));
}
catch (std::exception e)
{
    std::cout << "Failed to create Graphics Pipeline!";
}

Encoding Commands

In order to execute draw calls, you're going to need a place to write commands. A Command List is an object that can encode a number of commands to be executed by the GPU, be it configuring barriers, setting root signatures, etc.

• Reference: ID3D12GraphicsCommandList5

// 👋 Declare handles
ID3D12CommandAllocator* commandAllocator;
ID3D12PipelineState* initialPipelineState;
ID3D12GraphicsCommandList* commandList;

// 📃 Create the command list.
ThrowIfFailed(device->CreateCommandList(0, D3D12_COMMAND_LIST_TYPE_DIRECT,
                                        commandAllocator, initialPipelineState,
                                        IID_PPV_ARGS(&commandList)));

Later, encode those commands and submit them:

// 🚿 Reset the command list and add new commands.
ThrowIfFailed(commandAllocator->Reset());

// 🖌️ Begin using the Raster Graphics Pipeline
ThrowIfFailed(commandList->Reset(commandAllocator, pipelineState));

// 🔳 Setup Resources
commandList->SetGraphicsRootSignature(rootSignature);
ID3D12DescriptorHeap* pDescriptorHeaps[] = {constantBufferHeap};
commandList->SetDescriptorHeaps(_countof(pDescriptorHeaps), pDescriptorHeaps);
D3D12_GPU_DESCRIPTOR_HANDLE
    cbvHandle(constantBufferHeap->GetGPUDescriptorHandleForHeapStart());
commandList->SetGraphicsRootDescriptorTable(0, cbvHandle);

// 🖼️ Indicate that the back buffer will be used as a render target.
D3D12_RESOURCE_BARRIER renderTargetBarrier;
renderTargetBarrier.Type = D3D12_RESOURCE_BARRIER_TYPE_TRANSITION;
renderTargetBarrier.Flags = D3D12_RESOURCE_BARRIER_FLAG_NONE;
renderTargetBarrier.Transition.pResource = renderTargets[frameIndex];
renderTargetBarrier.Transition.StateBefore = D3D12_RESOURCE_STATE_PRESENT;
renderTargetBarrier.Transition.StateAfter = D3D12_RESOURCE_STATE_RENDER_TARGET;
renderTargetBarrier.Transition.Subresource =
    D3D12_RESOURCE_BARRIER_ALL_SUBRESOURCES;
commandList->ResourceBarrier(1, &renderTargetBarrier);

D3D12_CPU_DESCRIPTOR_HANDLE
    rtvHandle(rtvHeap->GetCPUDescriptorHandleForHeapStart());
rtvHandle.ptr = rtvHandle.ptr + (frameIndex * rtvDescriptorSize);
commandList->OMSetRenderTargets(1, &rtvHandle, FALSE, nullptr);

// 🎥 Record raster commands.
const float clearColor[] = {0.2f, 0.2f, 0.2f, 1.0f};
commandList->RSSetViewports(1, &viewport);
commandList->RSSetScissorRects(1, &surfaceSize);
commandList->ClearRenderTargetView(rtvHandle, clearColor, 0, nullptr);
commandList->IASetPrimitiveTopology(D3D_PRIMITIVE_TOPOLOGY_TRIANGLELIST);
commandList->IASetVertexBuffers(0, 1, &vertexBufferView);
commandList->IASetIndexBuffer(&indexBufferView);

commandList->DrawIndexedInstanced(3, 1, 0, 0, 0);

// 🖼️ Indicate that the back buffer will now be used to present.
D3D12_RESOURCE_BARRIER presentBarrier;
presentBarrier.Type = D3D12_RESOURCE_BARRIER_TYPE_TRANSITION;
presentBarrier.Flags = D3D12_RESOURCE_BARRIER_FLAG_NONE;
presentBarrier.Transition.pResource = renderTargets[frameIndex];
presentBarrier.Transition.StateBefore = D3D12_RESOURCE_STATE_RENDER_TARGET;
presentBarrier.Transition.StateAfter = D3D12_RESOURCE_STATE_PRESENT;
presentBarrier.Transition.Subresource = D3D12_RESOURCE_BARRIER_ALL_SUBRESOURCES;

commandList->ResourceBarrier(1, &presentBarrier);

ThrowIfFailed(commandList->Close());

Render

Rendering in DirectX 12 is a simple matter of changing any constant buffer data you intend to update, submitting your command lists to be executed, presenting the swapchain so your Win32 or UWP window updates, and signaling your application that you've finished rendering.

// 👋 declare handles
std::chrono::time_point<std::chrono::steady_clock> tStart, tEnd;
float elapsedTime = 0.0f;

void render()
{
    // ⌚ Frame limit set to 60 fps
    tEnd = std::chrono::high_resolution_clock::now();
    float time =
        std::chrono::duration<float, std::milli>(tEnd - tStart).count();
    if (time < (1000.0f / 60.0f))
    {
        return;
    }
    tStart = std::chrono::high_resolution_clock::now();

    // 🎛️ Update Uniforms
    elapsedTime += 0.001f * time;
    elapsedTime = fmodf(elapsedTime, 6.283185307179586f);
    cbVS.modelMatrix = Matrix4::rotationY(elapsedTime);

    D3D12_RANGE readRange;
    readRange.Begin = 0;
    readRange.End = 0;

    ThrowIfFailed(constantBuffer->Map(
        0, &readRange, reinterpret_cast<void**>(&mappedConstantBuffer)));
    memcpy(mappedConstantBuffer, &cbVS, sizeof(cbVS));
    constantBuffer->Unmap(0, &readRange);

    setupCommands();

    ID3D12CommandList* ppCommandLists[] = {commandList};
    commandQueue->ExecuteCommandLists(_countof(ppCommandLists), ppCommandLists);

    // 🎥 Present, then wait till finished to continue execution
    swapchain->Present(1, 0);

    const UINT64 fence = fenceValue;
    ThrowIfFailed(commandQueue->Signal(fence, fence));
    fenceValue++;

    if (fence->GetCompletedValue() < fence)
    {
        ThrowIfFailed(fence->SetEventOnCompletion(fence, fenceEvent));
        WaitForSingleObject(fenceEvent, INFINITE);
    }

    frameIndex = swapchain->GetCurrentBackBufferIndex();
}

Destroy Handles

If you're using ComPtr<T> data structures, then just like with shared pointers, you don't need to worry about destroying any handles you create. If you aren't, you can call the Release() function built into every DirectX data structure.

Conclusion

DirectX 12 is a feature rich and robust computer graphics API, perfect for commercial projects. The API is similar to modern graphics APIs in its design, while also being the primary API maintained by both hardware driver engineers and engineers of commercial projects. This post reviews raster based drawing, however there are more aspects of DirectX not discussed here worth reviewing, such as:

• DirectML - Hardware accelerated machine learning model execution.

• DirectX Raytracing - Hardware accelerated ray tracing and scene traversal.

• Compute Shaders - GPGPU based execution of arbitrary tasks, such as image processing, physics, etc.

• Mesh Shaders - an alternative way of building primitives from traditional raster based rendering techniques.

• Variable Rate Shading - More fine tuned control over the shading rate of a given set of commands.

I also wrote more about DirectX:

• The Raw DirectX 12 book goes over the compute pipeline, ray tracing pipeline, and much more.

• The Raw DirectX 11 blog post reviews the previous version of DirectX 11 and could serve as a great start to learning DirectX since many of the concepts transfer over to DirectX 12.

More Resources

Be sure to check out some of the following blog posts, tools, and projects:

Articles

• Microsoft's documentation on DirectX 12 features a page with all it's data structures for initialization here.

• Microsoft maintains a specification document for D3D12 and 11 here.

• Jendrik Illner (@jendrikillner) wrote DirectX 12 learning plan here.

• Braynzarsoft, a community of DirectX tutorials.

• Riko Ophorst's DirectX 12 Raytracing Thesis.

• Riccardo Loggini's D3D12 blog posts.

• Intel maintains a library of D3D12 blog posts here.

• NVIDIA maintains a number of blog posts on D3D12 and similar topics here.

• Diligent Graphics (@diligentengine) wrote articles on DirectX 12 here.

• Jeremiah van Oosten's DirectX 12.2 Tutorial Series.

• Alex Tardif (@longbool)'s A Gentle Introduction to D3D12.

Samples

• Microsoft's official repo of DirectX 12 Samples

• Intel's official GitHub Organization Game Tech Dev.

• NVIDIA's official GitHub Organization GameWorks.

• Matthäus G. Chajdas (@NIV_Anteru) released HelloD3D12, a part of AMD's GPUOpen Libraries & SDKs.

Tools

• The D3D12 Memory Allocator by Adam Sawicki (@Reg__), the same author behind the Vulkan Memory Allocator.

• Tim Jones (@tim_jones) released a VS Code Plugin called HLSL Tools that lets you lint your shaders more easily.

You'll find all the source code described in this post in the Github repo here.

DirectX 11 is the previous version of Microsoft's proprietary graphics API, designed to be simple to work with, while at the same time capable of high performance rendering. While DirectX 11 is an older API, it's still possible to use DirectX 11 with 12 through the library DirectX 11 on 12 or directly interfacing with the API.

That being said, given that there's a newer version of DirectX available to engineers, it may be best to start any greenfield projects with DirectX 12, though 11 is much simpler and can serve as a good entry point to graphics programming.

Visit my blog post on DirectX 12 to learn more about the latest version of the DirectX API.

Overview

DirectX's documentation recommends the use of ComPtr<T> as an alternative to std::shared_ptr<T>, with the benefit of better debugging and easier initialization of DirectX 11 data structures.

Regardless of whether or not you choose to use ComPtr<T>, the steps to rendering raster graphics with DirectX 11 are pretty similar to other modern graphics APIs:

1. Initialize the API - Create your IDXGIFactory, IDXGIAdapter, ID3D11Device, and ID3D11DeviceContext.

2. Setup Frame Backings - Create your IDXGIOutput to determine what your back buffer color space/size should be, IDXGISwapChain for your back buffers, get your back buffer ID3D11Texture2D and create ID3D11RenderTargetViews for those back buffers.

3. Initialize Resources - Create your Triangle Data such as your ID3D11Buffer Vertex Buffers, ID3D11Buffer Index Buffer, ID3D11Buffer Constant Buffer. Compile or load your ID3D11VertexShader and ID3D11PixelShader, your constant buffer ID3D11Buffers and and define your Raster Pipeline State with ID3D11DepthStencilState, ID3D11RasterizerState, etc.

4. Render - Update your GPU constant buffer data (Uniforms), write commands to your ID3D11DeviceContext, and Present your swapchain.

5. Destroy - Destroy any data structures you're done using with Release() or rely on ComPtr<T> deallocating for you.

The following will explain snippets of a DirectX 11 application, with certain parts omitted, and member variables (mMemberVariable) declared inline without the m prefix so their type is easier to see and the examples here can work on their own.

Window Creation

We're using CrossWindow to handle cross platform window creation, so creating a Win32 window and updating it is very easy:

#include "CrossWindow/CrossWindow.h"
#include "Renderer.h"

#include <iostream>

void xmain(int argc, const char** argv)
{
  // 🖼️ Create Window
  xwin::WindowDesc wdesc;
  wdesc.title = "DirectX 11 Seed";
  wdesc.name = "MainWindow";
  wdesc.visible = true;
  wdesc.width = 640;
  wdesc.height = 640;
  wdesc.fullscreen = false;

  xwin::Window window;
  xwin::EventQueue eventQueue;

  if (!window.create(wdesc, eventQueue))
  { return; };

  // 🎨 Create a renderer
  Renderer renderer(window);

  // 🏁 Engine loop
  bool isRunning = true;
  while (isRunning)
  {
    bool shouldRender = true;

    // ♻️ Update the event queue
    eventQueue.update();

    // 🎈 Iterate through that queue:
    while (!eventQueue.empty())
    {
      //Update Events
      const xwin::Event& event = eventQueue.front();

      // 💗 On Resize:
      if (event.type == xwin::EventType::Resize)
      {
        const xwin::ResizeData data = event.data.resize;
        renderer.resize(data.width, data.height);
        shouldRender = false;
      }

      // ❌ On Close:
      if (event.type == xwin::EventType::Close)
      {
        window.close();
        shouldRender = false;
        isRunning = false;
      }

      eventQueue.pop();
    }

    // ✨ Update Visuals
    if (shouldRender)
    {
      renderer.render();
    }
  }
}

As an alternative to CrossWindow, you could use another library like GLFW, SFML, SDL, QT, or just interface directly with the Win32 or UWP APIs.

Initialize API

Factory

Factories are the entry point to the DirectX 11 API, and will allow you to find adapters that you can use to execute DirectX 11 commands.

// 👋 declare handles
IDXGIFactory* factory;

ThrowIfFailed(CreateDXGIFactory(IID_PPV_ARGS(&factory)));

Adapter

An Adapter provides information on the physical properties of a given DirectX device. You can query your current GPU, how much memory it has, etc.

// 👋 declare handles
IDXGIAdapter* adapter;

ThrowIfFailed(factory->EnumAdapters(0, &adapter));

Adapter Output

An Adapter Output allows you to query display information for their color space and refresh rate.

// 👋 declare handles
IDXGIOutput* adapterOutput;
unsigned numerator = 0;
unsigned denominator = 1;

ThrowIfFailed(adapter->EnumOutputs(0, &adapterOutput));

// Get the number of modes that fit the DXGI_FORMAT_R8G8B8A8_UNORM display
// format for the adapter output (monitor).
unsigned int numModes;
DXGI_MODE_DESC* displayModeList;
ThrowIfFailed(adapterOutput->GetDisplayModeList(DXGI_FORMAT_R8G8B8A8_UNORM,
                                                DXGI_ENUM_MODES_INTERLACED,
                                                &numModes, NULL));

// Create a list to hold all the possible display modes for this monitor/video
// card combination.
displayModeList = new DXGI_MODE_DESC[numModes];
ThrowIfFailed(adapterOutput->GetDisplayModeList(DXGI_FORMAT_R8G8B8A8_UNORM,
                                                DXGI_ENUM_MODES_INTERLACED,
                                                &numModes, displayModeList));

// Now go through all the display modes and find the one that matches the screen
// width and height. When a match is found store the numerator and denominator
// of the refresh rate for that monitor.
for (size_t i = 0; i < numModes; i++)
{
    if (displayModeList[i].Width = width && displayModeList[i].Height == height)
    {
        numerator = displayModeList[i].RefreshRate.Numerator;
        denominator = displayModeList[i].RefreshRate.Denominator;
        break;
    }
}

// Release the display mode list.
delete[] displayModeList;
displayModeList = nullptr;

Device

A Device is your primary entry point to the DirectX 11 API, giving you the ability to create data structures such as buffers, textures, views, pipeline states, etc.

The Device Context is where you can execute commands, be it setting the raster pipeline, drawing triangles, or setting frame buffer attachments.

// 👋 declare handles
ID3D11Device* device;
ID3D11DeviceContext* deviceContext;

D3D_FEATURE_LEVEL featureLevelInputs[7] = {
    D3D_FEATURE_LEVEL_11_1, D3D_FEATURE_LEVEL_11_0, D3D_FEATURE_LEVEL_10_1,
    D3D_FEATURE_LEVEL_10_0, D3D_FEATURE_LEVEL_9_3,  D3D_FEATURE_LEVEL_9_2,
    D3D_FEATURE_LEVEL_9_1};

D3D_FEATURE_LEVEL featureLevelOutputs = D3D_FEATURE_LEVEL_11_1;

ThrowIfFailed(D3D11CreateDevice(adapter, D3D_DRIVER_TYPE_UNKNOWN, 0,
                                D3D11_CREATE_DEVICE_SINGLETHREADED |
                                    D3D11_CREATE_DEVICE_DEBUG,
                                featureLevelInputs, 7u, D3D11_SDK_VERSION,
                                &device, &featureLevelOutputs, &deviceContext));

You can optionally enable the Debug Controller to validate your commands on runtime.

// 👋 declare handles
ID3D11Debug* debugController;

#if defined(_DEBUG)
ThrowIfFailed(device->QueryInterface(IID_PPV_ARGS(&debugController)));
#endif

Frame Backing

Swapchain

Swapchains handle swapping and allocating back buffers to display what you're rendering to a given window. CrossWindow-Graphics makes creating a swapchain for a window very easy:

// 👋 declare handles
IDXGISwapChain* swapchain;
bool vsync = false;
unsigned width = 640;
unsigned height = 640;

DXGI_SWAP_CHAIN_DESC swapchainDesc;
ZeroMemory(&swapchainDesc, sizeof(DXGI_SWAP_CHAIN_DESC));
swapchainDesc.BufferCount = 1;
swapchainDesc.BufferDesc.Width = width;
swapchainDesc.BufferDesc.Height = height;
swapchainDesc.BufferDesc.Format = DXGI_FORMAT_R8G8B8A8_UNORM;

// 🏃‍♀️ Set the refresh rate of the back buffer.
if (vsync)
{
    swapchainDesc.BufferDesc.RefreshRate.Numerator = numerator;
    swapchainDesc.BufferDesc.RefreshRate.Denominator = denominator;
}
else
{
    swapchainDesc.BufferDesc.RefreshRate.Numerator = 0;
    swapchainDesc.BufferDesc.RefreshRate.Denominator = 1;
}
// Set the usage of the back buffer.
swapchainDesc.BufferUsage = DXGI_USAGE_RENDER_TARGET_OUTPUT;
// Turn multi-sampling off.
swapchainDesc.SampleDesc.Count = 1;
swapchainDesc.SampleDesc.Quality = 0;
// Set the scan line ordering and scaling to unspecified.
swapchainDesc.BufferDesc.ScanlineOrdering =
    DXGI_MODE_SCANLINE_ORDER_UNSPECIFIED;
swapchainDesc.BufferDesc.Scaling = DXGI_MODE_SCALING_UNSPECIFIED;
// Discard the back buffer contents after presenting.
swapchainDesc.SwapEffect = DXGI_SWAP_EFFECT_DISCARD;
// Don't set the advanced flags.
swapchainDesc.Flags = 0;

swapchain = xgfx::createSwapchain(window, factory, device, &swapchainDesc);

Render Targets

Render Targets are textures that will be written to during raster based rendering.

// 👋 declare handles
ID3D11RenderTargetView* renderTargetView;
ID3D11Texture2D* backbufferTex;

xwin::WindowDesc desc = window->getDesc();

// Get the back buffer texture.
ThrowIfFailed(swapchain->GetBuffer(0, IID_PPV_ARGS(&backBufferTex)));
// Create the render target view with the back buffer pointer.
ThrowIfFailed(device->CreateRenderTargetView(backBufferTex, NULL,
                                             &renderTargetView));

// Create the texture for the depth buffer.
D3D11_TEXTURE2D_DESC depthBufferDesc;
ZeroMemory(&depthBufferDesc, sizeof(depthBufferDesc));
depthBufferDesc.Width = width;
depthBufferDesc.Height = height;
depthBufferDesc.MipLevels = 1;
depthBufferDesc.ArraySize = 1;
depthBufferDesc.Format = DXGI_FORMAT_D24_UNORM_S8_UINT;
depthBufferDesc.SampleDesc.Count = 1;
depthBufferDesc.SampleDesc.Quality = 0;
depthBufferDesc.Usage = D3D11_USAGE_DEFAULT;
depthBufferDesc.BindFlags = D3D11_BIND_DEPTH_STENCIL;
depthBufferDesc.CPUAccessFlags = 0;
depthBufferDesc.MiscFlags = 0;
ThrowIfFailed(device->CreateTexture2D(&depthBufferDesc, NULL,
                                      &depthStencilBuffer));

// Create the depth stencil view.
D3D11_DEPTH_STENCIL_VIEW_DESC depthStencilViewDesc;
ZeroMemory(&depthStencilViewDesc, sizeof(depthStencilViewDesc));
depthStencilViewDesc.Format = DXGI_FORMAT_D24_UNORM_S8_UINT;
depthStencilViewDesc.ViewDimension = D3D11_DSV_DIMENSION_TEXTURE2D;
depthStencilViewDesc.Texture2D.MipSlice = 0;
ThrowIfFailed(device->CreateDepthStencilView(depthStencilBuffer,
                                             &depthStencilViewDesc,
                                             &depthStencilView));

Resources

Vertex Buffer

A Vertex Buffer stores the per vertex information available as attributes in your Vertex Shader. All buffers are ID3D11Buffer objects in DirectX 11, be it Vertex Buffers, Index Buffers, Constant Buffers, etc.

// 💾 Declare Data
struct Vertex
{
    float position[3];
    float color[3];
};

Vertex vertexBufferData[3] = {{{1.0f, 1.0f, 0.0f}, {1.0f, 0.0f, 0.0f}},
                              {{-1.0f, 1.0f, 0.0f}, {0.0f, 1.0f, 0.0f}},
                              {{0.0f, -1.0f, 0.0f}, {0.0f, 0.0f, 1.0f}}};

// 👋 declare handles
ID3D11Buffer* vertexBuffer;

// Set up the description of the static vertex buffer.
D3D11_BUFFER_DESC vertexBufferDesc;
vertexBufferDesc.Usage = D3D11_USAGE_DEFAULT;
vertexBufferDesc.ByteWidth = sizeof(Vertex) * 3;
vertexBufferDesc.BindFlags = D3D11_BIND_VERTEX_BUFFER;
vertexBufferDesc.CPUAccessFlags = 0;
vertexBufferDesc.MiscFlags = 0;
vertexBufferDesc.StructureByteStride = 0;

// Give the subresource structure a pointer to the vertex data.
D3D11_SUBRESOURCE_DATA vertexData;
vertexData.pSysMem = vertexBufferData;
vertexData.SysMemPitch = 0;
vertexData.SysMemSlicePitch = 0;

// Now create the vertex buffer.
ThrowIfFailed(device->CreateBuffer(&vertexBufferDesc, &vertexData,
                                   &vertexBuffer));

Index Buffer

An Index Buffer contains the individual indices of each triangle/line/point that you intend to draw.

// 💾 Declare Data
uint32_t indexBufferData[3] = {0, 1, 2};

// 👋 declare handles
ID3D11Buffer* indexBuffer;

// Set up the description of the static index buffer.
D3D11_BUFFER_DESC indexBufferDesc;
indexBufferDesc.Usage = D3D11_USAGE_DEFAULT;
indexBufferDesc.ByteWidth = sizeof(unsigned) * 3;
indexBufferDesc.BindFlags = D3D11_BIND_INDEX_BUFFER;
indexBufferDesc.CPUAccessFlags = 0;
indexBufferDesc.MiscFlags = 0;
indexBufferDesc.StructureByteStride = 0;

// Give the subresource structure a pointer to the index data.
D3D11_SUBRESOURCE_DATA indexData;
indexData.pSysMem = indexBufferData;
indexData.SysMemPitch = 0;
indexData.SysMemSlicePitch = 0;

// Create the index buffer.
ThrowIfFailed(device->CreateBuffer(&indexBufferDesc, &indexData,
                                   &mIndexBuffer));

Constant Buffer

A Constant Buffer describes data that we'll be sending to shader stages when drawing. Normally you would put Model View Projection Matrices or any specific variable data like colors, sliders, etc. here.

// 💾 Declare Data
struct
{
    glm::mat4 projectionMatrix;
    glm::mat4 modelMatrix;
    glm::mat4 viewMatrix;
} cbVS;

// 👋 declare handles
ID3D11Buffer* constantBuffer;

// Setup the description of the dynamic matrix constant buffer that is in the
// vertex shader.
D3D11_BUFFER_DESC constantBufferDesc;
uniformBufferDesc.Usage = D3D11_USAGE_DYNAMIC;
uniformBufferDesc.ByteWidth = sizeof(cboVS);
uniformBufferDesc.BindFlags = D3D11_BIND_CONSTANT_BUFFER;
uniformBufferDesc.CPUAccessFlags = D3D11_CPU_ACCESS_WRITE;
uniformBufferDesc.MiscFlags = 0;
uniformBufferDesc.StructureByteStride = 0;

// Create the constant buffer pointer so we can access the vertex shader
// constant buffer from within this class.
ThrowIfFailed(mDevice->CreateBuffer(&constantBufferDesc, NULL,
                                    &constantBuffer));

Vertex Shader

Vertex Shaders execute per vertex, and are perfect for transforming a given object, performing per vertex animations with blend shapes, GPU skinning, etc.

// 👋 Declare handles
ID3D11VertexShader* vertexShader;

ID3DBlob* vertexShaderBlob = nullptr;
ID3DBlob* errors = nullptr;

#if defined(_DEBUG)
// Enable better shader debugging with the graphics debugging tools.
UINT compileFlags = D3DCOMPILE_DEBUG | D3DCOMPILE_SKIP_OPTIMIZATION;
#else
UINT compileFlags = 0;
#endif
std::string path = "";
char pBuf[1024];

_getcwd(pBuf, 1024);
path = pBuf;
path += "\\";
std::wstring wpath = std::wstring(path.begin(), path.end());

std::wstring vertPath = wpath + L"assets/shaders/triangle.vert.hlsl";

try
{
    ThrowIfFailed(D3DCompileFromFile(vertPath.c_str(), nullptr, nullptr, "main",
                                     "vs_5_0", compileFlags, 0, &vertexShader,
                                     &errors));
}
catch (std::exception e)
{
    const char* errStr = (const char*)errors->GetBufferPointer();
    std::cout << errStr;
}

// Create the vertex shader from the buffer.
ThrowIfFailed(device->CreateVertexShader(vertexShaderBlob->GetBufferPointer(),
                                         vertexShaderBlob->GetBufferSize(),
                                         NULL, &vertexShader));

// Create the pixel shader from the buffer.
ThrowIfFailed(mDevice->CreatePixelShader(pixelShader->GetBufferPointer(),
                                         pixelShader->GetBufferSize(), NULL,
                                         &mPixelShader));

Here's the vertex shader:

cbuffer cb : register(b0)
{
    row_major float4x4 projectionMatrix : packoffset(c0);
    row_major float4x4 modelMatrix : packoffset(c4);
    row_major float4x4 viewMatrix : packoffset(c8);
};

struct VertexInput
{
    float3 inPos : POSITION;
    float3 inColor : COLOR;
};

struct VertexOutput
{
    float3 color : COLOR;
    float4 position : SV_Position;
};

VertexOutput main(VertexInput vertexInput)
{
    float3 inColor = vertexInput.inColor;
    float3 inPos = vertexInput.inPos;
    float3 outColor = inColor;
    float4 position = mul(float4(inPos, 1.0), mul(modelMatrix, mul(viewMatrix, projectionMatrix)));

    VertexOutput output;
    output.position = position;
    output.color = outColor;
    return output;
}

Pixel Shader

Pixel Shaders execute per each pixel of your output, including the other attachments that correspond to that pixel coordinate.

// 👋 Declare handles
ID3D11PixelShader* pixelShader;

ID3DBlob* pixelShader = nullptr;
ID3DBlob* errors = nullptr;

#if defined(_DEBUG)
// Enable better shader debugging with the graphics debugging tools.
UINT compileFlags = D3DCOMPILE_DEBUG | D3DCOMPILE_SKIP_OPTIMIZATION;
#else
UINT compileFlags = 0;
#endif

std::string path = "";
char pBuf[1024];

_getcwd(pBuf, 1024);
path = pBuf;
path += "\\";
std::wstring wpath = std::wstring(path.begin(), path.end());

std::wstring fragPath = wpath + L"assets/shaders/triangle.frag.hlsl";

try
{
    ThrowIfFailed(D3DCompileFromFile(fragPath.c_str(), nullptr, nullptr, "main",
                                     "ps_5_0", compileFlags, 0, &pixelShader,
                                     &errors));
}
catch (std::exception e)
{
    const char* errStr = (const char*)errors->GetBufferPointer();
    std::cout << errStr;
}

// Create the pixel shader from the buffer.
ThrowIfFailed(mDevice->CreatePixelShader(pixelShader->GetBufferPointer(),
                                         pixelShader->GetBufferSize(), NULL,
                                         &mPixelShader));

And here's the pixel shader:

struct PixelInput
{
    float3 color : COLOR;
};

struct PixelOutput
{
    float4 attachment0 : SV_Target0;
};

PixelOutput main(PixelInput pixelInput)
{
    float3 inColor = pixelInput.color;
    PixelOutput output;
    output.attachment0 = float4(inColor, 1.0f);
    return output;
}

Rendering

Pipeline State

The Pipeline State describes everything necessary to execute a given raster based draw call. DirectX 11 separates each of these individual states into easy to setup up sub-states.

// 👋 Declare handles
ID3D11InputLayout* mLayout;
ID3D11DepthStencilState* depthStencilState;
ID3D11RasterizerState* rasterState;

// ⚗️ Define the Graphics Pipeline

// 🔣 Input Assembly
D3D11_INPUT_ELEMENT_DESC polygonLayout[2];

polygonLayout[0].SemanticName = "POSITION";
polygonLayout[0].SemanticIndex = 0;
polygonLayout[0].Format = DXGI_FORMAT_R32G32B32_FLOAT;
polygonLayout[0].InputSlot = 0;
polygonLayout[0].AlignedByteOffset = 0;
polygonLayout[0].InputSlotClass = D3D11_INPUT_PER_VERTEX_DATA;
polygonLayout[0].InstanceDataStepRate = 0;

polygonLayout[1].SemanticName = "COLOR";
polygonLayout[1].SemanticIndex = 0;
polygonLayout[1].Format = DXGI_FORMAT_R32G32B32_FLOAT;
polygonLayout[1].InputSlot = 0;
polygonLayout[1].AlignedByteOffset = D3D11_APPEND_ALIGNED_ELEMENT;
polygonLayout[1].InputSlotClass = D3D11_INPUT_PER_VERTEX_DATA;
polygonLayout[1].InstanceDataStepRate = 0;

// Get a count of the elements in the layout.
unsigned numElements = sizeof(polygonLayout) / sizeof(polygonLayout[0]);

// Create the vertex input layout.
ThrowIfFailed(device->CreateInputLayout(polygonLayout, numElements,
                                        vertexShaderBlob->GetBufferPointer(),
                                        vertexShaderBlob->GetBufferSize(),
                                        &layout));

// 🌑 Depth/Stencil
D3D11_DEPTH_STENCIL_DESC depthStencilDesc;
ZeroMemory(&depthStencilDesc, sizeof(depthStencilDesc));
depthStencilDesc.DepthEnable = true;
depthStencilDesc.DepthWriteMask = D3D11_DEPTH_WRITE_MASK_ALL;
depthStencilDesc.DepthFunc = D3D11_COMPARISON_LESS;
depthStencilDesc.StencilEnable = true;
depthStencilDesc.StencilReadMask = 0xFF;
depthStencilDesc.StencilWriteMask = 0xFF;
// Stencil operations if pixel is front-facing.
depthStencilDesc.FrontFace.StencilFailOp = D3D11_STENCIL_OP_KEEP;
depthStencilDesc.FrontFace.StencilDepthFailOp = D3D11_STENCIL_OP_INCR;
depthStencilDesc.FrontFace.StencilPassOp = D3D11_STENCIL_OP_KEEP;
depthStencilDesc.FrontFace.StencilFunc = D3D11_COMPARISON_ALWAYS;
// Stencil operations if pixel is back-facing.
depthStencilDesc.BackFace.StencilFailOp = D3D11_STENCIL_OP_KEEP;
depthStencilDesc.BackFace.StencilDepthFailOp = D3D11_STENCIL_OP_DECR;
depthStencilDesc.BackFace.StencilPassOp = D3D11_STENCIL_OP_KEEP;
depthStencilDesc.BackFace.StencilFunc = D3D11_COMPARISON_ALWAYS;

// Create the depth stencil state.
ThrowIfFailed(device->CreateDepthStencilState(&depthStencilDesc,
                                              &depthStencilState));

// Set the depth stencil state.
deviceContext->OMSetDepthStencilState(depthStencilState, 1);

// 🟨 Rasterization
D3D11_RASTERIZER_DESC rasterDesc;
rasterDesc.AntialiasedLineEnable = false;
rasterDesc.CullMode = D3D11_CULL_NONE;
rasterDesc.DepthBias = 0;
rasterDesc.DepthBiasClamp = 0.0f;
rasterDesc.DepthClipEnable = true;
rasterDesc.FillMode = D3D11_FILL_SOLID;
rasterDesc.FrontCounterClockwise = false;
rasterDesc.MultisampleEnable = false;
rasterDesc.ScissorEnable = false;
rasterDesc.SlopeScaledDepthBias = 0.0f;

// Create the rasterizer state from the description we just filled out.
ThrowIfFailed(device->CreateRasterizerState(&rasterDesc, &rasterState));

// Now set the rasterizer state.
deviceContext->RSSetState(rasterState);

Draw Calls

DirectX 11 has a more direct approach to rendering, where you tell the device context exactly what you want to do and it handles queuing up those commands to be executed.

void render()
{
    // Framelimit set to 60 fps
    tEnd = std::chrono::high_resolution_clock::now();
    float time =
        std::chrono::duration<float, std::milli>(tEnd - tStart).count();
    if (time < (1000.0f / 60.0f))
    {
        return;
    }
    tStart = std::chrono::high_resolution_clock::now();

    // Update Uniforms
    elapsedTime += 0.001f * time;
    elapsedTime = fmodf(elapsedTime, 6.283185307179586f);
    cbVS.modelMatrix = Matrix4::rotationY(mElapsedTime);

    // Set the position of the constant buffer in the vertex shader.
    unsigned int bufferNumber = 0;

    // Lock the constant buffer so it can be written to.
    D3D11_MAPPED_SUBRESOURCE mappedResource;
    ThrowIfFailed(deviceContext->Map(constantBuffer, 0, D3D11_MAP_WRITE_DISCARD,
                                     0, &mappedResource));

    // Get a pointer to the data in the constant buffer.
    memcpy(mappedResource.pData, &cbVS, sizeof(cbVS));

    // Unlock the constant buffer.
    deviceContext->Unmap(constantBuffer, 0);

    // Finally set the constant buffer in the vertex shader with the updated
    // values.
    deviceContext->VSSetConstantBuffers(bufferNumber, 1, &constantBuffer);

    // Bind the render target view and depth stencil buffer to the output render
    // pipeline.
    deviceContext->OMSetRenderTargets(1, &renderTargetView, depthStencilView);

    deviceContext->RSSetViewports(1, &viewport);

    // Clear textures
    float color[4] = {0.2f, 0.2f, 0.2f, 1.0f};
    deviceContext->ClearRenderTargetView(renderTargetView, color);

    // Clear the depth buffer.
    deviceContext->ClearDepthStencilView(depthStencilView, D3D11_CLEAR_DEPTH,
                                         1.0f, 0);

    // Set the vertex input layout.
    deviceContext->IASetInputLayout(inputLayout);

    // Set the vertex and pixel shaders that will be used to render this
    // triangle.
    deviceContext->VSSetShader(vertexShader, NULL, 0);
    deviceContext->PSSetShader(pixelShader, NULL, 0);

    // Set the vertex buffer to active in the input assembler so it can be
    // rendered.
    unsigned stride = sizeof(Vertex);
    unsigned offset = 0;
    deviceContext->IASetVertexBuffers(0, 1, &vertexBuffer, &stride, &offset);

    // Set the index buffer to active in the input assembler so it can be
    // rendered.
    deviceContext->IASetIndexBuffer(indexBuffer, DXGI_FORMAT_R32_UINT, 0);

    // Set the type of primitive that should be rendered from this vertex
    // buffer, in this case triangles.
    deviceContext->IASetPrimitiveTopology(
        D3D11_PRIMITIVE_TOPOLOGY_TRIANGLELIST);

    // Render the triangle.
    deviceContext->DrawIndexed(3, 0, 0);

    if (vsync)
    {
        swapchain->Present(1, 0);
    }
    else
    {
        swapchain->Present(0, 0);
    }
}

Conclusion

DirectX 11 is a much easier to use graphics API than other modern APIs like DirecX 12, Vulkan, Metal, or WebGPU. That ease of use comes at the cost of some performance, but it's a great start to learning how to engineer graphics applications.

More Resources

• The DirectX 11 Programming Guide

• The DirectX Toolkit, a set of tools to help with loading textures, gamepad input, memory allocation, and much more.

• I wrote a blog post on DirectX 12 as well which is worth reviewing even if you plan on developing your project for D3D11.

This is not financial advice. I am not a professional financial advisor. This is my own opinion and observations.

Financial market forces can appear chaotic and unpredictable. It's as though the eyes of the public are obscured, and with more perfect information on purchases, buyers, sellers, and existing portfolios, we can better predict global markets. There are however some visible parts to this metaphorical blindfold.

Index funds can give an overview of the health of a given industry, Level 2 data gives individual traders access to current market asks/bids, CEOs and government officials must disclose their purchases and sales and their assets are public knowledge, and quarterly earnings reports provide a top level view of the company's finances.

Every bank is reinvesting the money deposited in your account in the form of assets such as securities (stocks/bonds), foreign exchange (Forex), mortgages, loans, leases, and credit, and making significant returns on the money you deposit.

Government monetary policy drives inflation, with interest rate control being so powerful it was able to reverse a potential crash in the stock market in March 2020. Governments can align their monetary policy towards an end goal, such as China's interest in maintaining the renminbi pegged against the US dollar to better compete in the international export market.

Market forces and monetary policy come together to bring impressive market trends such as $AMD multiplying its value 10 fold between 2017 and 2021, with the value of tech stocks such as $NVDA and $MSFT following similarly.

Digital assets such as the cryptocurrency DogeCoin $DOGE have gone from $0.005 to as high as $0.69 over the period of 5 months multiplied in value 138 times at the start of 2021. Tesla $TSLAhad a bull run at the start of May 2019 where the value skyrocketed from a low of$160, to over $1000, then had a 4-1 split and followed a similar trend, with the value hitting as high as $760 in April 2021, multiplying in value 17 times. Option trading has seen a boom over the course of the past few years, and publicly available data on currently traded puts and calls allowed for the famous short squeeze on GameStop $GME in late 2020, which caused the price to skyrocket from $4 all the way to $325, multiplying in value 81 times.

Stock market forces echoes into other markets, with the Nasdaq US Benchmark Real Estate Index showing a spike following the COVID 19' pandemic, and the United States House Price Index showing only 1 direction for prices, up. The value of construction assets is increasing, and the price of building a house reaching an average $120 a square foot in 2021. Land prices in the South Florida area increased suddenly, with a 10,000 square foot lot averaging $40,000 last year doubling in price in 2021, with this price remaining stable to 2024. The increase in online sales of housing has shifted the role of the traditional real estate agent becoming less necessary for some markets, leaving open a margin of 3-10% of the price of the sale to the seller. Real Estate mortgages had seen a drop in interest rates, with 3% rates becoming commonplace, though recently they've stablized to around 5-6% as the federal reserve adjusted their rate to combat inflation.

These echos can also be found in agricultural markets. The average price per hundred pounds per slaughter and feeder cattle has also seen an upward trend. The value of investing in cattle to grow (feeder cattle) in the Okeechobee Cattle Auction has increased over the past year, benefiting farmers with pregnant cattle. Recently, steers of feeder cattle averaged 370 pounds and cost $168 per hundred pounds ($622). Breaker cattle have seen an average price of $60 per hundred pounds and a weight average of 1400 pounds ($840). Milk cows have become much more valuable based on their genetic score in professional dairy operations, with sales reaching as high as $200K in private auctions hosted by the University of Michigan.

These increases in asset value make sense given the federal response to the global pandemic made loaning money incredibly cheap and easy. Small businesses receiving federal loans of hundreds of thousands at low interest rates is a recipe for an asset bull run. Anecdotally there have been rumors of widespread fraud in employee income assistance programs and in requesting investment loans from the US government, which only exacerbated this problem as individuals waste this money on sports cars and more property.

The democratization of data makes it possible for retail investors that have a mind towards software engineering to build tools to solve financial problems and make more educated financial decisions. Publicly available data on floating ask/bids in real time through Level 2 Data provides a real time view of puts and calls for stocks traded in the New York Stock Exchange (NYSE), and can enable a rough prediction on the short term behavior of a given asset. A stock's Price to Earnings ratio (P/E ratio), price history, trading volume, ask/bid history, derivatives behavior (options such as Calls and Puts), and earnings reports can help with determining long and short positions on a stock. and Real estate data such as property sales prices and regional market trends can allow buyers to better predict the price of their asset in the future, estimate the return on investment (ROI) with rental investments in agriculture/residential/commercial markets, and be aware of the overall history of a given region.

There's a scary underside to all this data. Public ownership records from real estate appraisals make it possible to estimate the net worth of an individual or business, information that could be used in cruel ways like offering less to new employees or estimating credit scores more harshly. Cryptocurrency public addresses associated with companies like Tesla make it possible to see exactly how much money they made by switching 10% of their cash to Bitcoin and communities like Twitter make it easier than ever for individuals to make big impacts on markets by simply tweeting about $DOGE and $BTC such as certain tech billionaires:

Let's discuss financial engineering in asset trading in brief detail.

Algorithmic Trading

There are public algorithmic trading tools that allow for you to simulate positions with a given trading bot or outright use those positions in a retail trading API.

Back Testing is using prior asset market trends to test portfolio positions/changes and predict where those positions would be today. You can use a strategy tester or a basic asset portfolio tester to determine what your wealth would have been had you made a financial decision years ago.

Crypto Trading

There's a lot of good things that cryptocurrencies can enable, suddenly transferring thousands of dollars instantly and securely (at least as long as (P \neq NP)) means you can easily transfer money to anyone for any sort of transaction, be it payroll, asset purchases, or simply buying common household goods.

Computational cryptocurrencies such as Etherium $ETH allow for programmable contracts that can allow unique identification for ownership of a given asset non-fungible tokens (NFTs), programmable loans, and more.

That freedom also enables bad behavior, from the wasted electricity that the global power grid now needs to support due to mining, paying ransomware or for illegal items, money laundering (With some NFT art sales serving that purpose), or gambling.

More Resources

Here's a few sites that focus on stock information and metric data:

• Quiver Quantitative exposes public information such as senator stock trades, Robinhood user behavior, r/wallstreetbets trends, etc. in an easy to read interface.

• Portfolio Visualizer let's you backtest a portfolio of stocks, in other words, test if a portfolio would have done well if you bought it X number of years ago.

• TradingView - Asset visualizations, backtesting strategies, and every indicator you can imagine.

• Dividend.Watch - Track portfolios, score the dividend performance of given stocks.

• HawkSight offers an easy way to try back testing. It's a bit like a game, have fun imagining the possibilities.

• Financial Modeling Prep is more of a back testing + machine learning training community.

• Unusual Whales provides a history of puts/calls and open interest.

For algorithmic trading there's a number of exchanges and apis with a wide range of assets to trade/study:

• Polygon - Real time stock data.

• Webull's Quantitative OpenAPI

• Alpaca Markets - An alternative API for algorithmic trading.

• QuantConnect - Open source algorithmic trading.

• Interactive Brokers API - A solid API for algorithmic trading.

• Kraken API

• Coinbase API

In addition, there's a few miscellaneous sites and articles:

• MarketWatch

• What I wish I knew about ESPP and RSUs sooner (company stock benefits. 2024, USA, California)

• Web3 is going just great is a website by Molly White (@molly0xFFF) that covers the failures of cryptocurrencies.

• Investments - A CLI pet project for managing stock portfolios. While not too useful here in the US, it does serve as a good reference for fintech engineering.

• Maybe - A financial tracking agrigator tool to manage portfolios and expenses.

• Wealthfront is an automated portfolio manager for US and foreign stocks.

• Vanguard is a popular investment manager transitioning to automation.

• Boomerang: Travels in the New Third World covers the history behind financial markets and their subsequent crashes throughout the turn of the 2nd millenium.

Control is a metaphysical 3rd person shooter by Remedy Entertainment, creators of Quantum Break and Alan Wake. You play as Ms. Jesse Faden, who finds herself in the Federal Bureau of Control, and upon discovering an odd gun and apparent suicide of the previous director, assumes the role of new Director of the FBC by the Board, a god-like group reminiscent of the employers from titles like Valve's Half Life. She's tasked to ridding the FBC from a metaphysical force she called the Hiss (which appears to be a force similar to the Dark Presence in Alan Wake) to both please the Board and further her own goals.

Control's gameplay loop is similar to that of From Software's Bloodbore or Id Software's Doom Eternal, with a strong emphasis on moving to dodge your enemies attacks and using abilities granted to you by objects of power (or OoPs) to both improve your combat abilities and navigate the world like a Metroidvania. Possessed enemies have similar combat abilities that they use to attack and lock off sections of the world.

Visually, the game looks stunning, with real time ray traced shadows, ray traced global illumination, ray traced reflections, Deep Learning Super Sampling, volumetric effects, and carefully tuned environmental effects like broken glass, and office debris. Let's analyze exactly what's happening in a single frame of Control, at its highest settings at 4K with Deep Learning Super Sampling enabled.

Frame Analysis

⚠️ Note: This is not an official analysis of the Northlight Engine renderer. Please support Control Ultimate Edition by grabbing a copy of it via Steam, Playstation Store, Xbox Store, or the Nintendo Store, thanks!

We'll be doing a high level analysis of a somewhat still frame mid combat in an area relatively early in the game, with plenty of lights, reflections, and global illumination.

Frame Setup

Each frame starts with setting up various data structures. Some early Draw calls build texture atlases for various UI elements, generate Acceleration Data Structures for the scene and Bounding Volume Hierarchies (BVHs) for lights.

Shadow Depth Pass

A light shadow depth pass renders to a large 16384x2048 texture atlas. Each segment of the atlas is relatively small, about 512x512 with the smallest being 256x256. While it might seem odd that this is here given that ray traced shadows are enabled, both ray traced shadows and raster based shadows are used to resolve shadows in Control, with ray traced shadows used for select lights as a means of better resolving contact shadows.

Prepass

The Prepass for Control renders view space normals with the b channel being used for edge softening/contact edges for corners, and a second render attachment encodes subsurface data in the r channel and roughness in g.

There's a second shadow and prepass step as well for smaller objects as well and to apply beveled edge effects to objects in the environment.

Velocity is encoded in its own separate pass afterwards. This appears to be NDC based velocity, encoded to an R16G16_FLOAT texture.

Lighting

The lighting passes begins with a few Dispatch calls to calculate bent normals, then renders reflections, global illumination, and shadows with DispatchRays.

Ray traced reflections renders at full resolution with samples denoised shortly after rendering. Reflected rays are based on the GGX specular bidirectional reflectance distribution function (BRDF). Control takes advantage of some approximation effects to help reduce the cost of evaluation, by only using reflected rays based on how far they are from the current surface - and that surface's roughness. [Sjöholm et al. 2021]

Ray Traced Global Illumination renders in a single pass at half the native resolution (so 960x560), and is then denoised. Control uses precomputed voxel based global illumination performed with an offline path tracer based on static objects, which helps with resolving global illumination with objects far away. Near field indirect global illumination is used based on an object's BRDF for surfaces relatively close by.

This pass functions as indirect diffuse lighting as well as a kind of ambient occlusion, since the precomputed GI values are used when rays miss.

Finally, Contact Shadows are rendered with the red channel representing spotlight index, and the green channel representing the point lights index of the current shadow sample.

All the denoising for these passes use data reprojected from previous frames and blurred with a bilateral filter, though each denoising pass is tuned slightly differently. Control uses a variant of the Spatio-temporal Variance Guided Filter (SVGF) [Schied 2017], with a firefly filter and spatial filter that is based on 4 pixels around a given sample. There's some issues with reprojecting at times, and you can occasionally catch the difference between regions with high accumulation and regions with low. There's also some noise dancing in the end result but it appears really charming and adds to the game's ascetic.

Lighting is then composited to a single render target on a per object basis to overlay material data like albedo, and even volumetric lighting contributions. The image is finally tone mapped.

Technical Art Passes

Control is full of technical art render passes that help it achieve it's overall look.

There's an enemy pass, where enemies are overlaid with various effects such as shields, which is also blurred to then give it the appearance of glowing.

Velocity	Pressure	Divergence

This is followed by a fluid smoke pass, probably the highlight of the renderer's technical artistry. There's a feedback loop that uses view space depth, a fluid buffer that consists of the current frame, a fluid velocity buffer that changes over a few DrawInstanced calls, geometric velocity, fluid divergence, and fluid pressure buffers to warp previous and current samples in real time, resulting in a screen space feedback effect that looks like rainbow gas. It's reminiscent of ShaderToys that emulate gaseous flow. [McAloon 2019]

A separate pass uses this fluid warped render pass with a hiss mask to result in a final output image.

Deep Learning Super Sampling

Deep learning Super Sampling (DLSS) takes in a smaller input image, and upscales it to a target resolution. In this case, it took a 1080p input image and upscaled it to 4K. This is how the game is able to average above 30 fps at 4K on the 2070 Super.

UI Generation

While this screenshot doesn't include it, first a Sobol outline pass is overlaid on top of the final composite for objects that you can control or interact with.

UI particles such as health chips are rendered piecemeal by drawing a billboard, and UI elements such as your health are rendered piecemeal as well in screen space. During my testing I found some of the UIs would draw at the same time, like text elements for your objectives. Given that Control is using Coherent Labs's HTML5 based Gameface Renderer for its user interface this makes sense, as the browser renderer could execute all of those draw calls with the CPU.

Finally the frame is finished and presented to the player.

Conclusion

Control was a really fun game, the metroidvania style level traversal, a variety of unlockables, and the mysterious, government controlled setting made for fun gameplay and a fascinating story. If you have any suggestions for a game to analyze, please let me know in the comments below or via Twitter.

More Resources

• Daniele Giannetti released a GDC Talk that covers in great detail a single frame of Halo Infinite/

• Angelo Pesce (@kenpex) wrote a blog post on Cyberpunk 2077's Renderer.

• Emilio López (@redorav) wrote about the new Unreal Engine 5 renderer in his post: A Macro View of Nanite.

• Chapter 46 of Ray Tracing Gems 2 discusses the ray tracer in Control in far more detail, the denoising section is particularly interesting.

• NVIDIA has released an SDK to integrate Deep Learning Super Sampling here. AMD's Fidelity FX Super resolution is also available for developers to integrate into their applications.

I've also released other articles describing the renderers of different titles:

• Minecraft RTX

• Mortal Kombat 11

• Overwatch 1

[Sjöholm et al. 2021] Ray Tracing in Control Juha Sjöholm, Paula Jukarainen and Tatu Aalto Apress 2021 developer.nvidia.com

[Schied 2017] Spatiotemporal Variance-Guided Filtering Christoph Schied (@c_schied) ACM 2017 research.nvidia.com

[McAloon 2019] A closer look at Control's reality-bending VFX Alissa McAloon Gamasutra 2019 gamasutra.com

Some artistic visions are hard to achieve without some level of engineering. Creating detailed characters and environments can benefit from using procedural materials to avoid visual repetition and speed up their design. Simulating natural phenomena like broken glass, water caustics [Brinck et al. 2021], newton's rings, or stylistic effects like volumetric hue shifting from Pixar's Soul requires some understanding of the fundamental design of a rendering system.

Let's discuss writing custom shaders with Marmoset Toolbag 4. We'll approach this from the perspective of rendering a raymarched scene, similar to what you might expect from a ShaderToy.

The goal here will be to use this raymarched scene as a generated background for our scene, but it's possible to take this a step further, such as using a raymarched scene as an environment map.

General Design

So the example we'll be using is an effect I made that was heavily inspired by the YouTube channel MKBHD.

The design here isn't too complicated, we use our own camera to ray march a scene of repeating spheres [Quílez 2013] that shift and move according to sinusoidal functions.

Marmoset Toolbag

Every material in Toolbag can have a Custom subroutine attached to it, which can then use a shader file to execute your shader. Custom shaders are executed after all the other logic for rendering has occurred, and can be used to overload that logic with your own behavior.

Creating UI Parameters

If you want uniforms to be controllable within the material settings UI, expose a uniform with a comment describing that uniform with the following format:

// 🎨 Color
uniform vec3 uColor; //color  name "My Color"  default 0.5,0.5,0.5

// 🗃️ Dropdown
uniform int uDropdown; //name "My Dropdown"  default 0  labels "Option 1" "Option 2" "Option 3"

// 🎚️ Slider
uniform float   uSlider; //name "My Slider"  default 1.0  min 0  max 10

Overloading a Render Pass

Toolbag has a series of categorized executable steps called Subroutines. Each render pass of Toolbag can use a different set of subroutines, allowing toolbag to easily permute the material shader to match the needs of that render pass. [Russell 2014]

You can actually see all subroutines you can overload with custom shaders, they're the drawers that make up the material settings, with the name of the subroutine on the right, and the currently active one on the left. You can also spot all the different subroutines over in the runSubroutines(...) function in <Your Toolbag Installation>/data/shader/mat/mat.frag and <Your Toolbag Installation>/data/shader/mat/mat.comp for ray tracing specific subroutines. Marmoset has a blog post that details each subroutine here.

Nearly all Toolbag subroutines follow a simple interface where they accept one argument, the shader state FragmentState, which stores relative information like eye direction, normals, etc. to be used across each subroutine in order.

Full Shader

So here's the full custom shader, basically, I'm feeding the output of the shader above to the albedo output, with a material's Diffusion subroutine set to unlit. The material is placed in an object covering the camera, a sphere with all normals pointing inward, and the signed distance field camera is set to match the camera in Toolbag.

#define DISTMARCH_STEPS 128
#define DISTMARCH_MAXDIST 256.

uniform vec3 uColor1; //color  name "Color 1"  default (14.0 / 255.0), (135.0 / 255.0), (163.0 / 255.0)
uniform vec3 uColor2; //color  name "Color 2"  default (11.0 / 255.0), (96.0 / 255.0), (178.0 / 255.0)

// 🌊 Distance Fields

float spheredf(vec3 pos, float r) { return length(pos) - r; }

float opRep(vec3 p, vec3 c)
{
    // Modulus Repeat on XZ
    vec3 spread = max(vec3(0.0, 0.0, 0.0), vec3(uCustomAnimationTime - 1.0,
                                                uCustomAnimationTime - 1.0,
                                                uCustomAnimationTime - 1.0));
    vec3 q = mod(abs(p), c) - 0.5 * c;
    q.y = p.y;
    // Wave effect
    float yy = sin(uCustomAnimationTime + .3 * p.x) +
               cos(uCustomAnimationTime * .9 + .2 * p.z);
    q += .5 * vec3(0.0, yy, 0.0);
    return spheredf(q, .1);
}

vec2 scenedf(vec3 p)
{
    p.y += max(2.0 / (uCustomAnimationTime + .001), 0.001);
    vec2 obj = vec2(opRep(p, vec3(2.5, 2.5, 2.5)), p.y);
    return obj;
}

vec3 distmarch(vec3 ro, vec3 rd, float maxd)
{
    float epsilon = 0.001;
    float dist = 10. * epsilon;
    float t = 0.;
    float material = -1.0;
    float iterations = 0.;

    for (int i = 0; i < DISTMARCH_STEPS; i++)
    {
        if (abs(dist) < epsilon || t > maxd) break;
        // advance the distance of the last lookup
        t += dist;
        vec2 dfresult = scenedf(ro + t * rd);
        dist = dfresult.x;
        material = dfresult.y;
        iterations++;
    }

    if (t > maxd)
    {
        iterations = -1.0;
    }
    return vec3(t, iterations, material);
}

void CustomAlbedo(inout FragmentState s)
{
    vec2 uv = s.screenTexCoord;
    uv.y = 1.0 - uv.y;
    vec4 outColor = vec4(0., 0., 0., 0.);

    // 🎥 Setup Camera
    vec3 camOrigin = vec3(45.0, -32.0, 45.0);
    vec3 camDir = normalize(-vertexEye);
    // aspect ratio
    vec2 st = uv.xy - .5;
    st *= (float(screenCoord.y) / float(screenCoord.x));

    // Perspective Transform
    float fov = 3.142 * .625;
    vec3 iu = vec3(0., 1., 0.);
    vec3 iz = camDir;
    vec3 ix = normalize(cross(iz, iu));
    vec3 iy = cross(ix, iz);
    vec3 dir = normalize(st.x * ix + st.y * iy + fov * iz);

    // 🥁 Scene Marching
    vec3 scenemarch = distmarch(camOrigin, dir, DISTMARCH_MAXDIST);
    float sceneCol = 0.0;
    sceneCol = clamp(sceneCol, 0.0, 1.0);

    // 🌫️ Fog
    if (scenemarch.x > 0.0001)
    {
        sceneCol = clamp(scenemarch.y, 0.,
                         mix(1.0, 0.0, smoothstep(10., 180., scenemarch.x)));
    }

    // 🖼️ Output
    outColor.xyz = mix(uColor1, .95 * uColor2,
                       smoothstep(0.0, 1.0, scenemarch.z)) *
                   sceneCol;
    outColor.a = sceneCol;

    // Write to albedo/baseColor
    s.albedo = outColor;
    s.baseColor = outColor.xyz;
}

#ifdef Albedo
#undef Albedo
#endif
#define Albedo CustomAlbedo

More Resources

• Every installation of Marmoset Toolbag has examples of custom shaders that can be found in <Your Toolbag Installation>/data/shader/mat/custom/.

• Marmoset released a blog post on custom shaders here.

• Marmoset maintains a library of user submitted custom shaders here.

• For more information on writing Python add-ons in Toolbag, I wrote a blog post here.

Developing Marmoset Toolbag plugins with Python can be challenging. There's important questions that you may need to solve like:

• How do you design UIs?

• How do you import models and textures?

• How do you export data like renders, materials, etc.

• How do you configure a material, texture project, or your render settings?

• How do you use external packages from PyPi?

With the advent of text editors such as Microsoft Visual Studio Code, AI assisted auto complete with Visual Studio IntelliCode, and language servers such as PyLance, iterating your plugin's design can be very intuitive, and the questions you might have about a given piece of software's API can be more easily answered by the text editor's auto-complete in addition to our documentation and examples.

Editor Setup

First, download Visual Studio Code with the link below:

• Microsoft Visual Studio Code

Then, install the latest version of Python to your computer. Marmoset Toolbag is designed to run on Python 3.6 and above. Make sure to add Python to your environment variables in the advance options.

• Python Download

From there you'll need to install python packages from the Python Package Index via the pip command. You will need the Marmoset Toolbag package mset for auto-complete support as well as remote execution.

Open up a terminal such as VS Code's Integrated Terminal with the shortcut Ctrl + Shift + ~ and run:

pip install mset

Linting is checking your code early for mistakes and let you know early where those mistakes are and how to fix them. There's a number of different packages available for linting. The Visual Studio Code team discusses the differences between different linters here.

Visual Studio Code will present you with an option for them to download a linter for you, but if that option doesn't work you can always open up the terminal with CTRL + Shift + ~ and type the following:

pip install pylint

Finally, install the PyLance Visual Studio Code extension as well as Intellicode:

• Pylance on the VS Code Marketplace

• Intellicode on the VS Code Marketplace

Now you should have everything you need for full autocomplete support!

Designing UIs

Marmoset Toolbag uses an imperative retained mode API for designing UIs, so you must create UI elements, then add them as children of other elements:

import mset
import os

# Initialize UI Objects

window = mset.UIWindow("UI Test")

# 🗄️ Setup Drawer

# 🔘 Button
button = mset.UIButton("Close Button")
button.onClick = lambda: mset.shutdownPlugin()
# ⏯ Icon Button
icon_button = mset.UIButton()
icon_button.setIcon(os.path.abspath(os.path.join(
    os.curdir, "data/gui/control/animationplay.tga")))

drawer = mset.UIDrawer(name="Settings")
drawer_window = mset.UIWindow(name="Drawer Window")
drawer.containedControl = drawer_window

drawer_window.addElement(button)
drawer_window.addReturn()
drawer_window.addElement(icon_button)

# 📜 Setup scroll box

scrollbox = mset.UIScrollBox()
scrollbox_window = mset.UIWindow(name="Scrollbox Window")
scrollbox.containedControl = scrollbox_window

# 📃 Lists
my_list = mset.UIListBox("List 1")
my_list.addItem("1")
my_list.addItem("2")
my_list.addItem("3")

scrollbox_window.addElement(drawer)
scrollbox_window.addReturn()
scrollbox_window.addElement(my_list)

# 🌟 Setup Main Window

# 🎚️ Sliders
slider = mset.UISliderInt(min=5, max=128, name="Slider")
# 🎨 Color Picker
picker = mset.UIColorPicker("Color")
# ✏️ Inputs
text = mset.UITextField()
text_float = mset.UITextFieldFloat()
# ✅ Checkbox
checkbox = mset.UICheckBox()

# 🍱 Add elements to Window
window.addElement(slider)
window.addReturn()
window.addElement(scrollbox)
window.addReturn()
window.addElement(picker)
window.addReturn()
window.addElement(text)
window.addReturn()
window.addElement(text_float)
window.addReturn()
window.addElement(checkbox)

# 2️⃣ Create Secondary Window

popup_window = mset.UIWindow("UI Popup Test")

Importing Models

Perhaps you're automating rendering a number of different models, or have a lot of models that need to be batch imported from different places. The mset.importModel(path: str) function is all you need to do to import a model to your scene. From there you can move it around by changing its position, or make it a child of something like say a Baker's Source children.

import mset

# 🏆 Model
model = mset.importModel("C:/mymodel.obj")
model.name = "My Imported Model"
model.position = [0.0, 2.0, 0.0]

Configuring Objects

There's a variety of different Objects you may want to configure in Marmoset Toolbag, let's go over a couple:

Materials

Materials can be configured to use a specific subroutine, and every parameter that exists can be modified to your liking.

import mset

# Create a Material
mat = mset.Material()

# 🔖 Set a subroutine
mat.setSubroutine("reflectivity", "Metalness")

# 🎛️ Edit a subroutine parameter
surface = mat.getSubroutine("surface")
tex = mset.Texture("C:/my_normal.png")
surface.setField("Normal Map", tex)

Cameras

All cameras in Toolbag can be controlled with Python, and you can adjust their transform parameters like their rotation, as well as lens settings like the camera limits (near, far), depth of field, post-processing.

import mset

cam = mset.CameraObject()
cam.name = "Python Camera"

# 🔬 Adjust camera limits
cam.limits.farLimit = 10

# 🔎 Adjust lens settings
cam.lens.dofEnabled = True
cam.lens.dofMaxBokehSize = 64.0

# ✨ Adjust post-processing
cam.postEffect.sharpen = 0.3
cam.postEffect.vignetteStrength = 0.4
cam.postEffect.bloomBrightness = 0.1
cam.postEffect.bloomSize = 4.0

Exporting Renders

Exporting renders is incredibly easy, there's 3 functions you can use to either render a single camera, or the images/video configuration set up in your Render Object.

import mset

# 📷 Render Main Camera
img = mset.renderCamera(path='C:/my_render.png')

# 📸 Render all images configured in the Render Object
mset.renderImages()

# 🎥 Render all videos configured in the Render Object
mset.renderVideos()

Using External Packages

Marmoset Toolbag PluginS can take advantage of the entire Python Package Index and use that ecosystem to help solve problems like hooking Toolbag up to an HTTP server such as Flask or to a machine learning framework such as PyTorch or TensorFlow.

A Toolbag Plugin that uses PyPi packages needs to have its files laid out as follows:

├─ YourPlugin/
│   ├─ __main__.py
│   ├─ requirements.txt
│   └─ ...
└─ ...

Your requirements.txt would be a text file like the following:

LinearAlgebra==1.4.1

The __main__.py file would serve as the entry point to the plugin, and similar to other Python modules, you can have any file/folder structure you want.

Conclusion

There's a lot more you can do with Marmoset Toolbag plugins than what we've discussed, from procedural generation of geometry to automating your baking, rendering, or technical art workflows.

For more examples of just what you can do, visit the Examples folder in your Toolbag installation:

• Windows: <Your Toolbag Installation>/data/plugin/Examples
• MacOS: <Your Toolbag Application.app>/data/plugin/Examples

As well as refer to our reference documentation here.

More Resources

• Marmoset maintains a library of user submitted plugins here.

• The procedural meshes were done using a library I wrote called Strange Attractors.

More general topics:

• There's Visual Studio plugins for relevant tasks you may want to do in Toolbag such as custom shaders. We would recommend the GLSL Lint plugin for linting GLSL shader code.

• The Theme used for Visual Studio Code in this blog post is called Github Dark Classic.

• The font used in this example was MonoLisa with ligatures enabled. For a free option that's similar try Fira Code.

• The Windows Powershell terminal was styled using Oh My Posh, which works on all platforms.

Low level Graphics APIs such as Vulkan, DirectX, Metal, and WebGPU, are converging to a model similar to the way GPUs are currently built. Graphics Processing Units (GPUs) are asynchronous compute units that can handle large quantities of data, such as complex mesh geometry, image textures, output frame buffers, transformation matrices, or anything you want computed.

GPUs weren't always like this, originally they were a set of fixed hardware based functions with very little programmability. This changed as applications pushed the limits of what these non-programmable systems could do, which warranted a race between GPU manufacturers and application developers constantly pushing the limits of their design [Peddie 2023]. Frame buffers and rasterizers [Fatahalian 2018] lead to programable shaders, General Purpose GPU (GPGPU) computing, and recently the addition of hardware for ray traversal acceleration and tensor processing for AI. Graphics APIs have evolved right alongside these changes with additions to the fixed graphics pipeline, compute shaders, and more recently ray traversal functionality (DirectX 12 and Vulkan Ray Tracing).

Let's look at some of the similarities and differences between graphics APIs. We'll be covering the C++ APIs for:

• 🌋 Vulkan

• ❎ DirectX 12.x

• ✖️ DirectX 11.x

• 🤖 Metal

• 🕸️ WebGPU

• ⚪ OpenGL

OpenGL's design originated during the early days of computer graphics and was designed as a state machine, so its interface differs greatly from modern graphics APIs. DirectX 11, while closer to modern GPU architectures than OpenGL, tries to be easy for developers by delegating tasks that Vulkan, DirectX 12, and Metal currently leave the developer responsible for to the driver. [Russell 2014]

It is useful to be aware of the legacy of modern graphics APIs so they are mentioned where relevant.

Execution Order

A graphics application normally follows the following order of execution no matter the API:

1. Initializing the API - Creating the core data structures needed to access the inner workings of the API.

2. Loading Assets - Creating the data structures needed to load things like shaders, describe the graphics pipeline, create and populate commands buffers for the GPU to execute, and send resources to GPU exclusive memory.

3. Updating Assets - Update any uniforms to your shader and perform application level logic here.

4. Presentation - Send your list of command buffers to your command queue and present your swapchain.

5. Repeat 2, 3, and 4 until a signal from the application to close.

6. Destroy - Wait for the GPU to finish any remaining work, and destroy all data structures and handles.

Thus we'll be following the creation and use of Graphics API data structures in this order.

Importing Dependencies

When starting a new application you need to include any dependencies you have to external APIs, and graphics APIs are no different. Depending on the API, you may also need other libraries in your project such as a shader compiler.

OpenGL stands out as the exception to every other graphics API, in that depending on the operating system and your personal setup, there can be a variety of imports from different locations.

Shader Compiler

Vulkan requires that you use an external shader compiler that generates SPIR-V such as glslang or the DirectX Shader Compiler.

For DirectX, It's recommended that you use the DirectX Shader Compiler rather than the included compiler since it supports newer shader model versions and more optimizations and speed.

Metal shaders can be compiled at runtime or compiled at build time with the metallib command line tool included in your MacOS path.

OpenGL doesn't require an external library to compile shaders since it's included with the library, though it also supports SPIR-V as an optional alternative to GLSL with OpenGL 4.6.

WebGPU shaders are plaintext strings, so there's no need to compile them, though it is a good idea to strip whitespace and minify/mangle symbols for production.

For a comparison of shader languages, visit my blog post reviewing shader languages.

Initialize API

Entry Point

The entry point to a graphics API generally allows you to access the API's inner classes.

Vulkan's entry point involves choosing what version of the API you intend to use as well as any extensions or layers you want, such as error checking, window surfaces, etc.

DirectX 11 and 12 requires that you create a Factory, and optionally a Debug data structure.

On Metal, an NSWindow is required to have an NSView with a CAMetalLayer in it (which is a part of QuartzCore). Once a layer exists and is attached to a window, that window can use the rest of the Metal API.

For OpenGL, the closest thing to an entry point is an operating system specific Context that you can request for after creating an OS Window.

Physical Device

Physical Devices allow you to query for important device specific details such as memory size and feature support.

Metal is the only outlier here since both the physical and logical device are shared by the same data structure.

OpenGL cannot query for any device details unless you use a manufacturer exclusive extension. You can get some miscellaneous data though like the driver vendor name, renderer, and OpenGL version.

Logical Device

A Device gives you access to the core inner functions of the API, such as creating graphics data structures like textures, buffers, queues, pipelines, etc. This type of data structure is the same for the most part across all modern graphics APIs with very few changes between them.

Vulkan and DirectX 12 offer control over memory via creating memory data structures via a device.

Queue

A Queue allows you to enqueue tasks for the GPU to execute. A GPU is an asynchronous compute device, so the idea here is to always keep it busy while having control over when items are added to the queue.

Vulkan queues require you to specify what queues the device will use before even creating it.

Command Pool

A Command Pool is a data structure that allows you to create command buffers.

Metal stands out by having the queue also be the data structure from which to allocate command buffers.

Frame Backings

Window Surface

A window Surface allows you to bind all draw calls to an OS specific window.

On DirectX, since there's only Windows / Xbox as targets for the API, the closest thing to a surface is the texture back buffer you receive from a swapchain. A swapchain receives your window handle and from there it'll create a surface internal to the DirectX driver.

Since MacOS and iOS windows feature a hierarchical structure where an Application contains a View, which can contain a layer, the closest thing to a surface in Metal is either a metal layer or view that wraps it.

Swapchain

A Swapchain flips between different back buffers for a given window, and controls aspects of rendering such as refresh rate and back buffer swapping behavior.

Metal and OpenGL stands out here in that the API lacks the idea of a swapchain, leaving that to the OS windowing API instead.

Frame Buffers

Frame Buffers Are groups of output textures used during a raster based graphics pipeline execution as outputs.

DirectX 12 and 11 don't feature an explicit data structure for this per say, rather you can pass a set of Views.

Initialize Resources

Texture

Textures are arrays of data that store color information, and serve as inputs/outputs for rendering. Vulkan, DirectX 12, and WebGPU introduce the idea of having multiple views of a given texture that can view that texture in different encoded formats or color spaces. Vulkan introduces the idea of managed memory for Images and buffers, thus a texture is a triplet of an Image, Image View when used (there can be multiple of these), and Memory in either device only or in CPU-GPU accessible space.

For a more traditional way of managing memory in Vulkan, I would highly recommend the AMD Vulkan Memory Allocator. For DirectX 12 the same authors released the AMD D3D12 Memory Allocator.

Buffer

A Buffer is an array of data, such as a mesh's positional data, color data, index data, etc. Similar rules for images apply to buffers in Vulkan and WebGPU.

Shader

A Shader tends to be a handle to a compiled blob of shader (HLSL, GLSL, MSL, etc.) code to be fed to a given Pipeline.

Shader Bindings

Most modern graphics APIs feature a binding data structure to help connect uniform buffers and textures to graphics pipelines that need that data. Metal is unique in that you can bind uniforms with setVertexBuffer in a command encoder, making it much easier to architect compared to Vulkan, DirectX 12, and WebGPU.

Pipeline