In computer graphics there exist two big categories that use a different approach to render an image to the display. These are online- and offline or realtime- and no realtime rendering. In computer games, or any other interactive media the computer needs to respond to the input of the user in a short amount of time to give the illusion of a continous media experience. Frames need to be rendered at about 60 times per second, so the time spent for one frame is very little. On the other hand for non interactive media, like movies or pictures, the time spent to calculate one frame is not restricted. This gives a different approach for rendering images. In realtime rendering, all rendering happens on the graphics card, the GPU. The GPU is a highly parallelized computer, that can run a lot of programs at once. This is used to calculate the color for every pixel on the monitor simultaneously. In order to program on the GPU there exists several API’s (like OpenGL, Vulkan, DirectX..) to communicate with the GPU. These API’s offer the ability to write shaders, which are programs that are executed on the GPU (see StevieJump, for an application with OpenGL).

For offline rendering a algorithm called raytracing is used. This algorithm tries to mimic the way light waves/particles behave when the illuminate a scene. A light ray gets projected into a scene from a lightsource and gets reflected, absorbed or scattered by interacting with surfaces or volume media. The basis for offline rendering are the physics laws that describe the behaviour of light (like Maxwell’s formulas). Many popular ray tracer run on the CPU. The difference between offline- and online rendering is getting smaller as time progresses. The computing power of modern computers is advancing, so the complicated algorithms of offline rendering become more and more feasible to use in real time (see real time ray tracing).

Path tracing and StevieTracer

The basis equation for offline rendering is the rendering equation (the brackets behind the equation denote the unit)

L(\textbf{x}, \omega) = L_e(\textbf{x}, \omega) + \int_{S^2} \f_r(\omega, \textbf{x}, \omega^\prime)L(\textbf{x}, \omega^\prime) \vert \textbf{n}(\textbf{x}) \cdot \omega^\prime \vert d\omega^\prime [\frac{W}{m^2sr}]

This equation describes the behaviour of light on surface in a vacuum. The term L(\textbf{x}, \omega) describes the radiance flowing from a point \textbf{x} in the direction \omega. This radiance is a combination of the direct light, being emitted from the point itself, and the indirect radiance from all other directions. Radiance is power per square meter per solid angle.

Solving this equation directly is very difficult because we cannot compute the integral and there is also a recursion inside the equation, because the term L(\textbf{x}, \omega) appears on bot sides. For integration we use a technique called monte-carlo integration. For the recursion we use the ray tracing algorithm.

A monte-carlo estimator for some generic function f(x) is given by

\int f(x)dx = \frac{1}{N}\sum_{i=1}^{N}\frac{f(X_i)}{p(X_i)}

To evaluate the integral we evaluate the function at random points X_i, that are distributed according to the probability density function p(X_i) (pdf). This pdf can be everything we want, since the function actually has no propabilites relate to it, but in order to have a fast convergence to the correct result it is best to choose the pdf similar to f(x).

In order to use the rendering equation for actual images, we need to relate it to the final pixel color of a monitor. This is done by using a Neumann-Reihe. If the rendering equation is written in operator notation one can see that it is actually a geometric series. This can be written as an infinite sum and finally an integral over the pathspace, a space that includes all possible combinations of paths. Also we switch the integration domain from directions to surfaces, to be able to sample points on lightsources.

I = \int_P f(X)dX [W]

This function sums up the contribution of all possible paths weighted with the measurement contribution function (MCF). The MCF calculates the contribution of a complete path from lightsource to the camera, along k vertices.

f(X) = L_e G(x_{k-1}, x_k) (\prod_{i=2}^{k-1}f_r(x_i)G(x_{i-1},x_i))W

G is the geometry term and relates solid angle to vertex area. It is needed to switch the integration domain from directions to surfaces and is the jacobian of that coordinate transformation.

To use monte carlo integration on the path space integral we need a way to sample paths and determine the pdf with it. One approach is to sample a direction according to the BRDF, so we choose a new direction that would give a high contribution according to the BRDF. Because this pdf would then be in solid angle measure, we also need to convert it with the geometry term to vertex area measure. Effectively the geometry term will cancel out in the estimator. In order to guarantee to hit the lightsource we use an algorithm called next event estimation (NEE). After we have an intersection with scene geometry, we will automatically sample a point on the lightsource and try to build a connection. Therefore we need to evaluate the visibility function, to check if the sampled point on the light source is actually visible from the vertex.

Finally our monte carlo estimate of the pixel intensity is

\frac{1}{N}\sum_{i=1}^{N}\frac{L_e(x_k) G(x_{k-1}, x_k) \prod_{i=2}^{k-1}f_r(x_i)}{p(x_k) \prod_{i=2}^{k-1} p(\omega_i)}

This are roughly the basic mathematics and thoughts that go into writing a path tracer. I tried to write a simple as possible version of a path tracer, called stevieTracer. It is based on “ray tracing in one weekend” by Peter Shirley and is upgraded to support all the things described above.

StevieTracer

StevieTracer is a simple path tracer I wrote after a university course about photorealistic rendering.

Overview:

  • StevieTracer uses the ppm image format, because it is so easy
  • only supports rendering of spheres, but by using very big spheres the illusion of a wall can be achieved
  • supports a perfectly specular and a perfectly diffuse material
  • next event estimation
  • Russian roulette to terminate paths

lambertian brdf:

The lambertian brdf is a perfectly diffuse brdf, this means light is scattered in all directions equally.

f_r(\textbf{x}) = \frac{\rho}{\pi}

Note the brdf does not depend on any direction. \rho is the albedo of the surfacepoint \textbf{x} (the color of the object) and \pi is needed for energy conservation. Because the probability of every direction is the same, the pdf for this brdf simply is the probability of sampling a direction in the sphere.

p = \frac{1}{4\pi}

In this case it would be better to sample the cosine term in the rendering equation and change the sampling procedure and the pdf accordingly.

metal brdf:

The metal brdf is a perfectly specular material, like a really shiny metal or a mirror. This means for every incoming direction, there is only one outgoing direction to transport energy. Mathematically this is achieved by using a delta function for the brdf, so in the rendering equation the integral only gets evaluated for one single direction. In code this direction can only be sampled, so the brdf should never be evaluated for any other direction.

Volume Rendering

In volume rendering a vacuum is no longer assumed, and there exists different processes inside a medium, like absorption or scattering, that affect photons. These things are modeled stochastically with different emission, absorption and scattering coefficients.

The basis equation for volume rendering is the volume rendering equation. It is derived by integrating the radiance transfer equation and using surfaces as a boundary condition.

L(\textbf{x},\omega) = \int_{0}^{z} T(\textbf{x},\textbf{y}) \sigma(\textbf{y})L^m(\textbf{y},\omega)d\textbf{y}+T(\textbf{x},\textbf{z})L^S(\textbf{z},\omega)

with

L^S(\textbf{z}, \omega) = L_e^S(\textbf{z}, \omega) + \int_{S^2} \rho_S(\omega, \textbf{z}, -\omega^\prime)L(\textbf{z}, \omega^\prime) \vert \textbf{n}(\textbf{z}) \cdot \omega^\prime \vert d\omega^\prime

for vertices on the surface and

L^m(\textbf{x}, \omega) = \frac{\sigma _a(\textbf{x})}{\sigma _t(\textbf{x})}L_e^m(\textbf{x}, \omega) + \frac{\sigma _s(\textbf{x})}{\sigma _t(\textbf{x})} \int_{S^2} \rho_m(\omega, \textbf{x}, \omega^\prime)L(\textbf{x}, \omega^\prime)d\omega^\prime

for points inside the medium.

In heterogenous media it is very difficult to sample distances and evaluate the transmittance term T(\textbf{x},\textbf{y}), because of the integral in the exponent. In order to simplify the medium the null-scattering algorithm is used. The heterogenous medium gets filled up with fictous particles, until a constant density is reached. Sampling from this homogenous medium with the null-scattering particles is now possible with a rejection sampling method. But by doing so the pdf for the sample is lost. This is no problem in the normal case, because the pdf with the transmittance will cancel out anyways, but using multiple importance sampling is no longer possible. In the paper “a null-scattering path integral formulation of light tranpsort” written by Bailey Miller, Iliyan Georgiev and Wojciech Jarosz, they describe a way to shift the entire rendering procedure into a null-scattering environment, which allows them to calculate the pdf and use MIS.