Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Project 5: Eric Chiu #22

Open
wants to merge 5 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
49 changes: 37 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,26 +3,51 @@ WebGL Clustered and Forward+ Shading

**University of Pennsylvania, CIS 565: GPU Programming and Architecture, Project 5**

* (TODO) YOUR NAME HERE
* Tested on: (TODO) **Google Chrome 222.2** on
Windows 22, i7-2222 @ 2.22GHz 22GB, GTX 222 222MB (Moore 2222 Lab)
* Eric Chiu
* Tested on: Macbook Pro (Retina, 15-inch, Mid 2015), OS X El Capitan 10.11.6, 2.5 GHz Intel Core i7, AMD Radeon R9 M370X 2048 MB

### Live Online

[![](img/thumb.png)](http://TODO.github.io/Project5B-WebGL-Deferred-Shading)
## Live Online

### Demo Video/GIF
[![](images/clustered-phong.gif)](https://echiu1997.github.io/Project5-WebGL-Clustered-Deferred-Forward-Plus/)

[![](img/video.png)](TODO)
## Description

### (TODO: Your README)
This project implements different types of GPU rendering using Javascript, WebGL, and GPU hardware. Features include forward rendering, forward plus rendering, clustered deferred rendering, lambert shading, phong shading, toon shading, iridescent shading, and sobel shading.

*DO NOT* leave the README to the last minute! It is a crucial part of the
project, and we will not be able to grade you without a good README.
## Performance Analysis

This assignment has a considerable amount of performance analysis compared
to implementation work. Complete the implementation early to leave time!
The following chart shows the frames per second of the rendering as the number of lights increase for forward, forward plus, and clustered deferred rendering. Clearly, clustered deferred rendering is faster than forward and forward plus rendering, and forward plus rendering is faster than forward rendering. This is because forward plus rendering categorizes lights into clusters, and the fragment shader will only need to check lights that influence the cluster the geometry fragment is in. Clustered deferred rendering takes this a step further and introduces extra buffers that store 2D textures of depth, normals, and color information. Lighting will then be applied to the 2D textures instead of every geometry to produce the final render. This means that the lighting loop will only have to be run once rather than every geometry.

![](./images/forward-clustered.png)

The following chart shows the frames per second of the rendering as the number of lights increase for default clustered deferred rendering, and optimized deferred rendering using 2-component normals. Since normals will always have a magnitude of 1, we can compress the normal's x and y components into two vec4s by multiplying it by the view matrix before packing, and then multiplying these vec4s by the inverse view matrix in the shader to uncompress them. The performance difference is not super significant but using fewer buffers clearly improves performance. I believe that the difference in frames per second between the default and optimized implementations will remain linear as the number of lights increase.

![](./images/default-optimized.png)

## Forward Plus

![](./images/forward-plus.gif)

## Clustered Lambert Shading

![](./images/clustered-lambert.gif)

## Clustered Phong Shading

![](./images/clustered-phong.gif)

## Clustered Toon Shading

![](./images/clustered-toon.gif)

## Clustered Iridescent Shading

![](./images/clustered-iridescence.gif)

## Clustered Sobel Shading

![](./images/clustered-sobel.gif)

### Credits

Expand Down
Binary file added images/clustered-iridescence.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/clustered-lambert.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/clustered-phong.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/clustered-sobel.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/clustered-toon.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/default-optimized.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/forward-clustered.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/forward-plus.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
2 changes: 1 addition & 1 deletion src/init.js
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
// TODO: Change this to enable / disable debug mode
export const DEBUG = true && process.env.NODE_ENV === 'development';
export const DEBUG = false && process.env.NODE_ENV === 'development';

import DAT from 'dat.gui';
import WebGLDebug from 'webgl-debug';
Expand Down
85 changes: 81 additions & 4 deletions src/renderers/base.js
Original file line number Diff line number Diff line change
@@ -1,8 +1,16 @@
import TextureBuffer from './textureBuffer';
import { mat4, vec4, vec3, vec2 } from 'gl-matrix';
import { NUM_LIGHTS } from '../scene';

export const MAX_LIGHTS_PER_CLUSTER = 100;

export default class BaseRenderer {
function clamp(num, min, max)
{
return Math.min(Math.max(num, min), max);
}

export default class BaseRenderer
{
constructor(xSlices, ySlices, zSlices) {
// Create a texture to store cluster data. Each cluster stores the number of lights followed by the light indices
this._clusterTexture = new TextureBuffer(xSlices * ySlices * zSlices, MAX_LIGHTS_PER_CLUSTER + 1);
Expand All @@ -11,9 +19,8 @@ export default class BaseRenderer {
this._zSlices = zSlices;
}

updateClusters(camera, viewMatrix, scene) {
// TODO: Update the cluster texture with the count and indices of the lights in each cluster
// This will take some time. The math is nontrivial...
updateClusters(camera, viewMatrix, scene)
{

for (let z = 0; z < this._zSlices; ++z) {
for (let y = 0; y < this._ySlices; ++y) {
Expand All @@ -25,6 +32,76 @@ export default class BaseRenderer {
}
}



// NEW: Update the cluster texture with the count and indices of the lights in each cluster
// This will take some time. The math is nontrivial...

var halfHeight = Math.tan(camera.fov / 2.0 * (Math.PI/180.0));
var halfWidth = halfHeight * camera.aspect;

for (let lightIndex = 0; lightIndex < NUM_LIGHTS; ++lightIndex)
{
// get light information
let light = scene.lights[lightIndex];
let lightRadius = light.radius;
var lightPos = vec4.fromValues(light.position[0], light.position[1], light.position[2], 1.0);
vec4.transformMat4(lightPos, lightPos, viewMatrix);
lightPos[2] *= -1.0;

// slice in x direction
let xDim = halfWidth * lightPos[2] * 2.0;
let xStep = xDim / this._xSlices;
let xStart = Math.floor((lightPos[0] - lightRadius + (xDim / 2.0)) / xStep) - 1;
let xEnd = Math.floor((lightPos[0] + lightRadius + (xDim / 2.0)) / xStep) + 1;

// slice in y direction
let yDim = halfHeight * lightPos[2] * 2.0;
let yStep = yDim / this._ySlices;
let yStart = Math.floor((lightPos[1] - lightRadius + (yDim / 2.0)) / yStep);
let yEnd = Math.floor((lightPos[1] + lightRadius + (yDim / 2.0)) / yStep);

// slice in z direction
let zDim = (camera.far - camera.near);
let zStep = zDim / this._zSlices;
let zStart = Math.floor((lightPos[2] - lightRadius) / zStep);
let zEnd = Math.floor((lightPos[2] + lightRadius) / zStep);

// make sure start and end x,y,z is within x,y,z slices
if((zStart < 0 && zEnd < 0) || (zStart >= this._zSlices && zEnd >= this._zSlices)) continue;
if((yStart < 0 && yEnd < 0) || (yStart >= this._ySlices && yEnd >= this._ySlices)) continue;
if((xStart < 0 && xEnd < 0) || (xStart >= this._xSlices && xEnd >= this._xSlices)) continue;

// clamp the start and end x,y,z
xStart = clamp(xStart, 0, this._xSlices-1);
xEnd = clamp(xEnd, 0, this._xSlices-1);
yStart = clamp(yStart, 0, this._ySlices-1);
yEnd = clamp(yEnd, 0, this._ySlices-1);
zStart = clamp(zStart, 0, this._zSlices-1);
zEnd = clamp(zEnd, 0, this._zSlices-1);

// iterate through start and end x,y,z
for (let z = zStart; z <= zEnd; z++) {
for (let y = yStart; y <= yEnd; y++) {
for (let x = xStart; x <= xEnd; x++)
{
let clusterIndex = x + (y * this._xSlices) + (z * this._ySlices * this._xSlices);
let numLightIndex = this._clusterTexture.bufferIndex(clusterIndex, 0);
let numLights = 1 + this._clusterTexture.buffer[numLightIndex];

if (numLights <= MAX_LIGHTS_PER_CLUSTER) {
let col = Math.floor(numLights * 0.25);
let row = Math.floor(numLights % 4);
this._clusterTexture.buffer[numLightIndex] = numLights;
this._clusterTexture.buffer[this._clusterTexture.bufferIndex(clusterIndex, col) + row] = lightIndex;
}
}
}
}
}



this._clusterTexture.update();
}
}
63 changes: 52 additions & 11 deletions src/renderers/clustered.js
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import { gl, WEBGL_draw_buffers, canvas } from '../init';
import { mat4, vec4 } from 'gl-matrix';
import { mat4, vec4 ,vec3, vec2} from 'gl-matrix';
import { loadShaderProgram, renderFullscreenQuad } from '../utils';
import { NUM_LIGHTS } from '../scene';
import toTextureVert from '../shaders/deferredToTexture.vert.glsl';
Expand All @@ -8,28 +8,42 @@ import QuadVertSource from '../shaders/quad.vert.glsl';
import fsSource from '../shaders/deferred.frag.glsl.js';
import TextureBuffer from './textureBuffer';
import BaseRenderer from './base';
import {MAX_LIGHTS_PER_CLUSTER} from "./base";

export const NUM_GBUFFERS = 4;
export const NUM_GBUFFERS = 2;

export default class ClusteredRenderer extends BaseRenderer {
constructor(xSlices, ySlices, zSlices) {
super(xSlices, ySlices, zSlices);

this.setupDrawBuffers(canvas.width, canvas.height);

// Create a texture to store light data
this._lightTexture = new TextureBuffer(NUM_LIGHTS, 8);

this._progCopy = loadShaderProgram(toTextureVert, toTextureFrag, {
uniforms: ['u_viewProjectionMatrix', 'u_colmap', 'u_normap'],
uniforms: ['u_viewProjectionMatrix','u_viewMatrix', 'u_colmap', 'u_normap'],
attribs: ['a_position', 'a_normal', 'a_uv'],
});

this._progShade = loadShaderProgram(QuadVertSource, fsSource({
numLights: NUM_LIGHTS,
numGBuffers: NUM_GBUFFERS,
xSlices: xSlices, ySlices: ySlices, zSlices: zSlices,
maxLightsPerCluster: MAX_LIGHTS_PER_CLUSTER,

}), {
uniforms: ['u_gbuffers[0]', 'u_gbuffers[1]', 'u_gbuffers[2]', 'u_gbuffers[3]'],
uniforms: ['u_gbuffers[0]',
'u_gbuffers[1]',
'u_gbuffers[2]',
'u_viewMatrix',
'u_invViewMatrix',
'u_clusterbuffer',
'u_lightbuffer',
'u_screenWidth',
'u_screenHeight',
'u_camera_near',
'u_camera_far'],
attribs: ['a_uv'],
});

Expand All @@ -43,7 +57,7 @@ export default class ClusteredRenderer extends BaseRenderer {
this._height = height;

this._fbo = gl.createFramebuffer();

//Create, bind, and store a depth target texture for the FBO
this._depthTex = gl.createTexture();
gl.bindTexture(gl.TEXTURE_2D, this._depthTex);
Expand Down Expand Up @@ -71,7 +85,7 @@ export default class ClusteredRenderer extends BaseRenderer {
gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, width, height, 0, gl.RGBA, gl.FLOAT, null);
gl.bindTexture(gl.TEXTURE_2D, null);

gl.framebufferTexture2D(gl.FRAMEBUFFER, attachments[i], gl.TEXTURE_2D, this._gbuffers[i], 0);
gl.framebufferTexture2D(gl.FRAMEBUFFER, attachments[i], gl.TEXTURE_2D, this._gbuffers[i], 0);
}

if (gl.checkFramebufferStatus(gl.FRAMEBUFFER) != gl.FRAMEBUFFER_COMPLETE) {
Expand Down Expand Up @@ -108,6 +122,9 @@ export default class ClusteredRenderer extends BaseRenderer {
mat4.invert(this._viewMatrix, camera.matrixWorld.elements);
mat4.copy(this._projectionMatrix, camera.projectionMatrix.elements);
mat4.multiply(this._viewProjectionMatrix, this._projectionMatrix, this._viewMatrix);
// NEW: find the inverse view matrix
let invViewMatrix = mat4.create();
mat4.invert(invViewMatrix,this._viewMatrix);

// Render to the whole screen
gl.viewport(0, 0, canvas.width, canvas.height);
Expand All @@ -123,10 +140,12 @@ export default class ClusteredRenderer extends BaseRenderer {

// Upload the camera matrix
gl.uniformMatrix4fv(this._progCopy.u_viewProjectionMatrix, false, this._viewProjectionMatrix);
// NEW: upload the view matrix
gl.uniformMatrix4fv(this._progCopy.u_viewMatrix, false, this._viewMatrix);

// Draw the scene. This function takes the shader program so that the model's textures can be bound to the right inputs
scene.draw(this._progCopy);

// Update the buffer used to populate the texture packed with light data
for (let i = 0; i < NUM_LIGHTS; ++i) {
this._lightTexture.buffer[this._lightTexture.bufferIndex(i, 0) + 0] = scene.lights[i].position[0];
Expand All @@ -153,7 +172,29 @@ export default class ClusteredRenderer extends BaseRenderer {
// Use this shader program
gl.useProgram(this._progShade.glShaderProgram);

// TODO: Bind any other shader inputs

// NEW

// Upload camera matrices
gl.uniformMatrix4fv(this._progShade.u_viewMatrix, false, this._viewMatrix);
gl.uniformMatrix4fv(this._progShade.u_invViewMatrix, false, invViewMatrix);

// Set the light texture as a uniform input to the shader
gl.activeTexture(gl.TEXTURE2);
gl.bindTexture(gl.TEXTURE_2D,this._lightTexture.glTexture);
gl.uniform1i(this._progShade.u_lightbuffer,2);

// Set the cluster texture as a uniform input to the shader
gl.activeTexture(gl.TEXTURE3);
gl.bindTexture(gl.TEXTURE_2D, this._clusterTexture.glTexture);
gl.uniform1i(this._progShade.u_clusterbuffer, 3);

// Bind any other shader inputs
gl.uniform1f(this._progShade.u_screenWidth, canvas.width);
gl.uniform1f(this._progShade.u_screenHeight, canvas.height);
gl.uniform1f(this._progShade.u_camera_far,camera.far);
gl.uniform1f(this._progShade.u_camera_near,camera.near);


// Bind g-buffers
const firstGBufferBinding = 0; // You may have to change this if you use other texture slots
Expand Down
28 changes: 25 additions & 3 deletions src/renderers/forwardPlus.js
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ import vsSource from '../shaders/forwardPlus.vert.glsl';
import fsSource from '../shaders/forwardPlus.frag.glsl.js';
import TextureBuffer from './textureBuffer';
import BaseRenderer from './base';
import { MAX_LIGHTS_PER_CLUSTER } from './base';

export default class ForwardPlusRenderer extends BaseRenderer {
constructor(xSlices, ySlices, zSlices) {
Expand All @@ -16,9 +17,24 @@ export default class ForwardPlusRenderer extends BaseRenderer {

this._shaderProgram = loadShaderProgram(vsSource, fsSource({
numLights: NUM_LIGHTS,
xSlices: xSlices,
ySlices: ySlices,
zSlices: zSlices,
maxLightsPerCluster: MAX_LIGHTS_PER_CLUSTER,
}), {
uniforms: ['u_viewProjectionMatrix', 'u_colmap', 'u_normap', 'u_lightbuffer', 'u_clusterbuffer'],
attribs: ['a_position', 'a_normal', 'a_uv'],
uniforms: ['u_viewProjectionMatrix',
'u_viewMatrix',
'u_colmap',
'u_normap',
'u_lightbuffer',
'u_clusterbuffer',
'u_screenWidth',
'u_screenHeight',
'u_camera_near',
'u_camera_far'],
attribs: ['a_position',
'a_normal',
'a_uv'],
});

this._projectionMatrix = mat4.create();
Expand Down Expand Up @@ -64,6 +80,8 @@ export default class ForwardPlusRenderer extends BaseRenderer {

// Upload the camera matrix
gl.uniformMatrix4fv(this._shaderProgram.u_viewProjectionMatrix, false, this._viewProjectionMatrix);
// NEW: Upload view matrix
gl.uniformMatrix4fv(this._shaderProgram.u_viewMatrix, false, this._viewMatrix);

// Set the light texture as a uniform input to the shader
gl.activeTexture(gl.TEXTURE2);
Expand All @@ -75,7 +93,11 @@ export default class ForwardPlusRenderer extends BaseRenderer {
gl.bindTexture(gl.TEXTURE_2D, this._clusterTexture.glTexture);
gl.uniform1i(this._shaderProgram.u_clusterbuffer, 3);

// TODO: Bind any other shader inputs
// NEW: Bind any other shader inputs
gl.uniform1f(this._shaderProgram.u_screenWidth, canvas.width);
gl.uniform1f(this._shaderProgram.u_screenHeight, canvas.height);
gl.uniform1f(this._shaderProgram.u_camera_near, camera.near);
gl.uniform1f(this._shaderProgram.u_camera_far, camera.far);

// Draw the scene. This function takes the shader program so that the model's textures can be bound to the right inputs
scene.draw(this._shaderProgram);
Expand Down
4 changes: 2 additions & 2 deletions src/scene.js
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,10 @@ import { gl } from './init';
export const LIGHT_MIN = [-14, 0, -6];
export const LIGHT_MAX = [14, 20, 6];
export const LIGHT_RADIUS = 5.0;
export const LIGHT_DT = -0.03;
export const LIGHT_DT = -0.1;

// TODO: This controls the number of lights
export const NUM_LIGHTS = 100;
export const NUM_LIGHTS = 150;

class Scene {
constructor() {
Expand Down
Loading