Toon Lighting
lighting
Part 10: Toon Lighting, Shaders For People Who Don’t Know How To Shader
Last time we learned how to use a dot product to add a fresnel rim to our shader. Now we’re going to learn how to use a dot product to do cel shading! If you don’t remember what a dot product is, reread Part 8 so you’re all set for this lesson!
Cel shading, also called toon shading, is a kind of technique that makes it so that there are hard cutoffs between light and dark on an object, like how shading is done in a cartoon. We can do this by writing a custom lighting function.
In forward rendering (which we are using right now) a lighting function is a calculation that is run for every light that hits the 3d object in question. Once we’ve applied our textures and normal maps and all that other good stuff, this is what’s used to apply darkness or brightness from the light to determine the final color of our 3d object.
Setup
If you’ve been following along, you should have the XibStandard shader, but if you don’t, download it now. (It’s also attached to this post as XibStandard.shader.) Copy XibStandard by clicking it in your project files and pressing ctrl + D. Name the new shader XibToon. Copy Claire’s current material the same way, and name it ClaireToon. Drag the ClaireToon material into the material slot on Girl_Body_Geo. Open XibToon in Visual Studio. Change the first line to
Shader “Xibanya/XibToon”
Save and go back to Unity. In the ClaireToon material, select XibToon from the dropdown. You should end up with something like this:
Custom Lighting
This entire time we’ve had a line in our shaders near the top that says this:
` #pragma surface surf Standard fullforwardshadows
this actually means something, let’s break it down!
#pragma is a pre-compiler directive, putting it at the start of the line signals to Unity that we’re about to tell it how it should treat this shader
surface means it’s a surface shader (there are other kinds of shader we haven’t looked at yet!)
surf is the name of the surface function (so you could rename it here if you wanted to as long as you made sure to change the name of the surface function down below too!)
Standard is the name of the lighting function that should be used
anything after that is options, so fullforwardshadows is just one of many options you could put there. You can see a full list of options in the Unity docs here. You can throw on as many as you want, or have none.
This entire time we have been using the Standard lighting model. This is the built-in default lighting model, and because it’s built-in, we can tell the shader to use it without having it in the file itself. If we want to use our own lighting function, we’ll have to add it to our shader code.
In this line in our XibToon shader, let’s replace Standard with Toon so the line looks like this.
#pragma surface surf Toon fullforwardshadows
By doing that, we’ve told Unity to look for a lighting function called LightingToon so now we need to add that or this shader will return errors and not work.
Let’s stub this out first. Add this LightingToon function below your #pragmas
I really wish Patreon let me have codeblocks! Since it doesn’t, if you’re lazy, copy and paste this, then add line breaks to taste.
` half4 LightingToon(SurfaceOutput s, half3 lightDir, half atten) { return half4(s.Albedo, 1); }
down in your surface function, replace SurfaceOutputStandard with SurfaceOutput
Save and go back to Unity to make sure you don’t get any errors. You should have this.
If you got errors look over your code carefully and make sure the parts you’ve altered so far look just like mine. (It can be easy to mess up here by forgetting to change something.)
Once you’re situated have a look at Claire. If you have the rim on, make the rim color black. You’ll find she has no shadowiness on her at all – she has the same brightness all over. In our lighting function we just said to keep the model the same color that it was before we tried to put lighting on it, which is like not bothering with lighting or shadow at all. Whoever made the texture was a skilled artist, as they made her neck darker than her face on purpose to make it look like there was shading even if the model was in a situation like this, so if you want to really see how there’s not any lighting being applied, click the texture slot in the material properties and hit backspace to remove it.
Interestingly this is often what it looks like when our shader has an error. That’s because an error often stops the lighting function from being run! (Then hit ctrl + z to put it the texture back!)
OK so what do we even want out of a lighting function? Well we want the parts being hit by light to be brighter and we want the parts not being hit by light to be darker. How can we do that? Well hm, last time when we were making the rim, the first thing we did was make the parts facing us directly lighter with the view direction, so if we had the light direction, we could do basically the same thing!
In our custom lighting function, we have our light direction given to us in the parameters, so we have that to start with. We also have SurfaceOutput being given to us, which will have all the same info stored in it as it does down in the surface function, which means we can get the direction any given pixel we’re coloring is facing in. So let’s add this to the top of our lighting function:
` half d = dot(s.Normal, lightDir);
Remember, our dot product will be bigger the more overlap there is between the directions and smaller the less overlap there is. So we could just multiply s.Albedo (our main color) by that and call it a day!
Well, not really. There’s some weird stuff happening here. I don’t know that this is totally accurate.
Like look under her chin, I think the shadowy areas kinda should be pushed back a bit. Let’s tweak our dot product a little bit. Change the first line to
` half d = dot(s.Normal, lightDir) * 0.5 + 0.5;
by making the dot product half value then adding 0.5, we’re basically squishing the possible ranges to be between 0.5 and 1. (because what would have been 0 normally is now 0.5, and what would have been 1 is still 1.)
OK, looks more reasonable.
But wait, there’s a problem. Look at her eyes! They’re purple! Why? Because my directional light is purple!
Remember, the eyes are on a different mesh that’s still using the standard shader! So we need to make sure our custom lighting function uses light color too!
Unity surface shaders basically invisibly add a lot of other code, which is extremely convenient because it lets us use a lot of cool built in functions without having to have them inside our shader files. (This is one of the reasons I’ve started us out with surface shaders instead of other kinds of shaders!) In the invisible code is a variable _LightColor0 that holds the light color, so we can use that right now!
Update your lighting function to look like this!
We’re declaring a half4 c so that our return line doesn’t get super long and hard to read. Since this function is a half4 instead of void, we have to return a half4 or the shader will throw errors. The half4 returned is the final color that will be applied to our 3d object!
_LightColor0, unlike the other variables we use in our shader, isn’t defined outside of a function – not by us anyway. It actually is defined, but in the invisible code we don’t see. We’ll return to this idea later when we start looking at vertex shaders. Anyway, save and head back to Unity.
Yay we have light color now!
Ahaa but we actually have another problem, although it’s not immediately obvious. You can see it if you create a point light with a short range near Claire.
The point light is being treated like a square, where the pixel is either inside the range or outside the range, and if it’s inside it has the light applied in full, and outside the light isn’t applied at all. That hard cutoff. That’s NOT the kind of hard cutoff we want for toon lighting because it doesn’t even follow the shape of the 3d object!
This is what the parameter atten is for! It stands for attenuation, and it means how close we are to the light. If this lighting function is being run on a directional light, attenuation is always 1, because directional lights in Unity have the same intensity no matter where they are, but if the lighting function is being run on a point or spot light, then the attenuation will be something between 0 and 1! Let’s update our lighting function to use it!
c.rgb = s.Albedo * d * _LightColor0.rgb * atten;
Ay there we go!
Well that’s great and all, but that is not toon lighting. So let’s get to work on that!
Toon Lighting
To review, a dot product is gonna be a value between 0 and 1 that shows how much overlap between directions there is. We need to establish a cutoff point - if the value is higher than this, there’s no shadowiness, and if it’s lower, there’s maximum shadowiness. Well, we do have the step function. Remember, with the step function, we put in two numbers - if the first is bigger, the result is 0. If the second is bigger, the result is 1.
If I do this
we get this
it’s very uh, Sin City or something. I’ll be honest I’m not crazy ‘bout it. Kinda harsh, don’t you think? Let’s add some sliders to adjust the shadow size and shadow smoothness like we did with the rim!
Since we’ll use these in the lighting function, we will have to declare them above the lighting function, since unlike in C#, the order in which things are defined matters in CG shader code (which is what we are using.)
Then we can modify our lighting function to let us adjust the shadow coverage (like we use _RimPower down in the surface function!) and the smoothness (like how we use _RimSmooth!)
Yeah there we go!
That’s all well and good, but the shadow being totally black is a bit harsh. Let’s have a shadow color to use as our maximum shadowiness. Add _ShadowColor to our properties like so:
_ShadowColor(“Shadow Color”, Color) = (0,0,0,1)
and declare it above the lighting function
half3 _ShadowColor;
Since multiplying anything by zero makes it zero, we don’t want to multiply our shadow color by the “shadow” variable we made from the dot product, because if the shadow variable is 0, it’ll just make the color black. Instead, we can use a lerp function with _ShadowColor as the lowest value in the gradient.
half3 shadowColor = lerp(_ShadowColor, half3(1, 1, 1), shadow);
Remember, half3(1,1,1) means white!
Yay! Now we have ourselves some basic toon lighting! There are a lot of really neat variations we can do on this, but for now, throw some lights into the scene and have some fun looking at how cool this stylized lighting looks!
The final shader code is attached to this post as XibToon.shader.
Have any questions, comments, or corrections? Let me know on Twitter or Discord! And if I helped you out, please consider becoming a patron!
This tutorial is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License. The code shared with this tutorial is licensed under a Creative Commons Attribution 4.0 International License.