Soma Games’ PC/console title, The Lost Legends of Redwall: The Scout is set within a wide expanse of deep forest. It’s the kind of design decision that sounds easy on paper but led to several issues.
Protip: Forests are hard.
There are tons of objects and you either need a crazy high poly count or lots of alpha cards to get those silhouettes. Lots of alpha cards however, mean lots of alpha overdraw. Plus, the nature of our small team means we couldn’t spend a ton of time making unique tree assets. By the end of our first iteration, we had several scenes that chugged along with hopelessly low frame rates, and with lots of muddy shapes and colors. We needed to try something else.
It was around this time that I was also trying to figure out how to make our ice look better. While that effect is a story for another time, in the course of that endeavor I came across the technique used in the Spider-Man game to give some depth to the windows in the buildings as you swing by. It was clever, and while subtle, that little bit of motion on your periphery lends a lot of life to the city.
I started thinking about this and realized that it could work on our trees. The problem with tree assets that use a solid shape is that they’re typically too cartoony for our art style, and look hard like a tin toy, not something soft where you could sink your hand into the foliage.
Something we strived to achieve in our game was a sense of warmth, of building a place you wanted to be in yourself. Hard-surface trees just didn’t feel like that to me. But what if I could get the visual benefits of the strong shape and the performance benefits of solid opacity, while ALSO giving it a sense of depth and softness?
I took an old concept of a small cypress tree, almost a Christmas tree, and whipped up a prototype mesh and texture. The first part of the shader, and frankly the foundation of the entire effect, was the parallax. I was going to take the effect that side-scrollers use on the background, and slap it on my tree to really make it feel like the branches disappeared into its depths. Remarkably, it worked! Especially up close, it really felt like there was a depth to the tree, and it shifted with your camera angle. Then I took a few steps back to really see the whole thing…. And my first problem showed itself. The tree’s silhouette was super hard and solid. Sure, the bits right in front looked like they had a lot of depth, but that illusion was quickly broken as soon as you could see the whole thing or even its side.
So I decided to introduce some cards to disguise the edges. Yes, this did introduce the alpha back in. However, since I wasn’t using those cards to give the tree its shape, just to soften the edges, I could use much smaller cards, and the overdraw would still be manageable. So that meant going back to Blender to add some cards along the edges, then back to Unity to map a new material on them, which is where I got some good news and bad news. On the upside, the cards did their job, and the silhouette was much more natural now! On the downside, the cards obscured the parallaxing effect in the middle of the tree… quite the pickle.
So, I next applied an inverse fresnel to the opacity, and viola – now when the card approaches the center of the tree, it’s more parallel to the camera viewing angle and is therefore hidden. When it approaches the edge of the tree, it becomes perpendicular to the viewing angle and shows itself! I added a gradient to its alpha channel so it would appear to grow instead of pop and the cards were finished.
You would think this would be the end of it, because the technique is working now, right? Well no, of course not. Being a creative person, I understand the impulse to not leave “well enough” alone, so I really should have been able to predict what came next. After proving the technique was possible to my Art Director Erin Marantette and our Project Head Chris Skaggs, the go-ahead was given to concept out a bunch of different trees that could benefit from this technique.
The original idea was made with a tree like the cypress in mind, which meant fairly uniform branches equally spread apart. Unfortunately we have some very talented concept artists on our team who came up with some strikingly irregular trees that I now had to create. It was challenging because these other trees had a variety of branches, in a variety of different patterns. The Douglas fir and Scots pine in particular introduced a slew of new difficulties. These branches were very directional, and so the cards used to obscure the edges couldn’t be little puffs like I used on my mini cypress (now named the “ice cream cone” among the team). The puffs seemed too random, and it needed to look cohesive. So I got a texture of a single branch card for a Douglas fir and split it down the middle, and laid those down along an edge of one of our canopy meshes for the Douglas fir tree.
This worked, but was labor intensive, so I needed a way to do it MUCH quicker than hand-snapping a bunch of verts together and tweaking them individually. So I figured out how to convert an edge loop to a bezier curve, and then mold the card strip to that curve. Having it duplicate itself to match the length of the curve was easy at that point, and so we just needed to get the edge loops duplicated out and it was fairly straightforward after that.
So we finally had a variety of trees, which had a solid canopy, an illusion of depth, and an edge of cards to soften their silhouette. Everything was perfect, right? Turns out, I had forgotten this game takes place in winter. They all needed snow. And of course you can’t just add snow wholesale to the top faces, because it’s supposed to be branches, not a solid ball. After some additional experimentation I decided to add snow to the top faces, but filter out the gaps in the texture where you would see the under layers when the snow lessens. For the silhouette-breaking cards, I had to filter by the red channel of the vertex colors for additional control.
And that’s basically the whole shebang. I experimented with some additional variations that had different textures for the various layers of the parallax within the tree, so you could have dead branches farther in and snowy branches at the top, and some other ways to use vertex coloring. But at some point I found the complexity of features became too much for the artists, and scaled back to the old adage of simple is better. Or at least, relatively simple. I found that it really works best with softer, clumpier shapes. It excels with bushes and brambles and hedges and that sort of vegetation. But there’s a lot of possibilities that I think we only just barely got to tap into. I’m excited to keep pushing this technique to new levels, polishing the tools for the artists to use it, and refining its rough edges. And I also hope that someone else sees this article and that it inspires them to do something creative with this shader technique. That’s probably my favorite part of the Tech Art community – shared, creative inspiration.
Written by Gavin Nichols
