Technology allows us to build fanciful kinds of worlds where toys can come alive, or fish can talk or monsters can roam the world. While it’s common knowledge that Pixar’s movies are the product of computer animation, you might be surprised by the amount of technology that was necessary to make them possible. In fact, Pixar is largely responsible for some of the most significant developments in computer graphics history. A lot of people look at the product and they say, oh it’s the artwork and so they focus on that.
But they don’t understand what went behind it. It’s a blend of the technology and the art that really makes it work. Behind every one of Pixar’s films and the visual effects of many iconic movies to come out in the past 30 years, is a piece of software called RenderMan. RenderMan is a renderer, the final tool at the end of a production pipeline that compiles all the 3D assets created for a film. It’s what translates the virtual camera the artists work with during the production of an animated or visual effect into the final image that we see of the movie.
There are lots of tools that companies and Pixar write to do the 3D modeling, the animation, motion, camera definitions, all flow into the very end of this software pipeline and that’s the renderer. Pixar got its start during computer graphics’ infant stages. When the most advanced CG images were primitive polygon shapes. At this time, the animation was strictly done through illustration. It was an artist’s medium, with every frame drawn by hand and photographed into a film reel. But in the late 70s, that began to change.
IN 1979 after the success of Star Wars, George Lucas wanted to bring high technology into the film industry and he was the only person in the film industry who thought this was a good thing to be spending money on. Ed Catmull was part of a small group of people in the industry who recognized the potential of computer graphics to create animated films. He joined LucasFilm to head its computer division and set to work solving some of the challenges in CGI, such as motion blur.
The engineers had to start square one, developing all the software and tools needed to create characters and animations, everything that’s commonplace today. During this process, they developed the precursor to RenderMan called REYES, which stood for “Renders Everything You Ever Saw.” They built this renderer and that’s what we used in early films like Star Trek II: The Wrath of Khan. We used it on Sherlock Holmes. We used it on our early short films, Luxor Jr.
Lucas’ group eventually spun off as an independent company purchased by Steve Jobs, which marked the beginning of Pixar. They would continue to go on making animated shorts, creating the iconic Luxo lamp characters, and receiving the first Academy Award for a computer generated short. With each and every project, they slowly evolved their tools and the REYES renderer, eventually creating RenderMan, which would go on to have a monumental impact on animation and visual effects.
As part of the sale to Steve Jobs, Lucas had the right to use Pixar’s technology for their VFX projects. Using it to create the groundbreaking CGI scenes for the genesis sequence in Star Trek II: The Wrath of Khan, the pseudopod creature in The Abyss, the T-1000 in Terminator 2: Judgment Day, and the dinosaurs of Jurassic Park. When Jurassic Park came out in ’93 it changed everything. Within about a two-year period, this industry which had been very digital averse threw the switch on everything, digital audio, digital video, and computer graphics.
People thought of us like this overnight phenomenon, but in fact, we’d been working on it for 20 years. And then Toy Story came out two years later which changed animation history. We always knew from day one, we wanted to do a feature. We wanted to get into that arena. We did all the short films because that was our stepping stone towards that. So when we finally got Toy Story out there that was sort of like, agh we did it. And then you go, well what’s next? Toy Story was a knockout success and launched the once-struggling company into a future filled with box office hits.
Pixar focused its efforts full-time on making more animated feature-length films. And with that came a slew of demands to push its animation technology to new heights. Every film represents both the directors and the artists wanting to push in some new technical direction. And so the rendering technology, and indeed all the tools, are sort of structured in a way that they can be remade and remade and remade over time to meet the artistic requirements that the directors and artists bring to the table.
New and more elaborate ways of specifying crowds or foliage or fur on animals. That stuff is constantly changing. RenderMan is basically our artist’s tool to create the images that we want. Every new thing they come up with it makes all these different images and worlds possible. It’s just sort of making the paintbrush better and better. A film like Monsters Inc. is like, OK, we want the main monster to have hair on him. Sullivan should have hair on him, and everyone’s like, yeah, we don’t know how to do hair.
OK, so we’re gonna have to R&D a project to figure out how to do hair. And then the next one is Finding Nemo and it’s underwater. How do we do underwater? OK, let’s figure out underwater. There are these huge hurdles that you’re getting over because none of those movies were possible in the beginning and there’s this big R&D project to figure it out. Making an animated film is an extremely time and labor-intensive process that draws upon the talents of hundreds of artists and computer engineers.
It’s on the order of 20,000 person weeks. We shoot for lower than that but we’ve had much worse actually. Once the story is figured out for a film, which can take months, even years itself. Artists start by creating all the different digital assets needed. Every character, prop and location must be sculpted and modeled. From there, the surfacing team adds all the relevant textures. Objects that are wood or metal are given the appropriate materials to reflect their composition.
Clothing is given a fabric appearance, skin an organic look, and so on. Character models have controls attached to their limbs so they move in a realistic way. Once all of these assets are complete, animators bring the characters and world to life, conveying the story through personality and action. Completed shots are then passed on to the lighting department, which gives the film its cinematic look and style. This is before lighting and you kind of get a sense of what the world looks like a little bit.
But then we throw the lighting in there and all of a sudden you can see WALL-E, where before you can’t even really see WALL-E, but you start to pull WALL-E out. For someone like me, the lighting DP might go on three years before it ever comes out in theaters and we create a whole three-dimensional world inside the computer. And if we don’t put lights in it actually comes out black because the computer is trying to mimic real life. And that we have little icons of lights we move around.
So if it’s sunset, I put the sun in, and I can make it kind of orange and put it at the horizon and start sort of building the scene up that way. The Coco set at Pixar was I think one of the most complicated that we’ve ever put into a film. And it was difficult because the cameras that they chose to fly through the world of the dead, or through the giant main Grand Central train station, or the big cemetery, each of those sets were hugely complex. In one scene in Coco, there were eight million light sources illuminating in this sort of interesting glow of candlelight and other light sources.
If you have more than a dozen lights it’s often a problem and so to scale that up into thousands and millions was a huge challenge for RenderMan. This is the shot where we have eight and a half million lights and it’s like, I don’t know, that’s probably eight million more lights than we’ve ever done in a scene before. Because the assets being worked with are extremely resource-intensive from a computing perspective, artists must work with lower-resolution versions of the film. The actual final image and look aren’t known until it’s processed through RenderMan.
Taking all the lighting, shading, and data aggregate into an image and turning it into a finished 2D frame. Compiling all these digital assets and processing them into their final form is an extremely intensive computing task. A typical frame takes hours to render, if you were going to render them all on your home computer it would take a couple of hundred years probably to make one of these movies. And so we solve that problem by having whole networks full of computers that all render simultaneously different frames at the same time.
The sequences being rendered by Pixar are so complex, even with state-of-the-art machinery, it can take days. An image is anywhere between 50 hours to 100 hours, on modern computers. That roughly translates to around 1,200 to 2,4000 hours of rendering for every second of a movie. I always try and remind people that we’re not pushing a button and a movie comes out the back end. That the huge number of people involved, from fine artists through the engineers that are writing RenderMan. It’s a very human-intensive task making one of these films.
The influence of Pixar’s work in computer animation had a profound impact on the industry, but its applications spread far beyond just entertainment. From the beginning, when we started building the renderer we were thinking about commercializing it. Making it available to anybody who wants to use it in doing visual effects, doing animation, doing scientific exploration. If they want to make images, it’s basically a way of making images. There’s a big advertising market. People use it in architecture visualization. One of our longest-standing customers is NASA.
There’s a group there that is tasked with visualizing some of the huge amounts of data they get back from their satellites. Over 30 years ago, Pixar started as a hyper-technical software company pioneering advances in computer graphics and animation. We’re able to tell the story we want to because technology enables us to.