Tutorial request: Scene optimization
I've noticed that some of the scenes we have received to the farm have been so well optimized that it's hard to believe that they would be producing graphics of such high quality. Unfortunately most of the scenes we receive are not that well optimized, so I now call you artists to drop a hint or two to this thread. I wrote something myself, but I'm hardly an artist and don't know the tricks that well. If you happen to have links to optimization tutorials already existing, then paste the urls here. Once we have some advice I'll make a page to Renderfarm.fi.
- Consider using compressed format textures (png or even jpg) instead of tga or bmp. However, keep in mind that jpg doesn't have alpha - use png if you use the texture for transparency.
- Use smaller resolution for textures. Can you use 1024x1024 instead of 2048x2048?
- Can you do some of the textures using the Blender's own procedural textures?
- Lots of lights -> lots of shadows -> lots of calculation. Use lamps wisely.
- Most likely you have your whole scene in the .blend file. You are going to film multiple shots using the same 3D scene. You can remove the data that isn't going to be shown in the current clip and split them to their own .blend files for rendering individually.
- Generally you have to "fake" as much as possible. Can you use a flat 2D plane as the background instead of complex 3D geometry? Do you think you can use texture based transparency on that fence instead of arraying the geometry?
- Enable free tex in render settings. This means that the texture files are released from the memory when they are not needed.
- Set octree resolution to low. The lower it is the longer it takes to render, but the less memory the render requires.
- Play with the raytrace acceleration structs. <- Someone explain these, for I have no idea how these work.
- Consider decreasing OSA to 5 from 8.
- Split the rendering to multiple parts per frame using the "Xparts, Yparts" -boxes. This allows one frame to be rendered on multiple computers at the same time. Use the powers of two for the setting (1x1, 2x2, 4x4, 8x8, 16x16). Think a bit before deciding the amount of parts, though. If you split a light scene to too little parts you are actually causing the render to take longer due to the slowness of file transfers (we are moving stuff over the Internet after all). Generally you should split most of the scenes to (less than hour) 4x4, a bit heavy (hour or so) to 8x8, and really heavy (multiple hours) to 16x16. Do remember that if you REQUIRE subsurface scattering or FILTER type composite nodes, then you can't use parts.
What I'm looking for in this thread is mostly small pieces of advice and tricks that you artists have used before. You don't have to start writing a big and large tutorial.
Procedural textures sound good, and it is probably a very good way to get a few megabytes of memory usage out.
What do you mean by faking by compositing? That sounds interesting :)
A basic scene setup might be both a bad and a good idea. Some people will probably want to do everything on their own and don't want the ground work to be there for them. There also might be issues when Blender updates. It might work if we save the scene in the Renderfarm.fi. Then it would always open with the application and there would be no need for all that work.
Speaking of which, what do you always have to do? What is the biggest problem when you are submitting scenes here? What tasks do you always have to do to each of the sessions?
We should probably also write a bit about particle simulation baking and some general text about how distributed computing works with Blender, including the limitations in more detail than in the preparation web page. There's a lot of material and it probably isn't that much of a task to write a summary to Renderfarm.fi.
EDIT: Oops, some quite old stuff in the first post which I wrote here. Of course, you don't use XParts & YParts, but the part setting in the Renderfarm.fi panel or web page
Ok, I got you wrong. Writing like short articles about one or two topics each sounds good to me!
I will start then with procedurals, going for compositing next, ASAP (could need couple of days)
The preparation page already looks better. A list of do's and dont's as far as distributed rendering would be a great reference. Specifically which nodes and/or settings will give a bad render.
I start with a first quick "trick" on image textures:
About 80% of the time I use image textures that have to repeat, like a wooden floor, stone material for a mountain, only bout 20% are really individual textures, like a face texture.
Making a texture "looping" often looks bad when made by hand, cause you see, that it is looped. Not because of the seems but due to the repeating structur. Commonly people then just create very big textures, so this effect becomes minimized.
There´s a lot easier solution that has the big advantage (especially for renderfarm) of getting rather small textures, that look great.
You need:
1. gimp
2. gimp-resynthesizer plugin
http://www.logarithmic.net/pfh/resynthesizer
(on ubuntu or debian its in the repos)
The plugin is (if installed correctly) found in the filters->map menu.
So try to load a wood (or anything else) texture in gimp, crop the image to a piece of 1024x1024 pixels, then just run the plugin with the presets. Most of the time (not always) you get a texture, which is seemles AND where the repetition aint visible (at least not that much).
This got some limitations, e.g. very linear structures like a brick wall dont work too well.
Now what I do often then is mixing an image texture with procedurales in blender, like adding a cloud tex with mostly grey and black colors for adding some dirt, darker areas etc.
Then I use another procedural as "stencil" (e.g. a blend gradient, or another cloud with different settings) to mask out the other procedural, so it´s only projected on some parts of my wooden floor.
Those kind of things are very important to distract the eye of the beholder and to get away from an artificial look.
more to come... :)
(p.s. it´ s really hard to get some kind of text formating in the editor)
Steps I always have to make before submitting to the farm:
- Pack all external files (should be on as default setting)
- Check for unused textures that came in during creation etc. and replace them with a 1 pixel blank texture (I never found an option to actually remove images from blender) - would be great if there was an option that just throws away images, which aren´t part of any material of the rendered scene objects, or a world texture
- check if compositing is on (should be on by default, if you dont have any compositing, nothing bad will happen, usually you either dont have a compositing or you want it to be rendered)
- check for parts and threads (this really is anoying, cause you have to change it back and forward, if you do another testrender after you twice checked all other things, I think threads should be overwritten by the farm in any case to "1" and parts should be adjustable online after the scene came in for automated check or alike)
-... (I´ ll complete this post next time I submit a scene, I think there are some more things that just dont come to my mind right now)
- I always have to change quality settings to the final rendering quality, like AA and Raytracing samples. This is too something I overlook frequently.
- Replace all image textures that could have copyright on with cc, free or my own textures. Thats a lot of work sometimes. Finding good textures/images is hard enough, if you try to find only cc-by, that also fit on your scene, it´s even harder. There are lot of texture sites out there, nearly all of them say "textures free to use for both commercial and private use. Only thing forbidden, is redistributing or selling them, like a texture-pack". But this aint a cc-licence, also redistributing could be understood as what is done here...
- Gather all cc-licence / copyright information/links for the description
I think thats mainly about it. The list maybe doesnt seem long, but for me it is (takes maybe 3h++). Main problem is, when I make a mistake somewhere, chances are good, that I overlook some things on the next round of "checkup", like changing quality settings back to final. And over and over again....
There aint no tool that could help me, like an alert("you have your rendersize set to 25% and switched of compositing, is that correct?"). Sometimes it feels like the checkup pilots make before plane starts :P
My proposal for this would be a "scene check" button (I know there is already something like that) that does only generate a simple text box, where only renderfarm relevant settings are displayed as conclusion, like:
Resolution: 640x480 100%
AA: 16
Raytracing-Samples: 8
(Not empty) Textures used: pic1.jpg pic12.jpg ..
Missing textures: pic4.jpg...
Packed textures: yes/NO!
Compositing: on/off
Parts: 16
Threads: 1
Scene description: Lorem ipsum...
maybe with some red coloring where a major setting is wrong, like threads was "4"
Prodigal_son once wrote to keith bruns cause of the hdr-maps and got the permission for renderfarm.
There are many free texture sites out there, maybe you can talk to some of them that they extend there licence for renderfarm distribution and we make a list of those resources.
Thanks a lot for all this information, Manu. It's useful and will surely help a lot of people facing these same problems.
The error messages definitely need to be outputted differently. On OS X people usually start their application using the .app shortcut and do not receive a console at all, which makes it a bit difficult to see the problems. I believe this should be the main point when we are creating the next versions of the script.
Printing all the scene information sounds good. That should be easy to add.
Hey!
I will write down my knowledge regarding those points.
But creating a whole tutorial along with your facts, seems like a pretty big task. I could write 2 pages only about procedural textures, as well as faking by compositing.
Besides I´m really not too good in writings tutorials and my english got some flaws. I mean there are people like Andrew Price (blenderguru.com) who are great "teachers", and there are lots of tutorials out there which are hard to follow and understand - to break it down short, this should be written by people who are better at that than me :D
But I have an additional idea you may find interesting:
After we gathered all points, we could create one ore more renderfarm basic scene setups for people to download.
This does not only help preventing some human errors people like me make, when submitting a scene to renderfarm, but also safes LOTS of time. There are some settings I have to go through everytime I submit a scene here,it´ s time consuming and naturally I oversee some important setting from time to time.