HOW TO SETUP AN OPEN SOURCE RENDERFARM WITH WINDOWS

(Version Française...)

There must be some translation errors and I apologize for it. I'll try to correct them as soon as I can ;)

         Introduction         

For all the "CG artists" who wish like to improve the productivity of their creative process, the use of a "renderfarm" is highly recommended. A renderfarm is simply computers connected in network to help you calculate your image sequences.

The title of this article can be in a way considered as misleading. A real open source renderfarm would only use open source software running on an open source OS like Linux. Nevertheless, for all of those who needs to work with software like After Effect or Cinema4D a windows/Mac running system is the only way to go. So waiting for the Linux big jump, we can already use some Windows compatible open source solutions like Natron ( compositing ) and Afanasy (renderfarm manager).

This tutorial based on my own experience will show you how to set up a render farm running on windows and how to use Afanasy to launch and control your rendering process through the network.

         1. Network Settings        

    Step 1: configure the physical network    

Required equipment:

1. a master computer called WORKSTATION.
2. several computers called CLIENTS.
3. "modem" and a "switch" to connect the whole network with ethernet cables.

Tip: I recommend to connect the modem to the switch rather than use the second ethernet port of your Workstation (if you have one). So the computers will all get an IP delivered by the modem and it will be easier to locate them on the network and control them by internet. Plus it will make possible the OS updates.

Your network should look like that:

     Step 2: Windows settings    

   1. Turn on "network discovery"   

 For every computer (workstation + clients) follow these steps:

1. "Start => Control panel => network => advanced parameters".
2. turn on "network discovery" and deactivate "protection by password", for Public and Home Network.

Other computers available on the network should be listed in the left tab.

Tip: It is not necessary to create a "bridge" through a "home group". We just need to map all the computers on the network.

 

 

 

   2. Create a work folder   

Create a folder on your workstation (wherever you want). This folder will have to contain all the elements of the project, necessary for its rendering. Whether it is for Maya, After-Effect, Nuke, Natron ...etc., the logic remains the same: the access path of your files has to be exactly the same for all your computers. This folder that we gonna named "WORK", will contain:

1. the project file
2. the external references (files, images, videos) 
3. the save folder (to set up in your export parameters)

So when your renderfarm manager will distribute a job to your clients, they will all use the same path and save your files to the same folder.

Do not forget to share this folder:

1. Right click on it => Properties => ...
2. In the drop-down menu select "Everyone" then "Add"
3. select "Reading / writing" then "Share"

   3. Map network drive    

Once your folder " WORK " is created on your workstation it is necessary, for each computer (workstation and clients) to follow these steps:

1. Open Windows explorer.
2. Click "Map network drive" then choose an arbitrarily letter, let's say "X", and select your folder "WORK".
3. "reconnect at logon" remains marked.

A new network drive should appeared with your local disk (C:). It has to be named:

" Name-file (\\ Host-Name) (letter of your choice:) "

The "Host-Name" is your computer name given at Windows install (If you don't remember it go to "Control panel => System", then below on the right). So in our case we have :

"WORK (\\WORKSTATION-PC) (X:)"

          

Repeat this operation on every clients. Be careful to choose the same letter (X in our example) as well as the same folder (WORK through the network by WORKSTATION).

Every computers have the same network drive with the same name targeting towards the same folder. Your computers are now set up in renderfarm! Now you have to install your softwares and plugins on every one.

        2. Installing AFANASY on Windows         

To help you work with your renderfarm you will find a lot of different manager software capable of submitting, distribute and organize your jobs. Most are paying like Deadline, Royal Render, PipelineFX etc...
... However, some of them are OPEN SOURCE, such as AFANASY, which is part of an open source toolkit dedicated to graphic creation called CGRU (http://cgru.info/home) and available for all platforms Windows, Mac and Linux.

 [ CGRU is a CG tools pack includes AFANASY - Free Open Source Render Farm Manager. Name "CGRU" - came from CG - Rules. It has two meanings: computer graphics principles and computer graphics is a cool thing. Name "Afanasy" - came from a Greek name meaning "immortal". ]

source: http://cgru.info/about

Afanasy proposes "submission scripts" for these applications:

• After Effect
• Blender
• Cinema 4D
• Houdini
• Max
• Maya
• Nuke
• XSI

Afanasy looks like this:

 

screenshots: http://cgru.info/afanasy/gui#page_top

 

 

   Step 1: server setup (workstation)  

The server is the computer that will dispatch automatically the different tasks to the other client computers. So Afanasy will be preferably installed on your workstation:

1. Download the last release of CGRU for MS Windows here: http://cgru.info/downloads
2. Unzip the folder and paste the content ("cgru-Windows") directly into the root of your local disk : C:\cgru-windows.

You're almost done! Two more things to specify:

• The IP address of the server.
• The path of the ".EXE" file of your softwares.

To do that you will need an embedded software called "KEEPER". Go to "cgru-Windows" folder and execute "start.cmd".  Keeper should now pop up in the notifications icons list. Then click right to get the options menu.

1. Right click => Afanasy => Set to server...
2. Indicate the IP address of your workstation (to know it go to "Start => write "CMD" in the search field then "enter", in the console write " ipconfig " then "enter", your IP appears on the 8th line in front of "Address IPv4 in ....... ").
3. Again Right click => Software => Setup Soft => select the one who concerns you (Maya, Blender ...) then indicate the path towards his .exe.

Tip: indicate your IP rather than your host-name. For any reason I can't explain it works for me only with the IP address.

That's it! Nothing more to do. Now, to launch the server, it is necessary to reach the Afanasy commands. There are two options:

• Go to C:\cgru-windows\start\AFANASY\
• On Keeper Right click => Afanasy => [browse]

You will find all the commands necessary to make AFANASY work:

• _afserver.cmd: launch the server.
• _afstarter.cmd: launch a job.
• Afwatch.cmd: to manage your Jobs, renders and users.
• Render.cmd: launch the render.

Follow these steps:

1. Execute "_afserver.cmd"
2. Then "afwatch.cmd" or on Keeper "Right click = > Afanasy = > Start Watch". A window will pop up with the tab "Renders" where you should see the name of your workstation. In my example : " workstation-pc " afwatch says that it's "Offline". It is normal, the render is not launched yet. The server communicates but render nothing, so it is necessary to launch also the render on my workstation to make it as a potential unit of calculation in afwatch.
3. Execute "render.cmd" or on Keeper "Right click => Afanasy => Local Render..."

You should see:

• Afwatch

• _afserveur

• Afwatch once the server launched

• Once the server AND the render both launched

Your server is now configured and your workstation is ready to make any job. Do a test job and execute "_afstarter.cmd" (don't forget to give the path of the .exe file your software). You can alternatively use the submission scripts to send the jobs directly from your applications.

  

 

   Step 2: renders setup   

Render will "make the job". You can start it on clients AND workstation.

To install it on your clients, follow the same steps as on your workstation (be careful on step n°4):

1. Copy-paste "cgru-Windows" folder directly into the root of your local disk (C:\cgru-Windows).
2. Browse "cgru-Windows" double-click on "start.cmd".
3. On Keeper: right click => Afanasy => Set server...
4. INDICATE THE IP ADDRESS OF YOUR WORKSTATION!!!
5. Again on Keeper: right click => Software => Setup Soft => select the software for which you want to use the render (indicate the path to its .exe, e.g. maya.exe or cinema4d.exe).
6. Go to C:\cgru-windows\start\AFANASY\ and execute "render.cmd" or on Keeper "Right click => Afanasy => Local Render".

Verify that  the window command ends with " Render registered ". It is very important, it means that your client has just been correctly located by the server. Last check : go bacl to afwatch on the workstation : the new client should appear in the tab "Renders" tab.

Your client is now configured and your workstation is ready to make any job. Repeat this operation on many clients as you need. Then send your jobs by executing "_afstarter.cmd " or using the submission sripts.

Tip: no need to use Keeper in the future. You can instead use shortcuts of Afanasy's commands. To create an icon on your desktop: right click => Send to => Desktop (shortcuts). A better way to launch quickly the render on your clients.


If you meet specific concerns you can always refer to the on-line user manual at http://cgru.info/home and if you still don't get it you can also ask questions on the dedicated forum at http://rules.cgru.info/*/Ask_Questions_Here.

   Step 3: launch a Job    

Proceed as follows:

1. On your workstation start "_afserver.cmd" then "render.cmd" then "afwatch.cmd".
2. Start "render.cmd" on your clients.
3. Back to the workstation execute "afstarter.cmd".

• File: select your project file (.mb, .ma, .aep, .ntp, .c4d, .nk ...etc).
• Frames: indicate your "frame range".
• FPT: "frames Per Task" i.e. the number of images calculated by task sent to the renders. If you enter 1 it means that there will be as many task as images to calculate. Knowing that every client will have first to load the project for every task it is better to increase this parameter at least to 10 (it actually depends on your project: if your project is too heavy it is better to decrease this parameter to avoid crashes. If your project is quite light with a big frame range then you can increase this parameter as you like).

Click "Refresh" to see the code sent to the server then "Start" to start the Job.

Tip : once afwatch is launched type "IDDQD" to change in "Super User" mode (don't write it in search tabs. Just type it over the window). afwatch must turn green. This will give you access to all available options when you'll right clic on Jobs, Renders and Users. 

Instead of using afwatch you can use your web browser :

• on Keeper => Right Clic => AFANASY => Web GUI
• or type in your browser adress => your IP:51000

It's recomended to use Firefox with the CGRU addon : http://cgru.info/firefox/cgru-addon.xpi

 

        3. Tricks and tools:        

   Remote control via your internet connection   

Now that everything is set up, you gonna need an easy way to access your clients from the workstation to check if everything is going correctly.

At the moment, the simplest solution I found, not open source but FREE, is " Teamviewer ".It gives you the possibility to control your computers throught the internet, including via your smartphone (very usefull when you need to leave the office and allows to restart any calculations you want in case of error or crash).

Easy to use also with a free account, It just has to be installed on each computer. Type your admin and password to connect them together with the same account. You're done! You can open as many Teamviewer windows as clients you have.

   More Tips   

• Desactivate your Windows updates to avoid your computers shuting down in the middle of a job.
• To save energy power, certain motherboard models switch off your network card alimentation. This result in interruption cuts in the network connection. To resolve it go to: "Start => Configuration panel => network Center and sharing => Modify the parameters of the card =>  Right click on your connection => Properties => Configure ... ". Sometimes you also have to deactivate this option in your BIOS.
• Verify that your firewall parameters is not blocking any of your softwares connections. "Start => Configuration panel => Firewall Windows => advanced Parameters".
• To save time at startup, remove the password user account protection. "Start => Configuration panel => Users Accounts".

   Recognition of the network drive   

Under Windows 7, the automatic recognition of the network drive at startup is not perfect. To help there is a simple solution:

1. Open any writer editor
2. write: "explore X:\ " (without quotation marks, and instead of X, the letter you chosed as shared network drive) save it with the name you want and change manually its extension in ".bat ". Before that, verify that your display options allow you to see your files extensions. In Windows explorer click "Organize => Options of the file and research => tab Display => deseable: Mask the extensions ... ".
3. Paste this file in your startup folder: "Start => All the programs => Right click on Startup => Open all users". You can paste here all the shortcuts of the programs that you wish to see running automatically at startup.

   Read images sequences   

Once your image sequence is calculated by your renderfarm, it is necessary to verify that nothing is missing and that it's what you expected. And thus, rather than make a preview with After-Effect (opening AE, import the image sequence, create composition, make a video etc.), there is a very practical open source soft which allows you to play directly in video an image sequence: DJV_VIEW

Install djv_view then select an image of your image sequence (not necessarily the first one) :
Right click => open with => djv_view

       The END        

Et voila !
I will continue to update this article according to your remarks and to my experiments.

JR
creature-studio