What is distributed rendering?
Distributed rendering (DR) is nothing new. It has been a part of V-Ray since version 1. The general idea is that renderings can be broken up into many little tasks. Render engines like V-Ray take advantage of this by distributing those tasks among the many cores (GPU or CPU) on your computer. The simplest way is by rendering small portions of the image, known as buckets. As each bucket is done, it moves on to the next one that is not being worked on by another core. Distributed rendering takes it a step further and adds more cores by talking to other computers on the network. Through the local network, it gets all the data that it needs to render a bucket, calculates it, sends that bucket back, and moves on to the next task.
How DR was implemented in the past?
In order to use DR, V-Ray had to be installed on every machine that you needed to render on. Then you would have to launch a Spawner program that would listen over the network if it had any tasks to do.
Then from the computer you are launching DR you need to know the network address (usually the IP) of every computer you want to use. Additionally, you would need to know the port used for DR. When going to render, you would need to select which computers you want to use, and then render.
Some of the limitations of this old system is that you had to know the port and all the addresses of the DR machines. Additionally, you would have to know how much power each DR machine had and if it was up to the task at hand. You also needed to make sure that every DR machine was using the exact same version of V-Ray.
Older Distributed Rendering Interface in V-Ray for Maya
How is V-Ray Swarm different?
Swarm is much smarter on how it communicates over the network. Here are some of the key differences:
Keep DR machines alive
Swarm is constantly monitoring the state of the computer to make sure that V-Ray is active and ready. If not, it will automatically restart it. This feature also existed in the old DR system but has been made more robust in Swarm.
Network discovery
Swarm machines automatically find each other over the network so you no longer need to know the address of each computer.
Automatically selects the master node
Using a peer-to-peer network, it automatically selects which computer will be the master node that controls and manages the entire Swarm.
Hardware profiling and tagging
Machines can be tagged as part of groups based on their resources available. It can also monitor the hardware available, such as the number of CPUs and GPUs available. Users can then include or exclude different computers or groups of computers based on tags from the swarm.
Always uses the correct version of V-Ray
The machine launching the render makes sure every other machine is rendering the same version. Each machine checks to see if their version coincides with the host machine. If it does not, then the host machine runs the correct version remotely on the Swarm machine.
Please Note: Since this feature essentially allows you to run an application from a remote computer, it uses cryptography to make sure that the application that is launched is actually V-Ray.
Highly simplified User Interface
Since a vast majority of the tasks previously needed to run DR are automatically negotiated now, the user interface is extremely simple. The user is presented with a slider that depicts the total amount of compute power available to him or her to do the rendering. By moving that slider to the right, Swarm dynamically adds more compute power. Sliding it to the left releases power.
Tagging Swarm Machines
Using a simple web interface, groups of machines can be tagged for different things. In doing so, when launching a job on the Swarm, you can use the tags to only use certain machines. For example, you may only want to tag machines that are faster than others. Or you may want to make a group of machines that are reserved for a certain job and tag them as such.
V-Ray Swarm machines can be easily managed and tagged in the web interface
What does this mean for the user?
With Swarm, you can now use every computer resource that your local network has to offer. Every machine on the network has the potential to contribute to the rendering, including administration and accounting computers. Because of the way that Swarm manages resources, the users on the Swarm machines will generally not even be aware that their computers are being used for rendering.
A few things to note
Swarm relies on a fast Local Area Network (LAN) to communicate between different machines. It needs this to keep open connections in order to move and distribute data. In its current state it is not well suited for communication over a Wide Area Network (WAN) such as between different offices, or to an external cloud resource. Swarm also needs machines to be on the same subnet of the network to work.
Additionally, each Swarm machine needs a V-Ray Render node license to run. This means that if you have 100 computers with Swarm on them, but only 5 render node licenses, only 5 will be used to render.
What is in the future?
The current state of Swarm is built to mainly work on local networks. We are looking at many other tools to drastically increase the compute power that is available to the user on a much larger scale. In doing so, rendering will essentially become instant.
