Apple Qmaster Render Farm Setup

Tested Compressor and Qmaster render farm setup.

Here you can look at the settings and configuration of the setup that I am currently using, Nov 29/07  
 

You need to install Compressor and Qmaster on all of the machines that you are going to cluster together.

I have installed Final Cut Studio 2 on all of the machines  
 

You need to have a fast connection to your shared storage

I am running a small render farm using 2x MacPro 2.66 GHz Dual-Core Intel Xeon Workstations and 1x Xserve 2 GHz Dual-Core Intel Xeon, The computers are connected together with single Fibre Channel cable to a 10TB Xsan. I have set the Shared Cluster Storage to a Folder on the Xsan, (volume01/admin/RENDERFARM) All computers have read and write access to this folder (777).  
 

In System Preferences you configure Apple Qmaster

Here are my settings in setup menu. I have my workstation (STATION11) setup as Services and Cluster Controller.  
 

Other computers in the cluster

The other client stations are set up to services only. (Nas01 and Station12)  
 

Here are my advanced settings

I have set my Cluster storrage to /Volumes/volume01/admin/RENDERFARM. It is set to delete Old files after 1 day.  
 

After you stop sharing you need to reset services before you restart again

Once you stop Qmaster services this button changes to Start Sharing. If you hold down the option key and click on this button it changes to Reset Services. I Reset services on all machines every time I shut down the services. If you don’t do this then often the machines do not show up in Apple Qadministrator > Qmaster Service Browser  
 

Open Apple Qadministrator and check to see if the machines you have set up are showing up in Qmaster Service Browser

If they are not showing up, then Reset the services as described in the above step.  
 

Create a new Cluster and choose a controller

Drag the Machines that are showing up in the Qmaster Service Browser up to the Services Window to assign them to the new Cluster. If you ever want to remove a machine from the cluster that you have set up, go here and drag them back down to the Qmaster Service Browser Window. Make sure to apply your changes  
 

Apple Qadministrator Preferences

My Cluster isn’t hooked up to the internet, so I have left these Preferences like this, the security settings are blank  
 

To speed up your rendering in Compressor make sure to change the Cluster Options from copy source to cluster as need to Never copy source to Cluster

 

Your new cluster should now be available in Compressor

Test your renders and see if the new cluster is working. try a 1min video and test to see how long it takes for your render.  
 

Here is one of my Compressor settings that I use all of the time for web previews with timecode and ID BUG

Name: H.264 800Kbps 16×9 Description: H.264 for high-bandwidth connections File Extension: mov Estimated file size: 295.31 MB/hour of source Audio Encoder AAC, Stereo (L R), 48.000 kHz Video Encoder Format: QT Width: 480 Height: 270 Pixel aspect ratio: Square Crop: (L: 0, T: 20, R: 0, B: 20) Padding: None Frame rate: (100% of source) Frame Controls: Automatically selected: Off Codec Type: H.264 Multi-pass: On, frame reorder: On Pixel depth: 24 Spatial quality: 50 Min. Spatial quality: 50 Key frame interval: 150 Temporal quality: 50 Min. temporal quality: 50 Average data rate: 0.688 (Mbps) Fast Start: on Timecode Generator Position: Lower Right–Title Safe Alpha: 1.000 Label: Start Timecode at 00:00:00:00 false Text Color: r:0.0000, g:0.7331, b:0.0020 Font: Helvetica, 18  
 

Here is a setting that I use for Hi Quality Previews of my HD content.

Description: H.264 for LANs/Intranets File Extension: mov Estimated file size: 1.07 GB/hour of source Audio Encoder AAC, Stereo (L R), 48.000 kHz Video Encoder Format: QT Width: 960 Height: 540 Pixel aspect ratio: Square Crop: None Padding: None Frame rate: (100% of source) Selected: 24 Frame Controls: Automatically selected: Off Codec Type: H.264 Multi-pass: On, frame reorder: On Pixel depth: 24 Spatial quality: 50 Min. Spatial quality: 50 Key frame interval: 150 Temporal quality: 50 Min. temporal quality: 50 Average data rate: 2.56 (Mbps) Fast Start: on Timecode Generator Position: Lower Right–Title Safe Alpha: 0.500 Label: Start Timecode at 00:00:00:00 false Text Color: r:1.0000, g:1.0000, b:1.0000 Font: Helvetica, 24 Watermark Position: Lower Left Scale By: 0.500 Alpha: 0.250 Repeat On File Name: volume01:admin:User:Logo_1920x1080_IDBUG.psd  
 

If I need to deliver the same content in a more compressed version I use this

Name: 960x540_Edison_Leo_Weekly_web Description: H.264 for web File Extension: mov Estimated file size: 439.45 MB/hour of source Audio Encoder AAC, Stereo (L R), 48.000 kHz Video Encoder Format: QT Width: 960 Height: 540 Pixel aspect ratio: Square Crop: None Padding: None Frame rate: (100% of source) Frame Controls: Automatically selected: Off Codec Type: H.264 Multi-pass: On, frame reorder: On Pixel depth: 24 Spatial quality: 50 Min. Spatial quality: 50 Key frame interval: 150 Temporal quality: 50 Min. temporal quality: 50 Average data rate: 1.024 (Mbps) Maximum data rate: 1.024 (Mbps) Fast Start: on Timecode Generator Position: Lower Right–Title Safe Alpha: 0.500 Label: Start Timecode at 00:00:00:00 false Text Color: r:1.0000, g:1.0000, b:1.0000 Font: Helvetica, 24 Watermark Position: Lower Left Scale By: 0.500 Alpha: 0.250 Repeat On File Name: volume01:admin:UserLogo_1920x1080_IDBUG.psd  
 

I am encoding video that has been exported from an Avid timeline in its native codec Avid DNX-36

To get this to work i needed to intall this codec on all the machines that I am using to render. put the components in Macintosh HD/Library/QuickTime/ Once you restart your machines they will now be able to play the Avid Files. I have installed a FibreChannel card in the avid so that it has access to the Xsan. We export a self contained movie from the avid onto a shared folder on the Xsan. This file is dropped onto a dropplet created in compressor. When the droplet starts up it asks what machine you want to use to render. Choose your new cluster. I will run a benchmark test for you with 1 min of video encoded with this setting Description: H.264 for LANs/Intranets Estimated file size: 1.07 GB/hour of source Audio Encoder AAC, Stereo (L R), 48.000 kHz Video Encoder Format: QT Width: 960 Height: 540  
 

1 Machine rending alone

Name: 1minDNX36.mov Job Type: Compressor Priority: High Time Elapsed: 0:17:22 Time Remaining: 0:00:00 Percent Complete: 100 Status: Successful  
 

3 Machines Clustered

Name: 1mincluster Submission Time: 29/11/07 6:12:44 PM Owner: station11 Time Elapsed: 0:04:13 Time Remaining: 0:00:00 Percent Complete: 100 Status: Successful Ive noticed that when using the cluster that the activity monitor is working much harder than 1 machine alone. My slowest machine is ususally the one that is the last to finish the render. In this case Nas01 took about 20 seconds longer to complete its render.  
 

Hmm… I wonder how fast it would be with 10 computers clustered together? Faster than real time?

Curtis Robinson VisionStudios.ca