Apple Qmaster Render Farm Setup
Tested Compressor and Qmaster render farm setup.
Here you can look at the settings and configuration of the setup that I am currently using, Nov 29/07
You need to install Compressor and Qmaster on all of the machines that you are going to cluster together.
I have installed Final Cut Studio 2 on all of the machines
You need to have a fast connection to your shared storage
I am running a small render farm using 2x MacPro 2.66 GHz Dual-Core Intel Xeon Workstations and 1x Xserve 2 GHz Dual-Core Intel Xeon,
The computers are connected together with single Fibre Channel cable to a 10TB Xsan.
I have set the Shared Cluster Storage to a Folder on the Xsan, (volume01/admin/RENDERFARM)
All computers have read and write access to this folder (777).
In System Preferences you configure Apple Qmaster
Here are my settings in setup menu.
I have my workstation (STATION11) setup as Services and Cluster Controller.
Other computers in the cluster
The other client stations are set up to services only.
(Nas01 and Station12)
Here are my advanced settings
I have set my Cluster storrage to /Volumes/volume01/admin/RENDERFARM.
It is set to delete Old files after 1 day.
After you stop sharing you need to reset services before you restart again
Once you stop Qmaster services this button changes to Start Sharing. If you hold down the option key and click on this button it changes to Reset Services. I Reset services on all machines every time I shut down the services. If you don’t do this then often the machines do not show up in Apple Qadministrator > Qmaster Service Browser
Open Apple Qadministrator and check to see if the machines you have set up are showing up in Qmaster Service Browser
If they are not showing up, then Reset the services as described in the above step.
Create a new Cluster and choose a controller
Drag the Machines that are showing up in the Qmaster Service Browser up to the Services Window to assign them to the new Cluster.
If you ever want to remove a machine from the cluster that you have set up, go here and drag them back down to the Qmaster Service Browser Window.
Make sure to apply your changes
Apple Qadministrator Preferences
My Cluster isn’t hooked up to the internet, so I have left these Preferences like this, the security settings are blank
To speed up your rendering in Compressor make sure to change the Cluster Options from copy source to cluster as need to Never copy source to Cluster
Your new cluster should now be available in Compressor
Test your renders and see if the new cluster is working. try a 1min video and test to see how long it takes for your render.
Here is one of my Compressor settings that I use all of the time for web previews with timecode and ID BUG
Name: H.264 800Kbps 16×9
Description: H.264 for high-bandwidth connections
File Extension: mov
Estimated file size: 295.31 MB/hour of source
Audio Encoder
AAC, Stereo (L R), 48.000 kHz
Video Encoder
Format: QT
Width: 480
Height: 270
Pixel aspect ratio: Square
Crop: (L: 0, T: 20, R: 0, B: 20)
Padding: None
Frame rate: (100% of source)
Frame Controls: Automatically selected: Off
Codec Type: H.264
Multi-pass: On, frame reorder: On
Pixel depth: 24
Spatial quality: 50
Min. Spatial quality: 50
Key frame interval: 150
Temporal quality: 50
Min. temporal quality: 50
Average data rate: 0.688 (Mbps)
Fast Start: on
Timecode Generator
Position: Lower Right–Title Safe
Alpha: 1.000
Label:
Start Timecode at 00:00:00:00 false
Text Color: r:0.0000, g:0.7331, b:0.0020
Font: Helvetica, 18
Here is a setting that I use for Hi Quality Previews of my HD content.
Description: H.264 for LANs/Intranets
File Extension: mov
Estimated file size: 1.07 GB/hour of source
Audio Encoder
AAC, Stereo (L R), 48.000 kHz
Video Encoder
Format: QT
Width: 960
Height: 540
Pixel aspect ratio: Square
Crop: None
Padding: None
Frame rate: (100% of source)
Selected: 24
Frame Controls: Automatically selected: Off
Codec Type: H.264
Multi-pass: On, frame reorder: On
Pixel depth: 24
Spatial quality: 50
Min. Spatial quality: 50
Key frame interval: 150
Temporal quality: 50
Min. temporal quality: 50
Average data rate: 2.56 (Mbps)
Fast Start: on
Timecode Generator
Position: Lower Right–Title Safe
Alpha: 0.500
Label:
Start Timecode at 00:00:00:00 false
Text Color: r:1.0000, g:1.0000, b:1.0000
Font: Helvetica, 24
Watermark
Position: Lower Left
Scale By: 0.500
Alpha: 0.250
Repeat On
File Name: volume01:admin:User:Logo_1920x1080_IDBUG.psd
If I need to deliver the same content in a more compressed version I use this
Name: 960x540_Edison_Leo_Weekly_web
Description: H.264 for web
File Extension: mov
Estimated file size: 439.45 MB/hour of source
Audio Encoder
AAC, Stereo (L R), 48.000 kHz
Video Encoder
Format: QT
Width: 960
Height: 540
Pixel aspect ratio: Square
Crop: None
Padding: None
Frame rate: (100% of source)
Frame Controls: Automatically selected: Off
Codec Type: H.264
Multi-pass: On, frame reorder: On
Pixel depth: 24
Spatial quality: 50
Min. Spatial quality: 50
Key frame interval: 150
Temporal quality: 50
Min. temporal quality: 50
Average data rate: 1.024 (Mbps)
Maximum data rate: 1.024 (Mbps)
Fast Start: on
Timecode Generator
Position: Lower Right–Title Safe
Alpha: 0.500
Label:
Start Timecode at 00:00:00:00 false
Text Color: r:1.0000, g:1.0000, b:1.0000
Font: Helvetica, 24
Watermark
Position: Lower Left
Scale By: 0.500
Alpha: 0.250
Repeat On
File Name: volume01:admin:UserLogo_1920x1080_IDBUG.psd
I am encoding video that has been exported from an Avid timeline in its native codec Avid DNX-36
To get this to work i needed to intall this codec on all the machines that I am using to render. put the components in Macintosh HD/Library/QuickTime/
Once you restart your machines they will now be able to play the Avid Files.
I have installed a FibreChannel card in the avid so that it has access to the Xsan.
We export a self contained movie from the avid onto a shared folder on the Xsan. This file is dropped onto a dropplet created in compressor. When the droplet starts up it asks what machine you want to use to render. Choose your new cluster.
I will run a benchmark test for you with 1 min of video encoded with this setting
Description: H.264 for LANs/Intranets
Estimated file size: 1.07 GB/hour of source
Audio Encoder
AAC, Stereo (L R), 48.000 kHz
Video Encoder
Format: QT
Width: 960
Height: 540
1 Machine rending alone
Name: 1minDNX36.mov
Job Type: Compressor
Priority: High
Time Elapsed: 0:17:22
Time Remaining: 0:00:00
Percent Complete: 100
Status: Successful
3 Machines Clustered
Name: 1mincluster
Submission Time: 29/11/07 6:12:44 PM
Owner: station11
Time Elapsed: 0:04:13
Time Remaining: 0:00:00
Percent Complete: 100
Status: Successful
Ive noticed that when using the cluster that the activity monitor is working much harder than 1 machine alone.
My slowest machine is ususally the one that is the last to finish the render. In this case Nas01 took about 20 seconds longer to complete its render.
Hmm… I wonder how fast it would be with 10 computers clustered together? Faster than real time?
Curtis Robinson
VisionStudios.ca