Deadline plugin

Overview¶

Conductor is integrated into Deadline via a set of plugins and scripts. This toolset is responsible for sending jobs to Conductor and ensuring these jobs only run on Conductor instances. The uploading of dependencies and the download of the resulting data is handled by Conductor's upload and download daemons respectively.

One of Conductor's underlying principles is to only spin-up an instance when it's needed by a specific job or task. This ensures that customers don't pay for idle machines. Traditional render farm managers' paradigm is that machines are running but idle until a job is assigned to them. Conductor's integration with Deadline was designed in a way to take advantage of only spinning up instances when they are needed, so that users only pay for what they use.

Just as with any other instance in a Deadline managed farm, Conductor instances run the Deadline Worker.

As of Deadline 10.1, the term Slave has been replaced with Worker**

Installation¶

Download the plugin from Github

Steps 2-4 require Super User access in Deadline Monitor

1. Copy plugins to the correct folders:

   deadline_plugin/custom/scripts/Jobs/* -> /custom/scripts/Jobs/
   deadline_plugin/custom/events/ConductorWorker -> /custom/events/
   

2. Copy the contents of deadline_plugin/src to a location of your choosing. Deadline will need to source this path.
3. Add the path of the conductor client tools to the python paths in your repo. Add the path to the src folder from step #1.
4. Add the Conductor paths to the MayaCmd plugin. You only need to add for the versions that you intend to use

           /opt/autodesk/maya-io/2016.5/maya-io2016.5.SP0/bin/Render
           /opt/autodesk/maya-io/2016/maya-io2016.SP0/bin/Render
           /opt/autodesk/maya-io/2017/maya-io2017.SP0/bin/Render
           /opt/autodesk/maya-io/2017/maya-io2017.SP3/bin/Render
           /opt/autodesk/maya-io/2017/maya-io2017.SP4/bin/Render
           /opt/autodesk/maya-io/2017/maya-io2017.SP5/bin/Render
           /opt/autodesk/maya-io/2018/maya-io2018.SP0/bin/Render
           /opt/autodesk/maya-io/2018/maya-io2018.SP1/bin/Render
           /opt/autodesk/maya-io/2018/maya-io2018.SP2/bin/Render
           /opt/autodesk/maya-io/2018/maya-io2018.SP3/bin/Render
           /opt/autodesk/maya-io/2018/maya-io2018.SP4/bin/Render
           /opt/autodesk/maya-io/2018/maya-io2018.SP5/bin/Render
           /opt/autodesk/maya-io/2018/maya-io2018.SP6/bin/Render
           /opt/autodesk/maya-io/2019/maya-io2019.SP0/bin/Render
           /opt/autodesk/maya-io/2019/maya-io2019.SP1/bin/Render
           /opt/autodesk/maya-io/2019/maya-io2019.SP2/bin/Render
           /opt/autodesk/maya-io/2020/maya-io2020.SP0/bin/Render

5. Add submit_to_conductor.py as a Job menu script. The path to the icon is deadline_plugin/custom/scripts/General/conductor_logo.ico
6. The Deadline Workers running on Conductor instances need to connect to your repository via a Remote Connection Server (RCS). See the Deadline documentation for more details.This will often require you to open ports in your firewall to ports 8000 or 4433 (TLS) - however the exact ports are configurable via the Deadline config.
7. To properly connect to your Deadline RCS, you need to provide the hostname/port and the client certificate (.pfx). Client certificates with passwords are not supported. To provide theses values, you can either set the environment variables CONDUCTOR_DEADLINE_PROXY and CONDUCTOR_DEADLINE_SSL_CERTIFICATE or modify the code directly in the submit script:

Usage¶

Prerequisites¶

1. The Conductor upload daemon must be running
2. The Conductor download daemon must be running
3. A dependency sidecar exists for the given job

Sending Jobs to Conductor¶

1. From the Deadline Monitor, select one or multiple jobs.
2. From the context menu, select Scripts->Send to Conductor
3. A dialog will appear where you can modify the default options for a job
4. Once you click Submit the job will be submitted to Conductor
5. The dialog will reappear for each job selected (ex: if you selected five jobs to submit, the dialog will appear five times)
6. It's imperative for a valid dependency sidecar file to be selected. For more details see the section on Dependencies
7. After a few minutes new Workers pop-up in the Worker panel that have the suffix -conductor. Each Worker will render exactly one Deadline task. After a Worker completes a Deadline task, they will shut down.
8. It's safe to Delete the Conductor Workers that have completed their task. They will have a state of Stalled. This can be done from the Workers panel.

How it works¶

The above is achieved by running a special Deadline Worker job on Conductor - which is the Deadline Worker software with some parameters to ensure it shuts down after it completes exactly one Deadline task. There's some additional logic to create a temporary Deadline group (conductorautogroup_<jobid>) to ensure that the submitted Deadline job only runs on the Conductor instances and that the Conductor instances only pick-up the given Deadline job.

An essential step is to ensure that a valid sidecar dependency file exists. If it doesn't, the Conductor instances will not have any of the necessary files to run the task. See the section on dependencies for more details.

Viewing reports and most other functions of a Deadline Worker will work with a Conductor Deadline Worker, except for any operation that involves sending a command directly to the host instance (see below for more details).

Dependencies¶

A task can not render if it does not have all of its required dependencies. The list of necessary dependencies is submitted to Conductor as part of the job submission process. The uploader daemon is responsible for performing the actual upload¹. Therefore, it's necessary to create this list of dependencies before the job is submitted. There are several ways to do this:

1. Generate the sidecar dependency file on job submission. This can be done manually or as part of the submission to Deadline. A modified Deadline Maya submitter is included in the repo (this should be used as a sample only). Below is a python code snippet on how to scan and save dependencies:

   def get_dependencies():
       from conductor.lib import common, maya_utils
       resources = common.load_resources_file()
       dependency_attrs = resources.get("maya_dependency_attrs") or {}
       return maya_utils.collect_dependencies(dependency_attrs)
   
   dependencies = {'dependencies': get_dependencies()}
   
   ma_path = maya.cmds.file(query=True, sceneName=True)
   dependency_sidecar_path = "{}.cdepends".format(ma_path)
   
   with open(dependency_sidecar_path, 'w') as fh:
       json.dump(dependencies, fh)
   

2. Submit an upstream job to Deadline that will generate the dependency sidecar file

Generally, if a DCC software is already open with the scene file loaded, scanning for dependencies is a quick task.

Package Mapping¶

Conductor also needs to know which software (DCC's, plugins, etc...) is required for the job. This can usually be obtained from the Deadline job. The DeadlineToConductorPackageMapper class is used for this purpose and will likely need to be modified by the customer to support the specific DCC's and plugins used by the customer. More details on extending DeadlineToConductorPackageMapper are available in the code-level documentation. Please contact Conductor support with any questions.

Licensing¶

As of Deadline 10.1, Deadline Workers running on AWS instances don't require any licenses. If running on GCP, the Deadline Worker licenses are the responsiblity of the customer and can be managed via their Deadline repository/server. Usage Based Licensing (UBL) is an option via the AWS Thinkbox Marketplace. Licensing of DCC's and all other software is managed by Conductor and included in the cost.

Supported software¶

Currently, the plugin is only set up to support Maya/Arnold renders that use the MayaCmd plugin. Additional packages will be available soon. If you require additional DCC's, please contact support@conductortech.com