Skip to content

Deadline plugin

Overview

Conductor and Deadlineare integrated via a toolset that sends jobs to Conductor and ensures they only run on Conductor instances. Upload and download tasks are handled by Conductor's upload and download daemons.

One of Conductor's underlying principles is to only spin-up an instance when a specific job or task needs it.

Just as with any other instance in a Deadline managed farm, Conductor instances run the Deadline Worker.

Installation

If you haven't already done so, Download the Deadline Plugin.

Steps 2-4 require SuperUser access in Deadline Monitor

  1. Copy plugins to the correct folders:
    deadline_plugin/custom/scripts/Jobs/* -> /custom/scripts/Jobs/
    deadline_plugin/custom/events/ConductorWorker -> /custom/events/
    
  2. Copy the contents of deadline_plugin/src to a location of your choosing. Deadline will need to source this path.
  3. Add the path of the conductor client tools to the python paths in your repo. Add the path to the src folder from step #1.
  4. Add the Conductor paths to the MayaCmd plugin. You only need to add for the versions that you intend to use
            /opt/autodesk/maya-io/2016.5/maya-io2016.5.SP0/bin/Render
            /opt/autodesk/maya-io/2016/maya-io2016.SP0/bin/Render
            /opt/autodesk/maya-io/2017/maya-io2017.SP0/bin/Render
            /opt/autodesk/maya-io/2017/maya-io2017.SP3/bin/Render
            /opt/autodesk/maya-io/2017/maya-io2017.SP4/bin/Render
            /opt/autodesk/maya-io/2017/maya-io2017.SP5/bin/Render
            /opt/autodesk/maya-io/2018/maya-io2018.SP0/bin/Render
            /opt/autodesk/maya-io/2018/maya-io2018.SP1/bin/Render
            /opt/autodesk/maya-io/2018/maya-io2018.SP2/bin/Render
            /opt/autodesk/maya-io/2018/maya-io2018.SP3/bin/Render
            /opt/autodesk/maya-io/2018/maya-io2018.SP4/bin/Render
            /opt/autodesk/maya-io/2018/maya-io2018.SP5/bin/Render
            /opt/autodesk/maya-io/2018/maya-io2018.SP6/bin/Render
            /opt/autodesk/maya-io/2019/maya-io2019.SP0/bin/Render
            /opt/autodesk/maya-io/2019/maya-io2019.SP1/bin/Render
            /opt/autodesk/maya-io/2019/maya-io2019.SP2/bin/Render
            /opt/autodesk/maya-io/2020/maya-io2020.SP0/bin/Render
  5. Add submit_to_conductor.py as a Job menu script. The path to the icon is deadline_plugin/custom/scripts/General/conductor_logo.ico
  6. The Deadline Workers running on Conductor instances need to connect to your repository via a Remote Connection Server (RCS). See the Deadline documentation for more details.This will often require you to open ports in your firewall to ports 8000 or 4433 (TLS) - however the exact ports are configurable via the Deadline config.
  7. To properly connect to your Deadline RCS, you need to provide the hostname/port and the client certificate (.pfx). Client certificates with passwords are not supported. To provide theses values, you can either set the environment variables CONDUCTOR_DEADLINE_PROXY and CONDUCTOR_DEADLINE_SSL_CERTIFICATE or modify the code directly in the submit script:

Usage

Prerequisites

  1. The Conductor upload daemon must be running
  2. The Conductor download daemon must be running
  3. A dependency sidecar exists for the given job

Sending Jobs to Conductor

  1. From the Deadline Monitor, select one or multiple jobs.
  2. From the context menu, select Scripts->Send to Conductor
  3. A dialog will appear where you can modify the default options for a job
  4. Once you click Submit, the job is shipped to Conductor
  5. The dialog reappears for each selected job. For example, if you select five jobs, the dialog will appear five times.
  6. A valid dependency sidecar file must be selected. For more details, see the section on Dependencies
  7. After a few minutes, new Workers pop-up in the Worker panel with the suffix -conductor. Each Worker will render exactly one Deadline task. After a Worker completes a Deadline task, they will shut down. 8.To delete Conductor Workers that have completed their task, visit the Workers panel. They will have a state of Stalled.

How it works

We run a special Deadline Worker job on Conductor and configure it to shut down after exactly one Deadline task. There's some additional logic to create a temporary Deadline group (conductorautogroup_<jobid>) to ensure that the submitted Deadline job only runs on the Conductor instances and that the Conductor instances only pick-up the given Deadline job.

An essential step is to ensure that a valid sidecar dependency file exists. If it doesn't, the Conductor instances will not have any necessary files to run the task. See the section on dependencies for more details.

Viewing reports and most other functions of a Deadline Worker will work with a Conductor Deadline Worker, except for any operation that involves sending a command directly to the host instance (see below for more details).

Dependencies

A task can not render if it does not have all of its required dependencies. We ship all required dependencies to Conductor as part of the submission process. The uploader daemon is responsible for performing the actual upload. Therefore, it's necessary to create this list of dependencies beforehand. There are several ways to do this:

  1. Generate the sidecar dependency file on job submission. Do this manually or as part of the submission to Deadline. You'll find a modified Deadline Maya submitter included in the repo. (this should be used as a sample only). Below is a python code snippet on how to scan and save dependencies:

    def get_dependencies():
        from conductor.lib import common, maya_utils
        resources = common.load_resources_file()
        dependency_attrs = resources.get("maya_dependency_attrs") or {}
        return maya_utils.collect_dependencies(dependency_attrs)
    
    dependencies = {'dependencies': get_dependencies()}
    
    ma_path = maya.cmds.file(query=True, sceneName=True)
    dependency_sidecar_path = "{}.cdepends".format(ma_path)
    
    with open(dependency_sidecar_path, 'w') as fh:
        json.dump(dependencies, fh)
    
  2. Submit an upstream job to Deadline that will generate the dependency sidecar file

Generally, if a DCC software is already open with the scene file loaded, scanning for dependencies is a quick task.

Note

There are a few key differences between regular on-site Deadline Workers and Conductor Workers:

  • Conductor Workers are not directly accessible like other Workers. They run from within the Conductor infrastructure. They don't have an exposed IP/hostname, and therefore, remote commands do not work.
  • Conductor Workers can only render the jobs they're assigned. If you try to render a different job, it will fail (as none of the files are available).
  • Conductor Workers will manage themselves. They'll start-up and shut down as needed

Package Mapping

Conductor also needs to know what software is required for the job. This can be obtained from the Deadline job. The DeadlineToConductorPackageMapper class is used for this purpose and will likely need to be modified by the customer to support the customer's specific DCC's and plugins. More details on extending DeadlineToConductorPackageMapper are available in the code-level documentation. Please contact Conductor support with any questions.

Licensing

As of Deadline 10.1, Deadline Workers running on AWS instances don't require licenses. If running on GCP, it is the customer's responsibility to provide Deadline Worker licenses via their Deadline repository/server. Usage-Based Licensing (UBL) is an option via the AWS Thinkbox Marketplace. Licensing of DCC's and all other software is managed by Conductor and included in the cost.

Supported software

Currently, the plugin is set up to support Maya/Arnold renders using the MayaCmd plugin.


  1. It is possible to upload the files simultaneously as submitting the job without using the uploader daemon. This can be configured in the Conductor config. However, this method is not recommended when submitting jobs via Deadline due to the potentially long wait times.*