Using Jobs

Jobs have been replaced by a more general framework called Workflows. For the latest info go to Workflows.

Creating a Job

Web UI

Jobs can be submitted via the Job Builder. There are a variety of optional and required options for your job including the instance type, the command, the container, etc. Note: At the top of the screen, you'll see a few job presets which will give a sense for how these options work.

• Machine Type. What type of instance to run your Job on. We recommend starting with a GPU+. Many Jobs benefit from a machine with a GPU, but some can run just using a CPU.

• Container. Jobs are run within a docker container. You can run a public or private container. Learn more here.

• Workspace. The workspace is the collection of code that is run. It can be a Git repository (public or private), your local working directory (if you are using the CLI) which is uploaded to the docker container during the job running process, or none (default value).

• Command. The command is the entry point to the container. This is the line of code that will kick off your Job. It could be a bash script ./run.sh or python main.py as just some examples.

• Custom Metrics. Enter a list of custom metrics to use with Gradient's statd client, such as percent_failure or percent_success.

Once you have examined or specified the parameters, hit "Submit Job" and watch the Job run!

The Job Builder is great for getting started with basic tasks. For more advanced workflows, we recommend using the CLI which offers more flexibility.

CLI

Syntax

gradient jobs create <namespace> <command> [options...]

Job Parameters Basics

• Machine Type: Such as --P100 or --C7 or --TPU

• Container: Such as --tensorflow/tensorflow:1.5.1-gpu

• Command: Such as "./do.sh"

Example Use

gradient jobs create \
    --name "my job" \
    --container "http://dockerhub.com/mycontainer" \
    --machineType "P5000" \
    --command "/paperspace/run.sh"
    --projectId "someProjectID"

Job Parameters Complete List

Argument	Default	Description
name	[required]	Job name
projectId	[required]	Project ID
machineType	K80	An optional machine type to run the job on: either 'GPU+', 'P4000', 'P5000', 'P6000', 'V100', 'K80', 'P100', or 'TPU'.
container	paperspace/tensorflow-python	A reference to a docker image in a public or private docker registry, or a container name provided by Paperspace. Docker image repository references must be in lowercase and may include a tag and a hostname prefix followed by a slash; if committed the hostname defaults to that of the public Docker Hub registry. An example docker image reference: docker.io/mynamespace/myimage:mytag. A container name may be mixed case. (Designated container names are currently only provided as part of various Gradient tutorials and samples.)
command		Job command/entrypoint
workspace		Path to workspace directory. (Soon also will support a path to a workspace archive or git repository URL.)
workspaceArchive		Path to workspace archive. (Currently being deprecated in an upcoming version.)
workspaceUrl		Project git repository URL. (Currently being deprecated in an upcoming version.)
workingDirectory		Working directory for the experiment
experimentId		Experiment Id
jobEnv		Environmental variables in JSON String Format. Example: { "HKS_EPOCHS": 1, "HKS_MAX_EVALS": 4, "DATASET_SIZE": 10000 }
project	$CWD	The name of the project for this job. If not provided, this is taken from the .ps_project/config.json file, or the current directory name.
apiKey		API key to use this time only
ignoreFiles		Ignore certain files from uploading

Environment variables are available for use within the context of your job. The following host config options are currently exposed within the container:

$PS_HOST_PUBLIC_IP_ADDRESS - the public IP address of the host machine running the job

$PS_HOST_PRIVATE_IP_ADDRESS - the private IP address of the host machine running the job

$PS_HOSTNAME - the hostname of the host machine running the job

These can be used in conjunction with the ports option to send HTTP traffic to the job while it's in progress for example.