Containers
Last updated
Last updated
When you run a Gradient Experiment you must provide a docker container within which your code will execute.
There are two types of containers you can provide: or .
The CLI expects that you provide a container using the standard Docker syntax. For example, any container on the Docker Hub registry will work out of the box. To pull the official Tensorflow container, you would check here:
And your container would take the form:
tensorflow/tensorflow:1.5.1-gpu where 1.5.1-gpu is the "tag" taken from here:
Example use:
If you specify a container that is hosted on a private registry, then you will need to authenticate using the optional experiment parameters: registryUsername and registryPassword.
Here are some details on the pulling from private docker repositories:
There are two new options on the gradient experiments run and gradient experiments createmethods to be used when specifying a container (aka a docker image) from a private docker repository: --registryUsername and --registryPassword.
Example use:
Notes: You need to specify the username and password each time you reference a private docker repository in an experiment submission. The credentials are transmitted over ssl and cached for the lifetime of the experiment. If you clone the experiment the credentials are automatically reused.
The credential options --registryUsername and --registryPassword should be sufficient for a number of private docker registry services, including
Docker Hub
Amazon EC2 Container Registry
Google Container Registry
JFrog/Bintray.io
Quay.io
Custom registries running Docker Registry v2 protocol, as long as they are publicly accessible on the internet.
The form of the repository link, and the format of the username and password are specific for some of these registries, but are publicly documented.
The following provides a summary for Docker Hub, AWS and GCP:
Username: <your Docker ID> Password: <your Docker ID password>
Container link format: "docker.io/<organization-or-username>/<repository-name>[:<tag>]"
Example:
Username: "AWS" Password: <value of '-p' option from output of the AWS SDK command "aws ecr get-login --no-include-email">
Container link format: "<aws_account_id>.dkr.ecr.us-east-1.amazonaws.com/[<namespace>/]<repository-name>" Example: Install and configure the AWS SDK on your local computer, then execute:
Note: if the --no-include-email is reported as not supported, you may need to update your aws cli using pip3.
It will produce output similar to the following:
Use the value of the '-p' option from the actual output as the password value in the gradient experiments method. Note: the value may span multiple lines:
Username: "_json_key" Password: <service account JSON Keyfile contents>
Container link format: "[us.|eu.|asia.]gcr.io/<project-id>/<repository-name>[:<tag>]"
Example:
Use the contents of the keyfile as the password value in the gradient experiments method:
Note: do not base64 encode the contents of the keyfile. It will be encoded by the Gradient Experiments job runner at runtime.
If you are running on a GPU-node your container will need to include the relevant NVIDIA GPU drivers. This can be done most easily by basing your container off of the official NVIDIA docker containers. i.e. the top line of your Dockerfile would be:
For more details and options see:
To get this create a service account JSON key file for your project, following these instructions: Once you have downloaded your JSON key file, rename it to "keyfile.json".
For more details and other options see: