Using private docker registry cause k8s deployment failure

[0xshawn]

hi,

I wanna to use private docker registry, which configured in /cluster-configuration/services-configuration.yaml.

Then run following command:

python paictl.py service start -p /cluster-configuration

can cause k8s deployment error like this:

drivers-one-shot-h2sks:
Failed to pull image "registry.aliyuncs.com/xxxxxx/drivers:latest": rpc error: code = Unknown desc = Error response from daemon: manifest for registry.aliyuncs.com/xxxxxx/drivers:latest not found

How can I configure the cluster-configuration to make gcr.io, OpenPAI's image like openpai/driver:latest and my private docker registry work properly.

Thanks so much!

[0xshawn]

services-configuration.yaml

# Copyright (c) Microsoft Corporation
# All rights reserved.
#
# MIT License
#
# Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated
# documentation files (the "Software"), to deal in the Software without restriction, including without limitation
# the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and
# to permit persons to whom the Software is furnished to do so, subject to the following conditions:
# The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED *AS IS*, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING
# BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
# NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM,
# DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

cluster:

  clusterid: openpai

  # HDFS, zookeeper data path on your cluster machine.
  data-path: "/data"

  # the docker registry to store docker images that contain system services like frameworklauncher, hadoop, etc.
  docker-registry-info:

    # If public, please fill it the same as your username
    docker-namespace: xxxxx

    # E.g., gcr.io. If public，fill docker_registry_domain with word "public"
    # docker_registry_domain: public
    docker-registry-domain: registry.aliyuncs.com
    # If the docker registry doesn't require authentication, please comment out docker_username and docker_password
    docker-username: docker_username
    docker-password: docker_password

    docker-tag: latest

    # The name of the secret in kubernetes will be created in your cluster
    # Must be lower case, e.g., regsecret.
    secret-name: regsecret


hadoop:
  # custom_hadoop_binary_path is required to build hadoop-ai.
  # More about hadoop-ai please follow the link: https://github.com/Microsoft/pai/tree/master/hadoop-ai.
  # Notice: the name should be hadoop-{hadoop-version}.tar.gz
  custom-hadoop-binary-path: /pathHadoop/hadoop-2.9.0.tar.gz
  hadoop-version: 2.9.0
  # Step 1 of 4 to set up Hadoop queues.
  # Define all virtual clusters, equivalent concept of Hadoop queues:
  #   - Each VC will be assigned with (capacity / total_capacity * 100%) of the resources in the system.
  #   - The 'default' VC can be used by any PAI user, i.e. a user will be automatically put into the
  #     member list of 'default' VC when it is created.
  #   - The system will automatically create the 'default' VC with 0 capacity, if 'default' VC has not
  #     been explicitly specified here.
  virtualClusters:
    default:
      description: Default VC.
      capacity: 40
    vc1:
      description: VC for Alice's team.
      capacity: 20
    vc2:
      description: VC for Bob's team.
      capacity: 20
    vc3:
      description: VC for Charlie's team.
      capacity: 20



frameworklauncher:
  frameworklauncher-port: 9086


restserver:
  # port for rest api server
  server-port: 9186
  # secret for signing authentication tokens, e.g., hello
  jwt-secret: HelloPAI
  # database admin username
  default-pai-admin-username: username
  # database admin password
  default-pai-admin-password: password


webportal:
  # port for webportal
  server-port: 9286


grafana:
  # port for grafana
  grafana-port: 3000


prometheus:
  # port for prometheus port
  prometheus-port: 9091
  # port for node exporter
  node-exporter-port: 9100
  # How frequently to scrape targets
  scrape_interval: 30
  # if you want to enable alert manager to send alert email, uncomment following lines and fill
  # right values.
  #  alerting:
  #    alert_manager_port: 9093
  #    alert_receiver: [email protected]
  #    smtp_url: smtp.gmail.com:587
  #    smtp_from: [email protected]
  #    smtp_auth_username: [email protected]
  #    smtp_auth_password: gmail_password


pylon:
  # port of pylon
  port: 80

[ydye]

@shawntian90 Have you built openPAI's image?

LINK：https://github.com/Microsoft/pai/blob/master/pai-management/doc/image-build.md

[0xshawn]

@ydye Thanks so much and it resolved my problem.