DGCLI

The DGCLI utility (also known as dgCtl) is used for getting license file, downloading, and updating the artifacts required for deploying and updating the On-Premise services.

Installation artifacts include:

• Docker images of services
• additional data required for the services to work

To learn more, see the installation artifacts lifecycle.

Architecture

DGCLI allows you to split the process of obtaining the installation artifacts into two steps:

1. Downloading the artifacts from the Internet (the pull mode).
2. Deploying services using the downloaded artifacts (the restore mode).

The artifacts can be stored in an S3 compatible storage or in a file system (see the storage.type setting of the configuration file).

You can use --only-apps, --only-data, and --only-license arguments to work with a specific type of artifacts only: application images, data, or license respectively. You can also combine these arguments to select multiple artifact types.

Operation modes

DGCLI can be launched in one of the following modes: license, pull, restore, save, or manifest.

Mode	Internet access	Source location	Target location
license	Required	Urbi public license servers	S3 compatible storage File system
pull	Required	Urbi public update servers	S3 compatible storage and Docker Registry File system and Docker Registry
restore	Not required	File system	S3 compatible storage and Docker Registry File system and Docker Registry
save	Not required	S3 compatible storage File system	File system
manifest	Not required	—	—

For the license and pull modes, Internet access is required only to reach the following addresses:

• datagateway.api.2gis.com to get license and manifest files
• origin-disk.2gis.com to get application files and data.

"license" mode

In the license mode, DGCLI requests a license file from Urbi servers and writes it to the selected storage. See Installing License service for more information.

The whole process of obtaining a license includes sending requests to an S3 compatible storage and using the internet:

1. License service generates files to request a license and stores them in an S3 compatible storage.
2. DGCLI utility reads these files from S3, builds a request with them, and sends it via the internet.
3. When the request is handled, a license is returned. The DGCLI utility writes it to the S3 compatible storage.
4. License service reads the license file and updates the state.

Note

To use the license mode with On-Premise versions 1.16.0 or lower, you must always add the --with-license-v1 argument to the commands.

"pull" mode

In the pull mode, DGCLI does the following:

• Downloads the latest installation artifacts from the update servers to the selected storage and (if necessary) to the Docker Registry.

  By default, the utility downloads artifacts for the components specified in the DGCLI configuration file. To download installation artifacts only for a specific service, make sure the parent component of that service is listed in the configuration file, and use the --services argument. Note that even when using this argument, the manifest file information is updated for all services.

  See the list of component and service names in the DGCLI command and argument reference.

• Writes the size of downloaded artifacts (service images, data, and others) to the log. Existing files in the storage are not counted (unless the --overwrite argument is passed).

  To check the expected size of downloaded artifacts, pass the --dry-run argument in the pull mode. DGCLI will not download artifacts but will write its size to the log. Compatible with --services, --only-apps, and --only-data arguments.

• Creates a manifest file in the JSON format that contains the information about all On-Premise services. The manifest is stored in the /manifests directory of the selected storage.

It is recommended to separate downloaded Docker images from additional service data by using the --apps-to-registry argument (see the reference of DGCLI command-line arguments). In this case, Docker images of services will be pushed into Docker Registry, and all other artifacts will be stored in the chosen type of storage. This approach gives more control over the placement of installation artifacts and avoids mixing them up.

"restore" mode

In the restore mode, DGCLI uploads the installation artifacts from the source file system to the selected storage and (if necessary) to the Docker Registry. Installation artifacts must be previously downloaded to the file system using the pull mode or the save mode.

It is recommended to separate downloaded Docker images from additional service data by using the --apps-to-registry argument (see the reference of DGCLI command-line arguments). In this case, Docker images of services will be pushed into Docker Registry, and all other artifacts will be stored in the chosen type of storage. This approach gives more control over the placement of installation artifacts and avoids mixing them up.

Note that when using the file system as the target storage (the storage.type: fs setting), launching the utility in restore mode requires two directories to be mounted: a source directory and a target directory. For more information, see the reference of DGCLI command-line arguments.

Note

To use the restore mode with On-Premise versions 1.16.0 or lower, you must always add the --with-license-v1 argument.

"save" mode

Functions of the save mode are opposite to the ones of the restore mode. In the save mode, DGCLI uploads the installation artifacts from the selected storage to the file system. Installation artifacts must be previously downloaded to the source storage using the pull mode.

Note that when using the file system as the target storage (the storage.type: fs setting), launching the utility in save mode requires two directories to be mounted: a source directory and a target directory. For more information, see the reference of DGCLI command-line arguments.

Note

To use the save mode with On-Premise versions 1.16.0 or lower, you must always add the --with-license-v1 argument.

"manifest" mode

The manifest mode is used for managing manifests in the S3 compatible storage when cleaning old data.

Dependencies

• On-Premise License that allows you to download the installation artifacts related to the purchased products.

  To get the license, see the instruction.

• Host with the Docker Engine installed, which will be used to launch DGCLI.

  To use DGCLI in the license or pull modes, the host must have Internet access to public update servers.

• Depending on the selected storage type:

  • S3 compatible installation artifacts storage.
  • Free disk space on the file system of the DGCLI host.

• Docker Registry, if it will be used to store service images.