Mirrors

Anaconda Server enables you to create a local copy of a repository so users can access approved software components from a centralized on-premise or cloud location.

Note

In order to mirror, you must first create a channel for your organization to use.

The mirror can be complete, partial, or include/exclude specific packages or types of packages based on CVE score, license type, and version. You can also create a mirror in an air-gapped environment, allowing access to approved packages and other artifacts without access to the larger internet.

Common use case of channel filtering with multiple mirrors configured to that channel:

You can set up multiple mirrors into a single channel of whatever artifacts you choose. Filters can be set at the channel level to apply to all mirrors of a channel, or at the mirror level to apply to specific mirrors.

For example, say you want to achieve the following mirroring goals:

  1. Prevent all packages with a GPL license from being mirrored
  2. Prevent a specific Windows package from getting into the channel

To achieve the first goal, you can exclude packages by all GPL License types via the channel editor. To achieve the second goal, you can create separate mirrors—perhaps one for each operating system—that filter by the subdirectory.

You can also exclude packages by CVE score. See Filtering Channel CVEs.

This topic provides guidance on the following actions:

Creating mirrors

To create a mirror, click the green Mirror button from any channel page.

../_images/mirrorbutton.png

You will be presented with the Create mirror page:

../_images/createmirror.png

Set the details, filters, and frequency you’d like, and then click Submit.

Via the CLI

Run the following command to create a mirror and specify where it receives packages from:

# Replace <mirror_name> your mirror and <channel_name> with your channel
conda repo mirror --create <mirror_name> --source=https://<example-website.com> -c <channel_name>

Active or passive mirroring

Mirroring can be either active or passive. An active mirror would clone channel artifacts with their binary content (files) and metadata. A passive mirror would only clone channel artifacts metadata, while the actual files would be fetched on demand (on first request).

Essentially, when you mirror actively, you’re really mirroring passively and manually going through and fetching files from the upstream channel.

Defining your external source and mirror type

Note

Please whitelist the following URLs before mirroring:

Select the mirror Type that matches the external source channel. You can mirror Conda, PyPI, or CRAN.

Conda

Use http://repo.anaconda.com/pkgs/main as the external source channel for conda type.

../_images/condatype.png

PyPI

Use https://pypi.python.org/ as the external source channel for PyPI (python_simple) type.

For the time being, you only need to fill in the Name, External Source Channel, Type, and Project when mirroring PyPI packages—no need to modify package filters.

Note

Make sure to add the specific PyPI packages you’d like to mirror to the Projects cell. Hit enter after typing in the name of each package. This will turn the name blue, indicating that the package you entered has been accepted.

../_images/pypitypeblue.png

CRAN

Use https://cran.r-project.org/ as the external source channel for CRAN type.

../_images/crantype.png

Mirroring filters

Package filters

You can filter which packages will be included in a mirror in the Package Filters section on the Edit mirror page. Include and exclude by package name and license type.

../_images/packagefilter.png

Filter packages By License Type to include or exclude them from your mirror. See License types for more details on licenses.

Note

When limiting the mirror to specific architecture, it’s a good idea to include the noarch package type to ensure that cross-platform packages are included. For example, if you have no need for Windows-compatible packages, you would include win-32, win-64, and noarch package types.

Date range filters

You can also filter packages by the date the package was modified on the source site.

../_images/daterange.png

View channel filters

Click the View Channel Filters button at the bottom of the Package Filters section on the Edit mirror page to view the filters applied to all mirrors within the channel.

../_images/viewchanbutton.png

Mirror frequency

You can modify how frequently a channel is mirrored at the bottom of the Edit mirror page. You can also run the mirror immediately by checking the box beside Run Now.

Custom mirroring frequency expressions are written in cron syntax. You can use https://crontab.guru to learn about and validate cron expressions.

Caution

Mirroring large channels, such as anaconda repository main for conda-forge, can take several hours. It is not recommended to set a frequency higher than daily due to mirror collision between current and updating mirrors.

../_images/mirrorfrequency.png

Note

Even if you want to run the mirror immediately by selecting the Run Now checkbox, you still need to set a frequency for how often you would like the channel updated to reflect the current state of the mirrored repository.

Editing mirrors

Click the three dots on a mirror, and then click Edit.

../_images/editmirrorbutton.png

You will be presented with the Edit mirror page:

../_images/editmirror.png

Make the changes you’d like implemented, and then click Submit.

Edit mirror frequency

You can modify how frequently a channel is mirrored at the bottom of the Edit mirror page. You can also run the mirror immediately by checking the box beside Run Now.

Custom mirroring frequency expressions are written in cron syntax. you can use https://crontab.guru to learn about and validate cron expressions.

Caution

Mirroring large channels, such as anaconda repository main for conda-forge, can take several hours. It is not recommended to set a frequency higher than daily due to mirror collision between current and updating mirrors.

../_images/mirrorfrequency.png

Viewing mirror history

You can view the history of mirrors you have created by clicking the History tab in a channel.

On the History page, you will see the following possible states for mirroring:

  • completed
  • failed
  • pending
  • running
../_images/mirrorhistory.png

Viewing all repository mirrors

View mirror progress and results globally for all users from the All Repository Mirrors view.

../_images/all_repositories_view.png

This view is available only to users whose role in Keycloak has the mirror attribute set to manage.

../_images/keycloak_role_attribute_mirror.png

View mirror status, which step is currently being performed, how long the mirror has been running, when it will complete, and the last time the state was updated.

../_images/mirror_status.png

Get statistics about packages as your mirror populates. You can also view which packages are active or passive and how many packages are being filtered out of your repositories due to license or CVE score restrictions.

../_images/mirror_details_modal.png

Via the CLI

Run the following command to view all repository mirrors and their details via the CLI:

conda repo mirror --list-all

Run the following command to show the synchronization state of a mirror:

# Replace <mirror_name> with the name of your mirror
conda repo mirror --sync_state <mirror_name>

Viewing mirror status by channel

View the status of a channel’s mirror from the channel view. Select a mirror, then click on Details to view the status details of the mirror.

../_images/channel-mirror-status-view.png