Stackvora.tech

Build a Local Package Mirror

Maintaining up-to-date software packages across an organization's servers, especially in environments with limited or no internet access, is a critical…

Maintaining up-to-date software packages across an organization's servers, especially in environments with limited or no internet access, is a critical operational challenge. Local package mirrors provide a robust solution, offering benefits such as accelerated deployments, reduced internet bandwidth consumption, and enhanced security through controlled package sources. This article details the strategies and tools for constructing and managing local package mirrors for Debian/Ubuntu (APT-based) and RHEL/CentOS/Fedora (RPM-based) distributions, including considerations for air-gapped networks and commercial repository managers.

Mirroring APT Repositories (Debian/Ubuntu)

For Debian and Ubuntu systems, the primary tool for creating a full local mirror is apt-mirror. This utility is designed to download entire or partial APT repositories, including main, updates, security, and proposed components for specified distributions and architectures.

Setting up apt-mirror

First, install apt-mirror on your designated mirror server:

sudo apt update
sudo apt install apt-mirror

The configuration file is located at /etc/apt/mirror.list. A typical configuration specifies the base path for the mirror, the architectures to include, and the repositories themselves. Here's an example for mirroring Ubuntu 22.04 LTS (Jammy Jellyfish) for AMD64:

# /etc/apt/mirror.list

##
# set base path for all repositories
#
set base_path /var/spool/apt-mirror

##
# set default distro
#
# set defaultarch  <release>

# set elements per sub directory.
#
# set limit_directory_number 5

# set run daily
#
# set run_postmirror 0

# set default log file
#
# set default_log_file mirror.log

# set clean up post mirror
#
# set clean_after_update yes

#
# APT repositories
#
deb http://archive.ubuntu.com/ubuntu jammy main restricted universe multiverse
deb http://archive.ubuntu.com/ubuntu jammy-security main restricted universe multiverse
deb http://archive.ubuntu.com/ubuntu jammy-updates main restricted universe multiverse
# deb http://archive.ubuntu.com/ubuntu jammy-backports main restricted universe multiverse

# If you want to mirror specific architectures, add:
# set defaultarch amd64

# Exclude source packages if not needed to save space
# set default_source_urls http://archive.ubuntu.com/ubuntu

# To mirror deb-src lines, ensure they are uncommented in your mirror.list, e.g.:
# deb-src http://archive.ubuntu.com/ubuntu jammy main restricted universe multiverse

# For PPA repositories, you'd typically add a line like this after adding its GPG key:
# deb http://ppa.launchpad.net/ondrej/php/ubuntu jammy main

After configuring, run apt-mirror:

sudo apt-mirror

This process can take many hours and consume hundreds of gigabytes of disk space, depending on the distributions and components selected. For Ubuntu 22.04 LTS (jammy) main/restricted/universe/multiverse for amd64, initial sync can exceed 200GB. Schedule subsequent runs using cron (e.g., daily) to keep the mirror updated:

# /etc/cron.d/apt-mirror
0 4 * * * apt-mirror > /var/spool/apt-mirror/mirror.log 2>&1

Clients can then be configured to use this mirror by modifying /etc/apt/sources.list:

# /etc/apt/sources.list on client
deb http://your-mirror-ip/ubuntu jammy main restricted universe multiverse
deb http://your-mirror-ip/ubuntu jammy-security main restricted universe multiverse
deb http://your-mirror-ip/ubuntu jammy-updates main restricted universe multiverse

Remember to run sudo apt update on clients after changing sources.

Mirroring RPM Repositories (RHEL/CentOS/Fedora)

For RHEL-based systems, reposync, part of the yum-utils or dnf-utils package, is used to synchronize repositories. Unlike apt-mirror, reposync downloads only the package files and doesn't generate the repository metadata (repomd.xml, etc.); this metadata must be created separately using createrepo_c.

Setting up reposync

Install the necessary tools on your mirror server (examples for RHEL 8/9 and CentOS Stream 8/9):

sudo dnf install yum-utils createrepo_c -y

For older systems (e.g., CentOS 7), you might use yum install yum-utils createrepo.

Create a base directory for your mirrors (e.g., /opt/repo). Then, synchronize the desired repositories. It's often helpful to define repository configurations in a .repo file first, e.g., /etc/yum.repos.d/centos-stream-mirror.repo:

[BaseOS]
name=CentOS Stream $releasever - BaseOS
baseurl=http://mirror.centos.org/$contentdir/$releasever/BaseOS/$basearch/os/
gpgcheck=0
enabled=1

[AppStream]
name=CentOS Stream $releasever - AppStream
baseurl=http://mirror.centos.org/$contentdir/$releasever/AppStream/$basearch/os/
gpgcheck=0
enabled=1

[Extras]
name=CentOS Stream $releasever - Extras
baseurl=http://mirror.centos.org/$contentdir/$releasever/Extras/$basearch/os/
gpgcheck=0
enabled=1

Now, synchronize each repository. For CentOS Stream 9 ($releasever expands to '9', $basearch to 'x86_64'):

REPO_BASE_DIR="/opt/repo"
mkdir -p $REPO_BASE_DIR/centos/9/BaseOS/x86_64/os
mkdir -p $REPO_BASE_DIR/centos/9/AppStream/x86_64/os
mkdir -p $REPO_BASE_DIR/centos/9/Extras/x86_64/os

# Sync BaseOS
sudo reposync --repoid=BaseOS --download-path=$REPO_BASE_DIR/centos/9/BaseOS/x86_64/os --newest-only --delete --arch=x86_64
sudo createrepo_c $REPO_BASE_DIR/centos/9/BaseOS/x86_64/os

# Sync AppStream
sudo reposync --repoid=AppStream --download-path=$REPO_BASE_DIR/centos/9/AppStream/x86_64/os --newest-only --delete --arch=x86_64
sudo createrepo_c $REPO_BASE_DIR/centos/9/AppStream/x86_64/os

# Sync Extras
sudo reposync --repoid=Extras --download-path=$REPO_BASE_DIR/centos/9/Extras/x86_64/os --newest-only --delete --arch=x86_64
sudo createrepo_c $REPO_BASE_DIR/centos/9/Extras/x86_64/os
  • --repoid: Specifies which repository to sync.
  • --download-path: The destination directory for packages.
  • --newest-only: Only download the newest version of packages.
  • --delete: Delete packages from the local repository that are no longer present in the upstream repository (use with caution).
  • --arch: Specify architecture (e.g., x86_64, aarch64).

After syncing, createrepo_c must be run in each package directory to generate the necessary metadata. This can be time-consuming for large repositories.

To automate updates, schedule these commands with cron, ensuring reposync runs before createrepo_c for each repository.

Clients can then be configured by creating or modifying /etc/yum.repos.d/local.repo:

# /etc/yum.repos.d/local.repo on client
[local-BaseOS]
name=Local CentOS Stream 9 - BaseOS
baseurl=http://your-mirror-ip/centos/9/BaseOS/x86_64/os/
gpgcheck=0
enabled=1

[local-AppStream]
name=Local CentOS Stream 9 - AppStream
baseurl=http://your-mirror-ip/centos/9/AppStream/x86_64/os/
gpgcheck=0
enabled=1

[local-Extras]
name=Local CentOS Stream 9 - Extras
baseurl=http://your-mirror-ip/centos/9/Extras/x86_64/os/
gpgcheck=0
enabled=1

Disable upstream repositories on the client to ensure only the local mirror is used.

Serving the Mirror via HTTP/HTTPS

Both APT and RPM clients expect repositories to be accessible via HTTP or HTTPS. Install and configure a web server like Nginx or Apache on your mirror server.

Nginx Example

Install Nginx:

sudo apt install nginx  # Debian/Ubuntu
sudo dnf install nginx -y # RHEL/CentOS

Configure a server block in Nginx, for example, by creating /etc/nginx/conf.d/repo.conf:

server {
    listen 80;
    server_name your-mirror-ip-or-hostname;
    root /opt/repo; # For RPMs, or /var/spool/apt-mirror/mirror for APT
    autoindex on; # Allows directory listing

    location / {
        # Optional: restrict access
        # allow 192.168.1.0/24;
        # deny all;
    }

    # For APT mirrors, the structure is /var/spool/apt-mirror/mirror/archive.ubuntu.com/ubuntu/
    # You might need a rewrite or alias if you want a cleaner URL for APT clients:
    # location /ubuntu {
    #     alias /var/spool/apt-mirror/mirror/archive.ubuntu.com/ubuntu/;
    #     autoindex on;
    # }
}

Restart Nginx:

sudo systemctl restart nginx

Commercial Repository Managers (Nexus/Artifactory)

For more advanced use cases, especially in larger organizations or those managing multiple package types (Maven, npm, Docker, PyPI, etc.), dedicated repository managers like Sonatype Nexus Repository or JFrog Artifactory offer significant advantages.

These tools act as proxy caches, downloading packages on demand and storing them locally. They also support hosting internal private repositories and can integrate with CI/CD pipelines.

Key Features:

  • Proxy Repositories: Cache packages from external sources (e.g., repo.maven.apache.org, pypi.org, archive.ubuntu.com).
  • Hosted Repositories: Store your own internally developed packages.
  • Group Repositories: Combine multiple proxy and hosted repositories into a single URL, simplifying client configuration.
  • Security: Granular access control, vulnerability scanning (with commercial add-ons), and GPG signature verification.
  • Scalability: Designed for high-performance and high-availability environments.
  • Multi-Format Support: Handles a wide array of package formats beyond just APT/RPM.

Setting up Nexus or Artifactory is a more involved process, typically involving Java runtime environments, database configurations, and detailed web UI administration. The general workflow for package mirroring involves:

  1. Install and configure the repository manager software.
  2. Create 'Proxy' repositories for each upstream source you wish to mirror (e.g., "Ubuntu Main" proxy, "CentOS BaseOS" proxy).
  3. Configure 'Group' repositories to combine these proxies into a single endpoint for clients.
  4. Point client systems to the appropriate group repository URL provided by Nexus/Artifactory.

For instance, an APT client would point its sources.list to http://nexus-server:8081/repository/ubuntu-group/. An RPM client's .repo file would use baseurl=http://nexus-server:8081/repository/centos-group/.

Air-Gapped Network Considerations

Air-gapped environments, completely isolated from the internet, require a specific strategy. A common approach involves a "jump-mirror" or "DMZ mirror" which has temporary or controlled access to the internet.

The process generally follows these steps:

  1. External Mirror: Set up a mirror server (as described above using apt-mirror or reposync) in a network segment that can access the internet. This server performs the initial and periodic synchronization with upstream repositories.
  2. Data Transfer: Periodically transfer the mirrored data from the external mirror to the internal air-gapped mirror. This is often done via rsync over a secure, temporary connection (e.g., a physically disconnected cable that is briefly connected), or by physically transporting storage media.
  3. Internal Mirror: The air-gapped network has its own mirror server that hosts the copied packages. Clients in the air-gapped network connect to this internal mirror.

Using rsync is efficient as it only transfers changed files. An example rsync command from the external to internal mirror:

# On the external mirror, when connection to internal mirror is established
rsync -avz --delete /var/spool/apt-mirror/mirror/ user@internal-mirror-ip:/var/spool/apt-mirror/mirror/
# Or for RPMs:
rsync -avz --delete /opt/repo/ user@internal-mirror-ip:/opt/repo/

--delete is crucial to remove packages on the destination that are no longer present on the source, ensuring synchronization. After an rsync for RPMs, remember to run createrepo_c on the internal mirror server to update metadata for any new or changed packages.

Common Pitfalls

  • Disk Space: Full mirrors consume significant disk space. APT mirrors for a single Ubuntu release (main, restricted, universe, multiverse, security, updates for one arch) can easily exceed 200GB. RHEL/CentOS mirrors are similarly large. Plan for terabytes of storage if mirroring multiple releases and architectures.
  • Network Bandwidth & Time: Initial syncs are bandwidth-intensive and can take days. Schedule these during off-peak hours.
  • Permissions: Ensure the web server user (e.g., www-data for Apache/Nginx on Debian/Ubuntu) has read access to the mirror directories.
  • GPG Keys: While local mirrors reduce external dependencies, clients still need the correct GPG keys for the original repositories to verify package signatures, even if served from your local mirror. Ensure these are distributed to clients.
  • Updating Metadata: For RPM mirrors, failing to run createrepo_c after a reposync will result in clients not seeing new packages.
  • Configuration Errors: Typos in mirror.list, .repo files, or web server configurations are common. Always test client connections thoroughly.
  • HTTP vs HTTPS: While HTTP is simpler to set up, using HTTPS for your local mirror is recommended for production environments to protect against tampering and ensure client trust. This requires setting up SSL certificates on your web server.

Back to the knowledge base · Ask the AI assistant