Technical Deep Dive: Portage & Binary Package Distribution

The single biggest complaint about Gentoo is compilation time. Firefox takes 45 minutes. KDE Plasma takes 3 hours. Chromium takes 6+ hours. LLVM/Clang takes 8 hours.

But here’s the secret: you only compile once.

This guide explains how Argo OS achieves 90%+ time savings by compiling packages on a dedicated build server and distributing pre-built binaries to all other machines.

The Architecture

┌─────────────────────────────────────────────────────────┐
│                 Binary Package Flow                      │
├─────────────────────────────────────────────────────────┤
│                                                          │
│  Tau-Ceti-Lab/Binhost (10.42.0.194)                           │
│  ├─ Compiles ALL packages from source                   │
│  ├─ Automatically creates .gpkg.tar binaries            │
│  ├─ Serves via HTTP (nginx) or SSH                      │
│  └─ 1,100+ packages, ~30GB repository                   │
│                     │                                    │
│                     ▼                                    │
│  Driver Workstation (10.42.0.100)                        │
│  ├─ Points PORTAGE_BINHOST to Tau-Ceti-Lab                   │
│  ├─ Downloads pre-compiled binaries                     │
│  ├─ Installs in seconds instead of hours                │
│  └─ Never runs the compiler                             │
│                                                          │
└─────────────────────────────────────────────────────────┘

Part 1: Configuring the Build Server (Binhost)

make.conf for the Binhost

The binhost needs to create binary packages automatically during every emerge:

# /etc/portage/make.conf (on binhost)

# === Critical: Binary package creation ===
FEATURES="buildpkg binpkg-multi-instance parallel-fetch candy"

# Compression (zstd level 9 = max compression, slower)
BINPKG_COMPRESS="zstd"
BINPKG_COMPRESS_FLAGS="-9"

# Modern package format (replaces old tbz2)
BINPKG_FORMAT="gpkg"

# === CPU Optimization ===
# Use YOUR CPU's native instruction set
COMMON_FLAGS="-march=native -O2 -pipe"
CFLAGS="${COMMON_FLAGS}"
CXXFLAGS="${COMMON_FLAGS}"

# Parallel compilation
MAKEOPTS="-j$(nproc)"
EMERGE_DEFAULT_OPTS="--jobs=4 --load-average=8"

# === GPU Support ===
# Compile for ALL GPU types your clients might use
VIDEO_CARDS="nvidia amdgpu radeonsi intel"

# === Accept all licenses (internal use) ===
ACCEPT_LICENSE="*"

Key insight: buildpkg is the magic flag. Every time you run emerge firefox, Portage automatically saves the compiled result to /var/cache/binpkgs/www-client/firefox-*.gpkg.tar.

Serving Packages via HTTP (Recommended)

HTTP is faster and more reliable than SSH for package distribution:

# Install nginx
emerge www-servers/nginx

# Create config
cat > /etc/nginx/sites-available/binhost << 'EOF'
server {
    listen 8080;
    server_name _;
    
    root /var/cache/binpkgs;
    autoindex on;
    
    # Enable directory listing for Portage
    location / {
        try_files $uri $uri/ =404;
    }
    
    # Optimize for large binary downloads
    sendfile on;
    tcp_nopush on;
    tcp_nodelay on;
}
EOF

# Enable and start
ln -sf /etc/nginx/sites-available/binhost /etc/nginx/sites-enabled/
rc-update add nginx default
rc-service nginx start

Generating the Package Index

After compiling packages, you must regenerate the index:

# Update the Packages file (required for clients to see updates!)
emaint binhost --fix

# Verify it worked
ls -la /var/cache/binpkgs/Packages

Tip: Add this to a cron job or run it after every emerge @world.

Part 2: Configuring Clients (Driver Machines)

make.conf for Clients

# /etc/portage/make.conf (on client machines)

# === Binary package server ===
PORTAGE_BINHOST="http://10.42.0.194:8080"

# Alternative: SSH (slower but works through firewalls)
# PORTAGE_BINHOST="ssh://[email protected]/var/cache/binpkgs"

# === Features ===
FEATURES="binpkg-multi-instance parallel-fetch getbinpkg"

# CRITICAL: Disable signature verification for internal network
# (Official Gentoo binaries are signed, yours are not)
FEATURES="${FEATURES} -binpkg-signing -binpkg-request-signature"

# === Default behavior: always try binary first ===
EMERGE_DEFAULT_OPTS="--getbinpkg --usepkg"

# === Match binhost optimization ===
# This MUST match what the binhost compiled with
COMMON_FLAGS="-march=native -O2 -pipe"
CFLAGS="${COMMON_FLAGS}"
CXXFLAGS="${COMMON_FLAGS}"

# GPU must match what binhost compiled
VIDEO_CARDS="nvidia"

Installing Packages from Binary

# Normal emerge now automatically uses binaries
emerge firefox

# Output will show:
# >>> Emerging binary (1 of 1) www-client/firefox-131.0
# >>> Downloading 'http://10.42.0.194:8080/www-client/firefox-131.0.gpkg.tar'
# >>> Installing (1 of 1) www-client/firefox-131.0

That’s it. A 45-minute compile becomes a 30-second download.

Part 3: Real-World Time Savings

Package	Source Compile	Binary Install	Savings
Firefox	45 minutes	30 seconds	99%
KDE Plasma (full)	3 hours	5 minutes	97%
LibreOffice	2 hours	2 minutes	98%
Chromium	6 hours	1 minute	99.7%
LLVM/Clang	8 hours	3 minutes	99.4%
gcc	2 hours	90 seconds	99%

Total system install: From Stage 3 to full KDE desktop drops from 20+ hours to ~2 hours.

Part 4: The GPU Compatibility Trap

This is where most people fail.

If your binhost has an AMD GPU (VIDEO_CARDS="amdgpu") and your client has NVIDIA (VIDEO_CARDS="nvidia"), packages like Mesa will be compiled without NVIDIA support. Your client will have broken 3D acceleration.

Solution: Configure the binhost to compile for ALL GPU types:

# On binhost /etc/portage/make.conf
VIDEO_CARDS="nvidia amdgpu radeonsi intel i965 iris"

Then recompile GPU-sensitive packages:

emerge --oneshot mesa vulkan-loader xorg-drivers
emaint binhost --fix

Part 5: USE Flag Synchronization

Another common failure: USE flags must match between binhost and clients.

Scenario: Binhost compiles firefox with USE="-wayland". Client has USE="wayland". Portage will refuse to use the binary and compile from source.

Solution: Keep /etc/portage/package.use/ synchronized across all machines. We use git:

# On binhost
cd /etc/portage
git init
git add .
git commit -m "Initial portage config"
git remote add origin [email protected]:commander/portage-config.git
git push -u origin main

# On clients
cd /etc/portage
git clone [email protected]:commander/portage-config.git .

Part 6: Maintenance

Cleaning Old Packages

Binary packages accumulate. Clean them periodically:

# Remove binaries for packages no longer in the tree
eclean-pkg --deep

# Check repository size
du -sh /var/cache/binpkgs/
find /var/cache/binpkgs/ -name "*.gpkg.tar" | wc -l

Full Update Workflow

# 1. Sync the repository
emerge --sync

# 2. Update world (creates new binaries automatically)
emerge -avuDN @world

# 3. Clean up orphaned dependencies
emerge --depclean

# 4. Rebuild anything with broken dependencies
emerge @preserved-rebuild

# 5. Regenerate binary index (CRITICAL!)
emaint binhost --fix

# 6. Optional: sync to mirrors
rsync -avz --delete /var/cache/binpkgs/ backup-server:/var/www/commander-repo/

Troubleshooting

”Binary package not found"

# Check if index is current
ls -la /var/cache/binpkgs/Packages

# Regenerate
emaint binhost --fix

# Verify package exists
ls /var/cache/binpkgs/www-client/firefox*

"USE flag mismatch”

# Check what USE flags the binary was built with
qtbz2 -l /var/cache/binpkgs/www-client/firefox-*.gpkg.tar

# Compare to local USE
emerge -pv firefox

Client compiling instead of downloading

Check FEATURES and EMERGE_DEFAULT_OPTS in client’s make.conf. Also verify the binhost is accessible:

curl -I http://10.42.0.194:8080/Packages

---

This system is the backbone of Argo OS. It transforms Gentoo from “compile everything yourself” into “compile once, deploy everywhere” — the best of both worlds.