Ubuntu Pro / ESM Servers Showing Large Numbers of Missing Security Updates

Overview:

ESM repositories are not enabled correctly
apt is not using the ESM repositories
unattended-upgrades is not configured for ESM
Packages are being held or pinned
Upgrade jobs are failing silently
Systems have not completed dependency transitions (full-upgrade)
Repository access issues are preventing downloads
The server is attached to Ubuntu Pro but ESM services are disabled

Typical indicators include:

Historical Apache, kernel, OpenSSL, or MariaDB USNs still appearing
Scanner output showing:
- “available with Pro”
- “update with Pro”
- “due from 2021/2022/2023”
Very large numbers of outstanding vulnerabilities despite Ubuntu Pro licensing

Recommended Host Checks

Verify Ubuntu Pro Status

pro status

Expected output should show services enabled:

SERVICE          ENTITLED  STATUS
esm-apps         yes       enabled
esm-infra        yes       enabled
livepatch        yes       enabled

Common Issue

Systems are attached to Ubuntu Pro but services are disabled:

SERVICE          ENTITLED  STATUS
esm-apps         yes       disabled
esm-infra        yes       disabled

Fix

sudo pro enable esm-infra
sudo pro enable esm-apps

Confirm APT Can See ESM Repositories

sudo apt update
apt list --upgradable

Then check package policy:

apt-cache policy apache2

or:

apt-cache policy mariadb-server

Expected repositories:

https://esm.ubuntu.com/infra/ubuntu
https://esm.ubuntu.com/apps/ubuntu

If ESM repositories are missing

Ubuntu Pro may not be integrated correctly with APT.

Check unattended-upgrades Configuration

Inspect:

cat /etc/apt/apt.conf.d/50unattended-upgrades

Ensure ESM origins are included:

"${distro_id}ESMApps:${distro_codename}-apps-security";
"${distro_id}ESM:${distro_codename}-infra-security";

Common Problem

Older configurations only contain:

"${distro_id}:${distro_codename}-security";

This allows standard security updates but excludes ESM updates entirely.

Check for Held Packages

Held packages can prevent critical updates.

apt-mark showhold

Look for packages such as:

apache2
linux-image
libc6
openssl

If present, investigate why they are held.

Check APT Pinning / Preferences

Inspect:

ls -la /etc/apt/preferences.d/
cat /etc/apt/preferences

Incorrect pinning can block ESM packages.

Also review package priorities:

apt-cache policy apache2

Watch for:

ESM repositories having lower priority
Installed versions pinned higher than upgrade candidates

Review unattended-upgrades Logs

Check for upgrade failures:

grep -i error /var/log/unattended-upgrades/unattended-upgrades.log

Also review recent activity:

tail -100 /var/log/unattended-upgrades/unattended-upgrades.log

Common findings include:

dependency conflicts
package kept back
dpkg interrupted
repository authentication failures
proxy failures
disk space issues

Check for Packages Being Kept Back

sudo apt upgrade

Then:

sudo apt full-upgrade

Packages kept back for long periods often indicate:

dependency transitions
obsolete packages
kernel meta-package issues

Verify Ubuntu Release and Kernel

Check OS version:

lsb_release -a
uname -a

Older releases may require:

HWE enablement
release upgrades
newer Ubuntu Pro client versions

This is particularly relevant for Ubuntu 16.04 and 18.04 systems.

Validate Repository Connectivity

sudo apt update

Look for:

TLS errors
proxy authentication failures
DNS issues
IPv6 connectivity problems
expired certificates

Test direct connectivity:

curl -I https://esm.ubuntu.com

Check for Cloned / Broken Ubuntu Pro Attachments

Cloned systems can retain invalid Ubuntu Pro state.

Possible symptoms:

Pro appears attached
ESM updates unavailable
Authentication errors

Reattach Ubuntu Pro

sudo pro detach
sudo pro attach <token>

Most Likely Root Causes

Based on environments commonly showing this behaviour:

Most Common Causes

1. unattended-upgrades Not Configured for ESM

Very common where Ubuntu Pro was enabled after deployment.

2. esm-apps / esm-infra Disabled

The server is attached to Ubuntu Pro but ESM services are not enabled.

3. Systems Only Running `apt upgrade`

Without periodic full-upgrade, dependency transitions can remain unresolved for years.

4. Held or Pinned Packages

Especially common on web servers with customised Apache deployments.

Quick Triage Command Set

Useful for rapid validation:

pro status

apt-cache policy apache2

apt-mark showhold

sudo apt update

apt list --upgradable

grep Allowed-Origins -A20 /etc/apt/apt.conf.d/50unattended-upgrades

Important Note Regarding Vulnerability Scanners

Some scanners may continue reporting historical USNs even after superseding fixes are installed.

Always validate installed package versions directly:

dpkg -l | grep apache2

Then compare against:

Ubuntu Security Tracker
USN fixed versions
Installed repository versions

However, if large numbers of historical advisories are still present across multiple packages and years, this usually indicates a genuine patch management issue rather than false positives. Do report any issues to your assessors where you have found this to be the case.

Additional Considerations

Be aware of environments where:

package pinning is intentional
systems are vendor-managed appliances
third-party PPAs override Ubuntu packages
Apache or MariaDB packages are sourced externally

In these scenarios, Ubuntu ESM updates may not apply normally.

Contact Cyber Tec Security

If and when you have checked everything and you believe you are a-ok, 100% patched and "all good", let your assessors know, and they will manually check to make sure you are, and if you are and this is a scanner false positive (due to backporting or similar) that can be ignored for your assessment.