23s2s

✏️ Editing: panel_data_snapshot.py

# Copyright © Cloud Linux GmbH & Cloud Linux Software, Inc 2010-2026 All Rights Reserved
#
# Licensed under CLOUD LINUX LICENSE AGREEMENT
# http://cloudlinux.com/docs/LICENSE.TXT

"""
Stat-validated snapshot of the cPanel authorization data consumed by
SimpleAgent._authorize_sender (the panel tenant-user set and the
domain -> owner mapping).

Why this exists. _authorize_sender resolves the tenant set (cpusers()) and
the reported domain's owner (domain_owner()) on every payload. On cPanel,
clcommon.cpapi has no cache for either call: cpusers() re-reads
/etc/userplans and domain_owner() re-parses /etc/userdatadomains IN FULL
(Python callback per line, no early break) per call. Authorization cost
therefore grows linearly with the account/domain count, and on large servers
it pins the GIL-bound agent at one full core while accepting only tens of
payloads per second (2570 accounts / 16K domain lines = ~55 ms of auth CPU
per payload measured on a customer box), silently dropping most telemetry at
the kernel accept queue. The DirectAdmin backend caches its domain DB and
Plesk resolves via local DB queries, so only cPanel needs this layer; every
other panel keeps fresh per-payload lookups.

Why a stat-validated snapshot is NOT the TTL cache that was removed in
0.4-28 (commit b6aec39). That cache expired entries by TIME, so for up to
the TTL it could serve answers that a fresh lookup would no longer return —
that stale-decision window was the root of all five staleness/race findings
against it (new-tenant trust-all, fail-open clobbering a resolved owner,
stale overwrite races). Here validity is proven by DATA on every use: each
lookup stat()s the backing panel files and serves the snapshot only while
their identity signature (inode, size, mtime_ns, ctime_ns) is unchanged —
i.e. only while a fresh parse would return byte-identical results. Any
change to the files (cPanel rewrites them atomically via rename, changing
the inode) invalidates the snapshot before the next decision, which then
triggers exactly one single-flight re-parse. There is no window in which a
served answer can differ from a fresh lookup, beyond the parse itself racing
the very write it is reading — a race fresh per-payload lookups have too —
and one theoretical blind spot the signature shares with every stat-based
scheme (git's "racily clean" problem): an IN-PLACE rewrite that keeps the
inode and size and lands within the same kernel coarse-clock tick as the
previous signature (~ms granularity on mtime_ns/ctime_ns) is undetectable
until the file next changes. cPanel itself always rename-replaces these
files (new inode), so only a non-panel writer editing them in place with
equal-length content could hit it.

Decision parity with the fresh path is preserved exactly:
- a domain absent from /etc/userdatadomains -> None (caller fails open);
- a domain listed more than once -> DuplicateData with clcommon's message
  (caller treats it as unresolvable, fails open) — raised from the snapshot
  WITHOUT re-parsing, so repeated payloads for a duplicated domain cannot
  force per-payload parses;
- any snapshot machinery failure (stat or parse errors) raises
  SnapshotUnavailable and the caller falls back to the fresh clcommon call.
"""

import os
import pwd
from threading import Lock

from clcommon.cpapi import cpusers, getCPName
from clcommon.cpapi.cpapiexceptions import DuplicateData

class SnapshotUnavailable(Exception):
    """
    The snapshot machinery itself failed (backing files could not be read or
    parsed). The caller must fall back to a fresh clcommon lookup — this is
    never raised for data-level outcomes (unknown domain, duplicated domain),
    which are authoritative and mirror the fresh path.
    """

class _StatKeyedValue:
    """
    A value derived from a set of files, memoized against their stat
    signature. get() stat()s every path on every call (a few microseconds)
    and returns the memoized value only when the signature of ALL paths is
    unchanged; otherwise it re-runs the loader under a lock (single-flight:
    concurrent callers wait and reuse the fresh result instead of parsing in
    parallel).

The signature covers inode, size, mtime_ns and ctime_ns per path, plus
    the path's absence (a missing file participates as None, so a file
    appearing or disappearing also invalidates). The signature is taken
    BEFORE the loader runs: if a file changes mid-parse, the stored value is
    keyed under the pre-change signature, so the very next get() re-validates
    and rebuilds — the memo self-heals instead of trusting a torn read. It is
    re-taken under the lock: a waiter whose pre-lock stat predates a
    concurrent change would otherwise mismatch the fresh entry a parallel
    rebuild just stored and re-run the loader once per waiter (evicting the
    correctly-keyed entry) instead of reusing the single-flight result.
    """

def __init__(self, paths, loader):
        self._paths = tuple(paths)
        self._loader = loader
        self._lock = Lock()
        # (signature, value) installed atomically; read without the lock.
        self._entry = None

def _signature(self):
        sig = []
        for path in self._paths:
            try:
                st = os.stat(path)
            except OSError:
                sig.append((path, None))
            else:
                sig.append((path, st.st_ino, st.st_size, st.st_mtime_ns, st.st_ctime_ns))
        return tuple(sig)

def get(self):
        entry = self._entry
        if entry is not None and entry[0] == self._signature():
            return entry[1]
        with self._lock:
            # Re-stat under the lock (see class docstring): the entry a
            # concurrent rebuild stored must be compared against the CURRENT
            # signature, not the one this caller computed before waiting.
            sig = self._signature()
            entry = self._entry
            if entry is not None and entry[0] == sig:
                return entry[1]
            value = self._loader()
            self._entry = (sig, value)
            return value

class CpanelAuthDataSnapshot:
    """
    O(1) per-payload provider of the two panel lookups _authorize_sender
    needs, each memoized against the stat signature of the cPanel files that
    are its single source of truth:

- tenant_users(): frozenset(cpusers()), backed by /etc/userplans;
    - domain_owner(domain): exact-key lookup in a {domain: [owners]} map
      built in ONE pass over /etc/userdatadomains (via the same clcommon
      parser the fresh path uses), instead of one full parse per payload.

Loader callables and backing paths are injected by
    build_panel_auth_snapshot (and by unit tests).
    """

def __init__(self, logger, tenants_paths, tenants_loader, owners_paths, owners_loader):
        self._logger = logger

def _load_tenants():
            value = frozenset(tenants_loader())
            logger.info('[PanelSnapshot] tenant set rebuilt from %s: %d account(s)', list(tenants_paths), len(value))
            return value

def _load_owners():
            value = owners_loader()
            logger.info(
                '[PanelSnapshot] domain-owner map rebuilt from %s: %d domain(s)', list(owners_paths), len(value)
            )
            return value

self._tenants = _StatKeyedValue(tenants_paths, _load_tenants)
        self._owners = _StatKeyedValue(owners_paths, _load_owners)

def tenant_users(self) -> frozenset:
        """
        The panel hosting-account set, equal to frozenset(cpusers()) over the
        current /etc/userplans bytes. Raises SnapshotUnavailable on machinery
        failure (caller falls back to a fresh cpusers()).
        """
        try:
            return self._tenants.get()
        # Any loader/stat failure, whatever its type, means "snapshot
        # machinery failed" -> deliberately wrapped for the fresh fallback.
        except Exception as e:
            raise SnapshotUnavailable(f'tenant set: {e}') from e

def domain_owner(self, domain: str):
        """
        The owner username of *domain*, with exact clcommon parity: None when
        the domain is not in the map (caller fails open), DuplicateData —
        clcommon's own exception and message — when the file lists it more
        than once (caller treats it as unresolvable and fails open; raised
        from the map so a duplicated domain cannot force per-payload parses).
        Raises SnapshotUnavailable on machinery failure (caller falls back to
        a fresh domain_owner()).
        """
        try:
            owners = self._owners.get().get(domain)
        # Any loader/stat failure, whatever its type, means "snapshot
        # machinery failed" -> deliberately wrapped for the fresh fallback.
        except Exception as e:
            raise SnapshotUnavailable(f'domain-owner map: {e}') from e
        if not owners:
            return None
        if len(owners) > 1:
            raise DuplicateData(f"domain {domain} belongs to few users: [{','.join(owners)}]")
        return owners[0]

def build_panel_auth_snapshot(logger):
    """
    Build the snapshot when the running panel is cPanel and the clcommon
    internals it relies on are present; return None otherwise (the agent then
    keeps fresh per-payload lookups — the status quo for DirectAdmin, whose
    backend caches internally, for Plesk, which resolves via local DB
    queries, and for any clcommon whose internals moved).

The map loader reuses clcommon's own parser (_parse_userdatadomains) and
    path constant, so line handling — comment/garbage skipping, the
    'domain: user==owner==type==...' split — is clcommon's, not a copy, and
    the backing paths in the stat signature are exactly the paths the fresh
    path would read: CPANEL_USERDATADOMAINS_PATH is a ';'-joined list whose
    '{user}' template entry is expanded to the daemon's euid name the same
    way the parser expands it before opening. Paths that do not exist are
    skipped and participate in the signature as absent; a path that cannot
    be opened OR fully read raises (surfacing as SnapshotUnavailable ->
    fresh fallback) instead of letting the parser's quiet error handling
    freeze an empty or truncated map under a valid signature.
    """
    try:
        from clcommon.cpapi.plugins import cpanel
    # Whatever fails here, and however it fails, the answer is the same:
    # no snapshot, keep fresh per-payload lookups (deliberate broad catch).
    except Exception as e:  # noqa: BLE001
        logger.warning('[PanelSnapshot] disabled, cPanel plugin unavailable: %s', str(e))
        return None
    try:
        if getCPName() != getattr(cpanel, '__cpname__', 'cPanel'):
            return None
        userplans_path = cpanel.CPANEL_USERPLANS_PATH
        userdata_path = cpanel.CPANEL_USERDATADOMAINS_PATH
        parse_userdatadomains = cpanel._parse_userdatadomains
        if not isinstance(userplans_path, str) or not isinstance(userdata_path, str):
            raise TypeError('unexpected clcommon path constant types')
        # clcommon's parser substitutes '{user}' with the caller's euid name
        # BEFORE opening (cpanel._parse_userdatadomains) — expand the template
        # identically here, or the stat signature would watch the literal
        # template path (a permanent ENOENT) while the parser reads the
        # expanded file, whose changes would then never invalidate the
        # snapshot. The daemon's euid never changes, so expanding once at
        # build time equals clcommon's per-call expansion.
        if '{user}' in userdata_path:
            userdata_path = userdata_path.replace('{user}', pwd.getpwuid(os.geteuid()).pw_name)
        owners_paths = tuple(userdata_path.split(';'))
    # Same contract as above: any detection/constant/attr failure disables
    # the snapshot rather than the agent (deliberate broad catch).
    except Exception as e:  # noqa: BLE001
        logger.warning('[PanelSnapshot] disabled, falling back to fresh per-payload lookups: %s', str(e))
        return None

def load_owner_map():
        owners = {}

def collect(_path, domain, domain_data):
            owners.setdefault(domain, []).append(domain_data[0])

for path in owners_paths:
            # clcommon's parser swallows I/O failures ('except IOError:
            # continue' under quiet=True — and the except wraps its WHOLE
            # read loop, not just the open; os.path.exists is also False on
            # a stat error), which here would memoize an EMPTY or TRUNCATED
            # map under the current valid signature (a read error changes no
            # ino/size/mtime/ctime) and freeze fail-open answers until the
            # file next changes. Probe each backing file with a FULL read
            # first so both open-time and mid-read failures (EIO/ESTALE)
            # raise — the caller turns them into SnapshotUnavailable and
            # serves fresh lookups (which self-heal) until the file is
            # readable again. Binary mode keeps the probe about I/O only
            # (decode errors surface identically from the parse either way).
            # Residual window: an I/O error materializing between this read
            # and the parser's re-read of the just-cached pages is still
            # swallowed; that requires the error to first appear inside a
            # sub-millisecond window on page-cache-hot data.
            try:
                with open(path, 'rb') as probe:
                    probe.read()
            except (FileNotFoundError, NotADirectoryError):
                # Genuinely absent: the parser skips it and the stat
                # signature tracks it as absent.
                continue
            parse_userdatadomains(path, collect, quiet=True)
        return owners

return CpanelAuthDataSnapshot(
        logger,
        tenants_paths=(userplans_path,),
        tenants_loader=cpusers,
        owners_paths=owners_paths,
        owners_loader=load_owner_map,
    )

Sid Gifari From Gifari Industries - BD Cyber Security Team