Pentest Daemon

The Pentest Daemon is a FastAPI service that orchestrates security scans against target domains. It runs on three nodes — Sentinel VPS (default, external), Tarn-Host (Kali VM), and Izar-Host (Kali VM) — with automatic failover. You pick a target, choose an assessment profile, and it runs through a sequence of tools (nmap, testssl, nuclei, nikto, etc.) reporting findings back in real-time via WebSocket.

Stack

• Backend: FastAPI (Python 3.11+), uvicorn
• Database: SQLite with aiosqlite (WAL mode)
• Port: 8095 on all nodes
• Auth: API key via X-Api-Key header
• Repository: ~/Development/pentest-daemon/ → Gitea InovinLabs/pentest-daemon

Nodes

The same daemon codebase runs on three different hosts, each with a different network perspective and toolset:

Node	Host	IP / URL	Tools	Resources
Sentinel (default)	Hetzner VPS	https://sentinel.Arcturus-Prime.com/pentest-api	Recon subset (nmap, nikto, nuclei, subfinder, testssl, sslscan, dnsrecon, wafw00f, whatweb)	2c / 2GB
Tarn-Host	Kali VM (VMID 150, Tarn-Host)	192.168.20.229:8095	Full Kali (20+ tools)	4c / 8GB
Izar-Host	Kali VM (Proxmox Izar-Host)	10.42.0.203:8095	Full Kali (20+ tools)	varies

Sentinel VPS Specifics

The VPS runs a lightweight deployment focused on external reconnaissance. Heavy exploitation tools (sqlmap, hydra, ffuf, gobuster, amass, zaproxy, wpscan) stay on the full Kali VMs where RAM isn’t a constraint.

• Daemon binds to 127.0.0.1:8095 — not publicly exposed
• Nginx proxies /pentest-api/ → 127.0.0.1:8095 on the same VPS
• SSL via Let’s Encrypt on sentinel.Arcturus-Prime.com
• systemd service: /etc/systemd/system/pentest-daemon.service
• Max concurrent scans: 2 (memory constrained)

Tarn-Host Kali VM Specifics

Property	Value
Proxmox VMID	150
Name	pentest-tarn
Resources	4 cores, 8GB RAM, 64GB disk
Network	vmbr0 (192.168.20.0/24)
IP Config	Static: 192.168.20.229/24, GW 192.168.20.1
SSH	ssh [email protected]
Service	systemctl status pentest-daemon
DB Path	/var/lib/pentest-daemon/scans.db
Scan Output	/var/lib/pentest-daemon/scans/
Reports	/var/lib/pentest-daemon/reports/

Architecture

Arcturus-Prime Admin Panel
    ↓ (HTTPS via CF Pages)
/api/admin/pentest/[...path] (API proxy with failover)
    ↓ picks node: sentinel → Tarn-Host → Izar-Host
    ├─→ Sentinel VPS (https://sentinel.Arcturus-Prime.com/pentest-api)
    ├─→ Tarn-Host Kali VM (http://192.168.20.229:8095)
    └─→ Izar-Host Kali VM (http://10.42.0.203:8095)
         ↓ (subprocess exec)
    Security Tools (nmap, testssl, nuclei, nikto, etc.)
         ↓ (stdout capture + WebSocket broadcast)
    Live Output Panel in Admin UI

The API proxy at /api/admin/pentest/[...path] includes automatic failover: if the selected node returns 5xx or times out, it tries the next node in the chain (sentinel → Tarn-Host → Izar-Host). Response headers X-Pentest-Node and X-Pentest-Failover indicate which node handled the request.

Assessment Profiles

Three built-in profiles control which tools run and in what order:

Quick (3 phases)

Fast surface scan — nmap, whatweb, testssl. Takes 2-5 minutes.

Standard (5 phases)

nmap → whatweb → testssl → nuclei → nikto. Takes 10-20 minutes.

Comprehensive (15 phases)

The full sweep:

1. Reconnaissance: nmap, whatweb, wafw00f, subfinder, dnsrecon
2. SSL/TLS: testssl, sslscan
3. Vulnerability Scanning: nuclei, nikto
4. Discovery: ffuf, gobuster
5. Attack Simulation: XSS, SSRF, LFI, CSRF probes

Takes 30-60+ minutes depending on target surface.

Multi-Node Support

Three nodes are configured in src/config/pentest-nodes.ts:

// src/config/pentest-nodes.ts
{ id: 'sentinel', devUrl: 'https://sentinel.Arcturus-Prime.com/pentest-api' }  // External VPS
{ id: 'Tarn-Host',    devUrl: 'http://192.168.20.229:8095' }                // Andromeda
{ id: 'Izar-Host',       devUrl: 'http://10.42.0.203:8095' }                   // Milky Way

Pass ?node=sentinel (default), ?node=Tarn-Host, ?node=Izar-Host, or ?node=all to target specific nodes. The failover chain is sentinel → Tarn-Host → Izar-Host.

Tool Registry

Tools are configured in app/services/tool_registry.py. Each entry defines the binary path, default arguments, target flag, output flag, and structured output flags for parsers.

Important: The scan_runner captures stdout line-by-line and writes it to the output file. If a tool’s output_flag also writes to the same file, it creates a conflict. Tools that produce useful stdout should have output_flag: None.

Text Output Flags

Tool	Output Flag	Notes
nmap	-oN	Writes normal output to file
testssl	None	stdout captured; --logfile conflicts with pre-created file
dnsrecon	None	stdout captured; --xml conflicts with pre-created file
nuclei	-o	Writes findings to file
nikto	-output	Writes to file
gobuster	-o	Writes to file
ffuf	-o	Writes JSON to file
sslscan	None	stdout captured for raw viewer
whatweb	--log-verbose	Verbose text log
wafw00f	-o	Writes to file
subfinder	-o	Writes to file

Structured Output Flags (for Parsers)

Each tool that has a parser also produces structured output (JSON or XML) to a separate file. The scan_runner generates two output files per scan — text for the raw viewer, structured for the parser.

Tool	Structured Flag	Extension	Special Handling
nmap	-oX	.xml	Dual output: -oN + -oX simultaneously
nuclei	-je	.jsonl	Produces JSON array, not JSONL (parser handles both)
testssl	--jsonfile	.json	JSON file alongside stdout text
nikto	nikto_xml	.xml	Special: expands to -Format xml -output <file>
sslscan	sslscan_xml	.xml	Special: expands to --xml=<file> (equals syntax)
whatweb	--log-json	.json	Dual: --log-verbose + --log-json simultaneously
dnsrecon	--json	.json	JSON file alongside stdout text

Special flag handling in get_tool_command():

• nikto_xml — nikto doesn’t use a single flag for XML. The registry expands it to -Format xml -output <file>.
• sslscan_xml — sslscan uses --xml=<file> (equals syntax, not space-separated). Space syntax makes sslscan treat the file path as the scan target.

Structured Findings Parsers

The daemon parses structured tool output (JSON/XML) into normalized Finding objects with severity, title, description, affected component, evidence, remediation, CVSS score, and CVE IDs. This replaces the old keyword heuristic that just counted lines containing “vuln” or “critical”.

Finding Model

class FindingSeverity(str, Enum):
    critical = "critical"
    high = "high"
    medium = "medium"
    low = "low"
    info = "info"

class Finding(BaseModel):
    severity: FindingSeverity
    title: str
    description: str = ""
    affected_component: str = ""
    evidence: str = ""
    remediation: str = ""
    cvss_score: Optional[float] = None
    cve_ids: list[str] = Field(default_factory=list)
    metadata: dict = Field(default_factory=dict)

Parser Registry

app/services/parsers/__init__.py maps tool names to parser instances. get_parser(tool) returns the appropriate parser or falls back to TextParser (keyword heuristic).

Parser	File	Format	Strategy
NmapParser	nmap_parser.py	XML	Ports, services, script findings from <host>/<port> entries
NucleiParser	nuclei_parser.py	JSON array	Template matches — maps info.severity, info.name, CVEs, CVSS directly to Finding fields
TestsslParser	testssl_parser.py	JSON	Check entries filtered by severity. Skips OK entries and noisy info IDs (client sims, cipher hex)
NiktoParser	nikto_parser.py	XML	<niktoscan>/<scandetails>/<item> — severity from keyword matching on description
SslscanParser	sslscan_parser.py	XML	Weak protocols (SSLv2/v3), weak ciphers (RC4/DES/3DES/NULL), cert issues (self-signed, expired, weak key)
WhatwebParser	whatweb_parser.py	JSON	Plugin matches mapped to findings
DnsreconParser	dnsrecon_parser.py	JSON	DNS records as info-level findings
TextParser	text_parser.py	Stdout	Fallback: keyword heuristic for tools without structured output

Parser Integration

After a scan completes, scan_runner.py:

1. Checks if a structured output file exists for the scan
2. Calls get_parser(tool).parse(structured_file) → list[Finding]
3. Stores finding count and severity breakdown in the database
4. Returns findings in GET /api/scans/{id} response

The Arcturus-Prime SSR proxy at /api/admin/pentest/[...path] forwards findings to the frontend and persists them to Cloudflare D1 via the existing PentestFinding schema in scans.ts.

Known Parser Quirks

• nuclei -je produces a JSON array [{...}], not JSONL (one JSON per line). The parser tries array first, falls back to JSONL for compatibility.
• nikto on Debian 12 only supports htm, nbe, xml output — not JSON. Kali has JSON support but Sentinel (Debian) doesn’t. Parser uses XML for cross-platform compatibility.
• sslscan uses --xml=FILE (equals syntax). Space syntax --xml FILE makes it treat the file path as the scan target.
• sslscan EC keys: P-256 certificates report 128-bit security, not 256-bit key length. Parser distinguishes EC (minimum 112-bit) from RSA (minimum 2048-bit) to avoid false positives.
• testssl noise: Client simulations (clientsimulation-*), individual cipher hex entries (cipher_x*), and cipher order details (cipher_order_*) are filtered at info level to reduce noise. Medium+ severity entries from these IDs are kept.

Deployment

Tarn-Host / Izar-Host (Kali VMs)

# Build tarball (from workstation)
cd ~/Development/pentest-daemon
tar czf /tmp/pentest-daemon.tar.gz --exclude='.git' --exclude='__pycache__' --exclude='venv' .

# Push to Tarn-Host Kali VM
scp /tmp/pentest-daemon.tar.gz [email protected]:/tmp/
ssh [email protected] 'sudo tar xzf /tmp/pentest-daemon.tar.gz -C /opt/pentest-daemon/ && sudo systemctl restart pentest-daemon'

Sentinel VPS

# Build tarball (from workstation)
cd ~/Development/pentest-daemon
tar czf /tmp/pentest-daemon.tar.gz --exclude='.git' --exclude='__pycache__' --exclude='venv' .

# Push to VPS
scp /tmp/pentest-daemon.tar.gz [email protected]:/tmp/
ssh [email protected] 'tar xzf /tmp/pentest-daemon.tar.gz -C /opt/pentest-daemon/ && systemctl restart pentest-daemon'

The VPS deployment uses the same codebase. Tools that don’t exist on the VPS (sqlmap, hydra, etc.) will fail gracefully per-scan — the daemon doesn’t crash, it just returns an error for that specific scan.

Startup Behavior

On startup, the daemon:

1. Initializes the SQLite database (creates tables if needed)
2. Runs cleanup_zombie_scans() — marks any “running” or “pending” scans/assessments as “failed” with error “Daemon restarted while scan was in progress”
3. Starts the uvicorn server

This prevents zombie scans from accumulating across restarts.

API Endpoints

Method	Path	Description
GET	/api/health	Health check with uptime and active scan count
POST	/api/scans	Start a new scan
GET	/api/scans	List scans (optional ?status= filter)
GET	/api/scans/{id}	Get scan details
GET	/api/scans/{id}/output	Get scan output (streaming)
DELETE	/api/scans/{id}	Cancel a running scan
POST	/api/assessments	Start a multi-phase assessment
GET	/api/assessments	List assessments
GET	/api/assessments/{id}	Get assessment details with phase status
DELETE	/api/assessments/{id}	Cancel a running assessment
GET	/api/reports	List generated reports
GET	/api/reports/{id}	Get report content
GET	/api/diagnostics	Comprehensive node health diagnostics (auth required)
WS	/ws/scans/{id}	WebSocket stream for live scan output

Diagnostics Endpoint

GET /api/diagnostics runs six local health checks and returns structured results. Unlike /api/health (unauthenticated, minimal), this endpoint requires API key auth and returns detailed system state.

Checks

Check ID	What It Tests	Pass/Warn/Fail
daemon_health	Uptime, version, active scan count	Always pass if responding
database	SELECT count(*) FROM scans + write lock test (BEGIN IMMEDIATE + rollback)	Fail if query errors or lock held
tools	os.access(path, os.X_OK) for all 22 configured tool paths	Warn if any missing, lists which
disk_space	shutil.disk_usage(SCAN_OUTPUT_DIR)	Warn <10% free, Fail <5%
zombie_scans	Scans with status=‘running’ and started >30 min ago, cross-checked against scan_runner.get_active_scan_ids()	Fail if zombies found in DB but not in runner
scan_capacity	scan_runner.get_active_count() vs MAX_CONCURRENT_SCANS	Warn if at max

Response Format

{
  "node_id": "sentinel",
  "timestamp": "2026-02-27T20:31:41Z",
  "checks": [
    {
      "id": "daemon_health",
      "name": "Daemon Health",
      "status": "pass",
      "detail": "Up 2h 14m, v1.0.0, 0 active scan(s)",
      "fix": null
    },
    {
      "id": "tools",
      "name": "Tool Availability",
      "status": "warn",
      "detail": "14/22 tools available. Missing: sqlmap, hydra, amass, ...",
      "extra": { "available": ["nmap", "nikto", "..."], "missing": ["sqlmap", "..."] },
      "fix": "apt install sqlmap hydra amass"
    }
  ]
}

Each check includes a fix field with an actionable suggestion (or null if passing). The extra field on tool checks provides the full available/missing lists.

Arcturus-Prime Proxy

The Arcturus-Prime proxy at /api/admin/pentest/diagnostics fans out to all nodes and adds four proxy-side checks that the daemon can’t detect itself:

• env_configured — Are the URL and API key env vars set for this node?
• cloudflare_context — Detects Cloudflare Pages environment. Internal nodes (Tarn-Host, Izar-Host) are expected to be unreachable from CF edge.
• network_reachable — Did the fetch succeed? Distinguishes timeout, ECONNREFUSED, and HTTP errors.
• response_valid — Is the response valid JSON? Catches HTML error pages from reverse proxies (nginx 403/502).

Known Issues & Fixes

output_flag File Conflict Pattern

Symptom: Scan starts but hangs with empty output file, no process running, status stuck on “running”.

Cause: The scan_runner pre-creates the output file and opens it with open(file, "w") for stdout capture. If the tool’s output_flag also targets the same file, one of them wins and the other fails silently or the tool refuses to overwrite.

Fix: Set output_flag: None for tools where stdout capture is sufficient. This was hit by both testssl (--logfile) and dnsrecon (--xml).

AXFR Zone Transfers on Cloudflare Domains

dnsrecon’s -a flag attempts AXFR zone transfers which hang indefinitely on Cloudflare-protected domains. Removed from default args.

Database Location

The SQLite database lives on the VM’s root filesystem at /var/lib/pentest-daemon/scans.db. If the VM is destroyed, scan history is lost. Consider backing up periodically or moving to persistent storage.

Troubleshooting

Tarn-Host (Kali VM)

# Check service status
ssh [email protected] 'systemctl status pentest-daemon'

# View recent logs
ssh [email protected] 'journalctl -u pentest-daemon --since "10 min ago" --no-pager'

# Check health
ssh [email protected] 'curl -s http://localhost:8095/api/health'

# List active scans (needs API key)
ssh [email protected] "curl -s -H 'X-Api-Key: <key>' http://localhost:8095/api/scans?status=running"

# Kill all running scans and restart
ssh [email protected] 'sudo systemctl restart pentest-daemon'
# (zombie cleanup runs automatically on startup)

Sentinel VPS

# Check service status
ssh [email protected] 'systemctl status pentest-daemon'

# View recent logs
ssh [email protected] 'journalctl -u pentest-daemon --since "10 min ago" --no-pager'

# Check health (via nginx)
curl -s https://sentinel.Arcturus-Prime.com/pentest-api/api/health

# Check health (direct on VPS)
ssh [email protected] 'curl -s http://127.0.0.1:8095/api/health'

# Restart daemon
ssh [email protected] 'systemctl restart pentest-daemon'