Vulnerability Management and Patching
Reducing Attack Surface Through Systematic Remediation
TL;DR
Vulnerability management is continuous: scan for vulnerabilities, assess risk, prioritize by severity and exploitability, test patches, and deploy. Automated scanning in CI/CD catches issues early (shift-left). Critical vulnerabilities should be patched within days. Non-critical vulnerabilities require risk-based assessment and scheduled patching cycles. Without patches, attackers exploit known CVEs (Easy, public exploits available). With patches, attackers must find zero-days (Hard, expensive).
Learning Objectives
- Implement vulnerability scanning across infrastructure
- Assess and prioritize vulnerabilities using CVSS
- Design patch management processes
- Automate testing before production deployment
- Manage zero-day vulnerabilities and exploits
Core Concepts
Vulnerability Sources
- Operating system packages (OS, libraries)
- Application dependencies (npm, pip, Maven)
- Container base images
- Infrastructure configuration issues
- Exposed credentials or secrets
- Weak cryptography or algorithms
CVSS Scoring
Common Vulnerability Scoring System (v3.1):
- 0.0-3.9: Low severity
- 4.0-6.9: Medium severity
- 7.0-8.9: High severity
- 9.0-10.0: Critical severity
Factors:
- Attack vector (network vs. local)
- Attack complexity (low vs. high)
- Privileges required (none, low, high)
- User interaction required
- Scope of impact (unchanged vs. changed)
- Confidentiality, Integrity, Availability impact
Patch Management Process
Vulnerability Published
↓
Scan & Identify
↓
Assess CVSS
↓
Risk Assessment
↓
Prioritize (critical → scheduled)
↓
Test in staging
↓
Deploy to production
↓
Verify patch effectiveness
↓
Document & Close
Practical Example
Vulnerability Management Workflow
ScanningPolicy:
Frequency:
Critical Dependencies: Daily
Application: Weekly
Infrastructure: Daily
Containers: On every build
RiskAssessment:
CVE-2024-1234:
CVSS: 9.8 (Critical)
Exploitable: Yes (POC available)
Affected Systems: 3 (production, 2 staging)
Patch Available: Yes
Action: Emergency patch within 24 hours
CVE-2024-5678:
CVSS: 4.2 (Medium)
Exploitable: No (POC requires authentication)
Affected Systems: 1 (development)
Patch Available: Yes
Action: Include in next scheduled patch cycle
PatchingSchedule:
Critical: Within 24-48 hours
High: Within 1 week
Medium: Within 2 weeks
Low: Within 30 days
TestingRequirement:
Security updates: Smoke tests only
Feature updates: Full regression tests
Rollback plan: Documented and tested
Strategies
Proactive Vulnerability Management
- Scan all systems on regular schedule
- Scan every dependency during development
- Monitor for new CVEs affecting your stack
- Subscribe to vendor security advisories
Reactive to Exploits
- Monitor threat intelligence feeds
- Prioritize vulnerabilities with active exploits
- Emergency patching process for critical exploits
- Public incident communication plan
Zero-Day Management
- Compensating controls (WAF rules, segmentation)
- Isolation of affected systems
- Vendor communication for patch timeline
- Enhanced monitoring for exploitation attempts
When to Use / When Not to Use
Vulnerability Management Best Practices
- Regular automated scanning (daily/weekly)
- Vulnerability scanning in CI/CD pipeline
- Risk-based prioritization using CVSS + context
- Rapid patching for critical vulnerabilities
- Staging environment testing before prod
- Monitoring for active exploits
- Compensating controls for unpatched systems
- Vulnerability tracking and metrics
Common Mistakes
- Scanning only occasionally (miss new vulnerabilities)
- Treating all vulnerabilities equally
- No testing before deploying patches
- Patching only when forced (too reactive)
- No prioritization strategy
- No monitoring for exploits in the wild
- Ignoring low/medium severity vulnerabilities
- No tracking of patched vs. unpatched systems
Design Review Checklist
- Regular vulnerability scanning scheduled?
- Scanning covers OS, apps, containers, dependencies?
- Scan results aggregated in central dashboard?
- False positive rate acceptable?
- CVSS scoring used for severity?
- Context considered (exploitability, blast radius)?
- SLA defined for each severity level?
- Escalation process for missed SLAs?
- Patches tested before production?
- Rollback plan documented?
- Patch deployment automated?
- Patch effectiveness verified post-deployment?
- Track mean time to patch (MTTP)?
- Monitor vulnerability remediation rate?
- Executive reporting on vulnerability posture?
- Trends analyzed (improving or declining)?
Self-Check
- How do you calculate CVSS scores?
- What's the difference between vulnerability scanning and patch testing?
- Why is exploitability important beyond CVSS severity?
- What's a zero-day and how do you respond?
- How do you balance patching speed with stability?
Real-World Vulnerability Management Scenarios
Scenario 1: Critical RCE (Remote Code Execution) in Log4j
Severity: CVSS 10.0 (Critical)
Impact: Attackers can execute code on any Java system using Log4j
Timeline: December 9, 2021 (discovered), December 10 (exploits in wild)
Response:
Hour 0 (Discovery):
- Alert: CVE-2021-44228 published
- Scan: Identify 847 affected systems (Java apps using Log4j)
- Assessment: 12 production systems, 50 staging, rest non-critical
Hour 2 (Immediate Response):
- High-risk systems: Patch immediately or disable logging
- Medium-risk: Prepare patches, test in staging
- Low-risk: Schedule for next patch cycle
Hour 6 (Status):
- Production: 5/12 patched (some require app restarts, scheduled for maintenance window)
- Other 7: Mitigated with WAF rules blocking exploit patterns
- Staging: Batch patched
Hour 24 (Completion):
- 100% of systems patched or mitigated
- New builds include patched Log4j
- Threat scanning for active exploitation attempts
- Incident review: Could have been faster with pre-staged patches
Lessons Learned:
- Pre-stage critical patches (Log4j, OpenSSL, etc.)
- Maintain fast patch testing pipeline (reduce from 1 week to 24 hours)
- Document WAF rules for common exploits (buy time during patching)
- Subscribe to exploit threat feeds (not just CVE databases)
Scenario 2: Non-Critical Vulnerability in Rarely-Used Library
Severity: CVSS 5.3 (Medium)
Vulnerability: XSS in deprecated logging library (no one uses it)
Timeline: Discovered during regular scanning
Assessment:
- Affected systems: 3 (all dev/test environments)
- Actual risk: Low (library not in data flow, no user input processed)
- Patch available: Yes, but requires code change
- Testing burden: Moderate (must test entire logging pipeline)
Decision:
- Risk: Low (not exploitable in our usage)
- Effort to patch: 16 hours (analysis + testing + deployment)
- Cost/benefit: Not worth patching immediately
- Action: Include in next quarterly patch cycle (2 months from now)
Alternative:
- If library is truly unused: Remove it (better than patching)
- Document in risk register: Known vulnerability, mitigated by context
- Monitor for active exploits (if observed, elevate priority)
Timeline: Patch included in next release (2 months) with other changes
Vulnerability Scanning in CI/CD
Example Pipeline
name: Security Scanning
on: [push, pull_request]
jobs:
scan-dependencies:
runs-on: ubuntu-latest
steps:
# Scan npm dependencies
- uses: actions/setup-node@v3
with:
node-version: '18'
- run: npm audit
# Blocks build if critical/high vulnerabilities found
# Exit code non-zero = fail CI
# Scan Python dependencies
- uses: actions/setup-python@v4
with:
python-version: '3.10'
- run: pip install safety
- run: safety check --json
# Blocks on known vulnerable packages
scan-container:
runs-on: ubuntu-latest
steps:
- uses: aquasecurity/trivy-action@master
with:
image-ref: myregistry/myimage:latest
format: 'sarif'
output: 'trivy-results.sarif'
- uses: github/codeql-action/upload-sarif@v2
with:
sarif_file: 'trivy-results.sarif'
scan-code:
runs-on: ubuntu-latest
steps:
# SCA (Software Composition Analysis)
- uses: github/codeql-action/analyze@v2
# SAST (Static Application Security Testing)
- uses: securego/gosec@master
if: ${{ matrix.language == 'go' }}
Result: If scan fails, PR cannot be merged until vulnerabilities resolved
Metrics and KPIs
Track vulnerability management effectiveness:
class VulnerabilityMetrics:
def mean_time_to_patch(self, severity: str) -> timedelta:
"""Average time from CVE publish to patch deployment"""
vulnerabilities = self.get_by_severity(severity)
times = [v.patched_date - v.published_date for v in vulnerabilities]
return sum(times, timedelta()) / len(times)
def remediation_rate(self) -> float:
"""Percentage of vulnerabilities remediated within SLA"""
total = len(self.vulnerabilities)
on_time = len([v for v in self.vulnerabilities if v.patched_on_time()])
return (on_time / total) * 100 if total > 0 else 0
def vulnerability_density(self) -> float:
"""Vulnerabilities per 1000 lines of code"""
vuln_count = len([v for v in self.vulnerabilities if v.is_active()])
loc = self.get_lines_of_code()
return (vuln_count / loc) * 1000
def zero_day_exposure(self) -> int:
"""Days without patch for zero-day vulnerabilities"""
zero_days = [v for v in self.vulnerabilities if v.is_zero_day()]
return max([(datetime.now() - v.discovered_date).days for v in zero_days])
# Targets (best in class)
# Mean Time to Patch (MTTP):
# - Critical: 24 hours (99% of systems)
# - High: 7 days (99%)
# - Medium: 30 days (95%)
# - Low: 90 days
# Remediation Rate: > 95%
# Vulnerability Density: < 1 per 1000 LOC (well-maintained codebase)
# Zero-Day Exposure: < 30 days (patch available within month)
Vulnerability Management Tools
| Tool | Capability | Scope |
|---|---|---|
| Dependabot (GitHub) | Dependency scanning | npm, pip, Gradle, Maven, etc. |
| Trivy (Aqua) | Container scanning | Docker, OCI registries |
| OWASP Dependency-Check | SCA | Multiple languages |
| Snyk | Dependency + SAST | npm, pip, Docker, Kubernetes |
| Tenable Nessus | Network/app scanning | Infrastructure, cloud |
| Qualys | Cloud vulnerability scanning | AWS, Azure, GCP |
| Anchore | Container + app scanning | Full supply chain |
| Grype | Fast container scanning | Minimal false positives |
Next Steps
- Inventory systems — What OS, apps, dependencies do you have? (Spreadsheet or CMDB)
- Deploy scanners — OS (Nessus), app (Snyk), container (Trivy), dependency (Dependabot)
- Set up CI/CD scanning — Block builds with critical vulnerabilities
- Create SLAs — Define patch deadlines by severity and exploitability
- Test patches — Staging environment, smoke tests before production
- Automate deployment — Scheduled patch windows (Patch Tuesday), or on-demand for critical
- Monitor exploits — Subscribe to threat intelligence feeds (CISA KEV, Shodan, etc.)
- Measure and improve — Track mean time to patch, remediation rate, zero-day exposure