5.4 Upgrade Verification, Post-Upgrade Health Checks, & Troubleshooting

Post-Upgrade Verification Commands & Health Verification

Once SmartConsole Central Deployment, a CDT batch plan, or a local CPUSE upgrade finishes, the CCSE administrator must execute mandatory verification and diagnostic checks on all target gateways before declaring the maintenance window successful and returning traffic to production.

1. Verification of Major Release & Kernel Version

To confirm that the Security Gateway kernel has successfully upgraded to Check Point R82 and is running the expected build number:

[Expert@Gateway1:0]# fw ver -k
This is Check Point VPN-1(TM) & FireWall-1(TM) R82 - Build 042 Kernel version 6.10

• fw ver: Displays the major version and build number of Firewall-1 security software.
• fw ver -k: Displays the specific running kernel version, verifying that the kernel module loaded during boot corresponds to R82.

2. Detailed Hotfix & JHA Inventory (cpinfo -y all)

To list all installed hotfixes, Jumbo Hotfix Accumulator Takes, and minor packages currently active on the system:

[Expert@Gateway1:0]# cpinfo -y all
******************************************************************
HOTFIX INFORMATION:
  HOTFIX_R82_JHF_MAIN Take: 25
  HOTFIX_R82_PUBLIC_FIX_012 Take: 3
******************************************************************

cpinfo -y all queries the Gaia software registry and reports the exact JHA Take number, verifying that the intended hotfix build was applied.

3. ClusterXL Synchronization & State Verification

For cluster deployments, verify that ClusterXL membership, state synchronization, and interface monitoring are fully operational:

[Expert@Gateway1:0]# cphaprob stat

Cluster Mode: High Availability (Active-Attention / Standby)
Number Unique Address Assigned status
1      192.168.1.1    0        Active
2 (local) 192.168.1.2 1        Standby

[Expert@Gateway1:0]# cphaprob -a if
Required interfaces: 4
Required secured interfaces: 1
Interface Name: eth0   Status: UP
Interface Name: eth1   Status: UP
Interface Name: sync   Status: UP (Secured)

Both members must show valid states (Active / Standby), and all cluster interfaces must report Status: UP with state synchronization active.

---

Security Policy Re-installation & Compilation Requirements

Following a major OS upgrade (e.g., from R81.20 to R82), Security Gateways cannot process new inspection features until a updated Security Policy is compiled and installed from the upgraded Security Management Server.

+---------------------------------------------------------------------------------+
|                    SECURITY MANAGEMENT SERVER (R82 SMS)                         |
|                                                                                 |
|   1. Update Gateway Object Properties -> Change Version to 'R82'                |
|   2. Publish Database Changes in SmartConsole                                   |
|   3. Install Security Policy (Compiles R82 Kernel Inspection Binaries)          |
+---------------------------------------------------------------------------------+
                                        |
                         Policy Download Tunnel (TCP 18191 / 18192)
                                        v
+---------------------------------------------------------------------------------+
|                    UPGRADED SECURITY GATEWAY (R82 KERNEL)                       |
|                                                                                 |
|   Receives updated inspection code ($FWDIR/state/local/FW1/ policy binaries)    |
+---------------------------------------------------------------------------------+

Steps for Mandatory Post-Upgrade Policy Push:

1. Open SmartConsole and navigate to Gateways & Servers.
2. Open the properties of the upgraded Gateway / Cluster object.
3. Change the Version drop-down menu from R81.20 to R82.
4. Click Publish to commit management database changes.
5. Select Install Policy, choose target Access Control and Threat Prevention policy packages, and execute installation.

---

CDT Log Inspection & Debugging Framework

When a CDT deployment plan fails or encounters errors on specific target gateways, the administrator must inspect CDT's central log repository located on the Security Management Server at /var/log/cdt/.

Key CDT Log Files

• cdt_trace_<plan_name>_<timestamp>.log: High-level execution timeline showing plan progression, phase starts/finishes, and overall gateway pass/fail status.
• cdt_debug_<plan_name>_<timestamp>.log: Detailed debug output capturing raw SSH transactions, CPUSE XML API exchanges, pre-check failure reasons, and script output.
• /var/log/cdt/scripts/: Contains execution logs for custom scripts invoked by <execute_script> tags.

Common CDT Error Codes & Diagnostic Matrix

---

Rollback Strategies & Procedures

If a post-upgrade health check reveals severe service degradation, application incompatibility, or kernel instability, administrators must execute a clean, controlled rollback.

1. CPUSE Package Revert (Hotfixes & JHAs)

For Jumbo Hotfix Accumulator updates, CPUSE maintains automated revert snapshots that allow instant uninstallation of the package without full OS re-imaging:

# View installed packages and identify package ID to revert
Gateway> installer show-installed

# Revert specific JHA package in Gaia Clish
Gateway> installer revert 1

CPUSE restores pre-installation binaries, updates system registry metadata, and prompts for gateway reboot.

2. Gaia OS Snapshot Rollback (Major Release Upgrades)

For major OS upgrades, reverting via CPUSE may not be supported. Administrators rely on Gaia OS Snapshots created prior to upgrade initiation:

# Display available OS snapshots stored on local disk
Gateway> show snapshots

# Revert system to pre-upgrade snapshot image
Gateway> set snapshot revert Pre_R82_Upgrade_Snapshot

Comprehensive Rollback Method Comparison