4.3 Troubleshooting Workflow Runs & Debug Logging

Troubleshooting Workflow Runs & Debug Logging

When automated workflows fail in production CI/CD environments, engineers must rapidly diagnose root causes, isolate failing components, and remediate errors without introducing pipeline downtime. GitHub Actions provides powerful diagnostic logging controls, granular re-run capabilities, structured workflow annotations, and automated secret masking to streamline troubleshooting.

---

1. Systematic Troubleshooting Framework

When a workflow run fails or behaves unexpectedly, follow a structured four-stage diagnostic methodology:

2. Enabling Diagnostic & Debug Logging

By default, GitHub Actions suppresses internal runner agent communications and verbose step evaluations to keep logs clean. When standard console output is insufficient, you can activate two distinct debug logging channels.

1. Step Debug Logging (ACTIONS_STEP_DEBUG)

• Purpose: Prints extra verbose logging during step execution, including how environment variables are resolved, action input evaluation, shell script command traces, and detailed stdout/stderr streams.
• Activation: Set a repository secret, organization secret, or repository configuration variable named ACTIONS_STEP_DEBUG with the value true.

2. Runner Diagnostic Logging (ACTIONS_RUNNER_DEBUG)

• Purpose: Logs runner agent internals, including runner registration, listener heartbeat polling, capability matching, job dispatch messages, and runner machine environment telemetry.
• Activation: Set a repository secret, organization secret, or repository configuration variable named ACTIONS_RUNNER_DEBUG with the value true.

+-----------------------------------------------------------------------------+
|                        DEBUG LOGGING CONFIGURATION                          |
|                                                                             |
|   Setting Location: Settings -> Secrets and variables -> Actions            |
|                                                                             |
|   Variable / Secret Name       Value    Diagnostic Output                   |
|   ----------------------       -----    -----------------                   |
|   ACTIONS_STEP_DEBUG           true     Verbose step script traces          |
|   ACTIONS_RUNNER_DEBUG         true     Runner listener & host diagnostics  |
+-----------------------------------------------------------------------------+

3. Enabling Debug Logging On-Demand via the UI

Instead of setting permanent configuration variables, you can enable debug logging temporarily when re-running a workflow. In the GitHub Actions run view, click Re-run jobs -> check the box "Enable debug logging" -> click Re-run jobs.

[!NOTE] When "Enable debug logging" is checked in the UI, GitHub Actions temporarily injects ACTIONS_STEP_DEBUG=true and ACTIONS_RUNNER_DEBUG=true into that specific re-run execution without modifying repository settings.

---

3. Targeted Re-Run Strategies

In complex multi-job pipelines and matrix builds, executing an entire workflow from the beginning wastes compute minutes and delays feedback. GitHub Actions supports three granular re-run modes:

+-----------------------------------------------------------------------------+
|                         WORKFLOW RE-RUN STRATEGIES                          |
|                                                                             |
|   [1. Re-run all jobs]                                                      |
|       Restarts the entire workflow DAG from scratch.                        |
|                                                                             |
|   [2. Re-run failed jobs]                                                   |
|       Re-runs only jobs that failed or were cancelled, plus any downstream  |
|       dependent jobs. Succeeded upstream jobs are preserved!               |
|                                                                             |
|   [3. Re-run specific matrix job]                                           |
|       Re-runs only the individual failed matrix slice (e.g., Windows Node 20)|
|       while leaving all other successful combinations untouched.            |
+-----------------------------------------------------------------------------+

Re-Running Failed Jobs in a DAG

Consider a pipeline with jobs: lint -> build -> test -> deploy.

1. lint and build succeed.
2. test fails due to an intermittent network glitch.
3. deploy is skipped.

When you select Re-run failed jobs:

• lint and build do not re-execute. Their previous outputs and uploaded artifacts remain available.
• test re-runs from scratch.
• Upon test succeeding, deploy executes automatically.

---

4. Log Inspection, Step Annotations & Masking

Step Annotations

Workflow commands formatted as ::error::, ::warning::, or ::notice:: generate highlighted UI callouts and inline annotations on pull request code diffs:

# Creating an error annotation targeting a specific file and line
echo "::error file=src/auth.ts,line=84,col=12::Missing JWT expiration check"

Downloading Complete Diagnostic Log Archives

For offline analysis or compliance audits, click the gear icon (⚙️) on the workflow run page and select Download log archive. This downloads a .zip archive containing:

• Individual .txt stdout/stderr log files for every step in every job.
• A dedicated runner-diagnostic-logs/ folder containing runner service communication traces (when ACTIONS_RUNNER_DEBUG is active).

Secret Masking & Log Truncation Rules

1. Automatic Secret Masking: Any secret referenced via ${{ secrets.* }} is automatically registered in the runner's redaction filter. If printed to stdout, it is replaced with ***.
2. Dynamic Runtime Masking (::add-mask::): Tokens generated dynamically during script execution can be masked using:

   SESSION_TOKEN=$(vault-cli get-token)
   echo "::add-mask::$SESSION_TOKEN"
   

3. Log Truncation in the Browser: GitHub does not publish a numeric log-size limit, but the web UI will not render an unbounded step log. A very large step is displayed with its head only and the notice "This step has been truncated due to its large size. Download the full logs from the menu once the workflow run has completed." The full output is still in the downloadable archive - it is a rendering limit, not data loss.
4. Live-Log Lag on Noisy Steps: Streaming logs for a high-volume step can fall behind, so the tail of the output may only become visible after the job completes. When a failure message is being buried, fix it at the source: quiet verbose tooling, or write detailed diagnostics to an artifact with actions/upload-artifact instead of to stdout.

---

5. Common Workflow Execution Errors & Remediation