12.3 Verbose Logging and Diagnostics

12.3 Verbose Logging and Diagnostics

Quick Answer: Set TF_LOG to TRACE, DEBUG, INFO, WARN, or ERROR (most verbose to least). Logs go to stderr. TF_LOG_PATH appends the same stream to a file — but TF_LOG must still be set or nothing is logged. TF_LOG_CORE and TF_LOG_PROVIDER split Terraform-core logs from provider-plugin logs. Use this for provider bugs and API failures. Never leave TRACE on in CI that prints logs: traces can contain credentials.

Objective 7c on Terraform Associate (004) is: describe when and how to use verbose logging on Terraform 1.12. This is an operations question, not an HCL question. Official pages: Enable Terraform logs and CLI environment variables.

Why this objective appears on 004

When apply dies with a vague provider error, the next step is not terraform destroy. It is a controlled debug log. 004 checks that you know the level names, that TF_LOG_PATH alone is not enough, that you can isolate core versus provider, and that you treat TRACE as sensitive.

TF_LOG levels

Enabling TF_LOG makes detailed logs appear on stderr. HashiCorp documents these levels, in decreasing verbosity:

export TF_LOG=TRACE          # or DEBUG, INFO, WARN, ERROR
terraform plan
unset TF_LOG                 # or: export TF_LOG=off

The environment-variables page shows export TF_LOG=trace (case is conventionally upper-case in examples; the documented disable is off or unset). Levels are the exam fact. Setting TF_LOG to JSON is a special value: logs at TRACE or higher, formatted as JSON. HashiCorp warns that this JSON encoding is not a stable interface — it may change without notice and exists to support forthcoming tooling. Do not treat TF_LOG=JSON as the same thing as terraform show -json.

None of these variables is required for normal use. They change verbosity for debugging. They do not change how plan/apply compute actions, and they do not replace terraform validate.

TF_LOG_PATH does not enable logging by itself

export TF_LOG=DEBUG
export TF_LOG_PATH=./terraform.log
terraform apply

TF_LOG_PATH forces the log to be appended to that file whenever logging is enabled. Official wording, repeated on both the debugging page and the environment-variables page: even when TF_LOG_PATH is set, TF_LOG must be set for any logging to be enabled. A CI job that only exports a path and then wonders why the file is empty has not turned logging on.

Use a path you control (a job artifact directory, not a world-readable shared volume). The file can grow quickly at TRACE. Delete or restrict it after you capture the failure.

Split core versus provider: TF_LOG_CORE and TF_LOG_PROVIDER

Terraform CLI ("core") and provider plugins are separate processes. A 403 from AWS and a panic inside Terraform's graph walker look similar at ERROR on the console. HashiCorp lets you enable subsets:

When the symptom is "the provider returned InvalidParameterValue / 403 / 429," start with TF_LOG_PROVIDER=DEBUG or TRACE so you are not drowned in core graph TRACE. When the symptom is a crash or odd behavior inside Terraform before the provider is called (backend init, state lock, malformed config evaluation), use TF_LOG_CORE. You can set both to different levels. That split is the 7c skill: isolate who is noisy.

# Provider plugin only — typical cloud API failure
export TF_LOG_PROVIDER=TRACE
export TF_LOG_PATH=./provider-trace.log
# Official persist note: TF_LOG must also be set for file logging to start.
# Keep TF_LOG at a quiet level so CORE stays out of a provider-only capture.
export TF_LOG=ERROR

# After the capture
export TF_LOG=off
unset TF_LOG_PROVIDER

flowchart TD
    Fail["Unexpected CLI or apply failure"] --> Decide{"Where does it smell?"}
    Decide -->|Cloud API / plugin| Prov["TF_LOG_PROVIDER=DEBUG or TRACE"]
    Decide -->|Graph, backend, language| Core["TF_LOG_CORE=DEBUG or TRACE"]
    Decide -->|Not sure| Both["TF_LOG=TRACE"]
    Prov --> Path["TF_LOG_PATH=controlled file"]
    Core --> Path
    Both --> Path
    Path --> Capture["Reproduce once"]
    Capture --> Off["TF_LOG=off; restrict or delete the file"]
    Off --> Bug["Attach TRACE to a HashiCorp/provider bug report if needed"]

When to turn it on

HashiCorp's debugging page exists so you can debug unexpected behavior and, if you file a Terraform bug, include a detailed log (they suggest a gist). Exam-level situations:

• A provider operation fails with a raw API error and the human message is not enough.
• A plugin exits or hangs during refresh/apply.
• You need to see whether Terraform even reached the provider (core) or the provider reached the API (provider).
• Support or a GitHub issue asks for a TRACE.

Situations that are not 7c:

• "Make plan faster" — logging makes it slower and noisier.
• "Hide secrets from state" — that is sensitive, ephemeral values, and a secrets manager (4h), not TF_LOG=ERROR.
• "Pretty-print HCL" — terraform fmt.
• "See what would change" — terraform plan / terraform show.

Leave logging at default (unset) for everyday laptop and production apply. Turn it up for a single reproduction, capture the file, then turn it off.

Never leave TRACE on in CI with secrets

TRACE (and often DEBUG) can include:

• provider credentials and session tokens,
• request headers and bodies,
• resource attributes that are secrets,
• backend credentials used to reach state.

If a pipeline exports TF_LOG=TRACE and the job log is world-readable (or shipped to a log vendor with broad access), you have published credentials. That is why 7c is also a safety question.

Practical rules for 004:

• Do not set TF_LOG=TRACE as a standing variable in the CI image.
• If you must capture a failing job, write to TF_LOG_PATH on a restricted artifact, redact before sharing, and rotate any credentials that appeared.
• TF_LOG_PATH without access control is still a secret file.
• TF_LOG=JSON is still TRACE-level detail in another wrapper — same secret risk.
• Unset the variables or set TF_LOG=off when the debug run ends.

TF_IN_AUTOMATION only changes cosmetic "what to run next" hints. It does not enable logging and it does not redact TRACE. Do not confuse the two variables.

Scenario: the 403 that was not a VPC bug

Priya's apply fails: the AWS provider reports AccessDenied on CreateRoute. A teammate wants to apply -replace the route table. Priya instead sets TF_LOG_PROVIDER=DEBUG and TF_LOG_PATH=./aws.log with TF_LOG enabled, reproduces once, and turns logging off. The provider log shows the request used the wrong account alias, not a bad CIDR. She fixes the aliased provider credentials. She never commits aws.log. She never leaves TF_LOG=TRACE in the GitHub Actions env: block, because that workflow prints logs to a public fork pull request.

If the log had shown a Terraform core panic before any AWS HTTP, she would have captured TF_LOG_CORE=TRACE and opened a Terraform issue with that gist — the debugging page's intended use.

004 traps for objective 7c

• Levels are TRACE, DEBUG, INFO, WARN, ERROR — not VERBOSE / SILENT.
• TF_LOG_PATH without TF_LOG writes nothing.
• TF_LOG_CORE versus TF_LOG_PROVIDER isolates Terraform versus the plugin.
• TF_LOG=JSON is unstable TRACE-shaped JSON, not terraform show -json.
• TRACE belongs in a controlled file for a single reproduction, not in standing CI.
• Logging does not apply infrastructure and does not format configuration.