9.1 AWS Systems Manager Core Management & Run Command

9.1 AWS Systems Manager Core Management & Run Command

CloudOps Blueprint Focus: The AWS Certified CloudOps Engineer – Associate (SOA-C03) exam tests your ability to configure managed node prerequisites, administer hybrid fleets without inbound network access, execute administrative tasks at scale using Systems Manager Run Command rate controls, and enforce desired system configurations via State Manager associations.

AWS Systems Manager Architecture & Managed Node Prerequisites

Modern cloud operations require centralized, agent-driven management that decouples administrative control from traditional networking constructs such as bastion hosts, open SSH/RDP ports, and static credentials. AWS Systems Manager (SSM) serves as the operational hub for AWS and hybrid environments. To transition a virtual machine or physical server into an SSM Managed Node, three non-negotiable architectural prerequisites must be satisfied:

1. SSM Agent Installation: The open-source SSM Agent runs as an in-guest daemon. It is preinstalled on Amazon Linux, Amazon Linux 2, Amazon Linux 2023, Ubuntu Server LTS, and Windows Server AMIs. On other distributions (RHEL, Debian, SLES) or on-premises servers, the binary must be installed manually. The agent periodically polls Systems Manager endpoints for tasks and reports system metadata.

2. Outbound HTTPS Connectivity (Port 443): Systems Manager uses an outbound-only polling model. The SSM Agent initiates all communication; the AWS control plane never initiates inbound connections. Instances require outbound HTTPS (port 443) connectivity to three regional service endpoints:

   • ssm.<region>.amazonaws.com: Core Systems Manager API for configuration, command retrieval, and metadata.
   • ec2messages.<region>.amazonaws.com: Messaging service for Run Command and State Manager.
   • ssmmessages.<region>.amazonaws.com: Interactive channel used by Session Manager.

   In public subnets, traffic traverses an Internet Gateway. In private subnets, traffic routes through a NAT Gateway or Interface VPC Endpoints (AWS PrivateLink) for ssm, ec2messages, and ssmmessages. Crucially, instance security groups require zero inbound rules—neither port 22 (SSH) nor port 3389 (RDP) is opened.

3. IAM Instance Profile Permissions: The instance must assume an IAM role containing the AWS-managed policy AmazonSSMManagedInstanceCore. This policy grants minimum necessary permissions (ssm:UpdateInstanceInformation, ssmmessages:*, ec2messages:*). The legacy policy AmazonEC2RoleforSSM is deprecated because it granted excessive S3 and CloudWatch permissions.

---

Fleet Manager: Consolidated Console & Hybrid Node Oversight

AWS Systems Manager Fleet Manager provides a consolidated console for real-time node diagnostics and fleet health inspection across Amazon EC2, IoT devices, and on-premises environments without requiring interactive terminal logins:

• File System Exploration: Operators can browse directory hierarchies, inspect file permissions, and download or upload files directly via the AWS console.
• Process Management: View running processes, sort by CPU or memory utilization, and terminate unresponsive processes.
• Performance Metrics: View live OS performance counters (CPU, memory, disk I/O, network traffic).
• Windows Event Logs: View application, security, and system event logs natively.

Hybrid Activations for On-Premises and Multi-Cloud Servers

To extend Systems Manager governance to on-premises physical servers, VMware VMs, or virtual machines in external cloud providers, administrators configure SSM Hybrid Activations:

1. An administrator creates a hybrid activation in Systems Manager, generating an Activation Code and Activation ID.
2. The activation specifies an IAM service role (with AmazonSSMDirectoryServiceAccess or a custom role containing AmazonSSMManagedInstanceCore) and an expiration limit.
3. The administrator installs the SSM Agent on the machine and registers it using the activation credentials:

   sudo amazon-ssm-agent -register -code "activation-code" -id "activation-id" -region "us-east-1"
   sudo systemctl start amazon-ssm-agent
   

Registered hybrid servers receive an ID prefixed with mi- (distinguishing them from EC2 instances prefixed with i-) and appear in Fleet Manager with full operational parity.

---

Systems Manager Run Command: Fleet-Wide Asynchronous Execution

Systems Manager Run Command executes scripts and administrative tasks across managed nodes at scale without opening inbound firewall ports or managing SSH keys.

Executions use SSM Command Documents (JSON/YAML specifications):

• AWS-RunShellScript: Executes bash scripts on Linux instances.
• AWS-RunPowerShellScript: Executes PowerShell scripts on Windows nodes.
• AWS-UpdateSSMAgent: Updates the SSM Agent daemon fleet-wide.
• AmazonCloudWatch-ManageAgent: Deploys and configures the unified CloudWatch Agent.

Target Selection Mechanisms

Run Command supports three target selection methods:

• Tag-Based Targeting: Targets nodes dynamically by tag key-value pairs (e.g., Environment = Production). Auto Scaling instances launch with tags and inherit commands automatically.
• Manual Instance Selection: Explicitly targets a list of Instance IDs (i-0123456789abcdef0).
• AWS Resource Groups: Targets pre-configured collections of resources grouped by tag or CloudFormation stack.

Rate Control: Concurrency Limits and Error Thresholds

To prevent fleet-wide degradation or backend service saturation, Run Command provides two Rate Control parameters:

• Concurrency (RateControl Concurrency): Limits how many managed nodes execute the command simultaneously, specified as an absolute count (e.g., 5 nodes) or percentage (e.g., 25% or 50%). Run Command processes instances in rolling batches.
• Error Threshold (RateControl ErrorThreshold): Specifies the maximum number of failures allowed before Run Command halts execution across remaining nodes (e.g., abort after 1 error or 5%).

Output Logging, Auditing & Truncation

Console and GetCommandInvocation API responses truncate stdout and stderr at 2,500 characters. For persistent auditing and full output capture, Run Command streams un-truncated execution logs directly to an Amazon S3 bucket (with optional KMS encryption) and an Amazon CloudWatch Logs log group.

---

Systems Manager State Manager: Continuous Configuration Enforcement

While Run Command executes one-off imperative tasks, Systems Manager State Manager provides declarative, continuous desired-state enforcement.

State Manager uses SSM Associations to bind a configuration document to target nodes on a recurring schedule:

• Enforcing Desired State: Automatically ensures instances remain compliant (e.g., applying CIS benchmarks, updating antivirus definitions daily, or keeping the SSM Agent updated).
• Association Schedules: Executed on recurring schedules defined by rate expressions (rate(12 hours)) or cron expressions (cron(0 0 ? * SUN *)).
• SSM Compliance Integration: Execution results are recorded in the Systems Manager Compliance dashboard. Nodes failing associations are flagged as non-compliant, triggering automated remediation via Amazon EventBridge.