Manage EC2 Instance and the Local SSH Config with Bash

May 9, 2023

In the last post, I created an Ansible role to configure an EC2 instance to shutdown when no active SSH connection was detected.

I still had a couple of problems, though:

Problems

First, when an EC2 instance reboots, its public IP address will change (if configured to have one). This meant that my local ~/.ssh/config would be out of date. I’m connecting to an IP in the Hostname setting, so I won’t connect if the IP is wrong!

Second, the instance is shutdown, which means I can’t SSH to it without bringing it back up.

Possible Solutions

I had four thoughts on fixing things:

Elastic IP

I could use an Elastic IP Address.

These can be associated with an instance or an ENI, which would make it easy to set up. However, I believe your account is limited to 5 Elastic IPs, and you’re charged for your Elastic IP if your instance is not running. This is the opposite of what I want!

This also does not address the problem of bringing the instance back up.

ENI

An ENI (Elastic Network Interface) can also have a public IP address and attach to an instance. It will not bring the instance up.

Bash Script with AWS CLI

This is the solution I settled on for now. See the script below. The script allows me to start and stop an instance by name, and also updates the IP address of its associated ~/.ssh/config entry.

I like this solution because I could use it with another cloud provider.

One drawback is that it won’t automatically start an instance that is shutdown. I have to manually run my script to bring the instance up.

I believe I have to either go through a bastion server or set up a Lambda with ProxyCommand set in my ~/.ssh/config. I haven’t decided how or if I’ll set this up.

Use AWS SSM

AWS SSM (Systems Manager Sessions Manager) would solve my problems when working with AWS. I would set ProxyCommand in ~/.ssh/config to run some sort of SSM script that would check the instance’s status, boot if necessary, then start a session.

This solution won’t work with other cloud providers, but I might implement it in the future to try it out.

The Solution

I wrote a bash script to start and stop an AWS instance by name. It will also update the ~/.ssh/config so you can seamlessly SSH into your instance.

Make it executable and use it like this:

# Start an instance
./ec2 start <some_server_name>

# Stop an instance
./ec2 stop <some_server_name>

Here are notes on the script:

The name variable matches the Host value in my ~/.ssh/config and the instance name in the AWS instance console. In other words, I have a Host that looks like Host some_server_name.
--no-cli-pager is handy. This turns off the AWS CLI pager, which makes the script interactive. You can also do this by setting the env variable AWS_PAGER="".
The replacement of the IP address in line 61 is brittle. I’m grepping a hardcoded 4 lines after finding the name. If the SSH config changes in my Ansible role, this might fail.
Does a global replacement on the IP address in ~/.ssh/config.
There is no error handling (ex. you can call it without an argument).

 1#!/bin/bash
 2# Start and stop and EC2 instance.
 3# If started, replace the old IP address of the instance with the new one in ~/.ssh/config.
 4
 5# Parse command-line arguments
 6if [ "$1" == "start" ]; then
 7  action="start"
 8elif [ "$1" == "stop" ]; then
 9  action="stop"
10else
11  echo "Usage: $0 [start|stop] <instance-name>"
12  exit 1
13fi
14
15sleep_seconds=5
16
17# Get the instance name from the command line
18name="$2"
19
20instance_id=$(aws ec2 describe-instances --filters "Name=tag:Name,Values=$name" --query "Reservations[].Instances[].InstanceId" --output=text)
21
22# Check if the instance is already running or stopped
23status=$(aws ec2 describe-instances --filters "Name=tag:Name,Values=$name" --query "Reservations[*].Instances[*].State.Name" --output text)
24if [ "$status" == "running" ] && [ "$action" == "start" ]; then
25  echo "Instance $name ($instance_id) is already running"
26  exit 0
27elif [ "$status" == "stopped" ] && [ "$action" == "stop" ]; then
28  echo "Instance $name ($instance_id) is already stopped"
29  exit 0
30fi
31
32# Start or stop the instance
33if [ "$action" == "start" ]; then
34  aws ec2 start-instances --no-cli-pager --instance-ids $(aws ec2 describe-instances --filters "Name=tag:Name,Values=$name" --query "Reservations[*].Instances[*].InstanceId" --output text)
35elif [ "$action" == "stop" ]; then
36  aws ec2 stop-instances --no-cli-pager --instance-ids $(aws ec2 describe-instances --filters "Name=tag:Name,Values=$name" --query "Reservations[*].Instances[*].InstanceId" --output text)
37fi
38
39# Wait for the instance to start or stop
40# Polls every $sleep_seconds seconds
41while true; do
42  status=$(aws ec2 describe-instances --filters "Name=tag:Name,Values=$name" --query "Reservations[*].Instances[*].State.Name" --output text)
43  if [ "$action" == "start" ] && [ "$status" == "running" ]; then
44    echo "Instance $name ($instance_id) started"
45    break
46  elif [ "$action" == "stop" ] && [ "$status" == "stopped" ]; then
47    echo "Instance $name ($instance_id) stopped"
48    break
49  fi
50  echo Waiting...
51  sleep $sleep_seconds
52done
53
54# Update SSH config file with new IP address
55if [ "$action" == "start" ]; then
56  # Get instance IP address
57  instance_ip=$(aws ec2 describe-instances --instance-ids "$instance_id" --query "Reservations[].Instances[].PublicIpAddress" --output=text)
58
59  # Find the SSH config entry for the instance by name
60  # This is brittle! Change if Ansible ec2 role SSH changes.
61  config=$(grep -A4 -w "Host $name\$" ~/.ssh/config)
62
63  # Get the current hostname for the SSH config entry
64  old_hostname=$(echo "$config" | grep Hostname | awk '{print $2}')
65
66  # Replace the old hostname with the new IP address in the SSH config file
67  # Change every matching IP in the file
68  sed -i "s/$old_hostname/$instance_ip/g" ~/.ssh/config
69
70  echo "Instance started with ID $instance_id and IP address $instance_ip"
71fi
72
73exit 0