EC2, UFW and Broken SSH Access

I’m a big fan of Amazon’s EC2 platform and long ago migrated all of my servers to its “cloud”. I’m also a realist when it comes to security and take all of the reasonable precautions I can to limit my risks. One such precaution is two-factor authentication for SSH (see my article “Two-Factor Authentication System for Apache and SSH”, Linux Journal – Issue 239, March 2014). Another is to take full advantage of a firewall to limit access to services.

When leveraging the EC2 Platform there are two firewall options that can be leveraged. One is the EC2 firewall (e.g. Security Groups) and the other is the firewall running on the server instance itself. I tend to use both, with the EC2 firewall for broad configurations (e.g. shared security group across multiple machines) and the local firewall for refined server specific configurations.

All of my Linux servers are Ubuntu and when configuring the local firewall I use its UFW tool to build simple iptables rules to refine access controls. The challenge I ran into was that I had limited SSH (port 22) to two specific IP addresses, one for home and the other for the office. All was fine until I moved my office and moved into a new house both within a month of each other. Here I am nine or ten weeks later and I need to SSH to my server to do some maintenance and, as you might have guessed, I can’t access the SSH service. Normally one would work around this issue by going directly to the console, logging in and resolving the issue. As that was not an option in this case, I thought about my problem for a minute and quickly realized that it would be fairly easy to work around my seemingly insurmountable problem. Here are the steps I took:

Note: Most of the steps outlined below are executed via the EC2 Dashboard. All others steps are specifically called out and must be executed at the command line.

  • Launch another EC2 server instance
  • The best way to accomplish this is use EC2’s “Launch More Like This” feature. This will ensure that the OS type, security group and other attributes are the same thus saving a bit of setup time.

    EC2 Instance Management
    EC2 Instance Management
  • Stop the problem instance
  • Detach volume from problem instance
  • Attach volume to new instance
  • Note: Newer Linux kernels may rename your devices to /dev/xvdf through /dev/xvdp internally, even when the device name entered is /dev/sdf through /dev/sdp.

  • Mount the volume
  • cd ~
    mkdir lnx1
    sudo mount /dev/xvdf ./lnx1
    
  • Disable UFW
  • cd lnx1
    sudo vim ufw.conf
    

    Now find ENABLED=yes and change it to ENABLED=no.

  • Detach volume
  • Be sure to unmount the volume first:

    sudo umount ./lnx1/
    
  • Reattach the volume to /dev/sda1 on our problem instance
  • Boot problem instance
  • Reassign elastic IP address if necessary
  • Delete the temporary instance and its associated volume

Voilà – I have SSH access to my server again. Now I need to modify my UFW rules and I’m good to go.

Until next time – GEEK OUT!

~GT~

Leave a Reply

Your email address will not be published. Required fields are marked *

Time limit is exhausted. Please reload the CAPTCHA.