Centralized and reliable SSH client configuration

SSH'ing into several VMs in a daily basics can be challenging and error-prone. In most cases you have to remember a bunch of different options: ips/domains, users, ports, ciphers… It’s hard to be updated when servers are added/removed and the SSH tunneling commands are easy to forget. At Zaleos we try to minimize errors by adopting good practices or well-defined strategies. In this blog post, we’re going to share the solution we adopted to efficiently sharing SSH client configuration within our team.

1. The SSH Config file

Is the key of our setup. You can configure the SSH options you want to use using the SSH Config file, which is usually stored in your home folder. At Zaleos, we use the config.d folder: ~/.ssh/config.d/. This allows the user to have more than one config.This great tutorial explains pretty well the main magic behind the SSH Config file. In our case, we have a strict naming convention which we all follow. It starts with zaleos and then it covers the networking logic based on subnets (dev, qa, …). This categorization helps to quickly trigger BASH autocompletion and start typing where you want to SSH into.SSH Basic Configuration

Zaleos file

• This is a basic Zaleos SSH Configuration item.
• Stored at: ~/.ssh/config.d/zaleos.config

Host zaleos_dev_git1*
     HostName git1.dev.zaleos.net
     Port 1111
     Host zaleos_dev_git1_ROOT     

Host zaleos_dev_git2*
     HostName git2.dev.zaleos.net
     Port 2222
     Host zaleos_dev_git2_ROOT     
     Host zaleos_dev_git2_ATLSTASH

Host zaleos_*_ATLSTASH
     User atlstash

Host zaleos_*_ROOT
     User root

As you may already know, this configuration makes two following two set of commands identical:

$ ssh -p 1111 root@git1.dev.zaleos.net
$ ssh -p 2222 root@git2.dev.zaleos.net
$ ssh -p 2222 atlstash@git2.dev.zaleos.net

$ ssh zaleos_dev_git1_ATLSTASH
$ ssh zaleos_dev_git2_ROOT
$ ssh zaleos_dev_git2_ATLSTASH

Personal

• This is a basic Personal SSH Configuration item.
• Stored at: ~/.ssh/config.d/zzz.personal.config

Host solete_nas*
    ForwardAgent yes

Host solete_nas1_*
    HostName 192.168.0.10
    Host solete_nas1_BAR
    Host solete_nas1_FOO

Host solete_nas2_*
    HostName 192.168.0.20
    Host solete_nas2_BAR
    Host solete_nas2_FOO

Host solete_*_BAR
    User bar

Host solete_*_FOO
    User foo

Host *
    Port 22
    Compression yes
    CompressionLevel 9
    Ciphers aes128-cbc,3des-cbc,aes192-cbc,aes256-cbc
    ConnectionAttempts 1
    TCPKeepAlive no
    ServerAliveCountMax 500
    ServerAliveInterval 10

This configuration has a default wildcard (*), which actually sets the defaults to be applied.

2. Sharing the SSH config file

It’s very important this file to be centralized and version controlled. We store it in a internal git repo with the proper permissions. In our case, just the admins can edit this file. We all have a predefined functions in our .bashrc,

update_ssh_config() {
    conf_file_dir="http://git.zaleos.net/projects/myrepo/ssh/zaleos.config?raw"
    mkdir -p ~/.ssh/config.d
    curl -o ~/.ssh/config.d/zaleos.config ${conf_file_dir}
    cat ~/.ssh/config.d/* > ~/.ssh/config
    chmod 600 ~/.ssh/config
}

This function:

• Gets the latest zaleos.config from the repo and stores it in the ~/.ssh/config.d/ folder.
• Generates a new ~/.ssh/config file from all the files stored in the ~/.ssh/config.d/ folder (alphabetical order).

3. Complex SSH requirements

Let’s imagine we don’t reach a server called unaccessible, but we can get into a server called accessible which reaches the unaccessible one.

3.a) SSH Proxying

We need to SSH into the unaccessible one. In this case we can easily configure a ProxyCommand as follows:

Host zaleos_accessible*
     HostName accessible.zaleos.net
     Host zaleos_accessible_ROOT

Host zaleos_unaccessible*
    HostName unaccessible.zaleos.net
    Host zaleos_unaccessible_MYCOOL_BRIDGE_ROOT

Host zaleos_*_MYCOOL_BRIDGE*
    ProxyCommand ssh zaleos_accessible_ROOT nc %h %p

Host zaleos_*_ROOT
     User root

$ ssh zaleos_accessible_ROOT
$ # this gets us into the accessible server
$ ssh zaleos_unaccessible_MYCOOL_BRIDGE_ROOT
$ # this gets us into the unaccessible server (through the accessible one)

3.b) SSH Forwarding

We need reach the port 80 of the unaccessible as if it were local port 40080. In this case we can easily configure a LocalForward as follows:

Host zaleos_tunnel_accessible_FOOBAR
     HostName accessible.zaleos.net
     LocalForward localhost:40080 unaccessible.zaleos.net:80

Host zaleos_*_FOOBAR
     User foobar

$ ssh zaleos_tunnel_accessible_FOOBAR
$ # don't close this SSH connection until done
$ telnet 127.0.0.1 40080
$ # Or open the browser 127.0.0.1:40080 ;-)

4. Conclusions

This post shares the knowledge and technique used at Zaleos to be better. The mix of simple configuration files with the complex ones commented is a very powerful tool to SSH into boxes.

As you can see, launching our SSH config BASH function and then typing ssh <TAB> is a very efficient way to reach our destination. Happy SSHing! ;-)

$ update_ssh_config # just when network changes
$ # Open new terminal or source file
$ ssh <TAB>
solete_nas1_BAR
solete_nas1_FOO
solete_nas2_BAR
solete_nas2_FOO
zaleos_dev_git1_ROOT
zaleos_dev_git2_ATLSTASH
zaleos_dev_git2_ROOT
zaleos_accessible_ROOT
zaleos_tunnel_accessible_FOOBAR
zaleos_unaccessible_MYCOOL_BRIDGE_ROOT
...

                                                                                                                                ‌