Curlbomb - Server Automation for the Wayfaring Hacker

May 19, 2016 at 10:13 AM | categories: curlbomb | View Comments

The industry is full of configuration frameworks: chef, puppet, fabric, salt, ansible, etc. I've also personally worked on several proprietary ones too. Most of these frameworks are pretty good at what they do. It's a good time to be alive if you're into devops.

All of these frameworks require some amount of setup to get going (some more than others.) In this way, they are geared towards running in a preconfigured environment, and are not easily used with fresh hardware that is outside of this environment. Mostly I mean having SSH keys installed, software dependencies setup, and (usually) the client daemon for your chosen automation framework running:

  • chef - Requires chef-client preinstalled on the client.
  • puppet - Requires puppet agent preinstalled on the client.
  • fabric - Runs entirely through SSH, no client is necessary, but does require SSH keys to be installed.
  • salt - Requires salt-minion preinstalled, or to use salt-ssh which requires SSH keys to be preinstalled.
  • ansible - Runs via SSH, so requires SSH keys to be preinstalled.

If you're running an enterprise, this extra setup is not a problem. You just bake the setup into your OS image, including the SSH keys, software dependencies, and a systemd service file (I don't get the hate) to run your client. Spin up a new instance with that image, and chef, puppet, or whatever can see it and control it.

But I'm a wayfaring hacker. I encounter new systems, on new networks, all the time. Here's a few examples where it is not so easy to setup these tools:

  • Installing Linux on a computer using a Live CD. Live CDs typically have a bare minimum of packages installed and don't have an SSH service running, let alone have your SSH keys installed.
  • Being given hardware from an IT organization, or from a partner company. In these cases I've usually been told to use IPMI or I have been given a passphrase rather than an SSH key.
  • Using an Open Source image for your virtualization platform. Maybe you don't trust yourself to perform security updates on your images. Maybe you feel safer running the base image provided by the distribution.
  • Cloud instances where you forgot to give it your SSH key. Yea, typically I just smack my head and redeploy with the key, but this happens to me fairly regularly.
  • Embedded systems, or Virtual Machines, where an SSH service and automation client is overkill.
  • You're at someone elses desk, they're logged into the system with their own key. Maybe you don't even have an account.

It's not that hard to copy your SSH key into a ~/.ssh/authorized_keys file, so what am I ranting on about here? I'm about to propose something even easier.

The wayfaring hacker is very different than the enterprise. He has more limited resources, more limited time, and is doing things very often in a one-off fashion. The wayfaring hacker doesn't have time to create images for all the systems he uses. He may not have the resources to dedicate a server to configuration managment and prefers to just use his laptop. Wayfaring hackers still find homes within enterprises, but his default way of working is to ignore enterprise standards of setting up data centers and creating system images. The wayfaring hacker bootstraps from default Linux distros. The wayfaring hacker regularly tries novel things on cloud instances for only a few hours, then shuts them down, only to rebuild them again a bit later.

Ad-hoc server automation

I've been using the following method of scripting the bootstrap of new machines for several years now. I have to assume I'm not the only one who does it this way, but I don't see it referenced too often. It doesn't use any of the above mentioned configuration management tools. When you're doing something simple, on just one machine, they're overkill. I just do this:

gpg -d ~/git/machine_scripts/my_server_setup.sh.gpg | ssh my_server

This one-liner is great. Let me enumerate the ways:

  • It uses tools commonly installed almost everywhere.
  • It uses ssh, so everything is encrypted on the wire.
  • The script is executed directly, without saving it on the remote client.
  • The script is safely stored, gpg encrypted, on my laptop. It is only ever decrypted right before being sent over the wire.
  • I edit the script in Emacs, which supports gpg encryption out of the box.
  • I store the script in a git repository, so it's always ready to use. If I need to redeploy, just power cycle the server, wipe the drives (or spin up a new cloud instance), and rerun the script.
  • Passphrases or other credentials can be put directly into the script because of the security this affords.

This makes it really easy for me to write a script for a specific machine, and keep that script safe on my own laptop (where it's easy to edit with any editor I have installed locally I might add.) I can redeploy my cloud instance, and rerun my script on it to do whatever I want. It's got none of the niceties (or error handling) of the big configuration management tools, but it's great for simple scripts (including scripts that kick off jobs for those other tools.)

But this still has the same problem as before, the machine still requires an SSH server, and it requires that my key is setup.

Server automation with curlbomb

The gpg | ssh trick is very similar to another well-known trick: curl | bash. It's probably the most popular way of installing open source software these days. It's everywhere (a few examples: docker, sandstorm, even Salt.)

The great thing about curl | bash is that it has very minimal requirements on the client end... you guessed it: curl and bash. You don't need an SSH server (nor ssh keys). All you have to do is type the command and stuff happens. Magic! There's a lot of reasons to not do that though (and some that make a pretty good argument it's ok), which this article doesn't really get into, I'm just planting the idea.

curlbomb is a twist on this idea. Instead of hosting an install script on a public webserver (that anyone can download and run), curlbomb starts a private webserver that requires an authentication token to download a script. You run this on your personal computer, where you have stored a script:

curlbomb run ~/git/machine_scripts/my_server.sh.gpg

Which outputs the following text to the screen:

Paste this command on the client:

  KNOCK=KfG2LkxEkrj4 bash <(curl -LSs http://192.0.2.100:56423)

curlbomb stays running until you paste that bit of text into another computer's shell. That computer will connect to your server, provide the preshared key (KNOCK), download the script, and pipe it into bash. Once the script has been served one time, the server shuts down. If you ever want to run the script again, you have to restart curlbomb. In essence, curlbomb is a private, one-time-use, curl | bash that only you can run. You write the script, and curlbomb ensures that it's transported securely to your client machine over TLS (SSL.) You gain security benefits similar to the gpg | ssh trick (GnuPG+OpenSSL+knock vs GnuPG+OpenSSH), and you also gain the dependency-free nature of curl | bash.

curlbomb can forward it's HTTP port through another server via SSH. This allows you to run the server on your laptop, from anywhere in the world, as long as you can ssh to another server you control. Any client, that can access the internet, can then download the script from your laptop by using the server as a proxy.

Let's look at the same scenarios I said was hard to automate with traditional configuration tools, this time with curlbomb:

  • Installing Linux on a computer using a Live CD - Stick the usb drive/CD into the computer, setup the network connection, and type a curl command.
  • Being given hardware from an IT organization, or from a partner company - Log in via IPMI, type a curl command.
  • Using an Open Source image for your virtualization platform - Log in, type a curl command.
  • Cloud instances where you forgot to give it your SSH key - Log in via the vendor provided console, type a curl command.
  • Embedded systems, or Virtual Machines, where an SSH service and automation client is overkill - Log in, type a curl command.
  • You're at someone elses desk, they're logged into the system with their own key - Type a curl command.

In essence, your dependencies for curlbomb on the client are:

  • a network connection.
  • curl installed.
  • a way to type the curl command.

Isn't that easy? No opening up ports to the client. No running daemons on the client. No passwords or SSH keys need setting up. No maintaining images. No need to ask questions on the client to fill in passwords or other setup questions (just put them in the script.)

Consider the following script I use to do the initial setup for a server. This creates my user account, sets my password, sets up my SSH key, and installs salt-minion so that I can now control the machine with saltstack:

#!/bin/bash
user=ryan
pass=hunter2
sshkey="ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAQEAmo9sQ6iAHDH8Gg1vd1js3R...."

# Create user account:
create_user() {
    useradd $user
    mkdir /home/$user
    chown $user:$user /home/$user
    chpasswd <<EOF
$user:$pass
EOF
    # Install SSH Key:
    mkdir /home/$user/.ssh
    chmod 700 /home/$user/.ssh
    echo "$sshkey" > /home/$user/.ssh/authorized_keys
    chmod 600 /home/$user/.ssh/authorized_keys
    chown -R $user:$user /home/$user/.ssh
}


install_salt_minion() {
    # Saltstack GPG key from https://repo.saltstack.com/apt/debian/8/amd64/latest/SALTSTACK-GPG-KEY.pub
    cat <<EOF | apt-key add -
-----BEGIN PGP PUBLIC KEY BLOCK-----
Version: GnuPG v2.0.22 (GNU/Linux)

mQENBFOpvpgBCADkP656H41i8fpplEEB8IeLhugyC2rTEwwSclb8tQNYtUiGdna9
m38kb0OS2DDrEdtdQb2hWCnswxaAkUunb2qq18vd3dBvlnI+C4/xu5ksZZkRj+fW
tArNR18V+2jkwcG26m8AxIrT+m4M6/bgnSfHTBtT5adNfVcTHqiT1JtCbQcXmwVw
WbqS6v/LhcsBE//SHne4uBCK/GHxZHhQ5jz5h+3vWeV4gvxS3Xu6v1IlIpLDwUts
kT1DumfynYnnZmWTGc6SYyIFXTPJLtnoWDb9OBdWgZxXfHEcBsKGha+bXO+m2tHA
gNneN9i5f8oNxo5njrL8jkCckOpNpng18BKXABEBAAG0MlNhbHRTdGFjayBQYWNr
YWdpbmcgVGVhbSA8cGFja2FnaW5nQHNhbHRzdGFjay5jb20+iQE4BBMBAgAiBQJT
qb6YAhsDBgsJCAcDAgYVCAIJCgsEFgIDAQIeAQIXgAAKCRAOCKFJ3le/vhkqB/0Q
WzELZf4d87WApzolLG+zpsJKtt/ueXL1W1KA7JILhXB1uyvVORt8uA9FjmE083o1
yE66wCya7V8hjNn2lkLXboOUd1UTErlRg1GYbIt++VPscTxHxwpjDGxDB1/fiX2o
nK5SEpuj4IeIPJVE/uLNAwZyfX8DArLVJ5h8lknwiHlQLGlnOu9ulEAejwAKt9CU
4oYTszYM4xrbtjB/fR+mPnYh2fBoQO4d/NQiejIEyd9IEEMd/03AJQBuMux62tjA
/NwvQ9eqNgLw9NisFNHRWtP4jhAOsshv1WW+zPzu3ozoO+lLHixUIz7fqRk38q8Q
9oNR31KvrkSNrFbA3D89uQENBFOpvpgBCADJ79iH10AfAfpTBEQwa6vzUI3Eltqb
9aZ0xbZV8V/8pnuU7rqM7Z+nJgldibFk4gFG2bHCG1C5aEH/FmcOMvTKDhJSFQUx
uhgxttMArXm2c22OSy1hpsnVG68G32Nag/QFEJ++3hNnbyGZpHnPiYgej3FrerQJ
zv456wIsxRDMvJ1NZQB3twoCqwapC6FJE2hukSdWB5yCYpWlZJXBKzlYz/gwD/Fr
GL578WrLhKw3UvnJmlpqQaDKwmV2s7MsoZogC6wkHE92kGPG2GmoRD3ALjmCvN1E
PsIsQGnwpcXsRpYVCoW7e2nW4wUf7IkFZ94yOCmUq6WreWI4NggRcFC5ABEBAAGJ
AR8EGAECAAkFAlOpvpgCGwwACgkQDgihSd5Xv74/NggA08kEdBkiWWwJZUZEy7cK
WWcgjnRuOHd4rPeT+vQbOWGu6x4bxuVf9aTiYkf7ZjVF2lPn97EXOEGFWPZeZbH4
vdRFH9jMtP+rrLt6+3c9j0M8SIJYwBL1+CNpEC/BuHj/Ra/cmnG5ZNhYebm76h5f
T9iPW9fFww36FzFka4VPlvA4oB7ebBtquFg3sdQNU/MmTVV4jPFWXxh4oRDDR+8N
1bcPnbB11b5ary99F/mqr7RgQ+YFF0uKRE3SKa7a+6cIuHEZ7Za+zhPaQlzAOZlx
fuBmScum8uQTrEF5+Um5zkwC7EXTdH1co/+/V/fpOtxIg4XO4kcugZefVm5ERfVS
MA==
=dtMN
-----END PGP PUBLIC KEY BLOCK-----
EOF

    cat <<EOF > /etc/apt/sources.list.d/saltstack.list
deb http://repo.saltstack.com/apt/debian/8/amd64/latest jessie main
EOF

    apt-get update
    apt-get install -y salt-minion

}

main() {
    create_user
    install_salt_minion

    systemctl start salt-minion
}

# It's a really good idea to create main functions and call them at
# the very bottom of the file whenever you are streaming a shell script
# from a remote location into your shell. What happens if you want to
# run the command "rm -rf /home/guy_who_doesnt_work_here_anymore" and
# the connection dies right as it gets to the first slash? I'm developing 
# a buffering feature into curlbomb, to prevent this, but it's not done yet 
# (although the encryption features do accomplish that as a side effect.)

main 

curlbomb is useful for these kinds of quick scripts that setup the environment for a more robust configuration management tool. It's also a way to keep your entire server setup automated and documented, without having to resort to system images.

This article only scratched the surface of what curlbomb can do. Learn about it in more detail at the curlbomb github project page.

Read and Post Comments