Enabling direct programming of your provisioning and deployment automation, rather than obscuring it, is a principal goal of SyncWrap.

Getting Started

Lets consider, as an example, provisioning users (aka roles) for use with a PostgreSQL database. The database engine may be installed via the existing SyncWrap::PostgreSQL component. If doing this manually we might start by adding the first database user we need, simply by running (PostgreSQL CLI) createuser bob on each host. Lets start by simply doing the same with SyncWrap. We create a new MyDatabase component (here directly in the sync.rb, but it could also be require’d from lib/my_database.rb, see LAYOUT) and add it to the appropriate hosts (or roles):

class MyDatabase < SyncWrap::Component
  def install
    sh( <<-SH, user: 'postgres' )
      createuser bob
    SH
  end
end

host 'localhost', MyDatabase.new

The <<-SH here-document is arguably overkill for this simple Bash command, but the syntax will prove helpful as the Bash fragments become more complex. The above assumes we are testing this on a localhost (where PostgreSQL already installed), but you might also want to try plugging this into something like examples/ec2.rb from the SyncWrap git repo and test with a temporary EC2 host.

Initial output using syncwrap -v (–verbose) flag:

syncwrap -f db_setup.rb -v
== localhost #<Module:0x0000000321f6a8>::MyDatabase#install: enqueue
<-- sh localhost (-v coalesce user:postgres live)
createuser bob
--> Exit 0 (success)

Repeatability

Now if we run the above a second time, we’ll get an error:

syncwrap -f db_setup.rb -v
== localhost #<Module:0x000000028a7620>::MyDatabase#install: enqueue
<-- sh localhost (-v coalesce user:postgres live)
createuser bob
createuser: creation of new role failed: ERROR:  role "bob" already exists
== localhost error [1]:
CommandFailure: sudo exit code: 1
/home/david/src/syncwrap/lib/syncwrap/context.rb:261:in `capture_stream'
/home/david/src/syncwrap/lib/syncwrap/context.rb:189:in `run_shell'
...

In general, when automating provisioning and deployment, we need to make the process idempotent, meaning that we can run it multiple times and get the same desired result. Here’s the simplest fix:

class MyDatabase < SyncWrap::Component
  def install
    sh( <<-SH, user: 'postgres' )
      createuser bob || true
    SH
  end
end

This continues to show the error on verbose output but returns success and allows the automation to continue. In general this is preferable to no automation. If this is checked into version control, basic repeatable automation and self documentation has been achieved. A remaining problem is that createuser could fail for more reasons than just that we already created the user, but our script fragment ignores all errors. A more robust approach would be to first check if the user already exists:¹

class MyDatabase < SyncWrap::Component
  def install
    sql_test = "SELECT count(*) FROM pg_user WHERE usename = 'bob'"
    sh( <<-SH, user: 'postgres' )
      if [[ $(psql -tA -c "#{sql_test}") == "0" ]]; then
        createuser bob
      fi
    SH
  end
end

De-/Composition in Ruby

If at some point in the future we need to add a second PostgreSQL user, then the script fragment becomes a good candidate for a Ruby method:

class MyDatabase < SyncWrap::Component
  def install
    %w[ bob joanne ].each { |u| pg_create_user( u ) }
  end

  # Create the PostgreSQL user if not already present
  def pg_create_user( user, flags = [] )
    sql_test = "SELECT count(*) FROM pg_user WHERE usename = '#{user}'"
    sh( <<-SH, user: 'postgres' )
      if [[ $(psql -tA -c "#{sql_test}") == "0" ]]; then
        createuser #{flags.join ' '} #{user}
      fi
    SH
  end
end

The SyncWrap command queue will effectively combine any number of pg_create_user calls into a script to be executed in a single shell session, so there is no significant overhead vs. implementing the loop in Bash. Indeed we can see this by running it with verbose output enabled:

syncwrap -f db_setup.rb -v
== localhost #<Module:0x0000000300aed0>::MyDatabase#install: enqueue
<-- sh localhost (-v coalesce user:postgres live)
if [[ $(psql -tA -c "SELECT count(*) FROM pg_user WHERE usename = 'bob'") == "0" ]]; then
  createuser  bob
fi
psql -tA -c "SELECT count(*) FROM pg_user WHERE usename = 'bob'")
psql -tA -c "SELECT count(*) FROM pg_user WHERE usename = 'bob'"
if [[ $(psql -tA -c "SELECT count(*) FROM pg_user WHERE usename = 'joanne'") == "0" ]]; then
  createuser  joanne
fi
psql -tA -c "SELECT count(*) FROM pg_user WHERE usename = 'joanne'")
psql -tA -c "SELECT count(*) FROM pg_user WHERE usename = 'joanne'"
--> Exit 0 (success)

Note that the script echoing is via Bash’s own set -v. Since the psql test command is run in a subshell it is echoed more than once. If we drop users “bob” and “joanne” and run it again with the -x flag, which uses Bash’s set -x:

syncwrap -f db_setup.rb -vx
== localhost #<Module:0x0000000345ac60>::MyDatabase#install: enqueue
<-- sh localhost (-x coalesce user:postgres live)
++ psql -tA -c 'SELECT count(*) FROM pg_user WHERE usename = '\''bob'\'''
+ [[ 0 == \0 ]]
+ createuser bob
++ psql -tA -c 'SELECT count(*) FROM pg_user WHERE usename = '\''joanne'\'''
+ [[ 0 == \0 ]]
+ createuser joanne
--> Exit 0 (success)

This shows that createuser is indeed run based on the matching condition. If run again, we see that the conditional is working as intended and createuser is not run if the users already exist:

syncwrap -f db_setup.rb -vx
== localhost #<Module:0x000000028a2c60>::MyDatabase#install: enqueue
<-- sh localhost (-x coalesce user:postgres live)
++ psql -tA -c 'SELECT count(*) FROM pg_user WHERE usename = '\''bob'\'''
+ [[ 1 == \0 ]]
++ psql -tA -c 'SELECT count(*) FROM pg_user WHERE usename = '\''joanne'\'''
+ [[ 1 == \0 ]]
--> Exit 0 (success)

As a final example of composition, the fragment below taken from SyncWrap::PostgreSQL demonstrates the block form of sh (here the sudo variant) where Ruby function dist_service enqueues commands from within an outer Bash conditional.

sudo( "if [ ! -d '#{pg_data_dir}/base' ]; then", close: "fi" ) do
  unless pg_data_dir == pg_default_data_dir
    sudo <<-SH
      mkdir -p #{pg_data_dir}
      chown postgres:postgres #{pg_data_dir}
      chmod 700 #{pg_data_dir}
    SH
  end
  dist_service( 'postgresql', 'initdb' )
end

Conclusions

The Bash used above can be thought of as low-level anonymous functions², where all higher level logic is implemented in Ruby. The remote pre-provisioning requirements are limited to sudo access and the bash shell, which comes by default on target Linux distributions. Since provisioning tasks are typically composed from system commands, Bash is a reasonable choice at this level. To access remote state, we need to be able to write bash conditionals and some simple pipelines. The (Wooledge) BashGuide and Bash Hackers Wiki may be useful resources. As shown, syncwrap provides convenient diagnostic options in order to debug your bash fragments.

Should every PostgreSQL user need to implement pg_create_user like methods?. The SyncWrap::PostgreSQL component can and should (#1) be extended to provide these kinds of general utility methods, freeing MyDatabase of needing to implement the same. Given dynamic component binding, the pg_create_user method can simply be moved to the SyncWrap::PostgreSQL component class, with the install method remaining unchanged.

The example demonstrates the low barrier from manual provisioning in an interactive shell, to basic automation in SyncWrap using similar bash fragments, unobscured. For flexibility and reuse, functional decomposition and re-composition can be done in Ruby with real Component objects–no need to deal with Bash functions. Add the ability to push templated configuration files, coordinated in Ruby, and we have a complete and direct system for provisioning and deployment.