Provisioning, administration, and deployment of CouchDB, Java, Tomcat, etc., made easy with Pallet

As I briefly mentioned [in my last post], I've been working with Pallet to enable automated administration of, among other things, CouchDB. If you're wondering why I'm using Pallet instead of, say, Puppet or Chef, you can either read the "Why Write Another Tool?" section in Hugo Duncan's recent post on Pallet. My answer to that question is that I wanted a tool that would provide automated:

• Provisioning,
• Administration & configuration, and
• Application deployment

...all in one piece of kit that would neatly interoperate with the rest of our development stack (JVM, Clojure, Maven, Hudson, etc., etc). Pallet is the only option I found that thread that needle.

From bare metal to ready-for-production app deployment in 5 minutes or 5 paragraphs...

Using Pallet, we can automate everything necessary to provision and configure the resources needed to run our application. The following code defines, spins up, and configures an EC2 node; the steps listed below correspond almost exactly with each line of the defnode configuration that forms the majority of the code:

1. Use a specific Ubuntu AMI on a particular instance size
2. Use a standard firewall / security group configuration
3. Configure an "admin user" with a specific username that has only one authorized key (mine).
4. Tweak apt so that it's "sane". <snark>I like being able to install useful software, so multiverse it is.</snark>
5. Install the Sun JDK
6. Install the Tomcat application server
7. Install CouchDB and set two properties in its local.ini file (one to disable the javascript view server reduce limit – don't ape that if you don't know what you're doing – and one to change its default storage location to a different directory).
8. Create the aforementioned CouchDB storage directory.
9. Deploy our application as the ROOT application in tomcat and restart it (I've omitted the part that sets security policy in the same block, which is what actually necessitates the app server restart).

(I've simplified certain things in this rendition, but what I've elided are details that are pretty esoteric and/or miscellaneous – i.e. installing unlimited-strength crypto policy files in the installed JDK, setting VM parameters for Tomcat, etc.)

(pallet.core/defnode master
  [:ubuntu :X86_32 :size-id "m1.small"
   :image-id "ami-bb709dd2"
   :inbound-ports [22 80 443]]
  :bootstrap [(pallet.crate.admin/automated-admin-user +admin-username+)
              (sane-package-manager)]
  :configure [(pallet.crate.java/java :sun)
              (pallet.crate.tomcat/tomcat)
              (pallet.crate.couchdb/couchdb
                [:query_server_config :reduce_limit] "false"
                [:couchdb :database_dir] +couchdb-root+)
              (pallet.resource.directory/directory +couchdb-root+
                :owner "couchdb:couchdb" :mode 600)]
  :deploy [(pallet.resource.service/with-restart "tomcat*"
             (pallet.crate.tomcat/deploy-local-file "/path/to/my/warfile.war" "ROOT"))])

(def service (jcompute/compute-service "ec2" "AWS_ID" "AWS_SECRET_KEY" :ssh :log4j)

(pallet.core/with-admin-user [+admin-username+]
  (jcompute/with-compute-service [service]
    (pallet.core/converge {master 1} :configure :deploy)))

(Note that jcompute is an alias for the compute namespace provided by the excellent jclouds library, which Pallet uses for cloud-agnostic infrastructure provisioning as well as cloud-specific stuff, like EBS volume and snapshot management, elastic IP management, etc.)

Want to spin up 10 nodes instead of one? Change {master 1} to {master 10}. Other changes are similarly straightforward. Want to deploy an application update to existing nodes instead of creating new nodes? Instead of using converge, execute (pallet.core/lift master :deploy).

There's obviously a lot going on behind the scenes, but this is what the day-to-day configuration and usage of Pallet looks like. Using it means that I never have to use a command line or fiddly manual AWS tooling like their console or ElasticFox, or cobble together some combination of Chef/Puppet with Capistrano/Fabric and a pile of shell scripts to get a complete provision/configure/deploy solution.

Huge thanks to Hugo (who let me play in his sandbox ) and Adrian Cole (the crazy man behind jclouds) for making this all possible.