Recently appendTo has been leveling up many of our internal processes, procedures, and standards. This has been a long time coming, but has been given dedicated attention recently. One of these efforts, the focus of this article, is appendTo’s most recent upgrade to our continuous integration and deployment (CI/CD) procedure and systems. This is one big piece of a larger standards and practices effort that is ongoing within our organization, and which benefits both our customers as well as our staff. If you would like to read more about our general developer workflow, standard practices, and tools you should read Aaron Bushnell’s recent article titled “The Tools We Use“!
There are some unique challenges to solve in a professional services organization when it comes to project server management. Each client, and each project for those clients, inevitably has unique software requirements. One client may have a back end in Rails which requires a specific application and testing stack while another may use Node.js with MongoDB. Furthermore, we could have one client which needs to use Node.js 0.10.28 because 0.10.29 has some change that breaks their application. These widely varying requirements result in appendTo not being able to simply deploy all client applications to a single staging server, we have to have unique environments.
Unfortunately, we don’t typically have the freedom to assign a server administrator to each project, and even if we could, that would not be the best use of our time, and thus our client’s investment. The solution that appendTo has implemented balances the needs of our clients with ease of use for our developer staff. The system is highly automated, but allows for extensive customization of the entire build process by the developers through simple JSON configuration within the project repository. We use TeamCity for version control triggers, build step management, access control, and reporting and logging. On the other side of the process we use Digital Ocean (and their HTTP API) to dynamically generate virtual machines (droplets) when needed and host and serve application files.
Bridging the Gap
So what does this all look like? The diagram below identifies the big pieces, on the left we have Github, where all of our application code resides. TeamCity, on the server in the center of the diagram, will identify code changes in a repository and initiate a build process. The build process will send various commands, in a particular order, to the Node application which in turn interacts with the Digital Ocean HTTP API and directly with the droplets over ssh. The droplets are provisioned when necessary (usually on first run) from a standard image that gives projects access to commonly needed software (like Node). Finally, the Node scripts report the results of each action to TeamCity in order to create a single source for build logs and history.
Wrap It Up and Put a Bow On It
This setup works very well for our needs, but we wanted to make things easier on our developers. To that end we made each step of the build process configurable within the project itself. Many of our projects use Grunt to run tests, concatenate and minify files, and generally build a project for distribution. In order allow for the simplest integration of that process, the specific actions required for each build step are specified in the “scripts” block of the project’s package.json file (a Node artifact).
Here is what that might look like:
"provision": "apt-get install mongodb",
"install": "bower install",
"test": "grunt test",
"stop": "forever stopall",
"deploy": "grunt deploy",
"start": "forever server/main.js"
Each of the lines in this “scripts” block maps to a step (or part of a step) in the larger build process. They will all be executed within the code directory, meaning developers don’t need to worry about where the code is located and can focus on what needs to be done to make it work. Additionally, since each line is executed (essentially) on the command line over ssh, the developer could specify a shell script should their process be overly complex for a one-line statement. Note that using Grunt is not at all required, but this process simplifies using tools like it with continuous integration and deployment.
As we discussed earlier, the goal with this bridge application written in Node is portability. Should we need to transition to a new tool on either side of that diagram above, we can reuse what we have written and our existing projects will require no alterations. This makes our developers happy, our clients happy, and results in better solutions delivered faster.
By implementing better standards and practices, and properly vetting those through our development staff, appendTo is able to accomplish two simultaneous goals: rapidly deliver better solutions to our clients and make the lives of our developers easier. The first may seem obvious, better standards (and adherence to them) directly translates into cleaner, more manageable code. The second may not be as obvious, but it is just as important to us. We value our developers’ time as well as their opinions. That’s why we vet our standards through them. Any employee is able to submit a change to our standards and practices. This change will be discussed among the entire organization and ultimately is approved by an S&P board which is composed of upper level engineers.