Telling our CI story

October 4, 2012 § Leave a comment

I’ll be speaking at the Better Software Conference in Orlando this November. One of the sponsors, asked me to share a little more about what led AtTask to embrace a continuously integrated system. When it first came out, I thought our stuff was pretty novel. Now I think it’s just the cost of doing business if you’re selling software on the web.

You can read the whole interview here.

Selling to the Enterprise, a Product Dev Dilemma

September 22, 2012 § Leave a comment

I recently read an excellent article by Paul Graham that defines a startup (as distinct from any other new business) as the combination of two key attributes:

  1. It must make something lots of people want
  2. There must be a way to reach those people

Most businesses can do one of those things, but not both together. Software and the distribution medium of the web has opened up all kinds of possibilities for achieving both goals, which is why another key indicator for a startup is rate of growth.

The common unit of success for any startup, then, is the growth of the user base. This presents a problem for a business geared at selling to the enterprise: although the unit of growth is the user, the acquisition of users is done at the level of the corporate account. This creates conflicts all through the product development process. Building sexy and easy-to-use is great for the user, but they’re not the ones buying your software. Building rich business value will help you win accounts, but lack of adoption by the end-user within the business will doom you when it comes time to renew.

Because if this internal tension, I believe anyone doing product development geared at selling to the enterprise must consider one of two growth models:

  1. Design to maximize number of accounts sold
  2. Design to maximize number of licenses per account

You achieve growth in model one by selling to as many companies as possible. You accept as a ground rule that you will probably not achieve rapid growth by going wall-to-wall within any corporation that buys. As a product designer, you should focus your efforts on deep solutions for specific verticals (Marketing groups, Help Desk, HR). This focus allows you to solve the problems better than one-size-fits-all solutions that could be sold to a larger audience within a single company. You’ll win more accounts than Model Two, but with fewer seats sold per account.

In Model Two you grow by selling as many possible licenses into each account. The trade-off for going with Model Two will be that you must provide a solution that is either one-size-fits-all or extremely customizable. It must work across a huge variety of professional disciplines. Wall-to-wall adoption of a product across an enterprise is rare, and must typically focus on common problem-sets to all workers. Examples here include collaboration, management and productivity-enhancement tools. Selling a tool like this requires a value-proposition that makes sense to Senior Executives, and the adaptability of the product will likely mean that compromises must be made on ease of use for specific verticals. As a designer, you should solve problems in the most generic way possible, with an emphasis on strategic problems in the business rather than tactical ones.

So, what does this have to do with Continuous Delivery? In any start-up you must build a system that allows you iterate on concepts as fast as possible so that you eliminate designs that don’t lead to growth. If you are selling to the enterprise the importance of Continuous Delivery in your system will depend on which of the two models you choose. If you’re shooting for selling the maximum number of accounts, you can iterate on look-and-feel, and on solving the most common problem sets of your targeted verticals. This can happen rapidly because your have more total accounts to work with. If your ambition is to grow by deal size, your iterations will probably be longer because you will have fewer over-all deals to test your strategy against.

I think if you’re building enterprise software, deciding on one of these two paths to growth is key to focusing the Product Development process enough to iterate into a successful solution.


People work hard for money, harder for leaders… but hardest for a cause.

May 8, 2012 § 2 Comments

From my vantage, the Atlanta Scrum Gathering was largely a success. There were a few stand-out talks, but some duds as well. I was surprised by how light many of the presentations actually were–perhaps because agile methodologies tend to be pretty simple in the first place. That said, there were a couple of learnings that will stick with me as I move our organization forward.

Time boxing is not a pre-requisite for effective Agile. The closer we’ve moved to Continuous Delivery, the more the time-boxing of our Sprints has felt like an artificial construct. Lots of people have run into this problem when working on queue-driven activities. One person shared an example of an off-shore team whose entire job was to write regular expressions for website scraping. There was no reason to use a time-boxed release cadence. Instead, the team pulled work from a queue at a consistent pace, delivering the work as it was completed. The ER (Escalation Response) team at AtTask is already doing this, just without a mechanism for observability by external participants, or a formal improvement program. I believe that Kanban solves this problem in an elegant way, and would be a relatively painless model to explore.

“People work hard for money, and harder for leaders. People work hardest for a cause.” This quote was shared during this mornings keynote by a gentleman named Tanner Corbridge. He works with Partners in Leadership, the company behind The Oz Principle. He mentioned it almost as an aside, but it certainly resonated with me. The fire that we instill in our organization should not be about compensation, or about the charisma of those that lead our teams. It should be about building our culture around a common purpose. It’s something we should have laser like focus on in everything we do. I’m very proud of AtTask’s culture, and believe that our adoption of agile has helped us focus on our common cause. AtTask is about improving the work lives of knowledge workers across the world, and we can measure ourselves against that cause. I’m looking forward to discovering ways to accomplish that going forward.

Oh, and Zombie survival video games are a great platform for exploring agile. I’ll leave that to your imagination.

Continuously Integrated with Selenium

May 2, 2012 § Leave a comment

The video from my talk at Selenium Conference 2012 is now online. I discuss the challenges and opportunities in using cloud infrastructure to run large Selenium suites every check-in. Here’s the abstract:

Most engineering organizations include some level of continuous integration in their development process. The brutal truth is that far too often these efforts are plagued by unreliable tests, long feedback loops, and bad configuration management. Learn how AtTask decided to radically rethink their software development model, and by using open source and cloud solutions went from 3 days of acceptance testing to just minutes, providing feedback on thousands of selenium tests to every engineer in the organization. See how by leveraging publicly available tools, you can deliver hyper-scalability to your Continuous Integration framework and include selenium testing per commit to drive cycle time down in your organization.

Feature Toggling our way to a better future

May 1, 2012 § Leave a comment

Today we released the first Phase of a major new user interface for AtTask. This was the culmination of the hard work over several teams, over a period of several months, during which we transitioned from monthly to daily releases, hired entire teams of engineers, and created an entirely new deployment pipeline. Needless to say it was a great relief to see the system finally deployed, and released into the eager arms of our users.

With a change this large, one might be led to believe that we inherited a high degree of risk. The truth is that this release was particularly low risk, compared to previous releases of our product. One of the reasons was our adoption of a practice that is becoming more common in the Continuous Delivery community, called “Feature Toggling“.

A Feature Toggle is a relatively simple in concept, but devious in execution. At it’s core, it simply means that everything which is coded that has exposure to end users, must be access-controlled through configuration. When you control access to each developed feature by configuration, you decouple the deployment of your changes from release to customers. The process of releasing software then becomes stage-gated: you first deploy, monitor, and learn from a baseline that excludes all customer facing change. Then you release. The release of the software becomes a business decision, not a technology decision. Companies like Facebook and Etsy use this technology every day to split-test and validate features in live production environments.

The reason Feature Toggles are devious in execution is because the implementation path is highly dependent on the flexibility of the application stack to absorb this model. This means that Feature Toggling is more of a design pattern than a product that can be packaged and sold to any development org. In fact, many companies who have developed a Feature Toggling framework in-house will come up with a clever name for their creation. We call ours “PitBoss.”

There are two primary ways that PitBoss differentiates itself from more typical configuration tools. Instead of choosing a simple “Yes” or “No” for each feature, PitBoss gives us the ability to evaluate an expression. We can look at the specific user logging in, and control availability based on their profile, group, company, or any other value we want. For example, if we want to release a new feature to just our beta-test customers, we could create an expression like:

“pitboss.isEnabled(user.beta == true)”

This is evaluated once when the user logs into the system, and drives the availability of features in the product. These are all configurable while the system is up and running, so features that are intended for a single group, or which we discover to be unstable, can be deactivated just as easily.

The second way we’ve differentiated PitBoss is by how we expose it in different layers of our application. In java we can make calls directly into PitBoss methods, but we also have the ability to control access to tiles through PitBoss annotations, or to reference it using JSTL syntax in our JSPs. PitBoss can be accessed through an internal REST api for controlling features in client applications. We have even implemented a mechanism for test cases that allows us to override a particular PitBoss default, for easy A/B testing of features behind configuration.

I mentioned that we released the first phase of our new user experience today, but the code was actually deployed days ago. We’ve had an opportunity to test it by activating the release for a specific set of users, and smoke testing in a live production environment for the first time.

I’m excited by the possibilities of this great new tool, and would love your comments about similar systems you’ve found success with in your organizations!

Cloud Scale CI with Jenkins

April 27, 2012 § Leave a comment

I’m sitting on the tarmac at JFK, thinking about the next time I’ll be in town. The sponsors of Jenkins User Confence NYC have invited me to speak on Continuous Integration in the Cloud. This talk will be similar to what we shared in Armenia, but with a crowd of hard-core Jenkins hackers, I think we’ll enjoy diving into the weeds a little more.
One of the first things we did after forming a tiger team to tackle the CI problem, was evaluate the vendors in the space both commercial and open-source. We had been using TeamCity from Jetbrains because of its deep integration with IntelliJ (we are a java shop). TeamCity has a lot going for it: a slick UI, great claim/blame functionality, and a built in remote run feature which essentially tests every diff against a shadow branch before merging to mainline. Jenkins was adopted in our QA organization, but was used primarily as a job automation system. It wasn’t connected to our version control, and was running on a single virtual machine.
Comparing these two tools was tough. We could keep what we had been using and begin the process of building out integrations to stand up our vision of a cloud-scale CI pipeline, or we could switch to Jenkins, absorb the crappy UI, and leverage the efforts of the community to give us a head start.
In the end, we went with Jenkins, and I’m glad we did. Yes, we ended up having to write a few plugins ourselves, but the initial hacking required to connect with our chosen cloud vendor, Amazon EC2, both it’s single instance APIs and cloud formation endpoints, was largely finished by folks in the community. Jenkins still stinks at visualization, but we have created views which have increased adoption in our org, and which we are now sharing with the community.
I’m looking forward to joining the Jenkins guys at their conference in NYC, stay tuned for slides, photos, and other content from the show on May 17th!

On the Continuous Integration world tour

April 26, 2012 § Leave a comment

We’ve finished up our visit to AtTask’s offices in Armenia. It will be a long flight home, but it was great to get some face time with the managers and leads there. As part of this trip, I spoke at the American University of Yerevan to a group of students, project managers, and engineers. The talk was on Massively Continuous Integration, the practice of applying scaling principles to your CI system using the Cloud. Definitely the most purple room in which I’ve spoken.

Massively Continuous Integration: From 3 days to 30 minutes

Honestly, the team did a great job putting on the whole event. We had over 100 people there, with lots of participation both before and after. CI principles aren’t as commonly adopted as I had expected.

The engineering program at this university provides an accredited masters degree. Many of the engineers in our Armenian office are graduates, and we’re looking into the possibility of beginning an internship program as well. I’ve found that with the right mentoring, the delta on productivity between fresh engineers and those with a decade or more of experience can be quite small. It will be interesting to see what can be accomplished in Yerevan in the months ahead.

Where Am I?

You are currently browsing the Continuous Delivery category at The Continuous Deliverist.

%d bloggers like this: