In this article, I will share a Continuous Delivery Assessment, a tool applicable to all organisations that are seeking improvements on digital products creation via excellent software delivery practices.

>> Download the Continuous Delivery Assessment Google Excel

Digital transformation depends on Continuous Delivery

The high speed of Digital transformation requires digital products to be delivered faster and more often.

The ability to get digital products features and enhancements very quickly into the hands of the customers is imperative for organizations that want to stay competitive in the digital world.

What is Continuous Delivery?

Continuous Delivery is a concept that was first described in the 2010 book (of the same name as the concept) by Jez Humble and David Farley. Continuous Delivery provides a standard language for the activities of building, testing, and deploying software on the production path.

“ Continuous Delivery is a software development discipline where you build software in such a way that the software can be released to production at any time. “ – Martin Fowler

The importance to assess Continuous Delivery

Every organisation that is interested in the concept of Continuous Delivery, putting it into practice, inevitably undergoes major changes in the way it builds and makes its software available to its customers.

The perception of adopting Continuous Delivery can be extremely subjective. Although it is very important to visualize the progress in adopting and improving such practice.

The Continuous Delivery Assessment

The Continuous Delivery Assessment is a tool capable of measuring the improvements achieved by the adoption of Continuous Delivery. It is an excel spreadsheet that visually depicts 8 dimensions of maturity for the delivery process:

  • Delivery Process
  • Visibility
  • Organisational Alignment
  • Technical Architecture
  • Data storage
  • Environments and Deployments
  • Configuration Control
  • Quality Control

Typically, there is an assessment period, with many interviews using the questions written in the excel to guide the conversation, that undergoes for a given product context and then the result is visible in a spider chart, as in the example below:

The entire assessment process should be conducted by a person with good experience in Continuous Delivery concepts and practices, the interviewer.

Typically, the interviewer spends a few days conducting question-and-answer sessions and lots of conversation with all professionals representing the areas involved in the creation of the software product in question.

Using notes, and according to the answers obtained during this period, the interviewer qualifies each dimension on a scale of 0-5.

As a result, a graph will be generated, as shown above, for a visual understanding of the current stage in each of the continuous delivery dimensions.

Continuous Delivery for new and experienced organisations

For organisations new to Continuous Delivery practices, the assessment result is useful to help plan investments and process improvements. The results provide the first steps for this journey.

For organisations that already do continuous delivery, the assessment can help identify the parts that need attention, or that have not yet adapted to the processes.

Track each Continuous Delivery dimension score

Use the Continuous Delivery shared Google Excel with the questions to interview each of the interviewees. In the spreadsheet, it will be possible to score the answers for each of the following Continuous Delivery dimensions: Delivery Process, Visibility, Organisational Alignment, Technical, Architecture, Data storage, Environments and Deployments, Configuration Control, Quality Control). Each dimension score is a vertex in the spider chart graph.

The Continuous Delivery shared Google Excel has 9 tabs: one for the result and eight for the each of the Continuous Delivery dimensions. You should fill up the tabs for each dimension.

The result tab has a table and the spider chart graph depicting the Continuous Delivery score for each dimension. It  is automatically generated.

sample result: table and graph

For each Continuous Delivery aspect there are a few notes for the levels 0, 3 and 5; 5 being the best possible result.

These notes are to guide the conversation. Based on the conversation, you will decide the score, from 0 to 5.

Take note of the Continuous Delivery interviews

During conversations, take note of everything. This is a time to listen and understand the process. In addition to writing down the scores (0-5) for each dimension, the notes will help explaining the result of the assessment. Remember that the result is specific to the organisation, team, or product under assessment. It is not a generic result. Your notes should provide that specificity.

Interview everyone in the product context

Seek representatives from all areas involved in delivering the specific. For example, product managers, designers, developers, operations, and support people.

Be aware of people schedules

Schedule conversations in advance. Make sure to be flexible with people agenda and daily challenges. People fall behind, and priorities can change (especially in large organisations). Leave some interview times vacant to help these people and their schedules.

Leave space between conversations

Leave space between the scheduled conversations so you have time to reflect and organize your notes. This break between interview slots allows people to be late, arrive earlier, or to extend the conversation when desirable.

Understand the day-to-day activities

Ask questions wanting to understand the daily life of each person interviewed. Understand how she handles her activities related to delivering the product in question. Make it clear that you are not looking for culprits or are interested in hearing complaints from colleagues and areas of the company, but rather in understanding the process.

Structure and present your notes

Once you gathered data from all interviews, the next step is to structure your notes succinctly. Bullet Points and Mind Maps are good tools for this.

The Excel spider chart graph with the dimensions score is important, but the notes and the reasoning behind them is even more important. Most likely, besides an excel you will also create some other artefact with a summary of your structured notes.

Recognise the Continuous Delivery progress so far

You should bring visibility to the actions and initiatives that are already taking place in the organisation. Whether there is always room for improvements and extra stuff, the assessment result is an excellent opportunity to reinforce the message and amplify the visibility of the journey so far.

Repeat the Continuous Delivery assessment periodically

It is highly recommended to repeat the Continuous Delivery assessment periodically. If possible, do schedule the same assessment in the same product context with the same interviewees.

For example, a certain company with an IT department with a hundred people repeats this assessment every quarter.

Another example is for an amazing consultancy engagement, that do run such an assessment in the beginning of the engagement, and then repeat it towards the end of the engagement to measure and show progress.

Comparing the results from time to time will show progress and the next actions to be taken. This is essential to validate what has been done, understand the path the organisation has followed so far, and reassess the next steps.

How to download and use the Continuous Delivery assessment shared excel

1. Click on the shared excel link (this should open a view only shared google excel)

2. Make a copy (or download as an excel file)

3. Find the question on each Continuous Delivery (CD) dimension tab
4. Write the result for each dimension (if this is the average of several interviews, add the result)

5. View the result table and the spider chart graph on the result tab

Article history: This article is based on a few articles and blog posts originally written in Portuguese. Over the years (since 2011) , I have been involved in many assessments for Continuous Delivery and DevOps. The provided excel is a simplification of many other assessments styles I have seen / used before.

Lately I have been using assessments as part of inceptions for transformational projects. Therefore the need to explain it in more details.