Category Archives: DevOps

5 Definitions of #DevOps

In my talk at QCon last week I was lucky enough to be the first speaker on the DevOps track so I took the opportunity to poll my fellow speakers for THEIR definition of DevOps and the results where fascinating.

What I ended up with was different 5 Definitions of DevOps:


DevOps - Matthew

DevOps - Anna

DevOps - Amy

DevOps - Steve

So what are the key take away messages from these DevOps definitions?

Firstly – there is no single, definitive version of what DevOps “is” as you can see from the responses above!

DevOps clearly means different things to different people and we (as the DevOps community) have to ask ourselves is this a good or bad thing? Good in that it’s promoting a diversity of ideas that contribute to the DevOps movement, but “bad” in that is causes confusion amongst people trying to learn more about DevOps.

Secondly – there is a clear message around communication, collaboration and culture being a key part of the DevOps message, which is excellent” #NoMoreSilos! J

Thirdly – sadly the name “DevOps” might be seen as being exclusionary in itself and not embracing critical parts of the SDLC community – specifically in Amy’s case the Test community. Amy expands on this in her blog post “DevOps with Testers”.

A number of people would say that DevOps should be called “DevTestOps” or “BizDevOps” but at QCON Dave Farley was arguing that the correct term should be “Continuous Delivery” as DevOps is just a subset of CD. We discussed this earlier in our post on Continuous Obsolescence.

  • So what does DevOps, DevTestOps or BizDevOps means to you?
  • Has the term DevOps had it’s day and should we just extend the definition of Continuous Delivery to embrace the full application lifecycle?
  • Do we need to rename our company? :-)

Your thoughts in the comments please?


#DevOps = #ContinuousDelivery + #ContinuousObsolescence?

During my talk at QCon last week we had started an interesting debate about whether DevOps was a subset of Continuous Delivery or vice versa.

We were lucky enough to have Dave Farley in the audience and it was clear from his definition of DevOps versus Continuous Delivery that he felt that CD was the superset and DevOps a component of that set… but he admitted that he if he was writing the Continuous Delivery Book again in 2015 would change the definition of “finished” to embrace the full product lifecycle from inception to retirement.

This aligns nicely with his co-author Jez Humble’s thoughts on being “Product-Centric” that we discussed in an earlier blog post.

“Finished” should mean “no longer delivering value to the client and hence retired from Production”.



That was the we joined a new term (live on stage as captured by Chris Massey’s tweet below) – “Continuous Obsolescence”


So what is #ContinuousObsolescence?

Continuous Obsolescence is the process of continuously grooming your product/feature portfolio and asking yourself “is this product/feature still delivering value to the customer?” because if it isn’t then it’s “muda” (waste), it’s costing you money and should be retired.

Continuous Obsolescence is the process of continuously grooming your product/feature portfolio and asking yourself “is this product/feature still delivering value to the customer?” because if it isn’t then it’s “muda” (waste), it’s costing you money and should be retired.

Every product/feature that’s not delivering value costs you by:

  • Wasted resources – every build, every release, you’re compiling and releasing code that doesn’t need to be shipped, and consuming server resources once deployed.
  • Overhead of Regression testing – it still needs to be tested every release
  • Unnecessary complexity – every product/feature you don’t need just adds complexity to your environment which complicates both the software development process and the production support process. Complex systems are harder to change, harder to debug and harder to support. If it’s not needed, simplify things and get rid of it!

I’d argue that DevOps could even be defined as
“#DevOps = #ContinuousDelivery + #ContinuousObsolescence”

If you’re doing DevOps and you’re a believer in the CALMS model then you should be practising “Continuous Obsolescence”… in fact I’d argue that DevOps could even be defined as “#DevOps = #ContinuousDelivery + #ContinuousObsolescence” – what do you think?

Send us your comments or join the debate on Twitter @DevOpsguys


Why aren’t more women in tech jobs?

Androgyny Edited

Last month we asked some questions about gender quotas in the workplace and whether or not that was the way to encourage more women into the world of IT professionals.

Our poll indicated that 86% of our Blog readers think that gender quotas are NOT the most effective way to equalise the IT gender gap, as one comment on our post stated:

To me, quotas seem like the well-intentioned, but ill-conceived path to diversity by avoiding cultural change and avoiding the problem of biases and institutions that create monoculture- Jyee

But what is the answer? Or perhaps we need to reconsider the question; instead of exploring how to encourage more women to embark on an IT-based career, maybe we first need to find out why they don’t already consider it an option.

  • In a traditionally male-dominated industry fully equipped with its own negative stereotypes it’s difficult for young women embarking on their careers to find role models in IT: ‘you can’t be what you can’t see’. Often women report hostility, or lack of opportunity in the workplace, finding themselves overlooked in favour of male counterparts:

The top two reasons why women leave are the hostile macho cultures — the hard hat culture of engineering, the geek culture of technology or the lab culture of science … and extreme work pressures – Laura Sherbin, a director at the Center for Work-Life Policy

  • It isn’t obvious what sort of tech roles are available until you start in the industry. A tech knowledge is a major part of almost all professional roles in contemporary society – it’s not just coding. The sector is developing rapidly, but a huge amount of this is not visible externally – girls in school may never consider that they could be in tech jobs creating robots, designing games, running start-ups, because this side of the industry is rarely touched upon as a career choice in schools. If young women in secondary education were aware of the potential of developing tech skills they might be more inclined to engage with tech-related subjects at school.
  • Women in tech have not had the best time of it. In the public eye women can see things like GamerGate and the ruthless persecution that can follow women who actively become involved in tech or web-based activities. Although this is far from the norm it’s what is predominantly shown in the media.

Clearly a culture shift is required, rather than short-term measures. Actively encouraging school-aged girls to take up an interested in technology is a way of opening up routes into the industry early on. Eliminate the notion of ‘gendered subjects’ and highlight the opportunities that a tech role brings both to individuals and to the future: it’s an ever-developing, changing exciting industry to be a part of and that’s something that will inspire boys and girls alike.

Technology pervades almost every element of contemporary culture and at least a basic knowledge of technology is becoming standard for the majority of professional roles. Using this as a way to destroy the trope that tech jobs are done by geeky men in a basement with no social skills and a penchant for online gaming.

But what are industries currently doing to tackle the problem of gender imbalance?

Companies like Google have introduced training schemes which aim to fight cultural biases and make the workplace a more positive and neutral environment. Participants take part in word association games and are often surprised at how often women are habitually associated with less technical roles. Hackathon are running a series of women-only events in India, Microsoft are running ‘DigiGirlz’ days all over the world and the Welsh Government’s WAVE scheme aims to help women improve their career prospects though training opportunities, networking opportunities and career development.

But what can smaller businesses do to help?

Companies need to research the biases that prevent women from getting ahead and then devise ‘interrupts’. Instead of single training sessions companies need to make systematic changes. – Joan C Williams.

This, to us, sounds like a DevOps approach. Like optimising your cloud-computing potential, gender equality in the tech industry is not something you can ‘do’ once; it’s an ongoing series of cultural and attitude changes that will deliver the best results for companies and employees. And remember: change takes time

We’d love to hear the strategies companies in the UK are undertaking to address this gender imbalance in the workplace. If you have any thoughts, suggestions or news you’d like to share, please get in touch.


How Do Databases Fit Into DevOps?



Towards the end of 2014, we started getting to know the folks over at Redgate, and learning about what they’re doing around Database Lifecycle Management. It’s prompted some interesting conversations.

Redgate have been spending a lot of their time thinking about continuous delivery for databases. One thing we’ve discussed recently is how it’s not uncommon to see tensions mount when databases are added to the mix of a Continuous Delivery process. Particularly relational databases, and especially transactional relational databases. As Redgate thinks about the opinions among the DevOps community on how to manage and deploy databases, it’s prompted a few interesting questions – for example:


Why do the tensions around databases seem less pronounced in the DevOps community?


It’s About Perspective

Now I’m not 100% convinced that there are fewer tensions – I suspect they may just be different tensions. That said, there are plenty of good explanations – both cultural and technical – for why the CD & DevOps communities perceive databases differently. Perhaps some of it comes down to what they’re respectively is focused on.




Naturally, Redgate are coming to this from their established position in the RDBMS space (predominantly SQL Server, but with a twist of Oracle and MySQL as well), and so the second question that comes to light is:


How do DBAs fit into a DevOps world?

If we look at how the roles and responsibilities break down, it looks like DBAs are simply Ops specialists. Does that mean that database developers and DBAs should be blended in a single DBDevOps team? Having two flavours of DevOps team running side-by-side on the same overall application feels like a poor compromise, but are databases special enough to warrant it?

Alternatively, does this mean that the DBA dissolves into just another Ops facet of the overall team? Given the amount of specialist training and knowledge we so often see invested in DBAs, it’s hard to imagine how a team could be high-performing without some degree of specialization when it comes to managing data. This could be the straw that breaks the Full-Stack Engineers back, or it could be that you can actually get a very long way before you have to dip into the skills of a true specialist. I’m sure that, like so many questions, the answer is “It depends”

Either way, the fact that we’re even talking about Database Lifecycle Management (and uttering sentences which include both “database” and “DevOps”) suggests that software engineering as a whole is maturing at the pace of Moore’s Law, and starting to embrace the ideas that were obvious to W. Deming. Not only are we tackling these questions thoughtfully, but we’re also being more aware of the practices that have emerged in software engineering. As a result of that maturity, we’re also starting to see tools emerging which are actually fit to be used in a DevOps environment. In the case of databases, part of that is about treating them as stateful artifacts, and I particularly like Redgate’s SQL Lighthouse in this regard – a free little tool currently in beta for monitoring when database states drift.

So Many Questions

The sharp-eyed among you will have noticed that I’ve suggested several possible ideas, but avoided stating which one is the Right One™. That’s mostly because a) I don’t know yet, and b) it really does depend. And c) it’s hard to put together a solid case in a relatively short blog post!

We’ve seen a few different approaches to the problem, but what kinds of practices are you forming around your databases (and how are your DBAs being involved)?

And given that databases are tricky and stateful, what kinds of tools and processes are you using to monitor and deploy them? Open source libraries sanctified by Etsy, Netflix et al, something home-grown (and quite possibly a bit of a snowflake), or something off-the-shelf?

Moving with the times: roll with the punches

Everything is changing! It’s 2015 and the digital landscape already looks radically different compared to this time last year. 2014 brought a lot of changes to IT and the way we work online, so let’s look back on some of the things we learned and use them to make 2015 a year of change and innovation.

Tech Republic’s 10 IT lessons learned in 2014 article tells us that:

At the end of 2014, application maintenance was no different than it was at the end of 1984. IT application staffs are still spending more than half their time maintaining existing application code bases. This leaves less time for development when the demand for new apps is at unparalleled levels 

But what if CEOs stopped spending time and money firefighting growing issues within software and web applications and implemented a crucial cultural change in the way the business is run as a whole? Adopting a DevOps approach could be the way to move managing application maintenance issues into the 21st century.

Here are 3 DevOps services that could transform the way you run your business in 2015:

Application Performance Management (APM)

Customers today expect instant response times, a simple, stress-free online experience and 100% availability. Implementing APM tools and processes will help to repair root causes of application performance issues before your customers encounter them and can reduce the cost of fixing the problems later on down the line, when they’ve impacted on your other processes and services.

DevOpsGuys operate along the CALMS principals. APM tools can help to automate processes that would usually require human interaction and allows you to monitor your Metrics and your customers’ online behaviour. This means you can speed up your response time and tailor your software and strategies to meet the exact needs and expectations of your audience

Continuous Delivery

Adopting continuous delivery will streamline your software delivery process, so that you can get new features and software to market faster and more reliably. It can reduce your deployment costs, pick up problems quickly and ship code up to 50% faster – seriously it could change your life…read more

Understanding your DevOps Maturity

If you’ve decided that a DevOps approach is the way to go it’s important to understand how far along the way your business currently is. Are your testing, building and management processes carried out manually? Are you constantly battling just to keep your website running as usual? There’s a lot we can do to help you. Take our maturity assessment to find out exactly where you stand and how we can help you optimise your business in 2015


“Last week we hosted the first AnsibleFest conference outside of the US – and I’m pleased to say it was a roaring success. With 320 attendees on the day, it was our largest attendance, despite the rather grey and wintery London weather! 

The crowd made it for us – fielding great questions, and even interacting with one another during the Q&As. One person’s attendance particularly stood out – for anybody watching Twitter there was an amazingly detailed and informative stream of the day’s events being tweeted. If you didn’t catch Matt Macdonald-Wallace’s stream on the day, here’s a super summary…” -Mark Phillips, EMEA Director of Business Development, Ansible, Inc

No time to read through? Check out these direct links to the festival’s talks:

“Ansible 2 is coming, and it’s going to be awesome!” was the message at AnsibleFest London last week. Talks covered the latest Ansible 2.0 news (scheduled to be released at the end of March); using Ansible Tower to give developers the ability to stand up their development environment at the push of a button; using Ansible to manage Windows and why Rackspace – one of our partner companies – moved from Chef to Ansible for standing up OpenStack (Spoiler Alert: it’s quicker and simpler).

We did our best to share the event online by live-tweeting the whole thing from the front row and here’s our summary of what went down

The day opened with a welcome and “state of the nation” address from Saïd Ziouani, Ansible’s CEO, covering everything from the news that the London AnsibleFest was the biggest one they had ever done with 325 attendees to the impressive stat that “1 out 3 forks of the Ansible code-base results in a contribution back to the community”.

Saïd also stated that the goals for the coming year all focused around compliance, hyper-scale computing, Microsoft Windows support, container technologies such as Docker and Google Kubernetes and the ability to configure your network from Ansible plays.

The first tech-talk was by Marius Boeru from BigStep and covered the ways in which Bigstep are using Ansible to provision large-scale, big-data solutions for multiple customers.

Marius and quite a few other speakers mentioned Vagrant as a good option for standing up development environments (something that we use heavily here at DevOpsGuys) and ensuring that your Ansible plays work as expected.

One of the points that Marius made, and one that was repeated throughout the day, was that Ansible is simple, easy to extend and easy to integrate with, so it’s a very good choice to get developers up and running, as well as providing the operations teams with a flexible tool to reduce their technical debt.

The next talk was by Sebastian Göttschkes.  Titled “Vagrant and Ansible, a love story”, the talk covered the point made by Marius about using Vagrant to test things and ensure your development and operations teams can iterate quickly through the development cycle.

The key topic that most people seemed to take away from Sebastian’s talk was that “Works on my machine” is no longer an acceptable response to broken/undeployable code.

Tools such as Vagrant and Ansible allow your teams to recreate production on their local workstation and model how changes to code and configuration would affect your running environment; simply writing code and claiming it’s ready for production without full testing is not an option anymore!

Another point that Sebastian was keen to emphasise was that if you are doing any part of your configuration/installation manually then you have effectively failed at DevOps.  Everything should be automated, and, with the various Vagrant providers available for RackSpace and other cloud providers, there really isn’t an excuse: you really should “Automate All TheThings!”.

The third talk was from Ali Asad Lotia and surrounded his work with the NFL on analysis of in-game statistics picked up from individual players on the pitch. Even before he had started talking about the challenges of intermittent connectivity, changing start times and developers who had never deployed code to production, Ali had everyone engaged with the statistic that they were tracking every player on the pitch to within 6 inches of their position with a latency of less than half a second!

Ali talked about a pattern familiar to most organisations when deploying a manual process in a README becomes a shell script and then the shell script gets out of control – and why this led them to decide on using Configuration Management tooling.  The next step was to decide where to focus their efforts and (in common with a number of other speakers) they settled on Ansible because they were looking for something with a low operational cost combined with simplicity.  Complex solutions were not an option here.

A large part of the reason for focusing on Ansible for Ali and his team was that NFL stadiums have incredibly bad internet connections, meaning that running an agent-based configuration management system such as Puppet or Chef was out of the question because they couldn’t guarantee that the system would call home and update the configuration at the right time. Ansible’s “push” model gave them the ability to control the connection as best they could and only push changes when they needed to.

As with Sebastian’s talk, one of the key points of Ali’s speech was that with Ansible you can start at a simple level.  The first playbook that Ali wrote effectively copied the README file so you could read it just as easily.  The team were then able to iterate and improve the Ansible plays so quickly that it actually freed them up to spend time looking at other tasks!

Lunch followed this talk, with all the usual free T-Shirts, stickers, cuddly toys and USB keys that accompany conferences such as this as well as a good opportunity to talk to some of the sponsors about what they are up to.

After lunch Jon Hawkesworth from M Modal opened the afternoon session with a talk on managing Microsoft Windows with Ansible. Whilst the talk was good it became apparent that there was more work to do on Ansible to get it working well.  The good news was that a lot of this work is being done as part of both a major refactor in the Windows modules and the on-going development of Ansible 2.

Some of the more useful modules highlighted by Jon included the ability to install packages via win_msi, download files or packages to a box via win_get_url and the fact that you can use PowerShell to write modules for Ansible, meaning that Windows developers and operations teams won’t have to learn Python.

The second talk of the afternoon was by Walter Bentley from Rackspace and centred on why he chose to take Rackspace away from Chef and towards Ansible for their OpenStack deployments.

Walter focused on the simplicity and ease that Ansible provides, as well as the fact that it is written in Python; they have a large team who can develop it further if required. Once again, not requiring an agent to run on each host also came up as a positive thing, because all you need is Python and an SSH connection – something that almost all modern Linux distributions have.

One of the key reasons that Walter wanted to automate the installation of Rackspace Private Cloud was to ensure that when he did a demonstration for a client, it took less than five hours reading 62 pages of documentation. Using Ansible and Vagrant, he and his team have managed to get the deploy fully automated and far more quickly: they now deploy at least ten Rackspace Private Cloud installations using this method each week

Walter pointed out that: “the great thing about Ansible is that you can run it over and over again and it only changes things that are different to their desired state”.  This is a vital part of any configuration management tooling as it avoids unnecessary service restarts or re-configuration of correct system settings.

After a short break, we were back with James Cammarata from Ansible talking about Ansible V2 and what we could expect to see there.  It was interesting to learn that the primary reason behind version 2 was to clear up a lot of technical debt in the code base, and also to see some of the new features.

The features that stood out in Ansible 2 were the ability to have “try/except” blocks in your plays, allowing for recovery from failures in plays and better error messages.

The final two talks of the day were about Ansible Internals and Ansible Tower. Ansible Internals deserves a blog post of its own and we hope to bring that to you in the next couple of weeks so we’ll leave that for now apart from to say that if you’re not looking at extending Ansible then you should – modules, plugins and filters are incredibly powerful and will help you.

Ansible Tower is the commercial offering from Ansible that provides auditing and role-based access control (RBAC) for your infrastructure. Mark Phillips explained how you can point Ansible Tower at your existing Git repositories and then assign permissions to various users to allow them to interact with your infrastructure in different ways.

One example given was that the Operations Team might need to be able to stand up any type of system, whereas QA might only require the ability to standup a dev/qa environment instead

Ansible Tower also provides dashboards that can be used as part of a compliance program and give you an overview of what has failed and where, however the main part of Mark’s message was that Ansible Tower allows you to let “non-technical” people provision environments for their own use without the need for support tickets!

All in all the conference was a great day out.  We learned loads, met up with friends and we will definitely be back!

Ghostbusters! Using monitoring to drive system vulnerability patching

Our GHOSTbusting DevOps Engineer Matt Macdonald-Wallace  (@ProfFalken) on the recent Linux security scare. He talks us through how an innovative approach to DevOps can solve these problems and prevent them from happening in the future.

The recent GHOST vulnerability struck fear into the heart of a lot of Linux systems administrators recently when it was revealed that for a couple of years one of the core libraries in the Linux operating system (libc) had been vulnerable to attack.

The vulnerability revealed that you could easily take control of a Linux system attached to the internet if you could send a well-crafted message to a webserver or email server running on that host.

At DevOpsGuys we’re big fans of test-driven development (TDD) and we’re always looking for ways to share knowledge between Dev and Ops so when this gave us an opportunity to try out a new way of managing system upgrades across a large number of customer we leapt at the chance!

The plan was to follow TDD methods to create a monitoring check that would fail (the test) and then patch systems until it passed (the “development”).

The monitoring scripts were written as a “minimum-viable” check for those Linux distributions that we use in house and then added to our monitoring system.


We use for monitoring and this meant that rolling out our scripts to all our hosts was easy – write the script, add it as a dataloop plugin, add that plugin to the relevant tag and wait for Dataloop to deploy the check.  In total it took less than three minutes to roll out our check to all of our systems and have it reporting back.

The next step was to create a custom dashboard to see all the hosts that had the plugin enabled.  Again, this was trivial to do using Dataloop’s drag and drop dashboard creation tool and within minutes we had a complete overview of all the servers under our control that were being monitored for GHOST – many of which were showing as “critical”, meaning that they needed the update applying.


The great thing about approaching the problem this way was that we were able to see if any hosts were already secure (anything running Ubuntu 14.10 or later was not affected by this patch) and that the check worked correctly.

We then ran our update procedures as normal, however we were able to prove that all of our servers had received the update by watching the dashboard turn green as each server updated and the check was executed again.


We’re going to keep the checks in place so that we know in future if any of our servers for some reason revert to a “bad” version and from now on, any major vulnerability like this will be dealt with in the same way.

Special thanks to Steven Acreman of for helping us with assisting in modifications to the scripts and ensuring the data showed up on the dashboards in the way we required it

If you want to follow our example and check your systems in the same way, then we’ve open-sourced the checks and you can download them from our Public Checks Git repository.