Category Archives: This week in DevOps

This week in DevOps #4

State of the Union

This week saw the publication of the “2013 State of DevOps” report from PuppetLabs. It’s a pretty in-depth survey of over 4,000 IT Operations professionals and Developers, which highlights an understanding of the state of DevOps today.

Key Findings

  • 63% of organizations implementing DevOps saw improved quality of software deployment, with 50% seeing fewer failures in production – Software bugs in production have always been expensive in-comparison to early stages of the SDLC, but today bugs are becoming increasingly more expensive to fix and their impact greater with the adoption of Cloud. Adopting a DevOps model, where Development and Operations combine to take greater ownership of the software delivery process and software quality is already starting to show benefits to organisations who have adopted it. It’s also important to note, that the research shows organizations implementing DevOps are likely to be able to fix production problems 12 times faster than organizations that don’t.
  • Organizations implementing DevOps are 5 times more likely to be high-performing businesses – The report defines high-performing organizations as those with the capability to deploy critical applications quickly without disrupting service. This is further categorized by organizations that deploy frequently with less change lead time and failure and have a lower mean-time to recover. Adopting a DevOps model was shown to improve all of these characteristics significantly and showed continued improvement over time.
  • 75% increase in demand for DevOps skills in the last year - The report found that DevOps skills were broadly in 4 categories; Coding/Scripting, People Skills, Process Re-Engineering and specific experience with “DevOps” tools. e.g. infrastructure automation. The overall head-line figure is a good one. More companies are looking for more people with skills to help them implement DevOps.  However, the report highlights the growing debasement of the term.  “DevOps” as a skill adverts on LinkedIn increased by 50 percent in the same time period. Recruiters are certainly jumping on the buzz word band wagon. We asked Gene Kim for his thoughts on this, and were surprised by his response;
  • “I’m ok w/ DevOps in job postings & LinkedIn profiles. It’s a great signal to match buyers/sellers of a needed skill” – @RealGeneKim

  • 48% indicated the value of DevOps wasn’t understood outside of their group – This reason was believed to be the biggest difficulty in implementing DevOps. Cultural factors are still seen as the biggest barriers to DevOps adoption. The report identifies improved communication as one of the best methods for overcoming DevOps adoption barriers.  We believe that there are 4 further key elements that can help to break down the barriers;  Build Trust, Create Champions, Build Confidence and Celebrate Success.

What we found this week

  • DevOps for Windows Azure – a comprehensive set of resources for DevOps teams wanting to deploy and support applications in Windows Azure.
  • DevOps: What’s All the Hype Really About? – A concise 60 second summary (unless your a slow reader) of DevOps with some interesting points for discussion raised. Two questions to ask yourself, So Everyone LOVES DevOps Right? and is DevOps Worth Pushing in My Organization?
  • DevOps and Cloud Management –  RightScale’s Cloud Evangelist Uri Budnik and Blackhawk Network’s Arindam Mukherjee give a real-world use case on using DevOps and RightScale to cut the time it takes to provision a spec environment by over 80%.

Tweets of the Week

This week in DevOps #3

Hey, it’s that time again and this week I start with an apology as we missed a week due to the awesome DevOpsDays London and a not so awesome bout of Tonsillitis.

DevOpsDays London

DevOpsDays London – easily the best conference/unconference I’ve ever attended. Great people, great speakers, OpenSpaces was awesome, food was good and to top it all off – free beer. Sadly, I missed the Saturday due to illness (not caused by the free beer, or the Wales Vs England rugby game).

Highlights;

  • @niekbartho gave a solid opening presentation, monitoring his heart rate as he went. Some good real world advice and a story from the trenches is always a nice touch. I felt the story was a little more focused on continuous delivery, rather than DevOps. Niek did clarify the term “Snowflake Server” very nicely as being “a server having configuration as unique as a snowflake, and no one has a clue how it was built!” Thanks Niek.
  • @johnC_bristol  gave us an insight into key cultural aspects of DevOps and how they monitored them at Nokia. It was a really interesting insight into how culture affects DevOps with some real information to backup theory. John completes a study with involvement of over 50 employees and presented some interesting findings.
  • @DavidMytton gave a great insight into bootstrapping a startup. Some good advice on life hacks relating to Startups relating to mobile usage, travelling, etc. His advice was wide ranging, but our favourite item was on task automation. “Do it once, Learn, automate!” — Thanks David. The ServerDensity team have produced some content DevOps Dayers which is worth a read.
  • In the final presentation before the ignite sessions, Deri Jones enlightened us on using metrics to calculate lost sales. His content focused on ensuring that all the great metrics captured by DevOps should be translated back to business measurements.
  • The ignite sessions brought some great fast paced presentations. I really enjoyed @simonvc on “How schemaless databases fit into the continuous delivery world”. His thoughts;

My final thoughts on DevOpsDays; awesome people, awesome event format but even at a DevOps event, it was surprising how many different ways people interpreted the terms DevOps. Here a just a few;

  • DevOps  … is a movement, a philosophy, a way of thinking.
  • DevOps  … is a person who can perform both Dev and Ops roles.
  • DevOps … means cross skilling people.
  • DevOps … is continuous delivery.
  • DevOps … is a team of developers and operation staff.
  • DevOps …is a culture movement.
  • DevOps … is monitoring.

I have my views on what DevOps is and more importantly is not. It seems that there is still some teaching to be done!

Thanks to all those at DevOpsLondon 2013. It was great to meet you.

What we found this week

  • Citrix outfits NetScaler with troubleshooting tools to help sysadmins troubleshoot the cause of poor application performance from multiple data centres, cloud services and networks. NetScaler can emit performance data in the AppFlow format, which third-party diagnosis tools such as Splunk.
  • The benefits of DevOps are numerous. First and foremost, there is more collaboration and trust within an organization. Where are you on the timeline? Where do you want to be?

Tweets of the Week

  • @patrickdebois: Mgmt wants us to replace shell scripts to become devops, /me Do they work now? Yes /me I suggest you find your real bottleneck.
  • @rposbo: Loving that there’s no DevOps Manifesto; just get everyone on the same goal, automate what you can, don’t be evil!
  • @withneedle: Pro tip: Changing everyone’s title to “#devops engineer” will not magically make it so.
  • @DevOpsBorat: In startup we are only use technology if is cover in blog of expert devops on benchmark is run on own laptop.
  • Interesting Idea. The Twelve Factor App is a methodology for building software-as-a-service apps  that can be applied to apps written in any programming language, and which use any combination of backing services.

This week in DevOps #2

#1: The Rise of a New Kind of Administrator

Luke can see the growing trend of admin empowerment within customers and partners from his view at PuppetLabs. His outlook on tools that support admins in complex environments has always been clear, creating tools which allow “you to focus more on the business value of your work and spend a less of your brainpower on implementation details.”

This article looks at the changing role of administrators within organisations as “tools better enable people to build teams to focus on their customer instead of their technology.”

Read more: http://insights.wired.com/profiles/blogs/the-rise-of-a-new-kind-of-administrator

#2: 8 Lessons in Deployment Tooling Lessons Learned

Another great article on learning’s from the UrbanCode guys. This one gives us a history lesson on AnthillPro and what the team learned from creating and using the product. In summary the 8 lessons are;

  1. Builds are about quality and checking quality.
  2. Deployment is a serious challenge.
  3. Developers know jack about audit and security.
  4. Deployment starts with production.
  5. Deployment requires co-ordination.
  6. Deployment starts with high-availability and scalability.
  7. “Release” is more than just deployment.
  8. Multiple integrated tools are not optional.

Read more:  http://architects.dzone.com/articles/8-lessons-deployment-tooling

#3: Playing Pitfall with DevOps and CMDB’s

One paragraph in and Sam delivers a stark warning that  “there are some underlying shortcomings; some so major they might render DevOps redundant in the near future.”

In summary, he outlines the following challenges;

  1. DevOps goes against organizational culture; which advocates for the separation of responsibilities.
  2. Departure from IT Service Management Process
  3. Configuration Management Database redundancy
  4. Inadequate Automation

And so it is, that Sam leaves us with one final point to ponder “Be prepared to say goodbye to DevOps and CMDB, as they are sure to be abandoned in favor of new and improved counterparts in the face of a future IT revolution.”

In response, we ask you to consider Gene Kim’s 3 ways as the underpinning to the DevOps philosophy;

  1. The First Way emphasizes the performance of the entire system – the value stream.
  2. The Second Way is about shorting and amplifying feedback loops.
  3. The Third Way is about creating a culture that fosters continual learning and understanding.

If these truly are the essence of DevOps, are they principles that can ever be abandoned?

Read more: https://www.scriptrock.com/articles/playing-pitfall-devops-cmdbs/?utm_source=twitterfeed&utm_medium=twitter

#4: DevOps Complete Certification Kit

Only 18 Hours and you could be certified! Sounds too good to be true, well it is. As Patrick Debois commented, “they are not getting it!” The geniuses who put this material have really missed the point. We’d love to be certified “DevOps” guys, but instead we’ve focused on recognizing what the practice requires to be implemented. Don’t get us wrong, education on the background to Agile process and the challenges facing Development and Operations teams is vital, but don’t think that 18 hours of study are going to equip you with all you need to become a “DevOps” engineer (a term in itself which is entirely flawed). Remember this…DevOps is a philosophy, not a qualification.

Read more: http://archive.aweber.com/theartofservice/JxFjA/h/DevOps_Complete.htm

#5: Agile Alliance has a DevOps Track

DevOps seeks to address the challenges organizations face when applying change to high velocity environments. Speedy software development handing work over to risk-averse operations need not result in bottlenecks of pessimism. The rise of PaaS, SaaS, IaaS combined with improved understanding of systemic dependencies and business risk is changing how collaboration, automation and operations work in today’s modern enterprises. Mental models are shifting from protection mode to enabling-change mode. The track is going to cover the processes, tools and culture shifts shaping the DevOps movement. We look forward to seeing the output from this. Get submitting!

Read more: http://agile2013.agilealliance.org/program/tracks/devops/

#6: Tweet of the Week

Jesse Keating ‏@iamjkeating

#DevOps lesson: things go well when you’re environment is in an expected state. Not so well when suffering from technical debt. #Fixit

This week in DevOps #1

TWIDO

We hate acronyms, but this one’s kinda catchy. This Week in DevOps (TWIDO) [[[ yeah, yeah, we promise not to use it again]]], is going to be a Friday round up of stuff we found interesting this week.

We’d love you to contribute, if you have articles, research, tweets on any other stuff you’d like to share, please get involved.

#1:  Q&A From Building a DevOps Team That Isn’t Evil

Q: How have auditors been ok with a devops model where devs have access to prod? e.g., continuous testing, automation, version control, logging, etc?  is one single dev allowed to push a code chg thru the entire process?

Few public companies allow a developer to go from change to production, or access production databases directly. There may still be roles with responsibility for building new features and others that operate on production. The goal is enhance the flow of knowledge between the people in those roles, and for there to be some joint responsibility. For instance, logs may be exposed to developers without giving access to the production box. I think the team at Stackify is trying to make a business around exactly that.

http://architects.dzone.com/articles/qa-building-devops-team-isnt 

Commentary

DevOps does not equal developers managing production. It’s aimed at promoting understand and collaboration between groups.  Thanks to Eric Minick from DevOps Zone and UrbanCode for supporting this.

#2: IT stability and business innovation are not enemies.

Q: DevOps promises a more responsive, more collaborative IT department that can realize business ideas faster. So what is holding back its widespread adoption? What’s the challenge or downside?

DE: There was a movie called “Charlie Wilson’s War” that had a great line between Tom Hanks, playing a U.S. Congressman, and Philip Seymour Hoffman, playing a CIA agent. Hoffman asks, “Why is Congress saying one thing and doing nothing?” Hanks replies, “Well, tradition mostly.”

All jokes aside, tradition is a powerful thing and hard to break. Tradition, or “what we’ve always done,” in IT is no different. There was a thread on Slashdot just this past month that asked whether developers should be allowed to deploy their own applications. You should have seen the outcry. The sheer number of commenters who shot down the idea as pure heresy was shocking. And the richest part of all of their denunciation was that the mob said over and over that the idea would “never work at a real company with real revenue at stake.” I thoroughly enjoyed sending that to John Allspaw, who runs all of technology at Etsy, and Jesse Robbins, who was in charge of risk and disaster planning for operations at Amazon. Etsy does over $600 million of transactions per year, and Amazon does about $50 billion in revenue. In both companies, developers are the ones who deploy and own the uptime for their own code. John’s reaction to the thread was a simple yet priceless one: “OMG.”

http://dev2ops.org/2013/02/

Commentary

Empowering development, operations and even QA teams to take ownership of code deployment is crucial for DevOps teams. It ensures that developers work with operations to understand the impact of change and take accountability/ownership for the “uptime of thier own code”. It also ensures that operations understand what changes are coming and how best to support the change. Uptime is a shared responsibility  with the environment the code executes within and the code that is executing both factors that must be managed correctly. 

#3 – StartOps: Growing an ops team from 1 founder

Bootstrapped startups don’t have the luxury of a full team of ops engineers available to respond to issues 24/7, so how can you survive on your own? This talk will tell the story of how to run your infrastructure as a single founder through to growing that into a team of on call engineers. It will include some interesting war stories as well as tips and suggestions for how to run ops at a startup.

http://devopsdays.org/events/2013-london/

Commentary

We are really looking forward to this track at DevOps Days London. DevOpsGuys are attending. Would be great to meet some of you.

#4: Engineering running the production environment

http://devopsreactions.tumblr.com/post/43882100705/engineering-running-the-production-environment

Commentary

Both funny and awesome! Nuff said!

#5:  Top 10 Practices for Effective DevOps

Practice 1: Active Stakeholder Participation

A fundamental philosophy of DevOps is that developers, operations staff, and support people must work closely together on a regular basis. An implication is that they must see one other as important stakeholders and actively seek to work together. A common practice within the agile community is “onsite customer,” adopted from Extreme Programming (XP), which motivates agile developers to work closely with the business. Disciplined agilists take this one step further with the practice of active stakeholder participation, which says that developers should work closely with all of their stakeholders, including operations and support staff–not just business stakeholders. This is a two-way street: Operations and support staff must also be willing to work closely with developers.

Practice 6: Integrated Deployment Planning

From the point of view of development teams, deployment planning has always required interaction with an organization’s operations staff; in some cases, via liaison specialists within operations typically called release engineers. Experienced development teams will do such planning continuously throughout construction with active stakeholder participation from development, operations, and support groups. When you adopt a DevOps strategy, you quickly realize the need to take a cross-team approach to deployment planning due to the need for operations staff to work with all of your development teams. This isn’t news to operations staff, but it can be a surprise to development teams accustomed to working in their own siloed environments. If your team is not doing this already, you will need to start vying for release slots in the overall organizational deployment schedule. Furthermore, to support continuous deployment, release engineers will need to increase the number of release slots available to agile teams that are disciplined enough to continuously and consistently meet the quality requirements for release.

Practice 7: Continuous Deployment

Continuous deployment extends the practice of continuous integration. With continuous deployment, when your integration is successful in one sandbox, your changes are automatically promoted to the next sandbox, and integration is automatically started there. This automatic promotion continues until the point where any changes must be verified by a person, typically at the transition point between development and operations.

Continuous deployment enables development teams to reduce the time between a new feature being identified and being deployed into production. It enables the business to be more responsive. However, continuous deployment increases operational risk by increasing the potential for defects to be introduced into production when development teams aren’t sufficiently disciplined. Successful continuous deployment in an enterprise environment requires all the practices described earlier.

Commentary

Overall we found this article rather in conflict with itself. It’s interesting how the author views Operations and Development as separate stakeholders within a process. The principles of DevOps are to break this view and to combine Development and Operations as a single entity and in this context they should be considered a single StakeHolder.

We fully agree that Operations should be managers of production with development participation, as Developers should be mangers of development with Operations participation, but on deployment we differ with the author. It’s easy in our view. If operations are a bottleneck to deployment, expand the role of “release engineers” to encapsulate the entire team. Anyone should be able to take accountability for and action deployment if the correct principles and practices have been applied.

The author also battles with Continuous Delivery principles. “release engineers will need to increase the number of release slots available to agile teams”. If you adopt a continuous delivery process your actually going to reduce the number of release slots, to one. One continuous release slot. Hence Continuous Delivery.

http://www.drdobbs.com/architecture-and-design/top-10-practices-for-effective-devops/240149363

#5: Tweet of the Week

Every #ITmanager, no matter how strong he is, lies to himself about his #DevOps initiative. I will find your lie. I will break you