Tag Archives: Continuous Delivery

DevOpsGuys Launch Workshops & Training

DevOpsGuys, a leading provider of DevOps services for Enterprise clients, is pleased to announce the launch of their new DevOps Education platform which provides DevOps, Continuous Delivery and Agile training courses and workshop, enabling organisations to build the knowledge they need to succeed with DevOps transformation.

As part of DevOpsGuys on-going commitment to impart knowledge and expertise through the company’s training programmes, and to support organisations developing internal capabilities themselves, DevOpsGuys has built a series of public and onsite training solutions where people of all levels can experience DevOps.

“Enterprise IT has a critical need to identify, select, learn and implement new platforms.  Being a digital enterprise is so critical to the future of every company that knowledge must be built and managed internally”, said James Betteley, Head of DevOps Education at DevOpsGuys.

DevOpsGuys courses are delivered through a combination of interactive workshops and practical, engineering led classes, allowing people to experience DevOps process and practices or get hands-on with Automation, or understand how to design software to facilitate Continuous Delivery.

“We’ve been listening to what people in our industry have been asking us for and are excited to launch our DevOps from the Trenches workshop”, announced Betteley. “This workshop is based on our DevOps experiences working with dozens of leading organisations across Europe – from small start-ups to large enterprises.”

The course can be booked online using EventBrite at http://devopsguys.eventbrite.com

DevOpsGuys offer a range of DevOps and Agile training courses and workshops, which can be found online at https://www.devopsguys.com/devops_training

About DevOpsGuys

DevOpsGuys deliver DevOps-as-a-Foundation to underpin digital transformation, achieving greater customer enablement and unlocking new revenue streams. Our solutions make enterprise organisations agile, scalable and responsive to rapidly changing business requirements.

Our platform is built on a foundation of education and DevOpsGuys offer a series of DevOps, Continuous Delivery and Agile training courses and workshop, enabling you to build the knowledge you need to succeed.

Our courses are delivered through a combination of interactive workshops and practical, engineering led classes, where people of all levels can experience DevOps, get hands-on with Automation, or understand how to design software to facilitate Continuous Delivery.

#DevOps = #ContinuousDelivery + #ContinuousObsolescence?

During my talk at QCon last week we had started an interesting debate about whether DevOps was a subset of Continuous Delivery or vice versa.

We were lucky enough to have Dave Farley in the audience and it was clear from his definition of DevOps versus Continuous Delivery that he felt that CD was the superset and DevOps a component of that set… but he admitted that he if he was writing the Continuous Delivery Book again in 2015 would change the definition of “finished” to embrace the full product lifecycle from inception to retirement.

This aligns nicely with his co-author Jez Humble’s thoughts on being “Product-Centric” that we discussed in an earlier blog post.

“Finished” should mean “no longer delivering value to the client and hence retired from Production”.

DaveDefinitionOfDevOps

DaveOnCD

That was the we joined a new term (live on stage as captured by Chris Massey’s tweet below) – “Continuous Obsolescence”

 MasseyTweet

So what is #ContinuousObsolescence?

Continuous Obsolescence is the process of continuously grooming your product/feature portfolio and asking yourself “is this product/feature still delivering value to the customer?” because if it isn’t then it’s “muda” (waste), it’s costing you money and should be retired.

Continuous Obsolescence is the process of continuously grooming your product/feature portfolio and asking yourself “is this product/feature still delivering value to the customer?” because if it isn’t then it’s “muda” (waste), it’s costing you money and should be retired.

Every product/feature that’s not delivering value costs you by:

  • Wasted resources – every build, every release, you’re compiling and releasing code that doesn’t need to be shipped, and consuming server resources once deployed.
  • Overhead of Regression testing – it still needs to be tested every release
  • Unnecessary complexity – every product/feature you don’t need just adds complexity to your environment which complicates both the software development process and the production support process. Complex systems are harder to change, harder to debug and harder to support. If it’s not needed, simplify things and get rid of it!

I’d argue that DevOps could even be defined as
“#DevOps = #ContinuousDelivery + #ContinuousObsolescence”

If you’re doing DevOps and you’re a believer in the CALMS model then you should be practising “Continuous Obsolescence”… in fact I’d argue that DevOps could even be defined as “#DevOps = #ContinuousDelivery + #ContinuousObsolescence” – what do you think?

Send us your comments or join the debate on Twitter @DevOpsguys

-TheOpsMgr

Dzone Guide to Continuous Delivery from @AgileSteveSmith & @Matthewpskelton

Our good friends and organisers of PipelineConf @AgileSteveSmith and @Matthewpskelton have contributed to the DZone Guide to Continuous Delivery.

The guide covers:

  1. SUMMARY & HIGHLIGHTS
  2. KEY RESEARCH FINDINGS
  3. INTRODUCING: CONTINUOUS DELIVERY BY STEVE SMITH
  4. CONTINUOUS DELIVERY PITFALLS BY J. PAUL REED
  5. THE CONTINUOUS DELIVERY TOOLCHAIN BY MATTHEW SKELTON
  6. CONTINUOUS DELIVERY VISUALIZED
  7. INFRASTRUCTURE AS CODE: WHEN AUTOMATION ISN’T ENOUGH BY MITCH PRONSCHINSKE
  8. CONTINUOUS DELIVERY MATURITY CHECKLIST

Dzone 2014 CD guideWell worth a read!

 

Why companies are investing in DevOps and Continuous Delivery

A thought-provoking infographic – with some interesting data points – shows how companies are reaping real rewards from investing in agile software delivery processes. Check out the graphic – from Zend – for more on how DevOps and Continuous Delivery are bridging the speed and innovation gap between business demand and IT.

Continuous Delivery Infographic by Zend Technologies.

Continuous Delivery Infographic

Upgrading to Octopus 2.0

On 20th December 2013, Paul Stovell and the Octopus Team released Octopus Deploy 2.0. We’ve had a chance to play with the product for sometime, and it’s a great next version.

Here’s 5 great reasons to upgrade;

  1. Configurable Dashboards
  2. Sensitive variables
  3. Polling Tentacles (Pull or Push)
  4. IIS website and application pool configuration
  5. Rolling deployments

So let’s take a look at how the upgrade process works.

Step 1 – Backup your existing database

Things to check before you upgrade;

  1. Your administrator username and password (especially, if you are not using Active Directory)
  2. Before attempting to migrate, make sure that you don’t have any projects, environments, or machines with duplicated names (this is no longer allowed in Octopus 2.0, and the migration wizard will report an error if it finds duplicates).
  3. IIS Configuration. Are you using ARR or rewrite rules of any kind.
  4. SMTP Server Settings (these are not kept in upgrade)

The first step is to ensure that you have a  recent database backup that you can restore in case anything goes wrong.

1 - Database Backup

Confirm the location of your backup file. You’ll need this file at a later stage in the upgrade process (and if anything goes wrong)

2-Database Backup Location

Step 2 – Install Octopus 2.0

Download the latest installer MSI from the Octopus Website at http://octopusdeploy.com/downloads

Next follow the installer instructions and the Octopus Manager process. This process is really straight forward, and if you’ve installed Octopus before you’ll be familiar with most of the process. There’s a great video here on how to install Octopus http://docs.octopusdeploy.com/display/OD/Installing+Octopus

Step 3 – Import Previous Octopus Database

Once you’ve installed Octopus 2.0, you’ll most likely want to import your old v1.6 database with all your existing configuration. To do this, select the “import from Octopus 1.6” option and choose the database back-up from the process you followed in step 1 and follow the instructions.

3 - Import Octopus Database

Step 4 – Post Upgrade Steps (server)

When you have completed the installation it’s essential to task a copy of the master key for backups. Instructions on how to backup the key can be found here http://docs.octopusdeploy.com/display/OD/Security+and+encryption

Once you have the server installed there’s a couple of items you’ll need to configure again.

  1. SMTP Server Settings are not kept, so these will need to be re-entered.

Step 5 – Upgrading Tentacles

If you are upgrading from version 1.6 – Octopus 2.0 server can no longer communicate with Tentacle 1.6. So in addition to upgrading Octopus, you’ll also need to upgrade any Tentacles manually. There are two ways to achieve this;

1. Download and Install the new Octopus Deploy Tentacle MSI on each target server

2. Use  a PowerShell script to download the latest Tentacle MSI, install it, import the X.509 certificate used for Tentacle 1.6, and configure it in listening mode. The powershell script is as follows;

function Upgrade-Tentacle
{ 
  Write-Output "Beginning Tentacle installation"
  Write-Output "Downloading Octopus Tentacle MSI..."
  $downloader = new-object System.Net.WebClient
  $downloader.DownloadFile("http://download.octopusdeploy.com/octopus/Octopus.Tentacle.2.0.5.933.msi", "Tentacle.msi")
  
  Write-Output "Installing MSI"
  $msiExitCode = (Start-Process -FilePath "msiexec.exe" -ArgumentList "/i Tentacle.msi /quiet" -Wait -Passthru).ExitCode
  Write-Output "Tentacle MSI installer returned exit code $msiExitCode"
  if ($msiExitCode -ne 0) {
    throw "Installation aborted"
  }
  
  Write-Output "Configuring and registering Tentacle"
   
  cd "${env:ProgramFiles(x86)}\Octopus Tentacle 2.0\Agent"
  Write-Output "Stopping the 1.0 Tentacle"
  
  & .\tentacle.exe create-instance --instance "Tentacle" --config "${env:SystemDrive}\Octopus\Tentacle\Tentacle.config" --console
  & .\tentacle.exe configure --instance "Tentacle" --home "${env:SystemDrive}\Octopus" --console
  & .\tentacle.exe configure --instance "Tentacle" --app "${env:SystemDrive}\Octopus\Applications" --console
  & .\tentacle.exe configure --instance "Tentacle" --port "10933" --console
  & .\tentacle.exe import-certificate --instance "Tentacle" --from-registry  --console
  Write-Output "Stopping the 1.0 Tentacle"
  Stop-Service "Tentacle"
  Write-Output "Starting the 2.0 Tentacle"
  & .\tentacle.exe service --instance "Tentacle" --install --start --console
  
  Write-Output "Tentacle commands complete"
}
  
Upgrade-Tentacle

Step 6 – Upgrade TeamCity Integration

The JetBrains TeamCity Plugin has also been released and requires an upgrade. To do this

  1. Download the plugin from the Octopus Website http://octopusdeploy.com/downloads.
  2. Shutdown the TeamCity server.
  3. Copy the zip archive (Octopus.TeamCity.zip) with the plugin into the <TeamCity Data Directory>/plugins directory.
  4. Start the TeamCity server: the plugin files will be unpacked and processed automatically. The plugin will be available in the Plugins List in the Administration area.

Please Note: The API Key’s for the users will have changed and will need to be re-entered into the Octopus Deploy Runner Configuration within the Build Step

Summary

We think that overall the Octopus Team have done a great job making the upgrade process easy. There’s still a few little issues to iron out, but overall the process is robust and simple. It’s a shame that automatic upgrade of the tentacles couldn’t have been supported as this may cause pain for customers with a large server install base, but Powershell automation makes for a great workaround.

As we learn more about Octopus 2.0, we’ll continue to let you know what we find. Good Luck with Upgrading — it’s definitely worth it!

DevOpsGuys use Octopus Deploy to deliver automated deployment solutions – ensuring frequent, low-risk software releases into multiple environments. Our engineers have deployed thousands of software releases using Octopus Deploy and we can help you do the same. Contact us today – we’d be more than happy to help.

The Top Ten DevOps “Operational Requirements”

Join us for “The Top 10 DevOps Operational Requirements” –  http://www.brighttalk.com/webcast/534/98059 via @BrightTALK

One of the key tenets in DevOps is to involve the Operations teams in the full software development life cycle (SDLC) and in particular to ensure that “operational requirements” (“OR’s”, formerly known as “non-functional requirements”, “NFR’s”) are incorporated into the design&build phases.

In order to make your life easier the DevOpsGuys have scoured the internet to compile this list of the Top Ten DevOps Operational Requirements (ok, it was really just chatting with some of the guys down the pub BUT we’ve been doing this a long time and we’re pretty sure that if you deliver on these your Ops people will be very happy indeed!).

#10 – Instrumentation

Would you drive a car with a blacked out windscreen and no speedo? No, didn’t think so, but often Operations are expected to run applications in Production in pretty much the same fashion.

Instrumenting your application with metrics and performance counters gives the Operations people a way to know what’s happening before the application drives off a cliff.

Some basic counters include things like “transactions per second” (useful for capacity) and “transaction time” (useful for performance).

#9 – Keep track of the Dependencies!

“Oh yeah, I forgot to mention that it needs [dependency XYZ] installed first” or “Yes, the system relies on [some 3rd party web service] can you just open up firewall port 666 right away”.

Look, we all understand that modern web apps rely on lots of 3rd party controls and web services – why re-invent the wheel if someone’s already done it, right? But please keep track of the dependencies and make sure that they are clearly documented (and ideally checked into source control along with your code where possible). Nothing derails live deployments like some dependency that wasn’t documented and that has to be installed/configured/whatever at the last moment. It’s a recipe for disaster.

#8 – Code defensively & degrade gracefully

Related to #9 above – don’t always assume the dependencies are present, particularly when dealing with network resources like databases or web services and even more so in Cloud environments where entire servers are known to vanish in the blink of an Amazon’s eye.

Make sure the system copes with missing dependences, logs the error and degrades gracefully should the situation arise!

#7 – Backward/Forward Compatibility

Existing code base with new database schema or stored procedure?

New code base with existing database schema or stored procedures?

Either way, forwards or backwards, it should work just fine because if it doesn’t you introduce “chicken and the egg” dependencies. What this mean for Operations is that we have to take one part of the system offline in order to upgrade the other part… and that can mean an impact on our customers and probably reams of paperwork to get it all approved.

#6 – Configurability

I once worked on a system where the database connection string was stored in a compiled resource DLL.

Every time we wanted to make a change to that connection string we had to get a developer to compile that DLL and then we had to deploy it… as opposed to simply just editing a text configuration file and re-starting the service. It was, quite frankly, a PITA.

Where possible avoid hard-coding values into the code;  they should be in external configuration files that you load (and cache) at system initialisation. This is particularly important as we move the application between environments (Dev, Test, Staging etc) and need to configure the application for each environment.

That said, I’ve seen systems that had literally thousands of configuration options and settings, most of which weren’t documented and certainly were rarely, if ever, changed. An “overly configurable” system can also create a support nightmare as tracking down which one of those settings has been misconfigured can be extremely painful!

#5 – “Feature Flags”

A special case of configurability that deserves its own rule – “feature flags”.

We freakin’ love feature flags.

Why?

Because they give us a lot of control over how the application works that we can use to (1) easily back out something that isn’t working without having to roll-back the entire code base and (2) we can use it to help control performance and scalability.

#4 – Horizontal Scalability (for all tiers).

We all want the Product to be a success with customers BUT we don’t want to waste money by over-provisioning the infrastructure upfront (we also want to be able to scale up/down if we have a spiky traffic profile).

For that we need the application to support “horizontal scalability” and for that we need you to think about this when designing the application.

3 quick “For Examples”:

  1. Don’t tie user/session state to a particular web/application server (use a shared session state mechanism).
  2. Support for read-only replicas of the database (e.g. a separate connection string for “read” versus “write”)
  3. Support for multi-master or peer-to-peer replication (to avoid a bottleneck on a single “master” server if the application is likely to scale beyond a reasonable server specification). Think very carefully about how the data could be partitioned across servers, use of IDENTITY/@Auto_Increment columns etc.

#3 –Automation and “scriptability”

One of the key tenets in the CALMS DevOps Model is A for Automation (Culture-Automation-Lean-Metrics-Sharing if you want to know the others).

We want to automate the release process as much as possible, for example by packaging the application into versionable released or the “infrastructure-as-code” approach using tools like Puppet & Chef for the underlying “hardware”.

But this means that things need to be scriptable!

I can remember being reduced to using keystroke macros to automate the (GUI) installer of a 3rd party dependency that didn’t have any support for silent/unattended installation. It was a painful experience and a fragile solution.

When designing the solution (and choosing your dependencies) constantly ask yourself the question “Can these easily be automated for installation and configuration”? Bonus points if you can, in very large scale environments (1,000 of servers) build in “auto-discovery” mechanisms where servers automatically get assigned roles, service auto-discovery (e.g. http://curator.apache.org/curator-x-discovery/index.htm) etc.

#2 – Robust Regression Test suite

Another think we love, almost as much as “feature flags” is a decent set of regression test scripts that we can run “on-demand” to help check/verify/validate everything is running correctly in Production.

We understand that maintaining automated test scripts can be onerous and painful BUT automated testing is vital to an automation strategy – we need to be able to verify that an application has been deployed correctly, either as part of a software release or “scaling out” onto new servers, in a way that doesn’t involve laborious manual testing. Manual testing doesn’t scale!

The ideal test suite will exercise all the key parts of the application and provide helpful diagnostic messaging if something isn’t working correctly. We can combine this with the instrumentation (remember #10 above), synthetic monitoring, Application Performance Management (APM) tools (e.g. AppDynamics), infrastructure monitoring (e.g. SolarWinds) etc to create a comprehensive alerting and monitoring suite for the whole system. The goal is to ensure that we know something is wrong before the customer!

#1 – Documentation

Contrary to popular belief we (Operations people) are quite happy to RTFM.

All we ask is that you WTFM (that’s W as in WRITE!) J

Ideally we’d collaborate on the product-centric documentation using a Wiki platform like Atlassian Confluence as we think that this gives everyone the easiest and best way to create – and maintain – documentation that’s relevant to everyone.

As a minimum we want to see:

  1. A high-level overview of the system (the “big picture”) probably in a diagram
  2. Details on every dependency
  3. Details on every error message
  4. Details on every configuration option/switch/flag/key etc
  5. Instrumentation hooks, expected values
  6. Assumptions, default values, etc

Hopefully this “Top Ten” list will give you a place to start when thinking about your DevOps “Operational Requirements” but it’s by no means comprehensive or exhaustive. We’d love to get your thoughts on what you think are the key OR’s for your applications!

Our experienced DevOps team provides a fully-managed “application-centric” website support service to your business and  your customers. Contact us to today, to find out how we can help.

image source: - CC  - c_knaus via Flickr - http://www.flickr.com/photos/soda37/6496536471

Continuous Integration & Delivery – Christmas Reading Essentials

Continuous Integration: Improving Software Quality and Reducing Risk
Continuous Integration: Improving Software Quality and Reducing Risk
by Paul M. DuvallSteve MatyasAndrew Glover
Learn more

Add to Wish List

Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation (Addison-Wesley Signature Series (Fowler))
Continuous Delivery: Reliable Software Releases through Build…
by Jez HumbleDavid Farley
Learn more

Add to Wish List

Agile Testing: A Practical Guide for Testers and Agile Teams
Agile Testing: A Practical Guide for Testers and Agile Teams
by Lisa CrispinJanet Gregory
Learn more

Add to Wish List

Continuous Delivery and DevOps: A Quickstart guide
Continuous Delivery and DevOps: A Quickstart guide
by Paul Swartout
Learn more

Add to Wish List

Test Driven Development: By Example
Test Driven Development: By Example
by Kent Beck
Learn more

Add to Wish List

Jenkins Continuous Integration Cookbook
Jenkins Continuous Integration Cookbook
by Alan Berg
Learn more

Add to Wish List

How Google Tests Software
How Google Tests Software
by James A. WhittakerJason ArbonJeff Carollo
Learn more

Add to Wish List

Jenkins: The Definitive Guide
Jenkins: The Definitive Guide
by John Ferguson Smart
Learn more

Add to Wish List

Growing Object-Oriented Software, Guided by Tests
Growing Object-Oriented Software, Guided by Tests
by Steve FreemanNat Pryce
Learn more

Add to Wish List

Succeeding with Agile: Software Development Using Scrum
Succeeding with Agile: Software Development Using Scrum
by Mike Cohn
Learn more

Add to Wish List

User Stories Applied: For Agile Software Development
User Stories Applied: For Agile Software Development
by Mike Cohn
Learn more

Add to Wish List

Refactoring to Patterns
Refactoring to Patterns
by Joshua Kerievsky
Learn more

Add to Wish List

Essential Scrum: A Practical Guide to the Most Popular Agile Process (Addison-Wesley Signature Series (Cohn))
Essential Scrum: A Practical Guide to the Most Popular Agile…
by Kenneth S. Rubin
Learn more

Add to Wish List

Service Design Patterns: Fundamental Design Solutions for SOAP/WSDL and RESTful Web Services
Service Design Patterns: Fundamental Design Solutions for SOAP/WSDL…
by Robert Daigneau
Learn more

Add to Wish List

Web Operations: Keeping the Data On Time
Web Operations: Keeping the Data On Time
by John AllspawJesse Robbins