Tag Archives: deployment pipeline

BamiWvqCIAAfBgw

Upgrading to Octopus 2.0

On 20th December 2013, Paul Stovell and the Octopus Team released Octopus Deploy 2.0. We’ve had a chance to play with the product for sometime, and it’s a great next version.

Here’s 5 great reasons to upgrade;

  1. Configurable Dashboards
  2. Sensitive variables
  3. Polling Tentacles (Pull or Push)
  4. IIS website and application pool configuration
  5. Rolling deployments

So let’s take a look at how the upgrade process works.

Step 1 – Backup your existing database

Things to check before you upgrade;

  1. Your administrator username and password (especially, if you are not using Active Directory)
  2. Before attempting to migrate, make sure that you don’t have any projects, environments, or machines with duplicated names (this is no longer allowed in Octopus 2.0, and the migration wizard will report an error if it finds duplicates).
  3. IIS Configuration. Are you using ARR or rewrite rules of any kind.
  4. SMTP Server Settings (these are not kept in upgrade)

The first step is to ensure that you have a  recent database backup that you can restore in case anything goes wrong.

1 - Database Backup

Confirm the location of your backup file. You’ll need this file at a later stage in the upgrade process (and if anything goes wrong)

2-Database Backup Location

Step 2 – Install Octopus 2.0

Download the latest installer MSI from the Octopus Website at http://octopusdeploy.com/downloads

Next follow the installer instructions and the Octopus Manager process. This process is really straight forward, and if you’ve installed Octopus before you’ll be familiar with most of the process. There’s a great video here on how to install Octopus http://docs.octopusdeploy.com/display/OD/Installing+Octopus

Step 3 – Import Previous Octopus Database

Once you’ve installed Octopus 2.0, you’ll most likely want to import your old v1.6 database with all your existing configuration. To do this, select the “import from Octopus 1.6″ option and choose the database back-up from the process you followed in step 1 and follow the instructions.

3 - Import Octopus Database

Step 4 – Post Upgrade Steps (server)

When you have completed the installation it’s essential to task a copy of the master key for backups. Instructions on how to backup the key can be found here http://docs.octopusdeploy.com/display/OD/Security+and+encryption

Once you have the server installed there’s a couple of items you’ll need to configure again.

  1. SMTP Server Settings are not kept, so these will need to be re-entered.

Step 5 – Upgrading Tentacles

If you are upgrading from version 1.6 - Octopus 2.0 server can no longer communicate with Tentacle 1.6. So in addition to upgrading Octopus, you’ll also need to upgrade any Tentacles manually. There are two ways to achieve this;

1. Download and Install the new Octopus Deploy Tentacle MSI on each target server

2. Use  a PowerShell script to download the latest Tentacle MSI, install it, import the X.509 certificate used for Tentacle 1.6, and configure it in listening mode. The powershell script is as follows;

function Upgrade-Tentacle
{ 
  Write-Output "Beginning Tentacle installation"
  Write-Output "Downloading Octopus Tentacle MSI..."
  $downloader = new-object System.Net.WebClient
  $downloader.DownloadFile("http://download.octopusdeploy.com/octopus/Octopus.Tentacle.2.0.5.933.msi", "Tentacle.msi")
  
  Write-Output "Installing MSI"
  $msiExitCode = (Start-Process -FilePath "msiexec.exe" -ArgumentList "/i Tentacle.msi /quiet" -Wait -Passthru).ExitCode
  Write-Output "Tentacle MSI installer returned exit code $msiExitCode"
  if ($msiExitCode -ne 0) {
    throw "Installation aborted"
  }
  
  Write-Output "Configuring and registering Tentacle"
   
  cd "${env:ProgramFiles(x86)}\Octopus Tentacle 2.0\Agent"
  Write-Output "Stopping the 1.0 Tentacle"
  
  & .\tentacle.exe create-instance --instance "Tentacle" --config "${env:SystemDrive}\Octopus\Tentacle\Tentacle.config" --console
  & .\tentacle.exe configure --instance "Tentacle" --home "${env:SystemDrive}\Octopus" --console
  & .\tentacle.exe configure --instance "Tentacle" --app "${env:SystemDrive}\Octopus\Applications" --console
  & .\tentacle.exe configure --instance "Tentacle" --port "10933" --console
  & .\tentacle.exe import-certificate --instance "Tentacle" --from-registry  --console
  Write-Output "Stopping the 1.0 Tentacle"
  Stop-Service "Tentacle"
  Write-Output "Starting the 2.0 Tentacle"
  & .\tentacle.exe service --instance "Tentacle" --install --start --console
  
  Write-Output "Tentacle commands complete"
}
  
Upgrade-Tentacle

Step 6 – Upgrade TeamCity Integration

The JetBrains TeamCity Plugin has also been released and requires an upgrade. To do this

  1. Download the plugin from the Octopus Website http://octopusdeploy.com/downloads.
  2. Shutdown the TeamCity server.
  3. Copy the zip archive (Octopus.TeamCity.zip) with the plugin into the <TeamCity Data Directory>/plugins directory.
  4. Start the TeamCity server: the plugin files will be unpacked and processed automatically. The plugin will be available in the Plugins List in the Administration area.

Please Note: The API Key’s for the users will have changed and will need to be re-entered into the Octopus Deploy Runner Configuration within the Build Step

Summary

We think that overall the Octopus Team have done a great job making the upgrade process easy. There’s still a few little issues to iron out, but overall the process is robust and simple. It’s a shame that automatic upgrade of the tentacles couldn’t have been supported as this may cause pain for customers with a large server install base, but Powershell automation makes for a great workaround.

As we learn more about Octopus 2.0, we’ll continue to let you know what we find. Good Luck with Upgrading — it’s definitely worth it!

DevOpsGuys use Octopus Deploy to deliver automated deployment solutions – ensuring frequent, low-risk software releases into multiple environments. Our engineers have deployed thousands of software releases using Octopus Deploy and we can help you do the same. Contact us today – we’d be more than happy to help.

Continuous Integration & Delivery – Christmas Reading Essentials

Continuous Integration: Improving Software Quality and Reducing Risk
Continuous Integration: Improving Software Quality and Reducing Risk
by Paul M. DuvallSteve MatyasAndrew Glover
Learn more

Add to Wish List

Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation (Addison-Wesley Signature Series (Fowler))
Continuous Delivery: Reliable Software Releases through Build…
by Jez HumbleDavid Farley
Learn more

Add to Wish List

Agile Testing: A Practical Guide for Testers and Agile Teams
Agile Testing: A Practical Guide for Testers and Agile Teams
by Lisa CrispinJanet Gregory
Learn more

Add to Wish List

Continuous Delivery and DevOps: A Quickstart guide
Continuous Delivery and DevOps: A Quickstart guide
by Paul Swartout
Learn more

Add to Wish List

Test Driven Development: By Example
Test Driven Development: By Example
by Kent Beck
Learn more

Add to Wish List

Jenkins Continuous Integration Cookbook
Jenkins Continuous Integration Cookbook
by Alan Berg
Learn more

Add to Wish List

How Google Tests Software
How Google Tests Software
by James A. WhittakerJason ArbonJeff Carollo
Learn more

Add to Wish List

Jenkins: The Definitive Guide
Jenkins: The Definitive Guide
by John Ferguson Smart
Learn more

Add to Wish List

Growing Object-Oriented Software, Guided by Tests
Growing Object-Oriented Software, Guided by Tests
by Steve FreemanNat Pryce
Learn more

Add to Wish List

Succeeding with Agile: Software Development Using Scrum
Succeeding with Agile: Software Development Using Scrum
by Mike Cohn
Learn more

Add to Wish List

User Stories Applied: For Agile Software Development
User Stories Applied: For Agile Software Development
by Mike Cohn
Learn more

Add to Wish List

Refactoring to Patterns
Refactoring to Patterns
by Joshua Kerievsky
Learn more

Add to Wish List

Essential Scrum: A Practical Guide to the Most Popular Agile Process (Addison-Wesley Signature Series (Cohn))
Essential Scrum: A Practical Guide to the Most Popular Agile…
by Kenneth S. Rubin
Learn more

Add to Wish List

Service Design Patterns: Fundamental Design Solutions for SOAP/WSDL and RESTful Web Services
Service Design Patterns: Fundamental Design Solutions for SOAP/WSDL…
by Robert Daigneau
Learn more

Add to Wish List

Web Operations: Keeping the Data On Time
Web Operations: Keeping the Data On Time
by John AllspawJesse Robbins

Continuous Delivery Adoption Barriers

For about a month, DevOpsGuys has been running a survey to understand what barriers exist for Continuous Delivery adoption. We’ve analysed the results and published our findings below. We’ve love to hear your feedback, so let us know what you think through the comments section.

barriers to cd

1. Organisation Culture (41%)

Organisational culture is probably the most important aspect to consider when adopting sustainable Continuous Delivery principles. 41% of respondents said that organisational culture was seen as the primary barrier to Continuous Delivery adoption.

Organizational culture is capable of blunting or significantly altering the intended impact of even well-thought-out changes in an organization.

Organisations seeking to adopt Continuous Delivery will primarily focus on evaluating and improving working practice, but to truly embrace Continuous Delivery, organisations must ensure that their employees encompass its primary principles. Continuous Delivery adoption is usually impossible unless management is bought in and at least some of the engineers are willing to change the way they work.

As organizations mature their Continuous Delivery adoption they must seek to embrace a culture which;

  • Is open, honest and transparent.
  • Encourages collaboration.
  • Promotes innovation, accountability and responsibility.
  • Builds and Rewards trust across organizational boundaries.
  • Enhances visibility of change and risk.

Organizational change is hard; it’s well known, theorised and documented and culture is the most difficult organisational attribute to change. For Continuous Delivery to be truly successful, the entire organisation must adopt its philosophy; to provide high-quality, valuable software as quickly as possible.

2. Lack of integrated development and operations (DevOps) (19%)

DevOps does provide an implementation strategy for Continuous Delivery principles, but neither philosophy is truly dependent on the other.  Continuous Delivery and DevOps are working towards common goals by providing business value through software delivery, within a culture that enables collaboration and understanding between the functional groups that deliver IT services where everyone is responsible for delivery.

Within the DevOps movement, Silo’s between teams are being removed so that software can be delivered quicker. That is, business value can be delivered quicker.

Developers are learning how to create production-ready software that can always be deployed, thus delivering high throughput and Operations are learning that Agile principles can enable effective and low risk change management, thus protecting stability.

recent study by Forrester Consulting and ThoughtWorks found that;

  • Companies are looking to prioritize innovation through developing software services.
  • Software development providers can’t deliver new services at the rate business leaders want.
  • Corporate culture and development process immaturity impede communication and slow service delivery.
  • Few IT organizations regularly perform advanced continuous delivery practices.

Therefore, it is not surprising that 19% of respondents consider a lack integrated development and operations capability as an adoption barrier for Continuous Delivery.

3. Technical hurdles (15%)

While Organizational Culture might be the primary adoption barrier for Continuous Delivery, there are also a number of technical hurdles to overcome. The technical challenges fall into 4 broad categories.

  • Configuration Management
  • Continuous Integration
  • Automated Testing
  • Automated Deployment

In each of these areas tools and associated best practice are becoming more advanced. Infrastructure-as-Code as promoted by OpsCode and Delivery Pipeline Management tools such as ThoughtWorks Go are being combined with an ever growing set of Cloud enabled services and platforms to make Continuous Delivery adoption easier from a technical viewpoint. However 15% of respondents believe that technical hurdles are significant in preventing adoption of Continuous Delivery.

4. Lack of understanding (15%)

Education of Continuous Delivery principles was seen as a big adoption barrier by 15% of respondents. Lack of understanding spans each of the possible responses in our survey from organisational culture through technical knowledge.

But there are more basic misunderstandings of the terminology, principles and practice of Continuous Delivery. As Paul Stack points out Continuous Delivery is not continuous deployment. Paul correctly defines Continuous Delivery as the process of having shippable software after each check in to source control, whereas Continuous Deployment is the process of shipping your product after every check in to source control. The difference is subtle but significant.

Another misconception is that Continuous Delivery and DevOps are basically the same thing. As Stephen Smith observes, Continuous Delivery and DevOps are interdependent, not equivalent. Some of this confusion may have arisen from the  misinterpretation of the DevOps principles leading to differing opinions of the relationship  between Continuous Delivery and DevOps, especially as “People often talk about DevOps and Continuous Delivery in the same breath.” as Jeff Sussna notes.

5. Business readiness to accept change at a fast pace (10%)

In many organizations, change can often feel constant. However as Isaac Asimov said “The only constant is change”. In general, people don’t like change, so methodology that promotes the acceleration of change is likely to face opposition. Only 10% of respondents saw business readiness to accept change at a quicker pace as a barrier to Continuous Delivery adoption. However, this may be influenced by the mainly technical community completing the survey.

Reducing the Mean-time-to-deploy can have significant business advantages. Continuous Delivery aims to provide high-quality and valuable software as quickly as possible. That is, to deliver business value is less time, whilst protecting the value quality. Given, respondents saw this reason as the least important factor is preventing Continuous Delivery adoption – could mean that organizations are aware of the benefits of getting to market faster.

For organizations aware of the benefits of delivering value at a quicker pace, and for those willing to embrace cultural change, Continuous Delivery provides a set of principles, which if implemented can provide significant benefit.

Maturing the Continuous Delivery Pipeline

The Maturity Model is a useful assessment tool for understanding your organizations level of Continuous Delivery adoption. Many organizations today have achieved what is needed to move from Level-1 (Regressive) to Level-0 (Repeatable), which is a significant accomplishment and as a reader of this blog post, you’re either about to start your journey of improvement or are already underway.

Continuous Delivery Maturity Model
Continuous Delivery Maturity Model

The Maturity Model

The advice for organizations wanting to adopt Continuous Delivery is ever more abundant, but for organizations that started adoption some time ago, the guidance on how to mature the process is still sparse. In this article, we explore one continuous improvement methodology that may help your organization mature its’ Continuous Delivery process.

Humble and Farley outline maturity Level 0 (Repeatable) – as one having process which is documented and partly automated.1 For this to be true, an organization must have first classified its’ software delivery maturity, identified areas for improvement, implemented some change and measured the effect. As Humble observes;

The deployment pipeline is the key pattern that enables continuous delivery.2

Humble also identifies that Deming’s Cycle is a good process to apply to initial adoption. 1 The process, according to Deming, should then be repeated so that further improvements can be planned and implemented; having the advantage the  data and experience from previous cycles is available. This process of continuous improvement is the first step to maturing the continuous delivery process.

Continuous Delivery and Kaizen

Continuous Delivery is a set of practices and principles aimed at building, testing and releasing software faster and more frequently.3 One key philosophy at the heart of Continuous Delivery, is Kaizen. The Japanese word, Kaizen, means continuous improvement. By improving standardized activities and processes, Kaizen aims to eliminate waste.  Within Kaizen are three major principles;

  1. 5S
  2. Standardization
  3. The elimination of waste; known as muda.

The 5S principle characterizes a continuous practice for creating and maintaining an organized and high-performance delivery environment. As 5S has the focus of waste reduction, it appears prudent to explore the 5s methodology as an approach to optimizing the Continuous Delivery pipeline.

What is 5S?

5S is a lean engineering principle focused on waste reduction and removal. The methodology is recursive and continuous and it’s primary goal is to make problems visible  so that they can be addressed.4

There are five primary 5S phases, derived from five Japanese words that translate to; sort (seiri), straighten (seiton), shine (seiso), standardize (seiketsu) and sustain (shitsuke).5

The 5s principle is more commonly applied to workplace organization or housekeeping. Since the delivery pipeline is a key component of the process, it is therefore a fundamental place where work will take place. The pipeline exhibits elements that are common with any physical work environment including;

  1. Tools, process, paperwork.
  2. Storage areas for artifacts, such as files.
  3. A requirement that components must be, physically or virtually, located in the correct location, and that they are connected by process.
  4. Tools and process need maintenance and cleaning. E.g. retention policies.
  5. Systems and procedures can be standardized and can benefit from standardization.

When applied to the delivery pipeline, the five primary 5S phases can be defined as;

  • Phase 1 – sort (seiri)
    • Analysis of processes and tools used in the delivery pipeline. Tools and processes not adding value should be removed.
  • Phase 2 - straighten (seiton)
    • Organization of processes and tools to promote pipeline efficiency.
  • Phase 3 – shine (seiso)
    • Inspection and pro-active/preventative maintenance to ensure clean operation throughout the delivery pipeline.
  • Phase 4 – standardize (seiketsu)
    • Standardization of working practice and operating procedure to ensure consistent delivery throughout pipeline.
  • Phase 5 – sustain (shitsuke)
    • Commitment to maintain standards and to practice the first 4S. Once the 4S’s have been established, they become the new way to operate.

Implementing 5s in the Delivery Pipeline

Experience has shown that there are several key factors that should be considered in the adoption of 5S.

  1. Begin with small changes. Remember that 5s is a continuous and recursive process.
  2. Involvement is crucial.6 Encompass everyone involved in the deployment pipeline.
  3. Create Champions – find leaders who love &  live the philosophy and encourage them to promote it.

5s implementation involves working through each phase (or S) in a methodical way. This is commonly achieved by Kaizen events, which are focused activities, designed to quickly identify and remove wasteful process from the value stream and are one of the prevalent approaches to continuous improvement.

Phase 1 – sort (seiri)

The first step of 5s is to sort. The purpose is to identify what is not needed in the delivery pipeline and remove it.

Additional process and practice can easily creep into the delivery mechanism, especially in the early phases of Continuous Delivery adoption, when you are experimenting.

The removal of items can either mean total elimination of a tool or process, since it is redundant, or the flagging, known as “Red Tagging”, of a tool or process which needs to be evaluated before it is disposed of. The review of red tagged tools and process should evaluate their importance and determine if they are truly required.

Suggested Steps

1. Evaluate all tools and process in the delivery pipeline and determine what is mandatory.

2. Ascertain what is not needed. Assess what it’s elimination status is, e.g. immediately or Red Flag.

3. Remove unnecessary or wasteful tools and  process.

Phase 2 - straighten (seiton)

The second step requires items in the delivery pipeline to be set in order. This requires the pipeline to be arranged so that  maximum efficiency is achieved for delivery.

To achieve seiton, the key components of the automated pipeline should be analysed, and should include the following;

  • Automated build process.
  • Automated unit, acceptance tests including code analysis.
  • Automated deployment process.

It is suggested that for the pipeline to be truly efficient the most commonly used items are the easiest and quickest to locate, such as;

  • Build status reports and metrics including KPI’s such as Mean-time-to-release (MTTR)
  • Automated test coverage, error and defect reports

In this step, actions can also be taken to visually label elements of the pipeline so that demarcation of responsibility  is clear. This may mean categorizing tools, process and target locations E.g. servers into  groups such as production and pre-production or development and test.

Suggested Steps

1. Analyse the value stream including tools and process.

2. Ensure the necessary information is available quickly to those who need it.

3. Visually distinguish or categorize key pipeline components.

Phase 3 – shine (seiso) 

Once the waste has been removed and only what is required has been kept and ordered, the next phase inspects the delivery pipeline to ensure that it’s operation is as clean as possible.

During pipeline execution large amounts of data can be produced. If this information is not dealt with efficiently it can become a waste product, detracting from the value of the delivery process.

Inspection of this data is vital. Data generated in the pipeline is likely to include instrumentation, metrics and logging information, as well as build artifacts and report data.

In context of the delivery pipeline, cleaning can be defined as archiving, roll-up or removal of delivery pipeline data. House keeping routines should be evaluated to ensure that data retention rules are applied and that waste, in the form of build artifacts, metric and report data and instrumentation data are correctly archived or removed as required.

Implementation of this phase requires assignment of cleaning responsibilities and scheduling of cleaning process.

Suggested Steps
1. Inspect data to understand what should be rolled-up, archived or deleted.

2. Ensure that responsibilities for data management are clearly assigned within the team.

3. Create a scheduled, automated cleaning processes.

Phase 4 – standardize (seiketsu)

The primary focus of step four is to ensure that  the working practice, training and operating procedures remain consistent. Standardization allows for continuous improvement and each of the first 3S’s should be regulated.

A critical element of seiketsu is to ensure that the process is reviewed on a regular basis or cycle. This ensures that best practices are maintained and areas  where standards have slipped are quickly identified.

A settled and regular process or practice becomes one that is hard to give up. Therefore,  if standardization, is implemented correctly, it can support culture change.

Suggested Steps

1. Ensure that the team have clear responsibilities for the tasks that need to be completed.

2. Create ownership and pride in the process.

3. Ensure champions provide guidance and support changes to the 5s process.

Phase 5 – sustain (shitsuke)

The purpose of the final phase is to ensure that the 5s process is not a one-time event. To achieve this,  the focus of the fifth  phase centers on culture. To sustain 5s organizations must engage in support, commitment and gainful communication, with the aim of keeping the team devoted to continuous improvement.

There are a number of elements which influence cultural support of 5s . Which elements take precedence varies within every organization and is heavily dependent on the existing culture of that organization. Key points are:

  1. Communication – aims & objectives need to be clear for all  team members.
  2. Education -  5s methodology, concepts and practices need to be communicated.
  3. Recognition -  team members need to feel that their efforts are recognized.

It is interesting to note that the literal translation of shitsuke is discipline. The final phase of 5s can also be one of the hardest to implement.

Suggested Steps

1. Establish an open channel for discussion and feedback, to facilitate continual learning.

2. Adopt the habit of auditing regularly and reward the best teams.

3. Ensure that all  problems are addressed quickly make sure that root cause analysis is completed to discover the cause.

Summary

The adoption of Kaizen events to implement 5s within the Continuous Delivery pipeline can provide significant support to maturing the model, by providing a continuous improvement process within a well-defined and standardized structure.

Lean manufacturing has been using the 5S pratice as a common method for lean implementation for years. Nearly 70% of of lean manufacturing companies use this approach 7, but the use of 5S as an adoption method for lean in software development would appear less common, as it has not be commonly applied to development practices.

Lean software development principles provide the promise of companies becoming more competitive by delivering software faster, increasing quality, and reducing cost leading to greater customer satisfaction, all of which align closely to the key principles of  Continuous Delivery.

– TheDevMgr

References

1. Humble, J. Farley D. (2010). Continuous Delivery: Reliable Software Releases Through Build, Test, and Deployment Automation. Indiana: Addison Wesley. 419.

2.  Humble, J. Farley D. Continuous Delivery: Anatomy of the Deployment Pipeline.  [September 7, 2010.]  InformIT. http://www.informit.com/articles/article.aspx?p=1621865

3. McGarr, Mike. Continuous Delivery. SlideShare.net [June 17, 2011.] http://www.slideshare.net/jmcgarr/continuous-delivery-8341276

4. Liker, Jeffery K. The Toyota Way. New York City : McGraw Hill, 2004.

5. MLG UK. 5S – The Housekeeping Approach Within Lean. MLG UK. [Cited: February 5th 2013] http://www.mlg.uk.com/html/5s.htm

6. Maready, Brian. Transparency = speed . Lean Lessons. [May 27, 2010] http://leanbuilt.blogspot.com/2010/05/transparency-speed.html.

7. Compdata Surveys. Lean Manufacturing and Safety Help Manufacturers Survive Tough Times.  Compdata Surveys. [December 13, 2010.] http://www.compdatasurveys.com/2010/12/13/lean-manufacturing-and-safety-help-manufacturers-survive-tough-times/