Continuous Testing with CI/CD, Rings and Impact Analysis

The key to building quality into a piece of software such as To-Do Studio is making sure we can get fast feedbacks on the impact of changes using a deployment pipeline with continuous testing.

In the last few months, using the Azure Pipelines service, we have worked hard to put this infrastructure in place. It relies on practices such as Continuous Integration and Delivery (CI/CD), deployment rings and impact analysis.

Continuous Integration is about automating build and test processes to make sure the resulting software is in a good state, ideally every time a developer changes code. Continuous Delivery goes one step further by automating a software release, which typically involves packaging the software for deployment in a production-like environment.

Deployment rings were first discussed in Jez Humble’s book. Rings promote a production-first DevOps mindset and limit impact on end users, while gradually deploying and validating changes in production.

Deployment rings are used to advance a software release, built in the initial stage of CI/CD, through a sequence of target users.

We identified four types of users:

• Ring 0 – Staging : Developers and testers who validate the release before deploying it to real users.
• Ring 1 – Alpha : Close circle of employees and volunteers who test new features as they become available.
• Ring 2 – Beta : Early adopters who voluntarily preview releases, considered more stable than the ones in Alpha.
• Ring 3 – Users : Users who consume the service, after passing through the alpha and beta early adopters.

Let’s look at how a change goes through the deployment rings.

1. A developer commits a change (checkin) to our source code repository.
2. The checkin triggers a Continuous Integration (CI) build.
3. The last step of the continuous integration launches a continuous delivery trigger, which automatically starts deployment in the staging environment.
4. The deployment publishes a new release to the Staging ring. Only the developers and testers are impacted by the change.
5. The deployment publishes a new release to the Alpha ring. Only the Alpha users are impacted by the change.
6. The deployment publishes a new release to the Beta ring. Only the Beta users are impacted by the change.
7. The deployment publishes a new release to the Users ring. At this stage, everyone is affected by the change.
8. It’s key to realize that the impact (“blast radius”) increases as your change moves through the rings. Exposing the change to the Alpha and the Beta users, is giving two opportunities to validate the change and hotfix critical bugs before a release to everyone.

At each ring, an impact analysis is performed. In case of insufficient quality, the deployment is rejected, and the developers must correct the problems. The process must then start over again. Impact (also called blast radius), is evaluated through observation, testing, analysis of telemetry, and user feedback. We rely primarily on the Application Insights and Azure Monitor services to perform the impact analysis.

Subscribe to my Newsletter

Since a few months I publish a newsletter for my family and friends. One of the objectives of this newsletter is to share my entrepreneurial journey. It contains the latest news about To-Do Studio, the startup I founded with Erik Renaud. Our company designs and markets a software service that allows leaders to empower others to take initiative. The newsletter is intended to offer a privileged look at the path of a startup company.

Encouraged by the positive comments I receive, I decided to give access to the newsletter to readers of this blog. That’s why I invite you to subscribe to my newsletter.

Here is the link to subscribe: Destination To-Do Studio

The newsletter is free. It is published every 2 months, 6 times a year. The subscription does not require any obligation on your part. The only information you must provide to register is your email address (it will never be shared with third parties). You can unsubscribe at any time. If you wish to preview the content of the newsletter, I invite you to read the first issue here.

PS. Following registration, if you do not receive quickly a confirmation email, check your Junk Email inbox.

Market Positioning Statement

As I stated in a previous post, the first challenge facing startups is to find their Product/Market Fit. Product/Market Fit means that the solution (the product) is well suited to its target problem (the market who experienced the problem). To find the right Product/Market fit, you must not only fulfill a job-to-be-done but also ensure you have an obvious competitive advantage.

Startups that have a competitive advantage create a sustainable future for their businesses, with products that can grow and generate profits long after the initial launch.

During my master’s degree in Innovation and Technological Entrepreneurship, I learned to capture and formalize a competitive advantage using a “Market Positioning Statement”. Positioning is a choice of attributes that provides a credible, different, attractive and sustainable position within a marketplace and in the minds of customers. The positioning communicates what makes a product unique and different.

This statement usually comes in a form that looks something like this:

For [Target Market] who [Statement of Need], the [Product Name] is a [Product Category] that [Key Benefit / Reason to Buy] unlike [Competitors] our product [Primary Differentiation]

Here is the rendition of the Market Positioning Statement for To-Do Studio:

[Target Market]
Modern leader

[Statement of Need]
Want to avoid the hassle of follow-ups and tough talks that occur when pushing and pulling others to take initiative

[Product Name]
To-Do Studio

[Product Category]
Task management software

[Key Benefit / Reason to Buy]
Guides teams to self-organization

[Competitors]
Other current solutions such as chats, emails, calendars, to-do lists, kanban boards, spreadsheets or project managers

[Primary Differentiation]
Get things done because each teammate works in tandem with an automated assistant (personalized coaching), that not only provides a daily schedule, but guides and directs the involvement (throughout the process)

Here is the positioning statement in sentence form:

For the modern leader who wants to avoid the hassle of follow-ups and tough talks that occur when pushing and pulling others to take initiative, To-Do Studio is a task management software that guides teams to self-organization. Unlike current solutions such as chats, emails, calendars, to-do lists, kanban boards, spreadsheets or project managers, To-Do Studio get things done because each teammate works in tandem with an automated assistant (personalized coaching), that not only provides a daily schedule, but guides and directs the involvement (throughout the process).

The key competitive advantage of To-Do Studio is its automated assistant (personalized coaching).

For five years we have been searching for the right Product/Market Fit

Often, I summarize the To-Do Studio journey saying that it took us five years of research and development (R&D) to find a simple and elegant solution to the following problem: How to allow teammates to complete a common to-do list? The reality is much more complex.

What I describe as R&D is rather a long quest to find the right Product/Market Fit. Of everything I discovered applying the Lean Startup practices, Product/Market Fit is the concept that remains the most inspiring for me. Compared to some other concepts such as MVP that I first discovered through reading The Lean Startup, Product/Market Fit is much less famous outside of the startup scene.

Product/Market Fit means that the solution (the product) is well suited to its target problem (the market who experienced the problem). Startups that achieve Product/Market Fit create a sustainable future for their businesses, with products that can grow and generate profits long after the initial launch. The life of any startup can be divided into two parts — ’before’ Product/Market Fit and ‘after’ Product/Market Fit. Obviously, to this day, To-Do Studio is in the ‘before’ part. Our goal in 2019 is to move into the ‘after’ part.

Over the last five years, using ongoing build-measure-learn cycles, we have progressed toward validated learning. Every day we had new assumptions for building the product, new measures to check results, and new learning opportunities that brought us closer to Product/Market Fit.

On the other hand, on three occasions, we abandoned perseverance and opted instead for a pivot. Pivoting is what you should consider when you conclude that you’re not making any progress (or not enough) toward Product/Market Fit. A pivot is a radical course correction, going in a new direction, with a new strategic hypothesis regarding the problem or solution space. In pivot mode, the unit of progress is creative destruction.

Here is a summary that presents the radical course correction of each pivot:

	Problem Space	Solution Space
Launch April 2013	New market For teammates who want to improve outcomes from shared accountability and aligns individual responsibility with team workflow	New product Boards that guides collaboration between teammates using sticky notes to represent work items and columns to represent each stage of the workflow
Pivot 1 May 2014	New market For individuals who want to schedule commitments and get a unified view of their “daily” commitments	New product Daily planner that organize work by pairing a daily calendar with a list of to-do items. It keeps track which to-dos remain to be started, which are in progress and which are done.
Pivot 2 Sept 2016	New market For people adopting work-life fusion who want to split their workload with their close ones	Product Evolution Collective workspace that expands on our previous product by allowing to team up with others through a common list of to-do items, synchronized among teammates
Pivot 3 Jan 2018	New market For leaders who need to empower others to get things done and want to avoid the hassle of follow-ups and tough talks that occur when pushing and pulling others to take initiative	Product Evolution Collective workspace that expands on our previous product by providing each teammate with an automated assistant (personal coach)

A first interesting observation is the elapse of several years between each pivot. This can be explained by the following reasons:

• The discovery process requires time. Making the right assumptions and finding out why they are not working is hard and time consuming;
• For entrepreneurs, deciding to persevere against pitfalls is always the first choice. It takes a long period of thinking and many failures to agree to pivot;
• In bootstrapping mode, since entrepreneurs divide their efforts on two fronts, the discovery process is much longer. They must complete the contracts that finance the business while continuing to pursuit the Product/Market Fit.

A second interesting observation is that each pivot involves a creative destruction of the target market. It seems that entrepreneurs are poorly equipped to define the problem space. As a convergent thinker, an entrepreneur is much more likely to focus on the solution than the problem.

The good news is that correctly identifying a problem space is a knowledge that can be learned. In our case, it was in December 2017 when I discovered the “Job to be Done” innovation approach. This acquisition of new know-how led us to our most recent pivot.

“Job to be Done” innovation approach presents a product not as a set of functionality but rather as a solution that a customer hires to get a job done.

Here are two blog posts I wrote following my discovery of the approach. It describes the positive impact it has on our quest to discover a better Product/Market Fit:

• Customers hire something to get a job done
• Customers don’t buy products, they buy better versions of themselves

Configuring Cypress in CI with Azure DevOps Pipelines

Cypress is a front-end test automation framework built for the modern web. It is open source and written entirely in JavaScript. It addresses the key pain points developers and QA engineers face when testing modern applications:

• A rich yet simple API for interactions with automatic waiting
• Mocha, Chai, and Sinon bundled in
• A sleek dashboard with automatic reloads for Test-Driven Development
• Easy debugging
• Network traffic control for validation and mocking
• Automatic screenshots and videos

It is a very powerful tool that enables developers to write End-to-End tests entirely in JavaScript, directly accessing everything within the browser. It is also a versatile tool that can be used to test REST APIs.

At To-Do Studio, we use Cypress to test End-to-End scenarios, as well as REST APIs.

On developers’ computers, we install the Cypress Test Runner and write tests locally. Everything works perfectly. However, for CI/CD testing, we must configure Cypress with Azure DevOps Pipelines. The recipe for configuring Cypress is not as simple as it may seem at first. I’m writing this post to make sure that a configuration recipe is documented somewhere on the web to explain how to configure Cypress in Continuous Integration (CI) with Azure DevOps Pipelines. Thus, with the help of Google search, those who will have the same goal in the future will find the right answer easily.

Here is diagram presenting an overview of the major elements of the solution:

First, ensure during the build phase that the cypress tests files are zipped and published to the artifacts drop.

Second, during the release phase hosted on an Ubuntu agent, extract the zipped test files,run the tests using Cypress and publish the tests results. Here are the detailed steps:

1. Extract the zip file containing the tests

Extract the zip file that was published to the artifacts drop by the build pipeline.

2. Run the tests

Start by creating the Cypress config file. Define the location of your tests with the integrationFolder configuration value. Do not use the testFiles configuration value or –spec command line option.

{
  "integrationFolder": "tests/e2e/specs",
  "baseUrl": "https://info-staging.to-do.studio",
  "projectId": "<insert your project Id>",
  "reporter": "junit",
  "reporterOptions": {
    "mochaFile": "tests/test-output-[hash].xml",
    "toConsole": true,
    "attachments": true
    },
  "video": false,
  "pluginsFile": "tests/e2e/plugins/index.js",
  "supportFile": "tests/e2e/support/index.js",
  "env": {
    "urlEnv": "staging"
  }
}

Follow by running the tests with Cypress using the command: npx cypress run –record –key <insert your record key>. npx install and run Cypress in a single step. npx is a npm package runner (x stands for eXecute). The typical use is to download and run a package temporarily. Please note that if you intent to record screenshots and videos with Cypresss Dashboard Service, you need to add the unique projectId into your cypress.json and pass the record key into the command. If you do not need visual results, simply omit the recording option and projectId.

3. Publish the tests results

Here you will publish the junit mocha files created during the tests run. Make sure that your merge test results and that it fails if there are test failures.

Virtual Studio: the Workplace of the 21st Century

The name of our software service To-Do Studio is a combination of two words: To-Do and Studio. We firmly believe that this name association correctly explains our product’s mission. We chose the word To-Do because the team work is performed via a common list of To-Do items, synchronized among teammates. We added the word Studio to indicate the creative and collective aspect of our service. A studio is a place of creativity, a collective workspace where teammates take initiative and assume responsibilities.

Just as Seth Godin, author and successful speaker on marketing issues, recently wrote on his blog, the studio is the modern workplace. Here’s how we paraphrase his words:

The success of leaders in a factory is based on compliance. More compliance leads to more profits. Do what you are told, faster and cheaper, and repeat.

The factory is a legacy of the twentieth century.

The studio, on the other hand, is a place of initiative and creativity. The initiative to create new things and better things. In short, a process that leads to better things.

It’s a peer to peer relationship. The hierarchy has virtually disappeared. All that remains is leadership.

This is the initiative coupled with responsibility. Authority is much less important, as are traditional measures of productivity.

You can distinguish a studio from a factory in minutes. Where do you work?

You can read the original post on Seth Godin’s blog here.

We are convinced that the studio is the future of work and that it will evolve toward a marriage between the physical and virtual world thanks to software such as our. We envision that each team will want to work with the help of a virtual studio. And we are determined to ensure that, for most companies, this virtual studio is To-Do Studio.

The virtual studio is the workplace of the 21st century.

How to clean up the wwwroot folder on the Azure Web App using PowerShell

I’m writing this post to make sure that a PowerShell recipe to remove all the files in the wwwroot directory on the Azure Web App is documented somewhere on the web. Thus, with the help of Google search, those who will have the same goal in the future will find the right answer easily.

Recently, when deploying the To-Do Studio website with Azure DevOps, I needed to clean the wwwroot directory before deploying. It is usually a simple configuration setting when running the release on a hosted VS2017 agent. When using the Azure App Service Deploy task with the Publish using Web Deploy option, there is an additional option to Remove Additional Files at Destination. Unfortunately, in order to roll the automated E2E tests with Cypress, we must run the release script on a Linux agent. This Remove Additional Files at Destination option is not available anymore on Linux. So I had to automate the cleanup  with an Azure Pipeline task.

At first, to fill this gap, I thought about using the “Azure WebApp Virtual File System Tasks” available in the Azure DevOps marketplace. Unfortunately, this task did not work. So I had to automate the cleanup with a Powershell script.

The non-automated (manual) way to delete files / folders in Azure Web App is to use the Kudu console. For those who like me want to automate this operation with PowerShell, there is KUDU Virtual File System (VFS) Rest API.

Without further ado, here is the script.  You can download it from my Personal GitHub repos or you can copy and paste without restraint from the script below.

$WebAppName = “insert Web App name”

$slotName = “insert Slot Name”

$username = ‘insert the username from the publish profile’ #From the publish profile

$password = “insert the password from the publish profile” #From the publish profile

# Initialize parameters for Invoke-RestMethod

$apiUrl = “https://$WebAppName.scm.azurewebsites.net/api/vfs/site/wwwroot/&#8221;

if ($slotName -ne “”){

    $apiUrl = “https://$webAppName`-$slotName.scm.azurewebsites.net/api/vfs/site/wwwroot/”

}

$base64AuthInfo = [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes((“$($username):$($password)”)))

$headers = @{

    Authorization=“Basic $($base64AuthInfo)”

    ‘If-Match’ = ‘*’

}

$userAgent = “powershell/2.0”

# Define a reursive function to delete files

function DeleteKuduDir ($content, $dir)

{

    foreach($c in $content)

    {      

        if($c.mime -eq “inode/directory”)

        {

            # Get listing of directory as an array

            $childContent = Invoke-RestMethod –Uri $c.href –Headers $headers –UserAgent $userAgent –Method GET –ContentType “application/json”

           

            # Delete directory

            $newDir = $dir + (Split-Path $c.href –leaf) + “\”

            DeleteKuduDir –content $childContent –dir $newDir

        }

        # Delete file

        $file = Split-Path $c.href –leaf

        Write-Host “Deleting” $dir$file    

        $result = Invoke-RestMethod –Uri $c.href –Headers $headers –UserAgent $userAgent –Method DELETE –ContentType “application/json”

    }

}

# Get listing of wwwroot as an array

$rootContent = Invoke-RestMethod –Uri $apiUrl –Headers $headers –UserAgent $userAgent –Method GET –ContentType “application/json”

# Delete files and directory in wwwroot

DeleteKuduDir –content $rootContent –dir “\”

Write-Host “Done!”

I wish emotional Intelligence and mental balance for my children

For almost two years, Yuval Noah Harari’s bestseller “Sapiens” has been on my reading list. Last summer, I started reading the book during a vacation with my youngest son. Since then, I have not only finished Sapiens but I have also devoured his second book, “Homo Deus”. These masterpieces really helped me to understand the world we live in a little bit better.

I love the mix of facts, simplicity and philosophical questions that makes you think a lot and reflect. It helped me put things in perspective and, in a way, extract myself from the current realty and think about meaning and purpose of life. These readings have fueled my thinking so much that I keep recommending their readings to everyone around me. Harari’s work should be taught in schools. I am currently reading his latest book “21 lessons for the 21st century“.

For those who are less familiar with the writings of Yuval Harari, here is an interview where he discusses his latest book. Unfortunately, the facilitator does not seem to understand the nuances of Harari’s work and he is completely unable to follow with significant questions, and especially to question his thinking intelligently. The result could be disappointing. But as there is something positive in everything, one of the advantages of this interview is that it is a very good introduction to Harari’s books.

Yuval Noah Harari: “21 Lessons for the 21st Century” | Talks at Google

Recently, thanks to the random algorithms of Youtube, I came across an interview of Harari with an intriguing title: The 2 Most Important Skills For the Rest Of Your Life. Harari is not only a macro-historian and professor but one of the world’s most innovative and exciting thinkers. I quickly made the decision to listen to the interview and I was not disappointed. There is a lot to learn from this video, especially for the youngest who start their adult life. In this regard, I will recommend it to my four children.

I like to believe that the main reason we are on earth is to create happiness. I have always wished that my children become initiators of happiness, that they flood their entourage with those little things that make life more pleasant. Obviously, we must begin by finding happiness deep within ourselves. Like Harari, I am convinced that in the 21st century, only those who have enough emotional intelligence and mental balance will be sources of happiness.

And in the end
The love you take
Is equal to the love you make

Beatles
The End (Abbey Road)

Here is the link to listen to the full interview: https://www.youtube.com/watch?v=x6tMLAjPVyo

Software as a service (SaaS) in 2018 revolves around JavaScript

In 2014, on this blog, I wrote a post in which I announced that we intended to create native mobile applications for tablets and phones (IOS and Android) and that we had planned to use Xamarin.Forms as the main programming tool. At the time, web tools were not mature enough to design applications that target all classes of personal devices be it laptops (Windows, Chrome and Apple), tablets (Android and iPad) or phones (Android and iPhone). This is no longer the case in 2018, change is the only constant and we must not be afraid to review our previous decisions. As developer polls in recent years show, the reality is simple, native development on mobile platforms is no longer important.

At To-Do Studio, with my business partner Erik, we are convinced that the best technologies in 2018 to launch software as a service (SaaS), revolves around JavaScript (at least for the front end).

We have decided to build our SaaS using one codebase, with familiar tools and web technology. In this regard, we chose Vue.js for web development. Vue.js is a library for building interactive web interfaces. It simplifies managing states in the front-end and provide composable view capabilities with an API that is as simple as possible.

I just spent the last four months learning to program with this framework. Vue is one of the easiest frameworks to learn and master, the learning curve is gentle and there is a well-defined ecosystem. Furthermore, since I already know the basics of the web, the transition was easy enough. I am not yet a pro in javascript but I am improving every day. Since I’m in charge of UI integration, I mainly focus on mastering Vuetify, the UX component framework for Vue.js that we have selected.

An assistant does not have to be a chatbot

Last winter, after realizing that one of the super powers we had to offer was a productivity assistant, we worked hard to find the best solution to include this feature in our product To-Do Studio. As a recap, the purpose of the assistant is not only to provide a daily schedule, but also to coach and direct teammates’ commitments throughout the process.

It was a long and winding journey and, today, I am happy to share the solution we have chosen.

As soon as we ruminate about a feature that resembles an assistant, we immediately think chatbot. A chatbot is a software program that converses with humans in a natural language, such as English or French, rather than through a graphic interface or via computer-language commands. Mainly because they are cool, seemingly very smart, potentially a little dangerous, and everyone wants to get to know them, recently, chatbots were widely talked up as the future of human interaction with technology. However, it is important to remember that the goal of most chatbots is not to match the capabilities of a human — to pass the Turing test — but to help people achieve specific goals without needing another human to be involved.

Conditioned by the bubble hype around chatbots, we initially tried to create a user experience driven by human-to-machine conversation.

Since we do not need a very sophisticated solution, we quickly identified that a simple notification assistant could meet the needs. A notification is an act of bringing something to the notice of someone, so he can act upon it. In addition to having to notify, we also identified the requirement for a rule-based bot with a limited set of commands that represent the potential responses to the notification.

Through all our experiments we discovered two improvements to simplify user experience (UX):

1. No need for a chatbot: Interactions with a notification assistant do not behave like a conversation. The conversation is limited to a notification (the starting context) and a response (a choice limited to a set of predefined commands). In this perspective why use a significative part of the screen space with a chatbot.
2. Need to display answer choices: As most bot developers could tell you, giving end users a box to type in rarely ends up being just a simple question and answer. We must provide shortcuts in the form of commands and commands must also be displayed in the conversation area if you want them to be easily visible and accessible.

In the end, after several iterations, we abandoned the user experience based on a chatbot. Similarly, we abandoned the idea of displaying the answer choices in the conversation area.

We opted for something simpler and proven, a menu-based conversation. The advantage of the menu-driven approach is that we can highlight the presence of the assistant with an extremely visible button, floating throughout the screen (Floating Action Button), and we can guide the user with a notification area.

The user has no difficulty discovering the commands because they are displayed as a menu that also floats above the screen.

Now that the bot hype is dying down, we recognize that a bot is just another frontend for accessing software services. An assistant is not necessarily a chatbot especially if it aims to help users perform specific tasks within a software service. A menu-based conversation is as valuable a choice as a chatbot.