Continuous Confidence in Software Development

There. I have said it. This is the elephant in the room. For years, we as Dev Ops professionals, and as an industry have been talking about how to up our game using Continuous Integration.  We talk about our Software Development Life Cycle (SDLC) or how we need to improve our QA Processes and deliver faster and with higher quality. We start implementing Continuous Integration and we try to implement Continuous Delivery and or Deployment. And you know what happens? We fail to do it the way others did. There are some examples that we try to wedge ourselves into. We try hard to mimic what others have done. And so we fail over time. Some of us, we get to where we want to be, but others...yeah...let's not talk about it! So what should we be doing? We should be talking about Continuous Confidence. We should be talking about what processes and what things are we doing that will help us get better confidence in our product. What will give others better confidence in our product. When...

Are there any of you out there who have attempted Continuous Integration, Delivery, or Deployment, and had it fail? Have you ever just given up because it was over complicating things? People were not happy? Believe it or not, this is very common. I have heard many stories where organizations have failed at implementing CI. So first and foremost I want to say: it's ok! We all recognize the value of what we get from doing CI, but in reality, in practice we need to be more practical and pragmatic in our approach. May I offer some counsel. Implement Countinuous Confidence This may seem strange at first, and probably expected since this blog is all about Continuous Confidence, but really, truly, this is he first step. Focus on your confidence points.  Generally, when I see failure in the CI world, it isn't the methodology, but rather the lack of confidence in the process that is causing the problem. Generally speaking, I see people wanting to implement CI in here organization becau...

Have you ever wondered why we have QA? I know, that's a weird question. When we look at what QA does, we are really asking them to do something we haven't done for ourselves: we ask for their vote of confidence. Consider the situation we put them in. We ask QA to review our application and tell us if there are any issues with it. We don't necessarily always tell them what we changed. If we tell them what we changed, hey don't always know how to test it properly, so instead, our founded confidence in them becomes, what I call, Bottleneck Confidence. Bottleneck Confidence Bottleneck Confidence is really just that, confidence that is restricted and not complete. This means that our confidence is throttled and limited to what we truly can achieve. So the question becomes: why do we need QA? Well, if we continue asking them to work in the same vein, then, we don't need QA. If we ask them to fill a different role, and allow us to reach our full potential, then they ...

The biggest challenge in today's Continuous Integration culture, is separating areas of concern and letting departments be autonomous and independent. What We find most often is that a Development organization adopts Continuous Integrationa and then starts to push their product and manner of releasing to the IT department. We need to start letting the IT organization be the professionals they are. This allows confidence to build across the teams and solidifies the organization as a whole. So how do we do this? Communication Listen to your teams. We need to recognize that Continuous Integration deals with the codebase. It is a practice to continuously integrate people's work with each other via continuously building and testing code. Development is responsible for the code. DevOps is responsible for building the assets that will be deployed, to the standard that IT expects. IT gets to determine what type of artifact they require to maintain the performance of he system. If t...

There are a lot of questions around legacy code. If we ask any developer who is new to the organization, they will most likely say: REFACTOR! Our first knee jerk reaction is to not touch the legacy code, so we kind of dismiss the new developer saying that. However, we should look more deeply at what they are saying. The Code is Messy Generally, when dealing with legacy code, it is messy! Let's face it, when we start writing code, we ignore certain design patterns for others because there isn't a need. Once we get large enough, something has to be done about it. We need to stay focussed. If we feel that we are assigning one senior developer to fix legacy code, while 50 others are still using it, we will,never fix the code. Continuous Confidence comes from looking at the code and feeling calm not erratic. The Code is Antiquated When we hear refactor, His might mean as well that the code doesn't follow today's standards. So how do we make it so? We need to refactor...

One of the hardest parts of Continuous Confidence is understanding what brings confidence to our testing. We strive, at times, to find that proper level of confidence. Too often, we talk about Code Coverage. We want to make sure that we are touching everything, and it lifts our confidence, falsely, to the same level of as our code coverage. If we have 60% code coverage, then we have 60% confidence. The issue, however, is after a while, we start to realize, once we get bugs, that we aren't really that confident in our code and then we dismiss testing almost as a chore instead of a tool. So how do we get more confidence? Well, the answer is simple, the process a little more difficult: write quality tests. Area Coverage The first thing we need to do, is understand what we are testing. We should be considering Area Coverage instead of Code Coverage. Area Coverage looks at the areas of the application that have tests. There aren't a whole lot of tools out there that will help yo...

The hardest question to answer, it seems, is what kind of test should we be writing? Within the industry today, we have a preconceived notion that integration tests are the best. When we have conversations with developers about what needs to be tested they like to say: test all of it. When we are hesitant that we may have broken something we turn to an integration or selenium test to verify it. This is a problem. We need to truly understand the cost and the confidence that we gain at each level of test. There are many mantras that say we need more unit tests. But the question is why? Also the question is why do we always think in terms of integrations and seleniums? Confidence Over Time Martin Fowler, presented a great post on the Test Pyramid . In the post, he speaks about he need to having a balanced testing suite, and the trade-offs for each type. So the question still remains: which do we use? To answer this question, let's look at this from the perspective of the devel...

Often, when we think about our release process, we think primarily about the core product we deliver.  We ponder on how we can deliver better, faster, and with quality. There is, however, other parts of the software business that need to improve as well. There are other deliverables that, when we take the time to understand and focus on, we can gain greater insight into confidence culture wide. Sales When we think about the Sales organization, how can we implement Continuous Confidence there? Well, notice that for a certain period of time, we spend a lot of time and effort training people to sell the product. We spend a lot of time focusing on how to answer questions and how to pitch the deal. May we in addition focus on the confidence of the deal? Do we focus on them having confidence and a solid relationship? Do we take the time to care about the other person? Do we genuinely seek to know why we didn't win the sale? Often sales fail because people don't have confidenc...

Continuous Confidence is an ideology, a process that changes and is different in each development shop. When we want to implement it, we need to flip our thoughts around and start by asking the question: Why do we release code? The Essential Question This question is the most essential question we can answer. This will ultimately drive our implementation. Some might answer this question with the easy route: to make money! Some might answer this with the statement: to rule the world! Whatever the answer is to this question, be honest and fair and precise. Answering this will show where the motivation is. Whatever the response, we find that we cannot do this without the customer. We find we cannot do a release without other departments and roles. We find that, in reality, we are far more impactful on others more so than just the development team. So we start to ask more questions from our experiences. How often should we release? Each software that is created is different. We mig...

There is a lot of talk on the Internet around Continuous Integration. There is a lot of push toward constant releases. Always releasing small portions, maybe even continuous deployment. There is a lot of tout people as towards releasing code 1000 times a day! People are in awe, and then, we try to reproduce the effects. We try to follow the example and release 100 times a day. So, we need to stop the insanity! Not Releasing Is Ok! We need to stop and look at our business model. We need to understand our requirements. We need to understand our customer base. Ask questions. Are we comfortable releasing code daily? Do our customers want us to release daily? Is our code base stable enough to release in a manner unnoticed by our customer? Does our product lifecycle make sense to release in this fashion? These questions are used to help guide discussions. We need to talk about and consider the importance of the confidence of the product. Without Confidence, Why Release? If we are not ...

Ok. We all have the same dilemma when we test. We all ask the same questions: what should we test?  If you want the greatest confidence in your code, test your changes. When we make changes to legacy code, test those changes. When we make new code, test those changes. The Status Quo When we have a legacy system, we have a hard time figuring out how to change it without breaking. We are not real confident in how our code will effect the system as a whole. So we move on and hand it off to QA saying "test it all!"  How To Kill Productivity If we want to kill our productivity in three small, short words, then by all means we should say to QA  "test it all". See when QA hears that, they will test it all. They will test things we never touched. They will test it in ways we never thought possible. They will take forever to do it. So there has to be a better way. There is: test the changes. What Do We Mean, Test the Changes When we test the changes, we assume the...

When we write code and we deliver software to people, there will always be mistakes made. It's OK. If we feel the pressures of delivery and performance to our customers, then we need to do something different. Slow Down When "fires" erupt in our daily software life, making a giant enormous deal out of it, is actually the opposite thing we should do. Be grateful for the moment to learn. Thank those who have brought it to our attention. Allow the fire to be dealt with in a methodical and smooth approach. Smooth is fast. Also, smooth is confidence. Identify The Source When we try to identify the source of the fire, we should not name the developer as the source. We name the real source: the code, or the process that had a problem. See, when we finally understand where the issue is, we can finally make the needed change, however, together with that, we can also inform the customer what the issue is and how we plan to fix it going forward. If we don't do hat, then ...

when we write software, often we talk about the customer. We talk about their interactions with our software. We talk heir behavior and we ultimately decide what is going to be best for the customer. So in all of that talking, however, do we ever talk about what will make he customer trust us the most? Trust: The Measuring Stick When we start talking about a new feature and he performance of the application, we start geeking out! We  are developers, it's what we live for. Never, in the history of the world have we ever seen so many people gathered around a small little box talking in a foreign language and having such a great time. This is coding! This is awesome! Yet, at the other end of the spectrum, at the end of he development process, the customer is waiting. They await he next code release, and if we have not done our jobs right, they dread the next release. See, if we were all about the feature, or the performance, and didn't bother to release seamlessly, the cus...

When we look at what we are selling and what we do as a company, have we ever noticed that customers just are not as loyal as they used to be? Have we ever seen people leave when our site gets unstable? Why is that? Confidence The answer is quite simple: they don't have a lot of confidence in us. When someone gets angry and sees that there are some problems that just went out, people start to lose confidence in our product. It's not that we did anything bad or vindictive, but it is the natural cause and effect. So as we continue to release code, sometimes, to gain trust back, a moratorium is a good thing. Stop. Let the customer breathe. Allow people to see and trust you once again. Work with them to help gain that trust once more. It's All About Relationships When we start servicing a customer, we are starting a relationship with them. When we don't care for anything but their money, we make sure that there is very little if any trust there. Once things get toug...

At the very heart of Continuous Integration and Continuous Confidence, is the automated test. Why do we write a test? If for whatever reason we write a test just to pass, then we are doing it wrong. Let's explore that for a moment. Superfluous Testing When we write a test, do we test a getter and a setter? If we answer yes to this question, then let's ask: why, followed by do we have to? Now, these questions need to be answered by each organization, as each requirement is different, but we want to look at something different. We want confidence in our code, and in our release. If we were to draw a line from left to right and say on the left is 0 confidence and on the right is 100% confidence, how much confidence do you gain by testing that a better or a setter worked? Put another way, would we ever write a test that simply returned true? Now, there are some getters or setters that have logic in them, points where decisions are made and a code bran is taken. That's o...

Quality Assurance has always been a conundrum since the day the first QA engineer came into existence.  We can imagine how they came to be. In The Beginning A bunch of developers were writing code. They all realized hat they needed to have some tests written and so they elected a specific developer to just write tests on their code so the others could keep going. Over time that poor developer lost their perspective and their knowledge of the code base. Once an engineer finished their work, they would pitch it over the wall and expect a report back by the end of the day, even if the end of the day was in 15 minutes. So over time, QA became viewed as a substandard position, a dead end for a career. Recently, that has started to change. Misunderstandings We want to focus for a moment on what QA is really doing, and perhaps we can find a better definition and role. Remember, in Continuous Confidence, we want to use people to their full potential and thus we will redefine what pe...

The Developer's role within Continuous Confidence, is a little bit different. The traditional look on what a software engineer does, brings us back to writing code and then having others verify it works. This traditional view is flawed and actually produces weakened confidence over time. Let's explore why. One Time Evaluation Over the last decade or so, new methodologies have taken form. Test Driven Development (TDD). Code coverage. Behavior Driven Development (BDD). These forms have been around for a while, but only in the last 10 years have they really started to gain traction. When the traditional model focusses on someone other than the developer verifying the build, it is verified in a single instance of time. That single instance will never be repeated, and so, naturally, bugs will come. Using the afore mentioned methodologies, we start to place the responsibility of testing on the shoulders of the developers. Abbreviated Timeline We have made a serious mistake. T...

When we first started down the Automated Build and Testing scene, we became very addicted to that wonderful thing: the green build. We installed tools like Jenkins or later GoCD or even rolled our own, but we loved the power of that green build. We said: ship it! Let's get that code out in the wild! Look it passed all the tests! Why Mythical? So we shipped that code and something broke. People in our company didn't like us too much. Whether it was customer service or whether it was sales for us breaking stuff in the middle of their demo. But all we could say was: it was green! So, evidently, when you have 100 tests on a 200,000 line code base, there just isn't a lot of confidence that something isn't going to break. So what does a green build really mean? That is why it is mythical. We have no real "green build" if the tests are not effective and have the "area coverage".  When we write tests, we need to cover areas of the code, not code cover...

When we look at the Continuous Integration model and the reason it has been so successful, one might think it is because we release more often and our commits are small. One might also think, the boredom of constant releasing is powerful as well. As we examine further, as well, there are times when in reality, even when practicing Continuous Integration, we are asked to stop releasing so often.  We have moments when the rest of the company or when the customer asks for us to not release as often. Why is that? Confidence. Trust. Let's be honest, we break stuff...often. It happens. It's ok. But when we lost trust, we need to gain it back. Determine The Cause The first step to Continuous Confidence is the council. Pull in people from every group and department of the company. Don't pull in managers. We can invite them, but we need people who are in the trenches and who know where the pain is. Be productive. Keep the council moving, but council  about why people don't w...

Have we ever asked ourselves "Why are we doing continuous integration/delivery?" A weird question right? It Makes Sense Almost every shop we know of says we need to go faster and deliver more content at higher quality. We grind and toil to make that happen. We learn that we can automatically build and test code. Tests pass and we are good! We are happy. We pass the hard work over to QA and expect them to give our code the final blessing. Sometimes, let's be honest MOST of the time, it comes back and we have to work on something to fix it. The final OK is given and we send it to DevOps/IT and it is released to the wild. We are proud. We got it out. And yet, there is sometimes a sense of impending doom. A little voice in the back of our heads saying: just wait. Or saying "What did we miss?" If It Makes Sense, Why The Doubt? So if all the effort to do continuous integration is worth it then why do we have doubts? Why do we find doubts in other parts ...