Continuous Confidence in Software Development

One of the hardest parts of Continuous Confidence is understanding what brings confidence to our testing. We strive, at times, to find that proper level of confidence. Too often, we talk about Code Coverage. We want to make sure that we are touching everything, and it lifts our confidence, falsely, to the same level of as our code coverage. If we have 60% code coverage, then we have 60% confidence. The issue, however, is after a while, we start to realize, once we get bugs, that we aren't really that confident in our code and then we dismiss testing almost as a chore instead of a tool. So how do we get more confidence? Well, the answer is simple, the process a little more difficult: write quality tests. Area Coverage The first thing we need to do, is understand what we are testing. We should be considering Area Coverage instead of Code Coverage. Area Coverage looks at the areas of the application that have tests. There aren't a whole lot of tools out there that will help yo...

The hardest question to answer, it seems, is what kind of test should we be writing? Within the industry today, we have a preconceived notion that integration tests are the best. When we have conversations with developers about what needs to be tested they like to say: test all of it. When we are hesitant that we may have broken something we turn to an integration or selenium test to verify it. This is a problem. We need to truly understand the cost and the confidence that we gain at each level of test. There are many mantras that say we need more unit tests. But the question is why? Also the question is why do we always think in terms of integrations and seleniums? Confidence Over Time Martin Fowler, presented a great post on the Test Pyramid . In the post, he speaks about he need to having a balanced testing suite, and the trade-offs for each type. So the question still remains: which do we use? To answer this question, let's look at this from the perspective of the devel...

Ok. We all have the same dilemma when we test. We all ask the same questions: what should we test?  If you want the greatest confidence in your code, test your changes. When we make changes to legacy code, test those changes. When we make new code, test those changes. The Status Quo When we have a legacy system, we have a hard time figuring out how to change it without breaking. We are not real confident in how our code will effect the system as a whole. So we move on and hand it off to QA saying "test it all!"  How To Kill Productivity If we want to kill our productivity in three small, short words, then by all means we should say to QA  "test it all". See when QA hears that, they will test it all. They will test things we never touched. They will test it in ways we never thought possible. They will take forever to do it. So there has to be a better way. There is: test the changes. What Do We Mean, Test the Changes When we test the changes, we assume the...

At the very heart of Continuous Integration and Continuous Confidence, is the automated test. Why do we write a test? If for whatever reason we write a test just to pass, then we are doing it wrong. Let's explore that for a moment. Superfluous Testing When we write a test, do we test a getter and a setter? If we answer yes to this question, then let's ask: why, followed by do we have to? Now, these questions need to be answered by each organization, as each requirement is different, but we want to look at something different. We want confidence in our code, and in our release. If we were to draw a line from left to right and say on the left is 0 confidence and on the right is 100% confidence, how much confidence do you gain by testing that a better or a setter worked? Put another way, would we ever write a test that simply returned true? Now, there are some getters or setters that have logic in them, points where decisions are made and a code bran is taken. That's o...