Where does innovation come from?

I have been involved in a rather interesting discussion this week with an individual I will refer to as Mr. Higherup, he’s one of the higher-ups here at the company (I refer to him in this way only because I haven’t asked his permission to use his name in this blog post).  There was a plea for more innovation to come from our company, and that sparked a debate between the two of us on ways to structure the company in such a way to allow for innovation to occur.

As I reflect on this topic, many of the arguments we have made can easily be applied to testing.  Exploratory Testing to me is a very innovative process.  The process of allowing your last step to  influence the next requires a steady stream of ideas on where the next step should be.  Below is my initial response to the request for the company to be more innovative:

Mr. Higherup,

I see a few cultural obstacles that need to be broken down for ideas to really flow out of this company.

The software we write exists to solve problems. In the case of [the division of the company in which I work], Mr. So-and-so and his companions noticed a problem in the justice sector, and they came up with an idea for a solution to solve the problem. This idea came from people that were very intimately connected to the problem that needed to be solved. I imagine that many of the ideas that this company currently monetizes have roots from ideas of people connected to different industries.

The first obstacle I see is that we don’t know what problems exist out there. A large portion of [this division], and by abstraction [the company as a whole], is made up of technical people that create and implement these ideas, but have no connection to the industries or problems we are solving. The members of the company that do have that connection to the industry in general don’t understand how the technology could solve the problems they are facing as well as the technical side of the company.

Steven Johnson talks about the coffee shop being the idea center of the Enlightenment because of the exchange of information and ideas that happen in that space leading to new and exciting ideas. I believe that the ideas you seek will breed in an environment where an understanding of existing problems mix with an understanding of technology solutions. The challenge is to create that environment.

I don’t believe that replacing [our e-mail system] or implementing corporate social networking will create that environment.

The next obstacle in creating these ideas is having time available to devote to creating ideas.. We don’t necessarily need to go the direction of Google’s 20% time or Atlassian’s model of quarterly idea days, but we need time to be able to look up from the day-to-day noise and talk about the problems that we could be solving. This isn’t going to happen passively, or by writing a couple hundred words on a blog. We all have dueling demands on our time, and if ideas are important to the company, then time needs to be dedicated to creating them.

I will offer one brief anecdote on this topic. One of our developers at [this division] just got back from a conference where he had the ability to sit and talk with people in the industry about some of the problems they are facing. He came back today with an idea that could be the killer app in the justice sector. He had time to mix his ideas with others with very positive results.

I will close with one last aspect that I see. For people to want to invest their ideas into the company, many people want to feel like the company is invested in them. [In this environment this is especially important].  Missteps in this area can lead to significant morale problems that will stomp out any desire to invest in a company that is not invested in its employees.

The take away message, it’s one thing to ask for ideas, it’s another to create an environment where ideas can be created. Hopefully this feedback is useful.

-Wade-

The main points I made in this initial response are that employees have basic needs to be innovative, namely an understanding of a problem, time and resources to focus on that problem, and a desire to allocate time and resources to solve that problem.  I believe these same needs apply to good software testing (for some definition of good).

Understanding the Problem

I had the opportunity last month to spend some time in Toronto at the Toronto Workshop on Software Testing (TWST).  At TWST we had some great conversations about identifying stakeholders and our ethical responsibility to these stakeholders as software testing professionals.  We started a mindmap on a flipchart to identify all of the potential stakeholders of a project and very quickly filled an entire page with probably 50+ individuals and groups and ideas were still flowing from the group as we moved on.  Each of these stakeholders represented another potential person our projects can impact.  Each of these people represent a potentially different problem set that we are facing as we develop and test software.  The trick here is to identify the key stakeholders in each specific project, understand what problems they have that need to be solved, then you can use that knowledge to guide your testing on that project.  New projects have new stakeholders and new problems that should be guiding testing.

Requirements and other documentation can be used heuristically to develop a rudimentary understanding of a problem while testing.  I often find that brief conversations with customers, managers, executives, marketing, support, and other important stakeholders can shed a lot of light on a project.

Most of my posts on this blog have covered this topic, and even though I have learned a lot on the topic since those initial posts, I will not cover it in much more detail in this post.  However, there is one important stakeholder that I have never before mentioned on this blog, that is yourself.  For me, it is critical to understand how specific projects and tasks fit into and affect my goals.  Earlier this year I packed my family up and moved across the country to start a new position in part because I had a firm understanding of what my goals were and the projects I was working on at the time did not fit into the goals I had set for myself.  I believe that we as individuals are the most important stakeholders in any project and we need to understand that.

Time and Resources to Focus on the Problem

Within certain frames and contexts, especially in business and software projects, time and resources are limited.  In these contexts we have to make decisions on where we allocate the resources and time that are available.  Often these are allocated based on our understanding of the problem.  In testing, resources are allocated to the portions of the software where someone understands there to be the most significant risks to solving the problem we are facing.

The problem with this method of allocation is that it only as good as our understanding of the actual problem.  With a perfect understanding of the problem, we can attack the most important items in order until the resources are spent at which point we call the project ‘done’ and move on to the next.  This perfect understanding is only possible in the most trivial of circumstances however.  In the more real-world and complex cases, our understanding is often flawed.  Going back to the map we identified at TWST, understanding the full problem set of those 50+ stakeholders is incredibly complex.  Given that as the situation, we can either choose to dedicate resources to developing a better understanding of the problem (which will likely lead to fewer resources and an understanding that is still flawed), or find a way to allocate resources in such a way to compensate for an imperfect understanding.

In my team, we use requirements and a brief regression checklist to drive most of our testing each sprint.  These are the things that we have identified as areas of the software that present the highest risks to reaching the goals of the company.  However, on each sprint I try to make sure that we have at least a few hours, if not a full day at the end, and a day or two at the beginning or each sprint to focus on areas that the testers feel need some attention.  I rely on the intuition and expertise of the individual testers to define what is tested during this time.  This time has helped us identify and track down quite a few important bugs that would not be found by only focusing on our flawed understanding of what certain stakeholders care about.

Desire to Allocate Resources

Understanding and have resources available to solve a problem don’t matter in the slightest if you don’t care enough to actually do anything about it.  In my letter to Mr. Higherup, I mentioned the responsibility of the company to manage itself in a way that promotes the desire of the employees to help the company grow.  This same dynamic exists in testing.

This is a really interesting point that I haven’t really heard much about in this context, but I feel is incredibly important.  I have read quite a bit lately on motivation and I think that it has such a large effect on our products that it should be talked about far more than it is.

In the TWiST podcast that just came out today I was part of a conversation with Matt Heusser, Michael Larsen, and Jon Bach on the topic of motivation.  In this conversation Jon gave an interesting bit of insight into how relationships affect his motivation to allocate his currently limited resources.  He says, “When I look at my todo list, it’s not a matter of, ‘oh look at all of this stuff to do’.  I think about the people that I’m serving.  In fact that often helps me to remember my todo list!  To just think about the people that I work with and go, ‘Oh. OK.  Probably isn’t the highest priority, but it is most motivating to me right now to get back to Thomas about something I told him I would get back to him about.’  I still have another 2 weeks but right now I feel like ‘hey he helped me out today.  I owe him, I want to do that now.’  That motivates me.  That’s part of respect and reputation I suppose, but it’s really a sense of being a part of what I think is cool.”

Right before that Jon defined cool as inspiring.  What this means to me is that he is, and by abstraction many of us are (including myself), inspired to do work based on relationships.  How does this relate to testing?  Several ways.

I often see the world from a Test Manager bias, because I function as a Test Manager.  The way that I see this connecting to testing is that I have a responsibility to create a relationship with my team.  Cem Kaner has stated, “Managers get work done through other people.”  There are many different styles of management, but the way that I choose to manage my team is by creating a relationship of mutual respect and leveraging that respect to get the job done.  In the time since we recorded TWiST, I have seen vast improvements in the team of testers I work with because I finally took a more proactive approach to investing in the relationships I have with my team.  I now understand them much better, they understand me, and we have a stronger relationship of mutual respect.

This respect has led to some great improvements on my team.  I see more initiative.  Based on the very positive response from customers and management over the last few months our testing has improved.  I have more trust in my team to get a job done when asked.  My team is doing a great job at the work I am asking them to do, and I believe that is a direct benefit of building relationships, and therefore desires to accomplish the work we are here to do.

I don’t yet know what the outcome of my conversation with Mr. Higherup will be.  His mind could be opened and significant changes could come, or he may not even respond again.  I do know that in this process I made some great connections and understand my world a little better.  Now that you ahve read this I hope you understand your world better as well.

One Response to this post.

  1. Posted by Tim Valenta on 28.10.11 at 12:20 pm

    (This got a little longer than I intended, but your commentary provoked some good thoughts.)

    Desire has been something weighing heavily on my mind in the last year. The company I’ve worked for was small, had lots of interesting software ideas, was immensely flexible in letting me manage myself and the specifics of the tools I used to program for them, and the owner gave me a lot of influence on making decisions for purchasing computing hardware and server technologies to run our project.

    Right at the beginning, I was learning at a mile a minute, and I was gladly taking my ideas home with me and would continue to explore solutions and familiarize myself with the problems that we needed to solve.

    But after a year, my motivation for working on the project started to drop off a cliff. There were lots of influences, I think, but in terms of my relationship to the company, I felt that two simple truths were preventing me from caring about the work I was doing anymore:

    1- As that year had progressed, all of the freedoms I had to tackle the problems meant less and less, as the company owner kept pulling me off of a project that we had previously accepted as “top priority” or generally “most important”, to stick me on something totally new and infantile in its current state as a business idea. I could never actually complete the jobs I was being put on, because without fail, there would be a new shiny idea that management would chase after before I could complete my current tasks. I was demotivated out of my mind, because in the (now) two years I had worked there, I had nothing to point to as a product of my effort.

    2- People are different from one another. (I know, shocker, right?) The things that motivated me had almost no overlap whatsoever with the things that motivated my boss. I’m not usually the idealistic type, but my motivation each day was to sit in front of a computer and be able to solve problems in a way that had so much grace that I could wow a fellow programmer. That doesn’t come easy, so I fully expected to work and stretch my brain to innovate new ideas for the purpose of the solution. My boss on the other hand was about marketing and selling ideas or solutions that we hadn’t actually designed yet, but that he was certain he had figured out. He talked about money and giving me a cut of what revenues the product would pull in if we finished it. Money motivates a lot of people to some degree, but he really couldn’t see past his own financial goals, and consequently could only offer me fame and fortune for a hypothetical product. Just thinking about it you could see how excited he was, but when I thought about it, I wanted to shut up about the nebulous what-if dreams, and start figuring out solutions. Each time he would come up with next week’s great idea, he would pitch the same old financial cut and suddenly expect there to be a working product.

    By the time I finally quit, he had a hard time understanding why I was “doing that to him”. I learned that there was virtually no overlap between what motivated us as human beings, and while I felt I understood his motivation for moving forward, he really couldn’t wrap his head around mine. But during all of that, I would still go home and program like a fool for other great ideas I had, outside of the context of my job. I could just never bring myself to work on something for the company that was developing a huge track record for demotivating me. It was tarnished after that, and I could never lift mysel back onto my feet for those projects.

    Some of my demotivation comes from personality flaws (sometimes I bore easily, for example), but the better a person can come to understanding what makes them tick, and understanding what makes those around them tick, the better those people can support each other and do great things.

Respond to this post