Innovations in Change Leadership

We enter into the high tech industry wanting to be that person with an idea that changes the industry. Few have had such ideas. The industry has been changed by such ideas. Somehow, even for those who have changed the industry, we create organizations, products, processes and roles that become resistant to change.

During my career, I've held various positions where I have been responsible for leading change. A few times, my team has been on the leading edge and we were operating in a future, potential state. Change was easy because we had no past. Getting others to see our potential was the difficult part. At other times, the business was forced to change. A near death experience is a good motivator. We reacted to correct for not changing sooner. Even then, change was difficult.

The drive for change was and is something that comes naturally to me. Early in my career, I felt that logical arguments wrapped with a little passion were sufficient to lead change. But, when asked how I led change, I could not describe it well. Unconscious-competent some would say. Reading studies on how leaders affect change, helped to explain methods and effects but they failed to create a workable model to shift an entire organization from one operating model to another. I read and practiced lean software and agile development methods. I read up on situational leadership by Hersey and Blanchard. I kept seeking better constructs to plan and affect change across a large engineering organization.

I came to a company to lead change. The executives wanted to pivot the organization from years of long-cycle development processes towards faster time-to-market processes for new enterprise products. This was important as we needed new, constantly improving products for emerging markets to return us to profitable growth. Previous product efforts missed their market windows. There were other issues facing engineering: focus on product features and not on customers, focus on waterfall checkpoints with lengthy periods for architecture and design, separated implementation from testing, multiple software control/build processes, and weak engineering skills in modern development, testing and tooling practices. The engineering organization was highly resistant to change after various predecessors and projects failed. Fortunately, a new management team was already in position was demanding change.

Building the case for change was easy as one of my product projects had already been established, the market conditions were right, and the company need was clear. The difficulty was, as a new leader, I had to use the people that I had while avoiding their passive-aggressive behavior. I needed to push change along multiple dimensions in parallel for a large, geographically dispersed organization. My dilemma was how to do this?

I was concerned with potential passive-aggressive behavior along two dimensions. The first being passive-aggressive behaviors vertically within the organization and, secondly, horizontally across multiple components of the product. I decided to orient the organization along four concepts: product, architecture, process and people. I would use Hersey-Blanchard situational leadership within of these areas to cultivate new leaders and methods. I selected Scrum for a development method due to my previous experience and readily available training. I made an appeal for change due to that fundamental reason why we joined high tech, to change the industry. I made the cause for change personal to each individual.

My innovation was to create three virtual organizations where the leader of each reported directly to me for product, architecture and process. A virtual organization leader had dotted line relationships to their organization but not direct management control. I created a fourth organization with people managers who had everyone else reporting to them. A leader was responsible for their area. This prevented any one of them overpowering another without me being notified. I selected a leader for each organization who had been open to change and had a strong understanding of their respective area. This organizational model gave me visibility into issues as those issues crossed between these virtual organizational boundaries. The issues escalated to me created teaching moments. This structure also mirrored Scrum roles.

The second step was to refactor the existing organization into the new structure while leaving the development process untouched. This allowed the organization to adjust while doing work as they had previously done it. Meanwhile, I started coaching of each leader on their roles and new expectations. I described to the engineers how the new organization would function in the future. Meanwhile, we kept current projects on track.

The third step was training. On product, training was focused at the use of customer stories with goals and definitions of done, as well as, release planning with shorter horizons and smaller increments. On architecture with the lead architect, we worked on a redesigned, flexible architecture. On process, I cultivated the notions of continuous integration, use of modern build and automated testing tools, test driven design, and early devops concepts. The goal was creation of a single repository, build, test and deployment model for the product. Lastly, I worked with the people managers on training (Scrum, virtualization, automation) and self-managing-team skill development.

The next step was practice. The product owners began to state requirements and SLAs as user-stories. The architects started to redesign the current architecture to allow refactoring and better incremental, independent changes. The process team began to pull together a single tool set using open source tools and modern methods. The next release that was heading into planning signed up for the first attempt at Scrum. The approach taken by the team was actually closer to spiral development while using Scrum terms. User-stories were closer to tasks than they were user-oriented. The back end was heavily focused on final integration and testing. Meanwhile, I continued to push product owners closer to user-stories, architects closer to enabling refactoring and independent development by component, and teams towards Scrum and self-management. The process team needed the least encouragement because they were building tools for future use and had already moved into the agile development mindset. Releases continued to flow out every 4 to 5 months.

I had the teams working on new releases incrementally move components on to the new tools and to automate aspects of builds and tests. This required shifting repositories and changing tools. It required finding ways to build things that were never automated. Meanwhile, we kept raising the quality of the definition of 'done' with each release.

The last step was to do a hard shift to only user-stories with a clear goal and definition of done, to being 'done within a sprint' and 'done-done' (or customer shippable quality). This created tremendous tension with the product owners and engineers as the whole organization had to shift from fast cycle, mini waterfall development to true Scrum development. The teams and product owners pushed back. They claimed it wasn't possible. I had the visibility to influence and coach the low-level product owners. I worked with the people managers and teams directly when needed. The architecture and the process were in place to enable this critical transition. The teams did make the switch. We moved into true agile development with user stories being started and done to customer shippable quality within a sprint. Releases continued to flow out under the new process

The quality of the product from our customers’ perspective improved with each release. The teams were able to deliver both incremental features and velocity improvements with each sprint. The product owners were able to remain focused on customers’ needs and stakeholder demands. Teams were able to show stakeholders feature demonstrations earlier. Sets of teams could work independently on different work streams than were integrated and released when done. Back-end testing moved from 3 sprints to 1 clean-up sprint with the number of serious bugs found during testing dramatically lowered. For a modest investment, the team created differentiated product features ahead of the industry leaders. From a change management perspective, the organization became fully bought into Scrum, its methods and its value. They pushed themselves to improve with each sprint cycle.

Looking back, there are things that I would change on my approach. For example, I should have been more demanding of the product owners to move to user-stories earlier. This would have provided a stronger motivation to break from the mini-waterfall development within a sprint to ‘potentially shippable’ user value being done for each story multiple times within a sprint.