Thursday, March 31, 2022

Large Organizations Affect or Effect Change?

Do you remember when to use the word ‘affect’ and when to use the word ‘effect’? ‘Affect’ is normally a verb and means to cause change, whereas ‘effect’ is normally a noun and means the result of the change. You got it! Easy, right? 

Let’s expand that thought a little. How about ‘affecting change’ and ‘effecting change’? Using ‘effect’ as a verb jumps out at you, doesn’t it? Let’s go to the all-knowing web, where we find, affecting change and effecting change are not the same. ‘Affecting change’ means the change was already in progress when an external force acted on it. ‘Effecting change’ means that an external force caused the change. 

This subtle but important difference is at the core of this blog post and helps explain why change in Scrum-based organizations is different than change in traditional, functional organizations. Simply put, one affects change in Scrum-based organizations whereas one effects change in traditional organizations.

Traditional Organizations

Many of today’s traditional organizations can trace their roots back to the explosive growth of the automotive industry and the days of Alfred Sloan around the 1920s. We still have many of the vestiges from those early organizations: functional organization, large product development cycles and formal processes. Generally, traditional businesses organize functionally. They have sales, marketing, operations, manufacturing, engineering and quality assurance departments. The concept is to bring together similar expertise, areas of responsibilities and hierarchical leaders to bring about highly efficient, effective and consistent decisions and results.

Traditional functional groups develop processes and decision frameworks to engage other functions. The cross-functional lifecycle processes, like a Product Development Lifecycle, are used to make coordinated, consistent agreements between functional groups. As functional organizations grow, they develop processes and decision frameworks to work consistently within their function. They create engineering lifecycles, engineering program management, and specialized functions within their organization, for example software engineering, hardware engineering and quality engineering. Within these specialized functions, they create processes and decision frameworks, for example, design reviews, code reviews, and product build processes.

As these processes are developed and used, they become well-known and common across the traditional business and within the functional organization. The process effectiveness is constantly evaluated and, when necessary, follows a modification process. Eventually, some processes become codified. Some processes even become standardized with an external committee who monitors for compliance.

Why are these processes needed? Simply put, they are needed for consistency of execution and to facilitate commitments between functions and groups. Traditional organizations develop products that are time phased in decision making between the functions.

For example, Product Marketing gains approval for funding and requirements at a commitment phase. Engineering takes the lead to develop the product. Product Marketing engages to price and position the resulting product. Manufacturing gears up production. Corporate Marketing organizes the product launch. Sales does sales/channel training and takes orders. Support fixes what breaks.

It is unclear if traditional organizations created large product development efforts or conversely, large product development efforts necessitated traditional, functional organizations. My guess is that they evolved together over time. Large development efforts or big batch releases often use a Waterfall development process. With big batch releases, we build up large amounts of momentum, or work-in-process (WIP) and use a Product Development Lifecycle process to keep functional organizations aligned.

This creates a constant fly-wheel of overlapping programs under constant development as the organization delivers new products to customers.

An unintended consequence of functional organization is a lack of transparency. As a functional organization develops its processes and codifies them, the deep understanding remains within the functional organization. The details of how a functional organization operates can be obscured behind this ‘process jargon’. The functional leadership can use their organizations’ process jargon to provide updates, thus undermining communication of information. This is a key reason why program managers, as arbiters of organizational jargon, are used to effectively translate to increase communication and visibility between functions.

Scrum-based Organizations.

Scrum-based organizations are relatively new. Scrum first jumped on to the scene in 1986 when proposed in a Harvard Business Review article. Then again, when the Scrum development process was published in 1995 by Sutherland and Schwaber at OOPSLA (Object-Oriented Programming, Systems, Languages & Applications) conference in Austin, Texas. The first Scrum book was published in 2001 and ScrumAlliance.Org formed in 2009. The industry really has yet to perfect how to build a large business based upon Scrum. Given the relative success of Amazon, Google and Microsoft, some large experiments are underway.

At the core of Scrum are transparency, small batches of work, constant improvements and constant change. All desired future work is kept in a visible, prioritized and incrementally/well-groomed backlog. All work progress and results are equally visible and inspectable.

The work batch size is kept very small where increments are completed by a team within 2 to 4 weeks, with the preferred time duration being 2 weeks. Each time duration, or sprint, has a retrospective at the end where the team seeks improvements to employ for the next sprint. Since the duration is short and the batches are small, the teams can wrap-up work in short order and pick up new work quickly.

This doesn’t mean that Scrum organizations are devoid of strategy, architecture, documentation, functional specialization, and processes. They have these too, however, what is at the core of a Scrum organization is small batches and constant change. The amount of work-in process, the amount of time from executive decision to initial results, and the amount of complexity is relatively lower.

Did you happen to notice that I didn’t say anything about the function using Scrum? Scrum isn’t just for product development. There are company boards of directors who use Scrum, as do marketing departments and sales departments. So, one could imagine a Scrum organization from top-to-bottom, across all functions.

Given the small batches, bringing each of those to market quickly means communication must happen frequently. Alignment of priorities happens transparently with dependencies and work ordering factored into the backlog grooming and planning. To make this happen, you can see that functional organization processes take a back seat to cross-functional work output. Transparency is required from team to team, function to function and lower-levels to upper-levels.

The small batches are key at keeping organizational risk minimal, since any loss due to failure or redirection is relatively small. Small batches require frequent communication between functions to keep the flow of value to customers going.

How change happens in traditional organizations.

Traditional organizations, realistically, don’t change much. If you look at large high-tech companies, we can all list companies who failed to see a technology shift and closed shop. Large high-tech companies aren’t the only ones to resist change. Many companies are struggling today with the shift to online shopping or Uber-ization of their industry.

Effective change happens in traditional organizations only when there’s strong executive support and concerted focus on making the change happen. For example, some companies have a Business Transformation organization that enables cross-organizational change management. The change process has been formalized to consistently and effectively engage with all organizations to make change happen. Once change has happened, the traditional organizations tend to standardize on the resulting adjusted process. To force change again, a new change effort must be undertaken.

By looking at how a traditional organization is built, we can see why a concerted effort is needed. There is a large amount of hysteresis (the dependence of the state of a system on its history) built into the functional organization. The cross-functional and internal processes being codified means that carefully applied efforts must be taken before modification can happen. The large batch size of development programs magnifies any impacts due to change. Lastly, with overlapping development programs, there never seems to be a good time with low risk for learning and improvement.

One can see why strong executive support with a specialized transformation organization is required to effect change in a traditional organization. Transformation effects change in a traditional organization.

How change happens in Scrum-based Organizations.

Scrum organizations effectively have change built into their DNA. With transparency and small batch sizes, organizations have a low-risk threshold to adjustments and the basic demand that change is required. Change is simply done by refining the backlog, adjusting the priorities, etc.

There’s no longer a need for an external force. The internal mechanism for change always exists. Thus, one affects change in a Scrum organization.

If Scrum organizations have strategy, architecture, documentation, functional specialization, and processes as well, why don’t they suffer from the same change problems as well?

Changing the cross-functional processes will be complex and will likely require the same senior executive support and an external function to drive change. Scrum organizations do have the benefit of transparency, small batches and no large, overlapping development programs that lowers the risk of change. The level of executive sponsorship for change and the risk of change can be taken on at lower levels of the organization.

Traditional Transformation and Scrum Organizations, can they co-exist?

Transformation for a traditional organization often requires an external transformation organization with senior executive sponsorship to take on the risk management of the resulting change. To do this, the transformation organization will create a support structure to make visible the plans and risk to mitigate impacts. As needed, the transformation organization will work cross-functionally and at all levels within the organization to achieve successful change. The transformation organization becomes a temporary scaffolding to support the traditional organization until the change is done and accepted.

In a Scrum organization, the visible product backlog with the Product Owner as the sole interface to a team, is key to keeping the complex system of small batches and teams functioning. If another, albeit temporary, transitional scaffolding is put into place for change, the Scrum organization will fail to function as teams will effectively have two or more PO’s.

Transformation takes place when the organization’s strategy necessitates change and new strategic steps, and theme outcomes are added with business outcomes clearly required. (Note: recall that strategic steps are achieved with 1 to 3 years of effort and are retired as themes/epics are achieved.) Themes can be added to facilitate change at a lower level and at a quicker pace. Even quicker and lower, epics and stories can be added to the backlog.

According to how the backlog priorities are set, Scrum organizations take on change. Hence, if a new shift in how a product is brought to market, with the right strategy, strategic steps, themes and epics, along with the change in priorities, the Scrum organization will respond within a few sprint cycles. The amount of disruption to existing work will be small. The impact to previously high-priority pillars, themes and, possibly, epics will be instantaneous. Caution is required when making quick changes at such a high level.

Transformation needs to be ‘groomed’ into the backlog just as other customer value is groomed. As the strategy changes, as business needs come to light, as competitors respond and as we innovate, transformation is integrated with planning to utilize the change mechanisms natively inherent in Scrum organizations.

Transformation affects change in Scrum organizations.