7

Release Trains Day-to-Day

You have an ART; now what? This is where the fun begins. There are so many things that happen every day in an ART that it is easy to quickly get overwhelmed. How to keep everything aligned and who is responsible for managing the everyday workings of the ART can be a challenge. The Troika, which we discussed in the previous chapter, plays an important part. In this chapter, we will look at the day-to-day activities that occur in the ART, as well as activities and actions for key ART roles, including RTE, Product Management, and the System Architect.

In this chapter, we’ll cover the following topics:

• Synchronization and Cadence
• The Continuous Delivery Pipeline (CDP)
• Tooling
• Day-to-day – Product Management
• Day-to-day – System Architect
• Day-to-day – the Release Train Engineer
• The Innovation and Planning (IP) Iteration

Synchronization and Cadence

One of the keys to ART success is synchronization and cadence. If you have ever taken a SAFe® class, you will have heard these two items discussed many times. These two aspects play a pivotal role in aligning teams and facilitating communication, contributing significantly to the overall success of the ART.

As a Coach, one crucial responsibility is to assist the RTE in ensuring that all teams within the ART schedule their events appropriately. By coordinating events, teams can effectively resolve dependencies and promote cross-team collaboration.

There are several benefits we receive from synchronization and cadence:

• Improved communication: Synchronization and cadence enable teams to work together more effectively by establishing a consistent rhythm for planning, development, and delivery. This promotes better communication and fosters cross-team collaboration, leading to a more cohesive working environment.
• Predictability and visibility: Maintaining a regular cadence helps teams establish a predictable pattern of delivery, making it easier to plan and manage expectations. This predictability enhances visibility across the organization, allowing Stakeholders to anticipate outcomes and make more informed decisions.
• Faster feedback and learning: Synchronization ensures that all teams within an ART are working in unison, allowing them to share knowledge, learn from each other, and adapt more rapidly. This process accelerates feedback loops and promotes Continuous Improvement, resulting in better overall performance.
• Efficient dependency management: By aligning teams within an ART, synchronization helps manage dependencies more effectively. This alignment enables teams to identify, communicate, and resolve dependencies promptly, reducing the risk of delays and ensuring smoother delivery.
• Higher quality: When teams follow a consistent cadence, they can better plan their work and allocate resources, leading to higher-quality deliverables. Synchronization also allows teams to identify and address issues sooner, resulting in improved product quality.
• Greater adaptability: A regular cadence allows organizations to respond to changing market conditions or customer needs more effectively. By maintaining synchronization across the ART, teams can quickly adapt to new requirements, prioritize work, and ensure that they are delivering the most valuable Features. This flexibility is crucial for organizations to stay competitive and responsive in a dynamic business landscape.

As organizations reach the stage where scaling is necessary, implementing additional patterns becomes crucial to preserve the connection between teams. To promote and sustain synchronization and cadence, we expand team events to encompass ART events.

We will delve into each ART-level event in depth, but an important point to remember is that ART events resemble team events, but on a larger scale. The following diagram illustrates the correlation between team events (inner loop) and ART events (outer loop). For instance, we can observe that PI Planning serves as the scaled ART event equivalent to team Iteration Planning. Similarly, Coach, PO, and ART Sync correspond to the Team Sync:

Figure 7.1 – ART event correlation to team events (© Scaled Agile, Inc.)

Over the next several chapters, we will review each of the ART events that are typically facilitated by the RTE and the outcomes we seek to achieve that help us to maintain synchronization and cadence. However, there are several other aspects of the ART we need to stay focused on that are not directly related to the ART events, such as the Continuous Delivery Pipeline.

Pro tip

As a Coach you will want to ensure that the teams on your ARTs are executing the appropriate team events and that the ART is executing the correlating ART events.

While SAFe® itself is a framework and thus isn’t prescriptive, by ensuring these events happen regularly and with the intended outcomes you will find greater success.

The Continuous Delivery Pipeline

The Continuous Delivery Pipeline (CDP) (sometimes referred to as the Continuous Integration/Continuous Deployment (CI/CD) pipeline) is an integral component of an ART:

Figure 7.2 – The Continuous Delivery Pipeline (© Scaled Agile, Inc.)

What is the CDP?

The CDP drives ARTs to follow principles over practices, particularly the following ones:

• Principle 4: Build incrementally with fast, integrated learning cycles
• Principle 5: Base milestones on an objective evaluation of working systems
• Principle 6: Make value flow without interruption

Historical tip

SAFe® Principle #6 was updated with SAFe® 6.0 to make value flow without interruption. Previously, it was to visualize and limit the WIP, reduce batch sizes, and manage queue lengths. These three items, when executed collectively, will allow for an increased flow of value that directly correlates to lean thinking principle #3.

The CDP is a four-part system representing the workflows, activities, and automation necessary for an idea to become a release of value. These four parts are as follows:

1. Continuous exploration (CE): This aspect of the CDP is focused on what needs to be built. Product Management and the System Architect spend time together in this area defining what needs to be delivered. We leverage Minimum Viable Products (MVPs) and/or Minimum Marketable Features (MMFs) to explore and then persevere in development or pivot based on the exploration. The items created comprise the ART backlog.
2. Continuous Integration (CI): CI is a software development practice that involves automatically building, testing, and integrating code changes into a shared repository on a regular basis. The goal of CI is to catch errors and issues as early as possible in the development process, so they can be addressed quickly and efficiently.

With CI, developers frequently integrate their code changes into a shared repository, triggering automated builds and tests. This ensures that code changes are compatible with the rest of the solution and that any issues are detected and resolved quickly.

3. Continuous Deployment (CD): CD is a software development practice that involves automatically deploying code changes to production environments once they have been tested and approved. In a mature CD, code changes are often automatically deployed to production as soon as they pass automated testing and other quality checks, without requiring any manual intervention.
4. Release on Demand (RoD): The Business chooses when to make the solution, or parts of the solution, available to customers. This can occur in many ways, including all at once or in a staggered fashion. By allowing the Business to choose when to release, they can ensure appropriate market timing and the ART can continue to work on its normal cadence.

The CDP is not a linear process; information flows between all four parts of the CDP continuously.

Who is Responsible for the CDP?

The simple answer is everyone. Depending on the aspect of the CDP, various roles and teams have larger roles in supporting the CDP.

Product Management and the System Architect play a key role in defining what should be built during Continuous Exploration. Subsequently, in CI, they ensure that what they wanted to be built is being built.

The System Team is often responsible for maintaining and maturing the DevOps pipeline, which enables Agile Teams to continuously integrate and deploy their solutions while following the architecture and engineering practices outlined by the System Architect.

The Agile Teams build their solutions while leveraging DevOps practices that allow the Business to choose when to release. In software development, a common practice is to use Feature toggles.

Pro tip

A Feature toggle, also known as a Feature switch or Feature flag, is a software development technique allowing certain Features and functionalities to be turned on and off without deploying new code.

Feature toggles are often used to enable the gradual rollout of new Features, to test new Features in production with a small subset of users, or to temporarily disable a Feature that is causing issues or errors.

Feature toggles work by wrapping sections of code with conditional statements that check whether a particular Feature should be enabled or disabled. When a Feature is disabled, the code associated with that Feature is not executed, and the application behaves as if the Feature doesn’t exist. When a Feature is enabled, the code associated with that Feature is executed, and the Feature becomes available to users.

Feature toggles are an important tool for creating more flexible and resilient software, allowing developers to control the rollout of new Features and respond quickly to issues or errors. However, they can also introduce complexity into the code base and require careful management to avoid creating technical debt.

Why do we need a CDP?

We have had the fortunate, or maybe unfortunate, experience in our careers of helping many different organizations with ART recovery. They have Agile Teams, they have ARTs, and they are executing but aren’t getting the results that they want. Most of the time, this is due to a poor CDP.

As a Coach, you will likely encounter organizations that want to “be Agile.” They will Train people in Agile practices, put them on Agile Teams, start writing User Stories, implement an Agile tool (such as Jira) to track the work, create ARTs, hold PI Planning Events, and fail to invest in the technology and development skills needed to deliver the solutions any differently than they have been.

For car enthusiasts, you could say we have done the bodywork to make the car look like a Maserati but didn’t upgrade any of the mechanical parts, including the engine, which is from an old Pinto. It’s like putting lipstick on a pig.

Integrate Often and Early

Integrating work from multiple teams (or even individuals) is one of the hardest things for most organizations to do. How many times have we heard, “It works on my machine?” Cadence, the CDP, and DevOps reinforce the importance of frequent integration:

“Nothing in the world is worth having or worth doing unless it means effort, pain, difficulty…”

- Theodore Roosevelt

Establishing patterns of frequent integrations, frequent deployments, and release on demand fundamentally change the culture of the organization. “We have always done it this way” is a hard habit to break.

Developers will need to learn new ways of working, including, but not limited to, Test-Driven Development (TDD), ensuring low coupling and high cohesion. They will need to leverage modularization and test doubles. The environments and tools will likely need to be improved and upgraded to support these new ways of working.

Pro tip

The Iteration System Demos create a forcing function for teams to integrate on a regular cadence. Because it is “hard,” this is an event that many ARTs try and skip or perform irregularly or only at the end of the PI.

Note: We have titled the System Demo that occurs during each iteration the Iteration System Demo to avoid confusion with the PI System Demo, which occurs at the end of the PI.

As a Coach, you will have to challenge the status quo and push the organization to invest both in technology and people development.

Pro tip

Ensure that the Iteration System Demos aren’t skipped and work with Product Management to show a seamless demo between teams, not just each team demoing what they completed in the last iteration.

The CDP is a crucial element for driving ARTs to prioritize principles over practices. It is a four-part system consisting of CE, CI, CD, and RoD. This non-linear process enables the constant flow of information between the four parts. While everyone has a role in the CDP, certain roles and teams have larger responsibilities over specific aspects. A well-executed CDP can lead to better results for organizations, while a poorly implemented one can hinder progress. To ensure the CDP’s success, it is essential to invest in technology and development skills and maintain a culture of frequent integration, deployment, and release on demand. Coaches must challenge the status quo and push organizations to make the necessary investments in both technology and people development.

Tooling

We aren’t about to tell you what tools are best, what tools we prefer, or even give you an exhaustive list of tools; we already listed some common industry tools to help with context and alignment. What you need to know is that everyone hates their tools and yet they are necessary and even more so in the distributed working environments in the post-Covid-19 era.

When you are considering your tooling suite to support the CDP, there are a few key items you will want to ensure your tools assist with:

• Workload management
• Automated testing
• Built-in-Quality
• Deployment automation
• Metrics

Workload Management

Workload management tools are essential in keeping the teams and ARTs aligned. Common workload management tools include Jira, Rally, Target Process, and VersionOne.

Automated Testing

Tools that support automated testing will vary widely, depending on the languages you are developing in. Selenium and Watir are two common software suites. There are also multiple levels at which you will need to leverage test automation, including unit tests, integration tests, and functional UI tests.

Built-in-Quality

Tools in this area intend to help developers improve the code that has been written. We often focus on code coverage for testing, but there are additional tools that will look for duplication of code and ensure proper documentation and formatting. A common code quality tool is SonarQube.

Deployment Automation

One of the most common tools for deployment automation is Jenkins. Deployment automation is key in allowing teams to move code quickly and easily from environment to environment and ensure that the proper automated tests are executed. By automating deployments, we minimize the risk of human error, likely decrease the amount of documentation, and reduce the time it takes, freeing up developers to do development.

Metrics

When people think of metrics in Agile, there tends to be a focus on team velocity and burndowns. See Chapter 15 for more information on metrics. However, many other metrics need to be considered. The tools in your CDP will have their own sets of metrics, including (but not limited to) the following:

• Code coverage: This metric quantifies the percentage of code that is covered by automated tests, indicating the extent to which the code base is tested and its overall quality.
• Deployment metrics: These measure the efficiency and effectiveness of the deployment process, including deployment frequency, deployment time, and deployment success rate.
• Test execution metrics: These metrics track the performance of the testing process, such as the number of tests executed, test pass rate, and test failure rate.
• Process time: This measures the time it takes for a task or process to be completed, helping to identify bottlenecks and inefficiencies in the workflow.
• Lead time: This metric captures the time elapsed from the initial request to the delivery of a completed Feature or product, providing insight into the overall responsiveness of the development process.
• Delay time: This metric calculates the amount of time for which a task or process is waiting in a queue before it is executed, helping to identify areas where work has been stalled or resources are being underutilized.
• Percent complete and accurate: This metric assesses the accuracy and completeness of work items, indicating the quality of the deliverables and the need for rework or additional effort.
• System up/down times: These metrics track the availability and reliability of systems, highlighting potential issues with infrastructure, application stability, or other factors impacting system performance.
• Utilization rates: These metrics measure the percentage of available resources (such as team members or equipment) that are actively engaged in work, providing insights into resource allocation and efficiency.

When using metrics, it is essential to avoid focusing too heavily on a single metric. Instead, apply SAFe® Principle #1: Take an economic view, considering a balanced set of metrics to understand the overall performance and make informed decisions.

Pro tip

As a Coach you will want to ensure that the tools selected by the organization will gather the metrics needed to mature the ART.

You will want to work with the teams to guide and influence what is being tracked and how frequently.

It’s important to watch out for metrics abuse, where metrics are misused or misinterpreted, leading to negative consequences and teams being punished.

Lastly, setting up tools isn’t a once and done activity, you will want to ensure that the tool suite is continuously reviewed, refined, and updated to stay current with industry practices.

So now that we understand the importance of the CDP, let’s shift gears and look at the people driving ART success and their day-to-day responsibilities and activities.

Day-to-Day – Product Management

While Product Management often spends a lot of their time looking forward and planning and coordinating work for future Planning Intervals (PIs), they still have responsibilities in the current PI.

ART Events and Product Management

Product Management is involved in many of the ART events that occur on a regular cadence.

Product Management and PI Planning

Product Management sets the vision and prioritizes the Features of the ART. Product Management will need to be available to answer questions that may arise. They participate in Management Review and Problem Solving and contribute to the Business Value assigned to team objectives and are consulted on plan acceptance by the Business Owners.

Product Management and the ART Sync

At the ART Sync, Product Management is an active participant and helps address and remove any problems and discusses opportunities with the teams.

Product Management and the PO Sync

While the PO Sync may be facilitated by either the RTE or Product Management, we have found that the event tends to be more effective when Product Management owns and facilitates it. This event ensures that the Product Owners and Product Management are aligned on the work being developed, as well as what is coming up in the next iteration.

Product Management and the Iteration System Demos

The Iteration System Demo (not the PI System Demo in I&A) is a key opportunity for Product Management to receive feedback on the current solution. Successful Iteration System Demos are facilitated by Product Management, who can tell the story of what is being delivered by the teams. It highlights and shows the integration of all the work that all the teams have delivered.

Product Management and Inspect & Adapt (I&A)

Product Management is a key player and often leads the PI System Demo portion of I&A. It is important for Product Management to actively participate in the Quantitative and Qualitative Measurement and Retrospective and Problem-Solving workshops. Product Management needs to ensure plans for improvement items are incorporated into the next PI.

Pro tip

Plan and dry run the PI System Demo. While we don’t encourage dry running every demo, especially team iteration demos, the PI System Demo warrants a little extra attention as it is often the only time when some key Stakeholders attend.

Product Management and ART Backlog Refinement

Product Management needs to continually refine the ART Backlog in the same fashion that a Product Owner refines the Team Backlog.

Pro tip

Encourage Product Management to set aside dedicated time each week to work on the backlog. Most Product Managers need at least 3 to 4 dedicated hours per week to stay ahead of the ART.

Part of refinement includes working with the Product Owners to help decompose the Features and Enablers into stories for the teams with appropriate Acceptance Criteria.

Product Management and Preparing for PI Planning

Product Management needs to ensure that they have the appropriate presentations ready for the PI Planning Event. This involves setting and refreshing the product vision, ensuring that Features and Enablers are ready and prioritized for PI Planning, and working with Product Owners to ensure Features are broken down into stories and refined with the teams.

Pro tip

As a Coach, you will want to ensure that Product Management is engaged with the ART and involved in the day-to-day work that is being delivered. Too often we encounter Product Management that doesn’t dedicate the time and effort necessary and we see those ARTs struggle.

Ongoing Activities for Product Management

In addition to attending the ART events, Product Management has five key areas of responsibility they should be continuously working on:

Figure 7.3 – Product Management responsibilities (© Scaled Agile, Inc.)

As a Coach, you will want to support Product Management as they navigate all the various areas and aspects of the role. Product Management gets pulled in numerous directions and often, it is a challenge for them to find enough time to manage and prioritize the ART Backlog – particularly when new to the role.

Pro tip

I encourage Product Management to make a habit of explicitly blocking time on their calendars to work on the backlog, conduct market research, meet the customer, and update roadmaps. This helps ensure that key activities aren’t skipped due to the tyranny of the urgent activities.

Day-to-Day – System Architect

Similar to Product Management, the System Architect is also looking ahead as well as keeping abreast of the current execution of the ART.

ART Events and the System Architect

Let’s look at some of the ART events the System Architect attends or facilitates.

The System Architect and PI Planning

The System Architect has a key role in the PI Planning Event – that is, preparing and sharing the architecture vision briefing. The System Architect is a key individual at Management Review and Problem Solving and influences the Business Owner in estimating Business Value and plan acceptance.

The System Architect and ART Sync

At the ART Sync, the System Architect is an active participant and helps address and remove any impediments and discusses opportunities with Feature and Enabler development. It’s also a good opportunity to review quality metrics.

Pro tip

While SAFe® doesn’t identify a Technical Sync, I have found that, in the same fashion, there is a Coach Sync (formerly the Scrum of Scrums) and a PO Sync, and that teams often have a technical leader, often in an unofficial capacity. I have had success in formalizing that role and establishing a Technical Sync that the System Architect leads, just as the RTE or Product Management would lead the Coach Sync or PO Sync, respectively.

This creates a similar and scalable context for sharing information regarding technical challenges in the teams, architectural design patterns, infrastructure discussions, and so on. The System Architect has a daunting role in keeping abreast of constantly changing technology, security threats, compliance needs, and performance demands.

If your ART decides to hold a Technical Sync, make sure you invite the RTE.

The System Architect and Iteration System Demos

At the Iteration System Demos, the System Architect can see the collective progress as they likely aren’t able to attend every team’s itineration demo. The System Architect will often use this opportunity to ask questions of the teams regarding technical practices and progress on Enablers.

The System Architect and Inspect & Adapt (I&A)

The System Architect actively participates in the PI System Demo and will often compliment Product Management and share some of the technical underpinnings of the system that have been developed. It is important for the System Architect to actively participate in the quantitative and qualitative measurement and retrospective and problem-solving workshops.

Pro tip

As a Coach, it is important to ensure that the System Architect is engaged daily with the ART. This includes actively attending and participating in the ART events.

In my experience, System Architects tend to be a bit introverted and likely don’t understand the full expectations of the role and will benefit from some extra knee-to-knee guidance and Coaching.

Ongoing Activities for the System Architect

In addition to supporting the ART in the various events, the System Architect has five key areas of responsibility:

Figure 7.4 – System Architect responsibilities (© Scaled Agile, Inc.)

The System Architect stays very busy and is constantly working with the teams and organization to deliver quality solutions.

While being embedded in to aligning architecture with business priorities and fostering Built-in Quality and attending to NFRs, the System Architect needs to continually refine the ART Backlog in conjunction with Product Management. They may need to work closely with the Product Owners to decompose both business and Enabler Features into consumable stories for the teams.

Pro tip

As a Coach, encourage the System Architect to set aside dedicated time each week to work on the backlog. Most System Architects need at least 3 to 4 dedicated hours per iteration to stay ahead of the ART. Often, the System Architect and Product Management will set aside time together every iteration to work on the backlog together in addition to independent work.

Day-to-Day – the Release Train Engineer

For a Release Train Engineer (RTE), no two days are ever the same. Attempting to lay out what an RTE should do each day would be come out of date the minute it was written. In this section, you will discover some typical activities that an RTE undertakes regularly. Bear in mind that depending on the ART and organizational maturity, these may vary widely.

Keeping the Train (ART) on the Tracks

There are many idioms that folks use to describe an RTE’s job, including “keeping the Train on the tracks,” “keeping the plates spinning,” “herding cats,” and “keeping balls in the air.” I’ve even heard the RTE called the “ART mom.” However, you want to describe it or whatever you want to call it, the RTE’s job is indispensable in ensuring success.

Much of what the RTE does is ensure that coordination and conversations are happening. They are working with Product Management and the System Architect to look ahead, as well as make sure that the teams are executing their day-to-day work.

The RTE removes the impediments that are escalated to keep the teams delivering value. The RTE is a Coach and helps ensure the teams continue to grow and mature and become high-performing teams.

Whatever comes up that the ART needs help, support, or guidance with is what the RTE does.

Facilitating the Events

The RTE is responsible for facilitating the ART-level events throughout the iteration. These events serve as the checkpoints to help the RTE make sure the ART is staying on track. Depending on the needs of the ART, holding all three sync events (ART Sync, Coach Sync, and PO Sync) might not be necessary.

The RTE and the ART Sync

Conducting the ART Sync is more an art form than a rigid, prescriptive process. During the ART Sync, the RTE plays a crucial role in guiding the conversation and ensuring that the event remains productive and focused.

In an ART Sync, the RTE utilizes the ART Planning Board from PI Planning to track and assess progress toward the agreed-upon commitments and dependency deliverables. This helps maintain a clear view of the current state and identifies any significant deviations from the plans.

This event also serves as an opportunity to review and address risks identified during PI Planning, as well as to discuss any new risks that may have emerged. By fostering an open and collaborative environment, the ART Sync encourages sharing insights and problem-solving strategies that can help mitigate these risks and maintain momentum.

Additionally, the ART Sync is a platform for discussing and resolving impediments that individual teams have been unable to overcome on their own. By bringing together the collective knowledge and experience of the ART, this event enables participants to devise creative solutions to challenges and remove any roadblocks that may be hindering progress.

The ART Sync is a dynamic and essential event that allows the RTE to facilitate effective communication, ensure alignment, and promote problem-solving across the teams in the ART. By mastering the art of conducting the ART Sync, the organization can maintain a well-coordinated and high-performing ART, ultimately delivering value to customers in a timely and efficient manner.

The RTE and the PO Sync

The PO Sync is a crucial event that enables Product Management and Product Owners to maintain alignment and foster collaboration with their teams. This event, facilitated by either the RTE or Product Management, creates the opportunity to discuss progress, address challenges, and ensure that the teams’ goals and objectives are aligned with the overall vision.

During the PO Sync, Product Owners come together to share updates on their respective team’s work, review dependencies between teams, and identify any potential risks or obstacles that need to be addressed. This event also provides an opportunity to discuss priorities, align backlog items, and coordinate any necessary adjustments to the overall plan. The exchange of information and insights during the PO Sync helps create a shared understanding of the status and direction, promoting a sense of unity and coherence across the ART.

The RTE and the Coach Sync

The Coach Sync, previously known as the Scrum of Scrums, serves as a vital forum for Scrum Masters/Team Coaches and other select members and Subject Matter Experts (SMEs) to discuss and collaborate on the challenges they face at the team level. In addition, it provides an opportunity to share successes and best practices that have positively impacted their teams. This event plays a crucial role in fostering Continuous Improvement and knowledge sharing across the ART.

The Coach Sync is an excellent platform for the RTE to offer guidance, support, and Coaching to the Scrum Masters/Team Coaches. By actively participating in these discussions, the RTE gains valuable insights into the day-to-day operations of the teams, allowing them to better understand their needs and challenges.

During the Coach Sync, participants can share their experiences, discuss common issues, and brainstorm solutions, which can be applied across the entire ART. This collaborative environment encourages learning from each other’s successes and mistakes, leading to more effective problem-solving and the implementation of improved practices.

Furthermore, the Coach Sync can help identify patterns and trends in team challenges, enabling the RTE to address systemic issues that may be affecting multiple teams. This proactive approach to identifying and resolving problems ensures that the ART operates efficiently and effectively, ultimately delivering value to customers.

The Coach Sync is a crucial event that promotes communication, collaboration, and learning among Scrum Masters/Team Coaches within the ART. By actively participating in these events, the RTE can provide valuable Coaching, identify trends and patterns, and gain a deeper understanding of the day-to-day challenges faced by the teams, ultimately contributing to the overall success of the ART.

Pro tip

Not all ARTs will need all three syncs: ART Sync, PO Sync, and Coach Sync. I have found that it often helps newer ARTS to have all three events as people learn how and what to communicate with each other in an Agile fashion.

The RTE and the Troika Sync

You won’t find this event in SAFe®, but we have found it to be a key event for the ART leadership to come together and have honest conversations about the state of the ART in delivering its commitments. We recommend holding this sync (typically 30 minutes) at least once per iteration. One format that works well is to time box the event into three 10-minute conversations with each Troika member (RTE, Product Management, and System Architect) leading the section with their pertinent agenda topics and discussions.

Pro tip

I have worked with ARTs that find this to be such a critical event that the RTE, System Architect, and Product Management treat it like a Daily Stand-up and hold it every day to ensure alignment.

The RTE and the Technical Sync

If your System Architect holds a Technical Sync, the RTE should be an active participant and listen to the technical challenges the ART may be having, as well as items that need to be included in the backlog for future PIs.

The RTE and the Iteration System Demos

The RTE’s role at the Iteration System Demo will vary from ART to ART, depending on the maturity and the strength of Product Management. At the very least, the RTE will ensure the demo is scheduled with appropriate attendees and will update the agenda based on feedback from Product Management.

General Responsibilities of the RTE

The five primary areas of an RTE’s responsibility are outlined in the following diagram. We called out a few key facilitation areas previously that aren’t explicitly listed here that are especially important as an ART is launched:

Figure 7.5 – RTE responsibility areas (© Scaled Agile, Inc.)

Pro tip

As a Coach, ensuring the ART is successfully launched and executed is of the utmost priority. First impressions are critical, especially for PI Planning. Ensure that adequate preparation has gone into the first PI Planning and that it continuously improves.

As a Coach, you may find yourself spending a lot of time helping the RTE get the ART going and keeping it on track. The RTE has a big job, and tooling and preparation are critical elements and areas.

The RTE and Agile Tools

The RTE should be regularly (daily) reviewing the ART metrics in the Agile tools and helping the Scrum Masters/Team Coaches as necessary to keep the tools up to date. If leaders aren’t comfortable with the Agile tools, the RTE may become the go-to source for information on the current “status” of the ART and its progress.

The RTE and preparing for PI Planning and I&A

The RTE has a big job in preparing for I&A and PI Planning every 10 to 12 weeks. A good RTE starts preparing for PI Planning as soon as the last one ends. While we will take a deeper look at what goes into PI Planning and I&A in subsequent chapters, we don’t want to overlook the amount of work and effort that occurs during the PI that the RTE does in preparation for these events.

In Appendix A, we have provided a spreadsheet where we have decomposed the large deliverables that feed into PI Planning and I&A into smaller activities, as well as provided a suggested timeline of when the various activities should be delivered.

Get Help!

One important thing the RTE must do is ensure they don’t try and do all the planning, coordination, and execution of the events by themselves. Too often, we see the RTE trying to do it all. The RTE should leverage Product Management and the System Architect for help, as well as the Scrum Masters/Team Coaches and Agile Coaches.

Calendars! Calendars! Calendars!

One of the most tedious activities that an RTE undertakes is working with calendars and schedules. Many factors come into play. Let’s look at some of the key calendars/schedules that an RTE is responsible for.

PI Calendar

The RTE is responsible for setting the PI calendar. They will need to coordinate with the organization to potentially align with other ARTs or ensure that PI events are off cadence from regular market rhythms.

The RTE should maintain a rolling PI calendar and, depending on the length of each PI, may need to make adjustments (usually by lengthening a PI) to avoid major holidays. Typically, we suggest setting the PI calendar to span at least a year, accommodating four to five PIs on average.

Pro tip

Pre-Covid-19, many RTEs found themselves needing a PI calendar that was 1.5 to 2 years out. This allowed them to find and schedule event spaces that could accommodate the entire ART for PI Planning. While Covid-19 has forced virtual PI Planning, consider that calendars may need to be extended as organizations are again returning to in-person PI Planning, even if the teams are remote.

Event Schedule

The RTE sets up and schedules the ART events. As ARTs launch, it can be difficult to find time to schedule these events; we’ve found it best to set them on a cadence and work the fallout. Often, the conflicts are meetings that already have a similar function. It will take work and effort to establish the new alignments.

The events the RTE needs to ensure are scheduled are as follows:

• PI Planning: Note that there may be multiple invites for this event; see Pro tip.
• ART Sync
• PO Sync: Encourage Product Management to schedule this event.
• Coach Sync
• Iteration System Demo
• I&A: Note that this event may also have multiple invites; see Pro tip.
• Troika Sync
• Technical Sync: If holding this event, encourage the System Architect to schedule this event.

Pro tip

Sending out multiple invites for PI Planning and I&A can help ensure attendance at appropriate times for key Stakeholders. Often, an executive can’t attend the full 2 days of PI Planning. Consider sending out invites for the specific timeboxes that would be critical for an executive to attend in addition to the full invite, such as the business context and vision sections, plan readouts, and management reviews.

Similarly for I&A, you may want a wider audience for the PI System Demo and then a Quantitative and Qualitative Measurement and Problem-Solving Workshop invite for ART members.

As a Coach you will want to work closely with the RTE and Scrum Masters/Team Coaches to set up and align all of the calendars, schedules, and events. This is especially critical for the ART launches.

Pro tip

I have had success in declaring meeting bankruptcy. To do this, meet with the Scrum Masters/Team Coaches and review and identify all the meetings that everyone in their teams participates in, including the PO. Any meeting that doesn’t correlate to an event must be negotiated to be attended moving forward. We effectively set a budget for meetings and are judicious in watching our meeting spend.

Team Calendars/Schedules

While the RTE isn’t directly responsible for scheduling and facilitating the team events, the RTE (as a Coach) must ensure the team events are happening with the appropriate outcomes.

Story from the real world

I was invited to help recover a failing ART that was already three quarters behind in delivery and forecasted that it would take at least four additional quarters to complete the work. If you are thinking that this is a very waterfall mindset already, you are correct. I could probably write an entire book on this ART and organization, but that is for another day.

For this ART, one area I looked at was their basic practices, including team events. I quickly discovered that the teams weren’t executing the basic events we would expect, and the ones they were holding didn’t meet the intent.

Most of the teams held 30 minutes of Iteration Planning, had no scheduled backlog refinement sessions, Team Syncs were over 1 hour long, iteration reviews had been skipped completely, and they had irregular retrospectives without actions.

The ART did have an Iteration System Demo; however, it was executed as a combined Iteration demo for the teams, with each showing work they had completed independently of one another.

The RTE was wearing multiple hats and didn’t have enough insight into the happenings of the teams on a day-to-day basis. Adding insult to injury, a couple of Scrum Masters/Team Coaches were new to their roles; they didn’t have formal training and supported multiple teams.

While I Coach teams to leverage principles over practices, this was a clear instance where the fundamentals were missing, and we re-instated standard ART and Scrum events and the ART quickly became more predictable and started delivering incrementally.

The lack of event execution is why the RTE needs to have a pulse on what is going on in each team and why the RTE needs to be aware of each of the team’s events and, in the ART Coach role, ensure they are being executed effectively.

Now that we understand what day-to-day activities Product Management, the System Architect, and RTE are responsible for during the PI, let’s take a look at the last iteration of the PI.

The Innovation and Planning Iteration

The Innovation and Planning (IP) Iteration occurs at the end of each Planning Interval (PI) and serves multiple purposes. The IP Iteration provides dedicated time for several different activities teams and individuals may undertake, depending on their needs and maturity:

• Innovation: Innovation gives the team time and space to research new technologies, try out new ideas, and create a thinking space to drive better solutions in the future. Teams often do this via hack-a-thon, leveraging design thinking, finding solutions to technical issues, and so on.
• Continuing education: Teams should try and learn new technologies or enhance and build existing skills. They can attend training classes and generally work to expand their T-shaped skills.
• Team/ART building: Improve collaboration between the teams with team-building exercises. Alternatively, they can work collectively to update and refine the various Definitions of Ready, Definitions of Done, and Working Agreements.
• Buffer to meet PI Objectives: Finish Features and refactor and/or fine-tune work that was delivered during the last PI. This may also include some whittling down of technical debt.
• Mature DevOps and CI/CD pipelines: This is an opportune time for upgrades to occur, including infrastructure, system, tooling, patching, and more.
• Measure and grow: Complete Measure and Grow Assessments and compare them with previous PIs. Common assessments include the Team and Technical Agility assessment and Agile Product Delivery assessment.
• PI Planning preparation: Teams meet and often draft initial User Stories, ensure Features are refined, and help create and set up the planning space.

Two key events occur during the IP Iteration that time needs to be set aside for:

• Inspect and Adapt: Typically, one day of the iteration is set aside for the Inspect and Adapt workshop.
• PI Planning: PI Planning typically takes 2 full days in the IP Iteration. If remote sessions, distributed teams, or travel are involved, additional time may need to be allotted.

Pro tip

A hack-a-thon is an event, usually lasting anywhere from a few hours to a few days, in which a large group of people, often with diverse backgrounds and skill sets, come together to collaboratively work on software or hardware projects. The goal of a hack-a-thon is typically to create a functional prototype, solution, or proof of concept for a specific problem or challenge, often within a limited time frame.

IP Iteration Schedule

Most ARTs align on a Wednesday through Tuesday iteration schedule. This schedule avoids many major holidays, and most team members are in the office on Tuesday, Wednesday, and Thursday for key events. Let’s look at some example event schedules.

Table 7.1 is an example IP Iteration schedule for in-person PI Planning. Monday and Friday can be used for travel if necessary:

Table 7.1 – Example In-Person IP Iteration schedule

Table 7.2 is an example IP Iteration when planning remotely. The same number of hours are used but are spread across multiple days to minimize “Zoom fatigue.” This schedule can also be used with distributed teams with limited overlap of working hours:

Table 7.2 – Example Remote IP Iteration schedule

Measure and Grow Assessments

Measure and Grow is a critical and often overlooked tool for the teams and ARTs to use to reflect on where they came from, as well as to identify areas for Continuous Improvement. We aren’t going to deep dive into the benefits, learnings, and opportunities that arise from using measure and grow assessments here, but we do need to touch on their value.

We encourage you to use the IP Iteration as a regular cadence for doing assessments with your teams. Then, you can incorporate the results into the Quantitative and Qualitative Measurement section of I&A or use them as potential inputs for the Problem-Solving Workshop.

We don’t recommend executing all the assessments with the teams every PI. We recommend selecting one or two at the most and executing them as a guided and facilitated conversation to ensure that the statements are understood and conversation is encouraged.

Consider using the Team and Technical Agility (TTA) self-assessment each PI and then pick one of the others on a rotating basis, potentially to get feedback on areas the ART might be struggling with where you want some additional insight from the ART or an area that was particularly low or high in the Business Agility assessment.

The Measure and Grow Core Competency Assessments align with the SAFe® Core Competencies and have questions that align with each of their dimensions. The SAFe® Core competencies and assessments are as follows:

• Organizational Agility (OA)
• Lean Portfolio Management (LPM)
• Enterprise Solution Delivery (ESD)
• Agile Product Delivery (APD)
• Team and Technical Agility (TTA)
• Continuous Learning Culture (CLC)
• Lean-Agile Leadership (LAL)

Pro tip

As a Coach, we often have to justify the expense/overhead that Coaching incurs. Leveraging Measure and Grow can help with this justification. You can show the impact Coaching is having on the ART in terms of the stability and growth of the metrics.

Responsibilities in the IP Iteration

It is incumbent on all ART members to ensure a successful IP Iteration.

RTE IP Iteration Responsibilities

During the IP Iteration, most RTEs barely have time to breathe. They are ensuring that everything is ready for I&A and PI Planning. It’s coordinated chaos to get all of the pieces in place.

Product Management IP Iteration Responsibilities

Product Management needs to ensure that the Product Owners are working with their teams to wrap up any last-minute items, preparing for the PI System Demo, and ensuring their briefings are ready for the PI event. Product Management may want to lead a hack-a-thon activity.

System Architect IP Iteration Responsibilities

In addition to preparing for the PI event briefing, the System Architect plays a crucial role in collaborating with teams to ensure a thorough understanding of the Enabler’s work for the upcoming PI. This involves working closely with the System Team to coordinate their efforts and provide guidance on the work they are executing. The System Architect also contributes to the successful closeout of the PI by addressing any outstanding issues and ensuring that all necessary work has been completed.

Another valuable opportunity for the System Architect is to team up with Product Owners in organizing and leading a hack-a-thon. A hack-a-thon is a focused event where cross-functional teams come together to work on specific problems or challenges, often involving innovation, problem-solving, or process improvement. By partnering with Product Owners, the System Architect can help identify areas where a hack-a-thon may be beneficial, align the objectives of the event with the overall vision of the product, and ensure that the outcomes of the hack-a-thon contribute to the success of the ART.

During a hack-a-thon, the System Architect and Product Owners can facilitate collaboration, offer guidance, and provide technical expertise to participating teams. This collaborative environment allows for the rapid development of creative solutions, the exploration of new ideas, and the fostering of team engagement and morale. Additionally, hack-a-thons can serve as a valuable learning experience for both the System Architect and Product Owners as they gain insights into team dynamics, individual skills, and potential areas for improvement.

Scrum Master/ Team Coach IP Iteration Responsibilities

The Scrum Masters/Team Coaches help with PI Planning preparation and ensure the teams are ready for all the events. One item that’s often overlooked is calculating the capacity for each iteration for the next PI; make sure not to skip this. The Scrum Masters/Team Coaches will often lead the measure and grow assessments for their teams. They will prepare their team spaces for planning and ensure that the plans are captured in the Agile tool after the event. The Scrum Masters/Team Coaches may also need to facilitate the additional activities that occur during the IP Iteration, such as a hack-a-thon, training, or team-building activities.

Product Owner IP Iteration Responsibilities

Product Owners need to ensure that the team is ready for PI Planning. There is often a lot of last-minute refinement that occurs, as well as ensuring that all work from the PI is closed out. The Product Owners will also work closely with Product Management to ensure successful briefings and will often help the Scrum Masters/Team Coaches in prepping the team spaces and facilitating events.

Team Member IP Iteration Responsibilities

The team members are fully engaged with all the wrap-up work, new skill development, preparation, and execution of the events. As a Coach, encourage the team members to help identify the work that they want to execute in the IP Iteration.

We don’t need an IP Iteration

As a Coach, one of the challenges you will likely face with every single ART you work with is regarding the IP Iteration. This challenge often stems from a misunderstanding of what occurs during the IP Iteration.

During PI Planning, we ask teams to not plan any stories or work in that iteration. As a result, people often mistakenly assume that the teams are not working. After all, we measure the teams by stories completed, right?

You will likely need to justify and push to keep the IP Iteration holistic. Often, Coaches will talk about using it to reduce technical debt, avoid burnout, and increase predictability. While these are noble efforts, and people agree that they are important, they typically don’t carry enough weight to quell the sentiment.

We recommend working with the Troika to identify specific activities that the teams will be doing during the IP Iteration and have them scheduled out in advance of PI Planning and then included in the PI plans as stories. This changes the visual of looking at an empty iteration on the team boards. It shows that the teams are still engaged and “doing work.” One other consideration would be to not have a board for the IP Iteration at all.

Pro tip

As a Coach, it’s critical to execute a successful IP Iteration. Work diligently with your organization to ensure this critical event occurs.

Summary

The day-to-day activities of the ART focus on keeping alignment and synchronization. The CDP allows the ART to deliver value and can be a significant impediment if it isn’t continuously matured. The tools that drive the CDP and the ART are important in helping keep track of where the ART is in its journey. In this chapter, we looked at the activities and expectations for Product Management, the System Architect, and the RTE during the PI as this group is largely responsible for ART success. We also covered the IP Iteration and now understand what occurs, why it’s important, and the responsibilities of everyone in the ART during the iteration.

Further reading

1. Continuous Delivery Pipeline: https://www.scaledagileframework.com/continuous-delivery-pipeline/
2. Innovation and Planning Iteration: https://www.scaledagileframework.com/innovation-and-planning-iteration/
3. Measure and Grow: https://www.scaledagileframework.com/measure-and-grow/
4. Product Management: https://www.scaledagileframework.com/product-management/
5. System Architect: https://www.scaledagileframework.com/system-architect/
6. Release Train Engineer: https://www.scaledagileframework.com/release-train-engineer/
7. Gene Kim, Kevin Behr, and George Spafford (2018). The Phoenix Project: A Novel about IT, DevOps, and Helping Your Business Win. IT Revolution Press.
8. Gene Kim, Jez Humble, Patrick Debois, and John Willis (2016). The DevOps Handbook: How to Create World-Class Agility, Reliability, and Security in Technology Organizations. IT Revolution Press.
9. Lisa Crispin and Janet Gregory (2009). Agile Testing: A Practical Guide for Testers and Agile Teams. Addison-Wesley.
10. John Doerr (2018). Measure What Matters: OKRs: The Simple Idea that Drives 10x Growth. Penguin Books Limited.