Boeing Safety Intelligence (BSI)

Summary

The aerospace industry represents a complex system with many data sources and potential safety issues. A holistic look at the data will reveal correlation and causation, rather than a needle-in-a-haystack approach. With this in mind it’s important to identify, the right signals in the midst of the noise. The Boeing Safety Intelligence (BSI) tool was developed to meet this need by combining systems engineering and domain knowledge with advanced analytics to model safety risk.

BSI is a product that provides proactive analytics to spotlight emerging risks and issues across the Boeing Ecosystem. The outcome being the prevention of future accidents and avoidable near misses. The tool was in part an internally at Boeing during safety reviews and externally used by the Federal Aviation Administration (FAA) to review Boeing’s progress towards a Safety Management System (SMS). SMS being a top-down, organisation wide approach to managing safety risk.

My Role

I was the lead Product Designer that developed BSI from initial discovery and framing to the UI implementation and product launch and delivery

Cross Functional Team Engagement

SMS Core team had members from Safety Practitioner SME’s, Data Modelling and Simulation Analysts, Data Engineers, Fleet Chiefs and Business Analysts. These various parties provided context to their impact to the Airplane Safety ecosystem and the various challenge and areas of opportunity that lay to help bridge the divide. The biggest focus being A/P Fleet Chief’s and Data and Modelling Analysts who could modify their safety recommendations and incoming data based upon real data to predict future behaviour.

Data Analytics included partners from Product Management, Information Technology Business Partners, Data Architects, Software Developers, and Information Security. These various teams provided their insight into the supporting systems and technical feasibility aspects needed to develop BSI. These teams provided their core set of resources across the software engineering functions to help initially validate and help integrate the application into the main Boeing Analytics system. This in turn helped identify the highest program impact and lowest risk tech elements to begin prioritise into the outcome roadmap.

Digital Enterprise IT were comprised of the Business and Digital Transformation Team (B&DT). These were the core groups to buy off on the final decision gate the integration of BSI being adopted enterprise wide. They provided their core requirements and the direct outcome of what a successful version of BSI looks like.

User Research

To start off the researching conducted an in depth Discovery & Framing approach to understand the multiple avenues of impact that BSI could take on.

During the framing phase I conducted an in depth user research study where I spoke to over 40 different safety practitioners. This provided insight into the capabilities and outcomes a tool such as BSI would have to proactively identifying hazards and analyse risk to improve safety. The goal of this contextual inquiry, was to get and on the ground understanding of the current work flow and eco system to Safety Analytics.

A service blueprint map was drawn up to start developing a user journey that highlighted a a users emotional engagement with safety analytics applications from initial start to ending, touch points, and back end system architecture. 3 main user groups of BSI were identified from that activity.

From that activity assumption gathering began where the team started identifying and documenting hypothesis we had about users needs and wants. User Personas were also archetypes to tangibly identify the tasks, pain points and frustrations, needs, behaviours, technology and areas of excitement to help tangibly map some areas of opportunity. From the user research I themed the research into main categories of information and data that would be valuable to know.

Core Users

Fleet Chiefs - Each A/P Program - the 737, 747, 767, 777, 777X, 787 and their variants have a Chief Engineering Leader who proactively reviews analytics. The individual Chief Engineering Leader reviews the performance reports of the inflight performance and also the production system analysis to understand challenges and areas of opportunity to close the gap. They would use BSI is to see where these emerging issues and hazards are today in their Airplane Program and where they might be trending in the near future to mitigate risks.

Business Unit Leadership - is composed of 3 main business units and several subsidiaries. The main ones being Boeing Commercial Airplanes (BCA), Boeing Global Services (BGS) and Boeing Defence & Space (BDS). Each BCA, BGS and BDS leader needs to be across the impacts happening in their fleet, and production system so they can proactively push changes into their business to mitigate risks. They would use BSI in their safety reviews to understand the present and future risks developing.

The Federal Aviation Administration (FAA) - which is the regulatory body for Aerospace governance in the United States and is responsible for oversight into the manufacturing and operation of commercial aircraft. As such - they needed insight into the emerging risks and hazards in the Boeing ecosystem to understand how Boeing is working towards mitigating those from developing into the fleet. They would use BSI to see the transparency and challenge areas

MVP - Core Experience

From an extensive Risk/Impact Generation workshop where the highest impact, low risk and vice versa actions the product team can take on, a final down selection was chosen. The first experience to be developed would the Fleet Safety (the current Boeing fleet), Conformance Assurance (Airplane Production System), Compliance Assurance (Alignment to Governing Laws & Regulations) and SMS Performance (an Safety Management System Internal Retrospective). This pages would focus around users being able to have the digital thread from top layer to the drilldown data. This meant finding the right interaction method to showcase how this gradual reveal experience and user journey might be realised.

UX Roadmap

Part of the user journey development was around the crafting of the 4 main pages of BSI - Fleet Safety Assurance, Compliance Assurance, Conformance Assurance and SMS Performance. Each of these were guided and constructed through the process of design studio sessions, prototyping, usability rounds and stakeholder share outs and buy in before finalising the incorporated product design

Material Design

Accessibility Considerations

Early Core Experience Wireframes

These were the initial BSI wireframes created before beginning usability. The concept was centered around gradual reveal of information to give users a high level view before delving further into the details. At a glance and trends fed up from the detailed pages provided the ability to see the emergent issues and to also provide further data details, akin to a News headline

Screen Shot 2022-07-02 at 2.25.46 pm.png

Screen Shot 2022-07-02 at 2.26.00 pm.png

Screen Shot 2022-07-02 at 2.26.10 pm.png

Screen Shot 2022-07-02 at 2.26.25 pm.png

Screen Shot 2022-07-02 at 2.28.23 pm.png

Screen Shot 2022-07-02 at 2.28.39 pm.png

Screen Shot 2022-07-02 at 2.29.33 pm.png

Screen Shot 2022-07-02 at 2.26.39 pm.png

Screen Shot 2022-07-02 at 2.26.53 pm.png

Usability Feedback

Usability rounds were conducted ahead of development to validate and or disvalidate assumptions that were made during research which had informed the prototype. One of the biggest learns was wanting to see the history of performance, the drivers and the data details at every level of key performance indicator level. Models were also important to be incorporated rather than just metrics to have the proactive elements of analytics. Management of data was also vital to keep levels of insight as it was integrating with the rest of the Boeing Fleet of apps. Finally there also was an interest in an aggregate roll up page to see in quick glance.

Design Studios

An insightful part of the design process for the progression of BSI considering its engineering complexity were proactively running studio sessions with various cross functional partners and users. There was an important focus on psychological safety to ensure everyone felt validated and heard. They were set up monthly sessions where they could share their perspectives from the hats of technical feasibility (development perspective), user practicality (SME experts, decision makers), viability (product teams) and usability (with other designers).

They were setup in a way to start with a '“How Might We” question focusing on what capability we might design to provide the solution to an outcome we’re trying to solve. 2 x 15 minute sketch sessions were practised where everyone could either describe or draw their ideas out (whilst a good ambient Spotify playlist was playing). Afterwards I would collect those artefacts, digitise them and reference them during the wireframe creations to understand intent and right size my design approaches.

Iterative Concepts

Screen Shot 2022-08-02 at 10.13.17 am.png

Screen Shot 2022-08-02 at 10.13.24 am.png

Screen Shot 2022-08-02 at 10.13.34 am.png

Screen Shot 2022-08-02 at 10.13.43 am.png

Screen Shot 2022-08-02 at 10.13.51 am.png

Screen Shot 2022-08-02 at 10.13.58 am.png

Screen Shot 2022-08-02 at 10.14.06 am.png

Screen Shot 2022-08-02 at 10.14.14 am.png

Screen Shot 2022-08-02 at 10.14.22 am.png

The core design was narrowed down based from usability after the main concept was drawn out and design studios iteratively conducted to understand best layout. 10-12 users composed of the direct or tertiary shadow users were tested to understand intuitiveness and how it stacked against the core heuristics. Feedback was around keeping visuals lean and also only presenting the most relevant information at the top level, leading to a gradual reveal of information.

KPI Metric Carousel

Gradual Reveal

Index Cards

Iterative Design Approach

Screen Shot 2022-07-29 at 8.05.06 am.png

Screen Shot 2022-07-29 at 8.05.28 am.png

First Core Experience

I took those learns from the usability and carried them over into the final designs. UI wise - BSI took shape from using the Material Design language. I used Sketch + Invision Studio to illustrate the interaction design and tested them with users before settling on these final designs.

Screen Shot 2022-07-02 at 4.32.31 pm.png

Screen Shot 2022-07-02 at 4.32.41 pm.png

Screen Shot 2022-07-02 at 4.32.49 pm.png

Screen Shot 2022-07-02 at 4.33.04 pm.png

Screen Shot 2022-07-02 at 4.33.15 pm.png

Screen Shot 2022-07-02 at 4.33.23 pm.png

Screen Shot 2022-07-02 at 4.33.34 pm.png

Screen Shot 2022-07-02 at 4.33.44 pm.png

This was the first page developed. Fleet Safety page provides insight into the current Boeing airplanes in operation and the associated key performance indicators. In here there’s the example of how a user can navigate the page and drill down to the specific KPI level to see what the performance history, drivers and data details provide. On the right hand side is an index card which is present on nearly every page providing a high level summary of what’s happening on the page. Very similarly structured pages of Conformance Assurance, and Compliance Assurance followed.

Responsive Design Breakpoints

Screen Shot 2022-08-30 at 6.02.24 pm.png

Screen Shot 2022-08-30 at 6.03.50 pm.png

Screen Shot 2022-08-30 at 6.02.40 pm.png

Screen Shot 2022-08-30 at 6.02.49 pm.png

Screen Shot 2022-08-30 at 6.02.59 pm.png

Simpler Card

Overview

Screen Shot 2022-08-30 at 7.42.49 pm.png

Screen Shot 2022-08-30 at 7.43.26 pm.png

Screen Shot 2022-08-30 at 7.43.09 pm.png

Screen Shot 2022-08-30 at 7.43.35 pm.png

Screen Shot 2022-08-30 at 7.43.18 pm.png

Screen Shot 2022-08-30 at 7.43.53 pm.png

An overview page was added to enabled users to source content from all the various portions of the tool to highlight trending risks and emerging hazards.

Modelling Pages - Close Calls

Screen Shot 2022-07-02 at 6.56.15 pm.png

Screen Shot 2022-07-02 at 6.56.26 pm.png

Screen Shot 2022-07-02 at 6.56.36 pm.png

Screen Shot 2022-07-02 at 6.56.46 pm.png

Screen Shot 2022-07-02 at 6.56.59 pm.png

Close Calls/Runways Overruns page is a specialised global perspective experience to see where there have been near incidents and misses. The filters at the top provide some down selection criterion so a user can decide for example, how a close call is being defined and a sort of the top 10 percentile. The cards provide a summary of the incident before clicking into the details of the airport or airline.

Admin

Screen Shot 2022-07-02 at 8.14.00 pm.png

Screen Shot 2022-07-02 at 8.14.11 pm.png

Screen Shot 2022-07-02 at 8.17.13 pm.png

Screen Shot 2022-07-02 at 8.17.21 pm.png

Screen Shot 2022-07-02 at 8.17.30 pm.png

Screen Shot 2022-07-02 at 8.17.43 pm.png

Screen Shot 2022-07-02 at 8.17.55 pm.png

Screen Shot 2022-07-02 at 8.18.05 pm.png

Screen Shot 2022-07-02 at 8.18.15 pm.png

Screen Shot 2022-07-02 at 8.18.26 pm.png

Screen Shot 2022-07-02 at 8.19.07 pm.png

Screen Shot 2022-07-02 at 8.19.16 pm.png

Screen Shot 2022-07-02 at 8.19.28 pm.png

Screen Shot 2022-07-02 at 8.19.39 pm.png

Screen Shot 2022-07-02 at 8.15.32 pm.png

Screen Shot 2022-07-02 at 8.15.42 pm.png

Admin is a page for regular users to set custom profile information, and as well for nominated users to manage access and users inside the tool and also setting role permissions.

Mobile

Dev/Design Pairing

As BSI reached its launch, regular implementation sessions were standardised to ensure the designs weren’t introducing beyond scope technical debt and also to help pixel perfection. This meant ensuring complete alignment to Material Design standards and to also evaluate its evolution. At the start of pulse, Material Design 2 was the referenced system, however towards the latter end of the initial MVP Material 3 had been launched. This evolved design had through research updated its design standards to simplify and also iterate the look and feel of common components. As a result when opportunity arose those elements were incorporated into the BSI Design.

Outcome

The Boeing Safety Intelligence application is now being utilised on the executive and airplane leadership program level. Monitored weekly for updates - users can use BSI to help guide their insight into current and future emerging risks and trends. In the next iteration the focus is the build-out of the Boeing Safety Intelligence Solution and the risk models that plug into it. Interrogating various types of data divulges more about correlations and causations of safety risk. There’s further interest actively identifying hazards and risks through data, then tracking and mitigating through the SMS processes. This development will be more integrated and visible outside the company.