Modern system engineering offers the vision of perfect integration – requirements seamlessly linking design and test, test updating models and returning results to designers, so that all stakeholders use the same live data. In practice, few – if any – organizations have achieved anything close to this ideal state, not because the technology isn’t there, but because the data is impossible.

The shift from hardware to data management

Systems engineering used to be about dealing with physical complexity – parts, connections, tolerances, steps in the process. That challenge is still around, but today’s systems engineers are overwhelmed by a different trend: extreme heterogeneity of data and tools across diverse disciplines. They’re using the best tools they have available to do their work, and keeping everything in sync with point-to-point connections and low-value exports and imports. Making something that feels like a truly integrated environment.

Open standards and ontologies as a foundation

The path to overcome this problem from a technical perspective is based on open standards. Specifications like SysML and exchange standards like ISO 10303 (STEP) are designed in such a way that they provide engineering data a common language in which no software vendor is the owner. Thus, when an organization stands on these standards as the main pillar of their data architecture, it is less important what software any specific unit adopts, as the data that any unit exposes is in a shape that anyone can read and validate.

Ahead of standardization is the ontology, which does not limit itself to defining formats, but defines the meaning. A formal ontology tells what a “requirement” is, what a “test case” is, and what the relationship among them signifies, in a way that different tools and different people can agree on. This is what prevents data from silently losing content as it moves between systems since the meaning is shared for both machines and humans.

The organizations that are taking these problems seriously are also turning to digital engineering services providers experienced in building the integration pipelines, legacy-to-cloud bridges, and MBSE environments that their internal engineering teams don’t have the time to design and maintain. These are tailor-made solutions. Integration has to land upon the tool stack and data architecture that each organization has.

How proprietary formats break the digital thread

The digital thread is a simple concept, it’s just really hard to implement in practice. This is a hard, if not impossible, problem to solve with a single-vendor solution. Data has to flow through dozens of incompatible tools on most product teams. There are thousands of tools available, new tools being adopted, and others being discarded, all the time. There is no one tool to rule them all, nor will there ever be.

Solving the digital thread essentially requires you to decouple your engineering data from its tools. Instead of relying on specific tools and the people who know how to use them to access your most essential corporate asset, you need dedicated data management that moves with your data as it flows in and out of disparate tools and organizations.

This isn’t purely theoretical. There are organizations that have done it. The Air Force has been developing its digital thread for over a decade now with a combination of data standards and middleware. The Navy used a similar approach to develop their Common Product Model. NASA has had one for even longer. These solutions work; they’re just out of reach for all but the largest organizations with the most to gain.

The legacy system problem

What further complicates the issue is that often the tools that create the most friction are the ones that are most difficult for the team to transition away from. You can’t keep years of change and configuration management data, often plain old files, and SharePoint drives, say nothing of IP-rich engineering content tied up in PLM, locked in an old simulation environment, or squirreled away in a proprietary database with a high learning curve.

So, the teams will be stuck for the foreseeable future with these legacy systems, and their overly complex editors, buggy file conversion utilities, and cumbersome export wizards. The same tools that were built from the ground up to maximize lock-in, adding friction to change with every wasted hour searching for an off-site backup created three years ago.

The cultural dimension

While the technical aspects are important, they are not the only obstacles to overcome. Data silos in engineering organizations often mirror organizational silos. Different teams share little data, tools, and processes because that’s how they maintain control over their domain. By asking everyone to contribute data to a shared model and adhere to shared standards, you are in turn asking people to relinquish ownership.

This means that interoperability needs to be considered as a managerial challenge as much as it is an engineering one. Program and project managers need to make the deliberate decision to foster a single source of truth and be prepared to force that decision even when this is not how teams are accustomed to operating. In the end, no technical solution can compensate if an organization rewards the hoarding of data in one’s own domain.

The long-term benefits of enabling genuine interoperability in your engineering ecosystem cannot be overstated. Systems are getting more and more complex and development cycles are shortening. Only considering the technical aspects, the cost of converting data manually will become unsustainable. Those who establish interoperability today are those who will be able to confidently race, tripping up less on integration problems, for the most rewarding programs.