Will Data Historians Die in a Wave of IIoT Disruption? A transformation in data historian thinking will happen!

A group of us were asked to comment on this article by Matthew Littlefield , President and Principal Analyst, LNS Research, on October 22, 2015. It certainly is an integrating questions, and valid question in the current industrial , operational transformation that is happening around us. As we answered it on email, I thought it is a valid topic for blog discussion.

http://www.automationworld.com/databases-historians/will-data-historians-die-wave-iiot-disruption

My immediate first response is “that the traditional thinking of industrial data historians will transform”. Actually it is already transforming, due to type , volume, and required access to the data. It is important to not look at the situation as a problem, but as a real opportunity to transform your operational effectiveness through increased embedded “knowledge and wisdom”:

The article raises the question of how or is this a disruptive point in the industrial data landscape, I would argue that is a “transformation point”.

Mathew states in the article:

“Even so, one area of the industrial software landscape that many believe is ripe for disruption is the data historian. The data historian emerged out of the process industries in the early 1980s as an efficient way to collect and store time-series data from production. Traditionally, values like temperature, pressure and flow were associated with physical assets, time stamped, compressed, and stored as tags. This data was then available for analysis, reporting and regulatory purposes.
Given the amount of data generated, a modest 5,000-tag installation that captures data on a per-second basis can generate 1 TB per year. Proprietary systems have proven superior to open relational databases, and the data historian market has grown continually over the past 35+ years.
The future may seem very bright for the data historian market, but there is disruption coming in the form of IIoT and industrial Big Data analytics.
As these systems have been rolled up from asset or plant-specific applications to enterprise applications, the main use cases have slightly expanded, but generally remained the same. Although there is undisputed incremental value associated with enterprise-level data historians, it is well short of the promise of IIoT.

In our recent post on Big Data analytics in manufacturing, I argued that Big Data is just one component of the IIoT Platform, and that volume and velocity are just two components of Big Data. The other (and most important) component of Big Data is variety, making the three types structured, unstructured and semi-structured. In this view of the world, data historians provide volume and velocity, but not variety.

If data historian vendors want to avoid disruption, expand the user base, and deliver on the promise of IIoT use cases, solutions must bring together all three types of data into a single environment that can drive next-generation applications that span the value chain.

It is unlikely that the data historian will die any time soon. It is, however, highly likely that disruption is coming, making the real question twofold: Will the data historian be a central component of the IIoT and Big Data story? Which type of vendor is best positioned to capture future growth—traditional pure-play data historian provider, traditional automation provider with data historian offerings, or disruptive IIoT provider?

If the data historian is going to take a leadership role in the IIoT platform and meet the needs of end users, providers in the space will have to develop next-generation solutions that address the following:

·         How to provide a Big Data solution that goes beyond semi-structured time-series data and includes structured transactional system data and unstructured web and machine data.

·         How to transition to a business/pricing model that is viable in a cheap sensor, ubiquitous connectivity, and cheap storage world.

·         How to enable next-generation enterprise applications that expand the user base from process engineers.”

The comments are very valid, that the data we now capturing is increased in both volume and variety, but I would argue that it needs to transformed into contextualized information, to knowledge so that  proportional wisdom growth can occur. The diagram below shows the potential direction many companies can go, of blowing out on data and not gaining the significant advantage of wisdom for operational efficiency from the increased data in the Industrial “sea”.

The way in which people will access and use data is transforming, they not using it just for analysis on traditional trends etc. They are applying big data tools, and modeling environments to understand situations early in assets condition, operational practices, and process behavior.

They are expecting to leverage this past history to predict the future through models that “what ifs” can applied. They are expecting access to their answers from people who with limited experience, in role or location (site/ plant awareness). They will not use traditional tools, they will expect “natural langue search” to transverse the information, and knowledge “ no matter where the location.

The article took me back to a body of work I collaborated on with one of the leading Oil and Gas companies around “Smart Fields” and in those conversations we talked about the end of the historian as we know it, due to the distributed nature of data capture, and the availability of memory, why would historise to disk vs leave the history in the device in memory.

I think this really drives the thought pattern around how the data is used, and the key 3 are:

• Operational “actionable decisions”
• Operational/ process improvements, through analysis and understanding to build models that transform situations in history to knowledge about the future.
• Operational, process records archiving.

The future is federated history that partitions the “load” between most-recent transient fast history in the device itself (introducing a concept of  “aggregators”) with periodic as-available uploads to more permanent storage. These local devices will have their own memory storage and can “aggregate” the data to central long term storage.

But when you are access information in the now you will not go to historian, you will go to the information model, that will navigate across this “industrial sea” of data and information, delivering it fast, and in a knowledge form.

So is the end of historian here, I would say no, but certainly as the article points out the transformation of the enterprise information system is happening, so are the models you will buy, manage, access the data.  

Applied Knowledge/ Wisdom Foundational to Internet of Things, and "Time to Performance" of Operational Teams

For the last couple of weeks, Stan DeVeries and I have been brainstorming around articulating this core area of the operational transformation, "the ability to have a system that can absorb workforce change/ turn over". Good example of this is with one company on the 2025 vision of "all knowledge/ experience in the system", this is capturing as much of the tribal "applied knowledge" that the experience operational staff are making decisions, and taking actions on and moving it to the system. If then applied in a "activity/task" based operational experience, a younger skilled user has the ability to select a "activity" and the associated knowledge/ information, and action are presented to him. Dramatically reducing the "Time to Performance" and increasing the consistency of operations, while increasing flexibility in operational workforce management.

The results of the discussions has brought the discussion around "Federated Wisdom, applied knowledge":

The explosion of information across industrial operations and enterprises creates a new challenge – how to find the “needles” of wisdom in the enormous “haystack” of information.

One of the analogies for the value and type of information is a chain from “data”, through “information” and “knowledge”, to “wisdom”.  In the industrial manufacturing and processing context, it may be helpful to use the following definitions:

·         "data” – raw data, which varies in quality, structure, naming, type and format

·         “information” – enhanced data, which has better quality and asset structure, and may have more useable naming, types and formats

·         “knowledge” – information with useful operational context, such as proximity to targets and limits, batch records, historical and forecasted trends, alarm states, estimated useful life, efficiency etc.

·         “wisdom/Applied Knowledge” – prescriptive advice and procedures to help achieve targets such as safety, health, environment, quality, schedule, throughput, efficiency, yields, profits etc.

The cost to store and share data has dropped significantly, and a simplistic expectation is that although storage is growing by a factor of millions in only a few years, that somehow the following pattern evolves:

Although the pattern might seem to be convenient, it is actually a nightmare, because it becomes much harder to discover and translate knowledge and wisdom from another operation, especially in another location, to the local needs.  But there is a solution.

To understand the problem better, let’s consider the definition of “knowledge” – it includes context.  This context begins with local context – time, location, process or machinery configuration, raw materials, energy and products being processed or produced.  It is already valuable to have “wisdom” to achieve and sustain best performance for the community, customers and the corporation.  This local context only needs to know its immediate information, if it has enough “wisdom”.

Now let’s consider what happens when a single site, a fleet of similar sites, or an enterprise have numerous similar operations.  How can local “wisdom” be enhanced by using “wisdom” from the other operations, especially when all of these operations are sufficiently different?

The reason that solving this problem is important is for operations transformation, such as operating physical assets as one (in a chain or as peers), and by supporting the multiple operations with a flexible team of remote experts.

One approach to solving this problem is to take advantage of a technique used in distributed databases, where a technique called “federated information” is used, especially in industrial operations management architectures.  This technique does not change the local information’s naming or structure, but provides multiple translations, both across the database for multiple similar structures, and for multiple contexts such as what financial, technical support, scheduling, quality and other functions require.  This technique is an alternative to the fragility and complexity of attempting to force a uniform and encompassing naming and structure that attempts to satisfy all applications and users.

The same approach can be applied for “wisdom”.  Currently, hobbyists and enthusiasts around the world share “wisdom”, for restoring cars, making furniture, playing a musical instrument, gardening etc.  Anyone with no experience at all can ask for “where do I get started?”, and most respondents will provide kind advice; in the same forum, experts can share wisdom that is valuable and understandable by them at their level of experience.  This “wisdom” is extremely decentralized, and the experts are providing the translation.

In the industrial operations environment, federating “wisdom” is partially automated by expanding the local context.  This expansion includes information about adjacent operations, information about the chain or peers if these operations are being managed as one, and then “knowledge” is expanded by applying the context of group targets and performance.

Some enterprises have hundreds or as much as tens of thousands of similar operations, supported by dozens or fewer experts.  Discovery of wisdom is greatly enhanced by maintaining an architecture which enhances local context without modifying or attempting to force burdensome structures on local operations.

Expect this discussion to continue as expand on the systems, and approaches to make this real, while enable sustainable operational innovation. This will be core to Industrial Internet of Things as we align smart devices, operational practices and humans into a dynamic but coordinated operational force.