Gartner Reveals the 2017 Hype Cycle for Data Management

SINGAPORE, October 2, 2017 — As data becomes ever more distributed across multiple systems, organizations have to cope with increasingly complex ecosystems and digital business requirements. The Gartner, Inc. Hype Cycle for Data Management* helps CIOs, chief data officers (CDOs) and other senior data and analytics leaders understand the maturity of the data management technologies they are evaluating to provide a cohesive data management ecosystem in their organizations.

“Data management continues to be central to the move toward digital business. As requirements change within the architecture of the organization and place greater demands on underlying technology, the maturity and capability of many of the technologies highlighted in the Hype Cycle will advance rapidly,” said Donald Feinberg, vice president and distinguished analyst at Gartner. “Recent years have seen many new additions to the Hype Cycle (see Figure 1), including in-memory, cloud, data virtualization, advanced analytics, data as a service, machine learning, graph, non-relational and Hadoop.”

Source: Gartner (September 2017)

Two technologies are of particular interest, in that they show the impact cloud computing is having on the data management discipline. Hadoop distributions are deemed to be obsolete before reaching the Plateau of Productivity because the complexity and questionable usefulness of the entire Hadoop stack is causing many organizations to reconsider its role in their information infrastructure. Instead, organizations are looking at increasingly competitive and convenient cloud-based options with on-demand pricing and fit-for-purpose data processing options.

As part of the same cloud-led trend, SQL interfaces to cloud object stores have appeared at the Innovation Trigger stage. “We expect these interfaces to represent the future of cloud database Platform as a Service (Paas) and reach the Plateau within two to five years because they are the focus of most cloud vendors and products in this space,” said Mr. Feinberg. “They enable organizations to interact with data stored in the cloud, using a familiar SQL syntax. Object stores are well suited to storing large volumes of multistructured data, typical of data lakes.”

Of the 35 other technologies highlighted on the 2017 Hype Cycle for Data Management, four are judged to be transformational in nature. Two — event stream processing (ESP) and operational in-memory database management system(IMDBMS) — are expected to reach the Plateau of Productivity within two to five years, while both blockchain and distributed ledgers are expected to take five to 10 years.

ESP is one of the key enablers of digital business, algorithmic business and intelligent business operations. ESP technology, including distributed stream computing platforms (DSCPs) and event processing platforms (EPPs), is maturing rapidly. Stream analytics provided by ESP software improves the quality of decision-making by presenting information that could otherwise be overlooked.

Operational In-Memory database management systems (IMDBMS) technology is maturing and growing in acceptance, although the infrastructure required to support it remains relatively expensive. Another inhibitor to the growth of operational IMDBMS technology is the need for persistence models that support the high levels of availability required to meet transaction SLAs. Nevertheless, operational IMDBMSs for transactions have the potential to make a tremendous impact on business value by speeding up data transactions 100 to 1,000 times.

Public distributed ledgers, including blockchain, continue to have high visibility, although organizations remain cautious about the future of public (permission-less) distributed ledger concepts due to scalability, risk and governance issues. Most business use cases have yet to be proven and extreme price volatility in bitcoin persists. Presupposing the technical and business challenges of distributed ledgers can be overcome; in the short term, organizations are most likely to use distributed ledger for operational efficiency gains via the use of shared information and infrastructure. Longer term, Gartner expects a complete reformation of whole industries and commercial activity as the programmable economy develops and ledgers contribute to the monetization of new ecosystems.

The requirements for more standards and enterprise-scale capabilities are evolving slowly, but distributed ledgers are still not adoptable in a mission-critical at-scale context. Their value propositions, compared with existing technology, are also not clearly established, making the widespread acceptance of the technology problematic. Private distributed ledger concepts are gaining traction, because they hold the promise to transform industry operating models and overcome some of the issues of scalability, risk management and governance that plague public ledgers. As with blockchain, however, many business use cases are unproven at this time.

Gartner clients can learn more in the report “Hype Cycle for Data Management 2017.” This research is part of the Gartner Trend Insight Report “2017 Hype Cycles Highlight Enterprise and Ecosystem Digital Disruptions.” With over 1,800 profiles of technologies, services and disciplines spanning over 100 Hype Cycles focused on a diversity of regions, industries and roles, this Trend Insight Report is designed to help CIOs and IT leaders respond to the opportunities and threats affecting their businesses, take the lead in technology-enabled business innovations and help their organizations define an effective digital business strategy. Information on further 2017 Hype Cycles covering data and analytics can be found on the Gartner Blog Network.

*Previously titled “Hype Cycle for Information Infrastructure, 2016, “Hype Cycle for Data Management, 2017” covers the broad aspects and technologies that describe, organize, integrate, share and govern data.

Gartner analysts will provide additional analysis on data and analytics leadership trends at the Gartner Data & Analytics Summits, taking place: November 20-21, 2017 in Frankfurt; February 26-27, 2018 in Sydney; March 5-8, 2018 inGrapevine, Texas; and March 19-21, 2018 in London. Follow news and updates from the events on Twitter using #GartnerDA.

Gartner, Inc. (NYSE: IT) is the world’s leading research and advisory company. The company helps business leaders across all major functions in every industry and enterprise size with the objective insights they need to make the right decisions. Gartner’s comprehensive suite of services delivers strategic advice and proven best practices to help clients succeed in their mission-critical priorities. Gartner is headquartered in Stamford, Connecticut, U.S.A., and has more than 13,000 associates serving clients in 11,000 enterprises in 100 countries. For more information, visit www.gartner.com.