Content area
[...]legacy systems and sprawling technical debt serve as inhibitors to modernization. "[E]ven elite data scientists cannot truly define a business problem, understand the data, nor deploy a model into a business process without their help. Current data professionals, even those well-versed or certified in advanced database skills such as Oracle or SQL Server, need to be kept current as enterprise data requirements expend.
While the database management space has been abuzz with activity lately, one thing is clear: Enterprises are depending on their data teams more than ever to support a dizzying array of new initiatives, from AI to robotics to edge computing. This is because, in the end, nothing can function without reliable, quality data.
"If there is a lesson to learn from Al breakthroughs so far, it is that in most cases data trumps algorithms," said Michael Bronstein, DeepMind professor of Artificial Intelligence at the University of Oxford, in a recent report. "Ensuring data quality standards, reproducibility, and tracking provenance are crucial to developing trustworthy AI systems."
Are data teams ready to meet this challenge and support the AT-driven enterprises that will dominate markets in the latter half of the 2020s?
Theres no shortage of tools and technologies to help data leaders and their teams move forward into the enterprise АТ frontier to support instances of generative Al, agentic AI, and more traditional computational AI. However, tools and technology are only one side of the story - these environments must be managed and aligned with their businesses, ensuring greater speed, scalability, and flexibility in how data is processed, stored, accessed, and analyzed.
The need is urgent, as data environments responding to these challenges are growing in size and complexity. This means data teams need to step up the modernization of their data infrastructures. "An average of 41% of systems are considered 'beyond end of life or support; and an average of 38% are incompatible with target architectures and new projects; a by Gartner found. As a result, legacy systems and sprawling technical debt serve as inhibitors to modernization.
The following are essential considerations in today's data transformation efforts:
Staying attuned to the business. Again, businesses are depending more heavily than ever before on the capabilities and services offered by their data teams. But data managers and professionals need to thoroughly understand what their businesses seek in terms of AI, real-time responsiveness, and other data-driven ventures-and work closely with the business side. "It is easy for non-technical employees to get lost in the shuffle" when it comes to implementing and using data technology, write well-known data analytics advocate Thomas Davenport and a group of co-authors in the June 7, 2023, edition of Harvard Business Review. "[E]ven elite data scientists cannot truly define a business problem, understand the data, nor deploy a model into a business process without their help. Nor can you make the needed improvements to data." They go on to say, "Instead, we advise senior leaders to recruit as many regular people involved as possible into their data effort. Practically everyone can bring small data and basic analytics to improve their team's performance. ... And, for many companies, they appear to be prerequisite for taking on bigger data and more advanced techniques."
Democratizing data, analytics, and AI. An intelligent enterprise requires information sharing at all levels. One of the keys to staying attuned to the business is by boosting self-service and the availability of data to all that need it to do their jobs. This is enabled by deploying user-friendly tools and platforms with which nontechnical users can conduct queries. Low- and no-code solutions may be a fit here.
Employing AI to develop and maintain AI. While AI is the reason data environments need to be modernized and expanded, it also will help facilitate such capabilities. For example, АТ can automatically classify data and help flush out anomalies that may be arising in the data or supporting systems. AI will also help ensure higher levels of data quality by catching duplicates or incomplete entries. Al also can be employed to sense and respond to security breaches or human errors. It can also help predict bottlenecks in data pipelines.
Firing up compute resources. When it comes to compute capacity, cloud- public or hybrid-offers the best options for data management. This includes employing more than one cloud provider, as well as mixing and matching data and resources through hybrid cloud and on-prem arrangements. Multi-cloud and hybrid strategies provide great flexibility to manage, process, and store the rising volumes of data.
Providing proper homes for data. Data can reside on-prem or in the cloud. Across both settings, there are a variety of places where data can land and be maintained, including data warehouses, data lakes, data lakehouses, and data fabric. Data warehouses can supply data in structured or transformed formats for use by analytics tools; data lakes can be deployed to capture unstructured data from any and all sources for later action by future applications. A data lakehouse brings both of these functionalities together, ensuring raw data storage as well as structuring for later analysis. A data fabric architecture serves as a highly distributed layer that enables unified access for data from all sources.
Ensuring observability and governance. As data in all its forms pulses through organizations, it's important that it be monitored and serves its intended purposes. There are numerous tools and platforms now on the market that facilitate such greater visibility to ensure continuous performance. In addition, data governance tools and platforms help manage data ownership and compliance with both corporate and governmental rules.
Preparing people for the transformation. In fast-changing data environments, it's critical to keep data teams up-to-date with tools, platforms, and processes. Current data professionals, even those well-versed or certified in advanced database skills such as Oracle or SQL Server, need to be kept current as enterprise data requirements expend. This includes offering training and coaching opportunities to support the new data initiatives sought by the business-be it AI, real-time capabilities, or edge computing.
The world is changing fast, businesses are changing fast, technologies are changing fast, and data teams need to keep up. Importantly, this is an opportunity for data managers and professionals to take leadership and advisory roles within their organizations.
"An entirely new 'management paradigm' for data is needed. [It] embodies a common language, a holistic vision of the ways data should contribute, a clearly defined organizational structure showing how data integrates across the organization, along with clear roles and responsibilities for all involved," Davenport and his co-authors advise.
"Eventually, it needs to incorporate corporate culture, relationships with universities and vendors, policy, and anything else that advances, or holds back the effective use of data. The new paradigm employs a more pervasive and integrated approach to using data, analytics, and AI in the business."
Copyright Information Today, Inc. 2025