While boards of directors are pushing for the adoption of artificial intelligence, chief information officers and other IT executives know it’s important to do more than establish solid use cases for AI.
Purkalpa Sankar, co-founder of data catalog and governance software Atlan, says the challenge holding back technology leaders from adopting AI is not actually generating and deploying models. Rather, she said, the data is not ready for AI. “Everyone is ready for AI, except data,” Sankar says.
A recent global survey of more than 1,300 technology and data executives found that only 18% of companies say they are fully ready for AI adoption, which is the lowest percentage of companies that have fully accessed their data. It means being accessible and integrated (another 40% think they are almost ready, but not quite ready).
Sankar said companies need to overcome several hurdles to reach that stage of preparation. The first is finding and organizing all the data, which is primarily the job of data engineers. “We’re looking at consolidating data that was siled across different business units and really putting it into specific use cases,” she said.
Companies also need to complete complex data tagging and classification, primarily to keep private data within reasonable bounds. “Depending on who is asking the question, you can change the data behind it,” Sankar said. For example, an HR chatbot may be able to use payroll data, but the entire chatbot may not be.
With AI, data governance becomes less strict
All of this falls under the umbrella of data governance, or how companies manage their data assets through policies, processes, and standards. Matt Carroll, CEO and co-founder of data security platform Immuta, said data governance is not new, but AI will change the way it is done.
“If you think about traditional business intelligence, which we’ve been working on for 30 years, governance has always been a structured, well-oiled machine,” Carroll said. “When you deploy AI, you can’t do it the same way.”
This is because companies must constantly add new data to support their AI models from both internal and external sources.
Ultimately, Carroll said, AI readiness comes down to three things. “You need to be able to find the data, you need to be able to use it, and you need to be able to observe how the data is being used.”
Having a mature data governance pipeline is not common across industries. At least, it’s not common yet. MIT’s 2024 AI Readiness Report found that data governance, trust, and security are more important in government and financial institutions than in other industries. Carroll said this practice needs to extend beyond banks and governments, as they are not the only industries that handle sensitive data. Any company pursuing generative or other types of AI solutions will need to dance with those departments, as well as IT, legal, and broader organizational executives.
Additionally, Carroll would like to see more companies implement continuous data preparation even after deploying AI. One way companies can do that is through an AI hotline. This could be a full hotline for larger companies, or a more accessible moderated Slack channel for smaller companies. Importantly, domain experts can report issues such as hallucinations or incorrect data tagging directly to the engineering team.
“They need a feedback loop, so maybe a model review board could remove it, reevaluate it, or flag it for retraining or revalidation,” Carroll said. Said. game. “
Of course, this is in addition to continuous testing of the model to check for any unusual behavior and ensure that the model meets the company’s quality standards.
Businesses are creatively preparing for AI
Sankar said he has seen companies create AI readiness scores to quantify the process of organizing their data since the beginning of their AI adoption efforts. A measurable score for AI readiness may rank data sets from 5.0 based on a variety of factors. “Nothing moves unless you measure it,” she said.
Another trend that experts are noticing is adding the secondary title of data steward to an employee’s primary role. “You’re in business and you happen to know the field, and all of a sudden you have this dataset that may or may not be used for AI,” Carroll said. Additionally, highly specialized data governance (e.g., people with formal titles such as data governance executive or data management engineer) is difficult to find, but increasingly important, and likely to become even more important in the future. He said that there is.
Sankar likens the data infrastructure ecosystem to a market. “On one side of the market are business-ready AI use cases,” she said. “On the other hand, you also have a complex data infrastructure.”
Experts agree that data preparation must be a top priority for organizations pursuing AI. But even the broad category of data preparation can be further broken down. Before tackling step one, Carroll said it’s worth asking questions that may be unpopular among executives. “In data preparation, there’s also the question of whether we need to do it in the first place.” This means that every company should be careful about whether they need to expose certain types of data to their systems or not. What Carroll means is that a decision has to be made. Only with this green light can companies truly pursue AI readiness.
