The data revolution has been a blessing and a curse for businesses and communicators, and the arrival of AI has only made things more complicated. And now, new research from cloud data engineering consultancy Hakkōda finds that virtually all companies are going to need a full data stack overhaul before year’s end.
The firm’s newly released State of Data 2024 report, based on a survey conducted by Lawless Research, indicates that with new AI capabilities quickly being integrated into business IT—a substantial 50 percent of organizations are actively implementing AI for limited use cases such as automation—every organization surveyed views generative AI as important, and nearly two in three organizations reported that generative AI will be very or critically important to their success by 2027. But to get the best results, 94 percent of all organizations polled say they need to upgrade their data systems this year.
The report also found that most organizations say they have the necessary AI skills and expertise to support the use of GenAI: 42 percent of respondents agreed strongly and 44 percent agreed moderately.
Among the survey highlights:
- 70 percent of organizations have already deployed some of their data in the cloud
- 64 percent of organizations intend to monetize their data in 2024
- 45 percent of C-level are extremely confident their data team can build the generative AI capabilities they need, while only 28 percent of their VP-level counterparts feel the same
To gauge how companies utilize their current data stack and prepare for the future in their respective industries, the report also differentiates organizational data maturity. Best practice-based evidence of harnessing and optimizing data systems is used to classify four stages of sophistication in a company’s data journey moving from chaos to order to Insight to Innovation. Those stages of maturity produced notable variance in survey responses.
Those in chaos don’t know they’re in chaos
Nearly all organizations say they need to modernize (94 percent), and nearly half of all organizations (48 percent) say they need to modernize their data systems “a great deal” in 2024. Only 24 percent of chaos organizations report needing to modernize their data systems “a great deal” in 2024, while 66 percent of Innovation organizations make that determination, indicating vastly different perceptions of capability challenges.
Maturity affords ROI bump
On average, organizations reported achieving 126 percent return on investment for their 2023 outlay in data technologies and tools. Chaos organizations reported averaging only 73 percent ROI. Innovation organizations reported an average ROI of 164 percent.
Companies with high ROI data strategies leverage outside expertise
Innovation organizations outsource 63 percent of data management on average. Chaos organizations outsource 39 percent.
Perception vs. reality in data strategy
While 83 percent of organizations rated their data strategy as effective in 2023, only 56 percent actually reported achieving their strategic goals. This gap is most pronounced among chaos organizations, where the achievement rate falls to 43 percent.
“The report underscores the imperative need for modernizing data strategies and a growing focus on AI integration,” said Erik Duffield, CEO of Hakkōda, in a news release. “It also highlights a chasm between organizations that are leading in innovation and those playing catch up in chaos. Those who use managed services move further and faster. At Hakkōda, we are focused on bridging this gap.”
Download the full report here.
The blind survey was conducted by Lawless Research in December 2023 of 500 data leaders from large organizations across major industries, holding director-level or higher positions in IT and analytics roles at US companies with 1,000 or more employees. 20 percent of respondents came from organizations with less than $500M annual revenue; 29 percent with $500M to $999.9M annual revenue; 37 percent with $1B to $9.9B annual revenue; and 14 percent with $10B or more annual revenue.
