The Frozen Middle: The Demand for Data-Driven Leadership

Organisations are increasingly aware that data-driven decision-making is here to stay. However, developing the necessary leadership skills within the workforce to leverage data and drive transformation remains a challenge. Middle managers are fast becoming a “frozen middle” and face particular difficulties in effective data leadership.

As AI continues to dominate the workplace and technologies transform at lightning speed, organisations that cannot rise to this challenge risk being left behind altogether. Achieving a skills synergy between data and AI will facilitate intelligent decision-making and creates an agile environment that organisations need to stay ahead in a landscape that constantly demands adaptation.

The UK Business Data Survey 2022 gathers evidence on the importance and awareness of digital data use for UK businesses; their 2022 survey revealed that nearly 7 out of 10 business leaders expressed the need for greater data skills to help improve their business growth. The primary barriers preventing businesses from becoming data-driven organisations include a lack of analytical skills among employees and governance issues.

Chief data and analytics officers (CDAOs) are poised to be of increasing strategic importance to their organisations. Still, many are struggling to make headway, according to data presented last week by Gartner at the Gartner Data & Analytics Summit 2023. Fewer than half (44%) of data and analytics leaders say their teams effectively provide value to their organisation. 

The crux of the issue lies in the scarcity of data-driven leaders who can effectively translate data insights into actionable change within their organisations. Many companies struggle to find talent that understands their business and has the technical abilities and skills to make informed decisions based on data. This problem is further exacerbated by the fact that it is nearly impossible for organisations to hire their way out of this issue, as the demand for such talent far outstrips the supply.

The frozen middle poses significant challenges to organisational transformations and data-driven initiatives. Many middle managers lack the technical abilities and foundational skills to make data-driven decisions and respond effectively to emerging issues. They will struggle to produce strategic results due to poor data literacy and ineffective change management.

As companies look ahead, they must ask themselves where these data-driven leaders will come from. A report by AND Digital, reveals that 72% of business decision-makers agree that the ability to collect, analyse, and make meaningful decisions with data has become more critical in the last 18 months. Despite this, only 24% of companies describe themselves as data-driven, meaning they use the data they collect and analyse to inform their decision-making.

While the importance of data-driven decision-making continues to grow, there is still a lack of data-driven leaders in the country. McKinsey claims that “by 2025, smart workflows and seamless interactions among humans and machines will likely be as standard as the corporate balance sheet, and most employees will use data to optimise nearly every aspect of their work.”

The lack of data leadership in organisations signifies a disconnect between the potential of data and its alignment with the broader context of organisational challenges and goals. To transform into a data-driven business, anchoring data discussions within the framework of organisational growth is imperative. By emphasising tangible outcomes driven by data, like revenue growth, cost reduction, profit growth, and the downstream benefits of enhanced customer experience and improved operational efficiencies, the way people work will transform. This approach ensures that data initiatives are not isolated endeavours but integral to the strategic vision that propels an organisation forward, empowering employees at all levels with data-driven skills.

The Data Sharing Governance Framework, released by GOV.UK, revealed in 2022 that only 59.5% of executives reported that their companies were driving business innovation with data, showing no change from four years prior. This highlights the urgent need for companies to take a strategic approach to developing data literacy skills among their employees, particularly middle managers, who play a crucial role in translating data insights into actionable decisions.

Data-driven leadership is essential for organisations looking to stay competitive in today’s business environment. Early adopters of data-driven skills development have already witnessed significant success; data-driven businesses are 20-plus times more likely to acquire new customers and six times more likely to retain them, highlighting the significant impact of data-driven leadership on an organisation’s success in the UK market (McKinsey). 

By 2025, AI technology is anticipated to create 12 million more jobs than it replaces, with 97 million specialists needed in the AI industry by 2025 (Gartner). AI adoption has led to a significant increase in productivity, with 64% of businesses reporting AI to boost their productivity levels, as companies seeing the highest financial returns from AI continue to pull ahead of competitors (McKinsey).

If middle managers have a data skills deficiency, it can only be expected that they will fail to rise with the tide of AI use in the workplace. Making informed decisions based on data, coupled with the efficiencies AI tools offer, will allow companies agility when the digital landscape continues to sift so rapidly. Managing data change from the middle will be crucial to any organisation’s success in the years ahead.