Introduction: Massive Budgets, Invisible Results
Most large companies have taken the plunge. They have deployed internal LLMs, launched AI upskilling programs, and invested in group training. Employees now know what ChatGPT is and have heard of “prompts,” “hallucinations,” and “RAG.”
And yet, one question remains unanswered in most boardrooms: does it actually change anything in our operations?
This is the paradox of AI training investments: they are growing fast, but success indicators are almost nonexistent. According to the Deloitte State of AI 2026, 53% of organizations prioritize general education over real AI skills restructuring because the data is missing. Companies measure the number of employees trained, module completion rates, and sometimes immediate satisfaction scores. But actual competency, job function by job function? Nobody maps it.
This article explores why generic AI training has reached its limits and how organizations that want a real ROI need to change their approach.
What the Upskilling Phase Conceals
AI upskilling is a necessary first step. It demystifies the technology, reduces resistance, and brings as many employees as possible into the transformation momentum. No serious program can skip this phase.
But it has a fundamental limitation: it treats everyone the same.
A lawyer evaluating the compliance of an AI-generated contract does not have the same needs as a buyer automating sourcing requests, or a data scientist fine-tuning an internal model. Yet most upskilling programs offer the same common foundation to all staff populations.
The result: teams that have “completed their AI training,” but whose operational proficiency remains unclear, inconsistent, and incomparable from one department to another.
Three Reasons Why Generic Training Is No Longer Enough
1. It does not speak the language of the business
Generic AI training teaches cross-cutting concepts: how LLMs work, bias risks, good prompting practices. That is useful. But it is not what changes the day-to-day behavior of an HR manager, a financial controller, or a customer relations director.
What these profiles need is to understand how AI concretely changes their roles: which tools to master, which risks to anticipate in their specific business context, and what value to create with the use cases that directly affect them. This contextualization is absent from virtually all current generic programs.
2. It cannot measure what it produces
This is the central problem with AI training ROI: inputs are measured (training hours, completed modules, amount invested) but not outputs (level of proficiency reached, ability to use AI autonomously and safely within one’s job role).
Without measuring real competencies, it is impossible to know whether training has had any effect, or to decide where to concentrate future investments.
3. It creates a false sense of security
An employee who has “done their AI training” ticks a box. But that box says nothing about their ability to identify model hallucinations, manage data confidentiality issues, or assess the reliability of an AI output in a high-risk context.
In an increasingly strict regulatory environment, particularly with the AI Act, this false sense of security can become a real risk. Article 4 of the European regulation now requires organizations to ensure a sufficient level of AI literacy for staff operating AI systems. A webinar attendance certificate does not constitute auditable proof of competency.
How to Measure AI Training ROI: The Right Indicators
Moving from an upskilling logic to a competency logic means changing the indicators. Here are the dimensions to measure in order to obtain an actionable picture.
Effective proficiency level, by job function. The goal is not to know whether your teams have heard of generative AI, but to measure their ability to use it autonomously, critically, and safely within their domain. This measurement must be standardized to allow comparison across teams and over time.
Gaps by competency and by population. Which pillars are mastered? Where are the priority gaps? Which departments are behind on AI governance? Which profiles are ready to become internal ambassadors? Without a detailed mapping, prioritizing action is guesswork.
Progress over time. AI competency is not a stable state: it is built, measured, and maintained. A meaningful ROI indicator compares the situation before and after a targeted training action, on the dimensions that matter for each job function.
Alignment with regulatory requirements. In regulated sectors, the ability to produce an objective, auditable proof of teams’ AI competency becomes an indicator in its own right, not just for ROI, but for compliance.
From Generic Training to Targeted Diagnosis: A Change of Paradigm
In practice, how do you break out of the “train everyone, invisible results” cycle?
The first step is to measure before acting. Before rolling out the next training program, map the real level of teams according to their job functions, seniority levels, and the competencies that matter for their roles. This baseline snapshot is the prerequisite for a skills development strategy that hits the mark.
The second step is to contextualize actions by job function. A priority area of improvement for the finance department is not the same as for the procurement teams or the IT department. Breaking action plans down by population rather than addressing everything in a single wave maximizes effectiveness while reducing per-head costs by avoiding irrelevant training.
The third step is to measure impact after the action. What gets measured improves. By re-evaluating the same populations after a targeted action, you have an objective, comparable, and boardroom-defensible progress indicator.
The Role of Diagnosis in an AI Training Strategy
This is precisely what AICET offers: a standardized AI competency diagnostic, contextualized by job function, that transforms previously invisible competencies into actionable data for decision-making.
Designed on a framework of more than 400 competencies covering 9 major job functions (Finance, HR, Legal, Procurement, IT/Tech, Marketing, Customer Relations, Operations, Data & AI) and 3 difficulty levels, it delivers a precise mapping of strengths, gaps, and high-potential profiles within the organization in under a month.
The AICET diagnostic does not replace training, it precedes and guides it. It answers three questions that CHROs and AI leads ask themselves without always having the data to answer them: Where do my teams actually stand? Where should I invest first? How do I prove progress over time?
Anchored to the AFNOR SPEC 2401 standard and aligned with the AI Act Article 4 requirements, it produces an objective, auditable measurement, not a declarative estimate, but a comparable, traceable, and defensible data point.
Conclusion: AI Competency Is Not Declared, It Is Measured
AI in the enterprise has entered its phase of operational maturity. The question is no longer “should we train our teams in AI?” the answer is yes, and it is already happening everywhere. The real question is: how do we ensure these investments produce a lasting effect?
The answer lies in breaking with the logic of generic upskilling. It lies in measuring real competencies, contextualizing by job function, and steering with indicators rather than completion rates.
The organizations that make this shift first will hold a decisive advantage: the ability to demonstrate, at any time, that their teams genuinely master the AI they deploy.
Would you like to map your teams’ AI competencies before planning your next training actions? Book an AICET demo
