The Skill Nobody’s Trained For Yet: Reviewing What AI Builds

Copilot and Fabric agents can now write the formulas behind your reports and plan the pipelines that feed them. That shifts the risk from building too slowly to approving too quickly: a plausible but wrong figure can reach a board pack before anyone questions it. This article is for leaders responsible for data teams and training plans. It explains the business risk, the review skill that reduces it, who needs that skill by role, and how PTR builds it into its Fabric training, with the benefits you should expect.

Most Fabric training teaches people how to build. AI tools now produce much of the first draft, and the person approving it needs to judge it, not just accept it. Without that skill, errors look professional. A calculation may filter in the wrong order, a data pipeline may do far more work than the job needs, or an agent may have wider access to data than intended. The costs show up as wrong figures in front of decision makers, rework once the mistake is found, and lost trust in reports that took effort to build.

PTR’s engineering blog on spec-driven development makes the point plainly: an assistant can draft a spec, a plan and the code, but people remain responsible for all three, and reviewers need enough Fabric, DAX and SQL knowledge to judge what’s proposed, not just enough confidence to approve it. Knowing how to write a measure (a calculation behind a Power BI report, written in a formula language called DAX) and knowing whether an AI-generated one matches your actual metric definition are separate competencies. The second does not appear automatically because someone is good at the first.

PTR’s Fabric training pathway, described in full in How PTR’s Fabric Training Pathway Gets Everyone Ready, is built around three goals: user adoption, data literacy, and improved design and quality. The third is the closest match to this problem. It covers defining standards, keeping logic central, building a business data dictionary, following guidelines on semantic model design and performance, and putting calculations in the right place. Those are the yardsticks a reviewer needs when checking AI output, because without agreed standards there is nothing to judge it against.

The data-literacy goal contributes as well. It teaches people to interpret data, spot patterns, issues and anomalies, and work from governed sources. A figure that looks wrong is often the first sign of an AI mistake, and the people who notice it need to know how to raise it.

The role-based stages already in the pathway are the natural home for this. The stage descriptions below reflect the current pathway; the review content is what we would add to each.

Report Creators. The pathway runs a one-day Getting Started overview covering data modelling, DAX basics and visualisation, followed by a one-day Next Steps course on DAX functions such as CALCULATE and FILTER, time intelligence and certified datasets. Next Steps is where people work with DAX at the level needed to judge a Copilot-written measure. The addition would be practising how to read that measure against the business data dictionary and spot where it has quietly diverged, for example a filter applied in the wrong order or a grain that does not match what the business means by the number.

Report Users. The Getting Started Power BI workshop already includes spotting trends and anomalies. Report users are often the first to see a number that looks off, so the addition would be a short, practical session on how to question a figure and who to raise it with.

Data Engineers. The pathway begins with a one-day instructor-led overview covering architecture, ingestion and pipelines, lakehouse and warehouse fundamentals, notebooks and governance, with deep-dive courses on lakehouse and warehouse design, orchestration and pipeline design, notebooks, Python and SQL. PTR’s approach is to recommend instructor-led training for new technology areas and to use e-learning for upskilling, and agent-built engineering is a new technology area. The addition would be reviewing an agent’s proposed design, including spotting over-engineering such as a full refresh where an incremental load would do.

Governance roles and platform administrators. Governance roles take a two-hour Getting Started workshop on ownership, stewardship, security, quality, lineage and classification, while administrators take a one-day overview that covers security, permissions and performance monitoring, followed by a hands-on break-and-fix workshop. Both are good places to add the question of whether an agent’s access to a dataset carries the rules it is supposed to inherit, and a break-and-fix scenario where it does not.

Leadership and stakeholders. The two-hour leadership briefing and the stakeholder Getting Started workshop already ask for support, sponsorship and commitment. Leaders who sponsor AI-assisted work should also know what good review looks like, so they can ask for it.

The pathway is organised around fusion teams, each combining a business expert, an IT data expert and a data specialist, with training cycles for stakeholders, report creators and report users repeated for each team. That structure suits review of AI-built work. The business expert confirms that a figure means what the business thinks it means, and the data specialist checks that it has been built soundly. Neither can do the other’s job, and AI output reviewed by only one of them is only half checked.

Bespoke report creator training in the pathway uses the client’s own semantic model, business data dictionary and data in custom lab sessions, in groups of up to eight over one to three days. Reviewing AI output against your own definitions is far more convincing than generic examples. Bite-size e-learning of five to ten minutes with a single learning objective suits refreshers, such as what to check in a Copilot-written measure. Monthly update sessions help teams keep pace with a platform that changes quickly, and user groups and clinics give people a place to compare real examples of AI output they have caught and corrected.

PTR’s AI Readiness Assessment asks, among its six questions, whether your people are ready to run with AI or whether there is a skills gap. Where it finds one, PTR’s AI Enablement service provides training and coaching that builds AI literacy, good habits and the human checkpoints that keep AI working for you. The review skills in this article are the practical form of those checkpoints. PTR’s AI Readiness & Data Foundations work also pairs naturally with blended consultancy and training, so people learn to use and oversee AI tools as the foundations go in.