13.1 Artificial Intelligence, Machine Learning, and Robotic Process Automation

13.1 Artificial Intelligence, Machine Learning, and Robotic Process Automation

Quick Answer: CIA Part 2 B3 asks you to evaluate technology options that internal auditors may use to develop and support engagement findings and conclusions. Robotic process automation (RPA) is rules-based bots executing defined steps. Machine learning (ML) is models that learn patterns. Artificial intelligence (AI) is broader — including natural-language processing and copilots. None of them waives Global Internal Audit Standards (GIAS) Standard 14.1: information still must be relevant, sufficient, and reliable. The auditor remains responsible. The classic fail is filing a chatbot summary as if it were audit evidence.

Section B of the 2025 syllabus is 40% of CIA Part 2. You have already chosen sources (Chapter 11) and filtered them for evidence quality (Chapter 12). B3 is the next proficient skill: pick efficient and effective technology to build findings, not to decorate the file with a vendor logo. Chapter 14 will cover process mapping and analytics methods (diagnostic, predictive, and the rest). This section stays on RPA, ML, and AI as options for developing and supporting conclusions. Function-wide technology strategy and the chief audit executive's resource model live in CIA Part 3 / GIAS Standard 10.3 — do not drag them into an engagement-findings item.

GIAS Principle 14, Conduct Engagement Work, is the spine. Standard 14.2 requires you to analyze relevant, reliable, and sufficient information to develop potential findings. Standard 14.3 requires you to evaluate significance. Standard 14.5 requires you to develop the engagement conclusion. A bot can extract, score, or draft. It cannot own the conclusion.

Three tools the exam will mix up

Memorize the differences as how the tool decides, not as brand names.

RPA is the right first tool when the test is a known rule you could explain on a whiteboard: same vendor, same invoice number, same amount, different payment document. ML is the right first tool when the pattern is too numerous or too fuzzy for a stable rule — and you are willing to explain and test the output. Generative AI is the right first tool when the job is language: find the indemnity cap in 400 PDFs so a human can read those pages. It is the wrong tool when you need a condition.

The IIA's Artificial Intelligence Auditing Framework (updated 13 September 2024) expanded coverage of generative AI and large language models and emphasizes transparency, traceability, and accountability. For Part 2 B3 you do not need to recite the framework. You need the implication: if you cannot trace an AI output to source evidence an informed, competent person could reperform, you cannot support a finding with it. Auditing the organization's AI program as the activity under review is a different engagement. Here the AI is your option, or management's output that you might use.

Efficient is not the same as effective

B3's expanded test specification says efficient and effective solutions. Efficiency is hours, coverage, and speed. Effectiveness is whether the output can carry a finding that survives Standard 14.1 and Chapter 12's reperformance test.

Worked example: five-plant accounts payable

Corporate policy AP-14 requires that a vendor invoice number not be paid twice. You need findings on duplicate payments, not a data-science case study.

RPA path (usually best here). After IPE tests — record count and dollars reconcile to the AP subledger; company codes 100–500 included; voided and reversed payments kept or excluded on purpose — a rules-based bot extracts the year and flags exact matches on vendor + invoice number + amount. Forty exceptions. You inspect source invoices and payment documents. Twelve are true duplicates totaling $186,400; the rest are legitimate reverse-and-repay or shared invoice numbers across plants that the rule should be refined to exclude. The finding is twelve paid duplicates against AP-14. The bot is in the workpapers as the method. The evidence is the IPE-tested population plus the twelve packages.

ML path (lead generator, not the condition). An unsupervised model flags 220 invoices as unusual. Efficient triage. You still test IPE, ask which features drove the score (amount versus new vendor versus weekend posting), inspect a sample of high scores and a sample of low scores (false negatives), and write findings only where conditions fail AP-14. If the model was trained on last year's suspicious list built by a terminated clerk who tagged every foreign vendor, bias will over-flag one class and miss domestic duplicates.

Generative AI path (trap). A staff auditor pastes AP-14 into a chatbot: Do we have a duplicate-payment problem? The model answers that controls appear mature and residual risk is low. That paragraph has no population, no IPE, no invoice, and may be hallucinated. Filing it as evidence is the B3 fail.

Model risk, bias, explainability, and IPE of AI output

Before a model or bot output can support — or even reliably point to — a finding, walk four questions. They map to evidence quality, not to a data-science elective.

Model risk. Is this the right technique for the objective? A clustering model cannot prove policy noncompliance; it can only surface items that look unlike the rest. Has anyone validated that the model still behaves after the ERP upgrade? Drift — live data no longer resembling training data — turns last quarter's useful scores into this quarter's noise.

Bias. Training data and feature choices can systematically miss or over-hit a vendor class, region, or employee group. A fraud model trained mostly on small-dollar travel claims will not support a conclusion about large vendor-master fraud. Bias is both an ethics issue and a sufficiency issue: the slice of reality the model saw was not the population in scope.

Explainability (black-box risk). GIAS 14.1 sufficiency is information a prudent, informed person would accept. Chapter 12's reperformance test asks whether an informed, competent person could reach the same conclusion from the file. If the workpaper says neural-net score 0.91 and nobody can say why invoice 88421 was flagged, the score is not reliable evidence of a condition. Demand features, rules, or at least an error analysis you can document. If the vendor refuses to explain, you have a limitation, not a finding against the invoices.

IPE of the AI output. Chapter 12 taught that IPE remains internal evidence even from a well-controlled ERP. An ML exception file, an RPA output table, and a copilot's clause list are reports. Test completeness (did the job see every in-scope document or posting?), accuracy (trace a sample of output lines to source), and parameters (period, company code, filters, prompt, model version). If management's AI produced the list, also weigh the control environment of that system (section 12.2). If internal audit's bot produced it, document logic, version, and who can change the rule. We used AI does not make the extract auditor-obtained in the strong sense until you obtained and tested it.

The auditor remains responsible

GIAS Standard 4.2, Due Professional Care, and Standard 4.3, Professional Skepticism, do not include an exception for automation. Standard 3.1, Competency, means if the team cannot evaluate the tool, obtain a specialist or do not rely. Principle 5, Maintain Confidentiality, is the other chatbot trap: pasting live contracts, payroll, or customer data into a public large language model is a protection-of-information failure even before it is an evidence failure.

Exam trap — accepting a chatbot summary as audit evidence. Generative models invent citations, skip exhibits, and smooth over exceptions. A summary can be a drafting aid after you have read the source. It cannot replace the source. If the prompt cannot be replayed, the model version is unknown, and the underlying PDFs are not in the file, an informed person cannot reperform the work.

Choose the simplest option that can produce evidence a prudent, informed person would accept. Known rule → RPA. Fuzzy pattern → ML used as a lead. Unstructured language → NLP or a copilot used to find pages you then read. In every case, the finding is still condition versus criterion, supported by relevant, sufficient, reliable information — not by the logo on the tool.