Every audit conclusion is actually based on a small amount of evidence. Auditors rarely have the time to examine all documents, all transactions, or all production records. What is examined is a sample, and then the report speaks about the entire population. This is where the risk lies: if the sample is not representative, the conclusion will be skewed, and there will not be a single line in the report that informs you of this.
Standards have set out how to close this gap. That section is usually rarely read to completion.
Six steps that must not be skipped
SNI ISO 19011:2018, the management system audit guideline, contains the sampling sequence in Appendix A.6.1:
- establish the purpose of sampling
- determine the scope and composition of the population to be sampled
- select the sampling method
- determine the sample size
- conduct the sampling activity
- compile, evaluate, report, and document the results
The second step is the one most often overlooked. As long as the population boundary is not clear, any number you take as a sample has no statistical meaning.
Two pathways, and the requirements for each
Judgment-based (A.6.2). Samples are selected based on the competence and experience of the audit team. The standard mentions six factors worth considering: previous audit experience in the same scope, the complexity of requirements including legislation, the complexity and interaction of organizational processes, the level of technological or management system change, significant risks already identified, and the outputs from management system monitoring.
The weakness is clearly stated in the same standard: there is no statistical estimate of the impact of uncertainty on audit findings and conclusions. This means intuition may be used, but the results cannot be claimed to have a certain level of confidence.
Statistical-based (A.6.3). Sample selection uses probability theory, and there are two forms with different purposes:
- Attribute-based, when the results have only two possibilities, such as compliant or non-compliant, pass or fail. Suitable for assessing the compliance of forms with procedures.
- Variable-based, when the results fall within a continuous range of values. Suitable for calculating the occurrence of food safety incidents or the number of security violations.
The number you need to hold on to: a 5 percent sampling risk is equivalent to a 95 percent confidence level. This means you accept the possibility that 5 out of 100 samples, or 1 out of 20, do not reflect the true value if the entire population were examined. The sample size moves according to that number, not according to preference. The standard also mentions six factors that influence the sampling plan, including the size and complexity of the organization, the number of competent auditors, the frequency of audits, the time for each audit, and the level of confidence requested by external parties.
In the financial realm, the rules are stricter
For financial statement audits, refer to SA 530 (Revised 2021) Audit Sampling from the Indonesian Institute of Public Accountants. It is part of the 2021 Audit Standards that became effective for audits of financial statements for periods beginning on or after January 1, 2022.
The definition: the application of audit procedures to less than 100 percent of elements in the relevant audit population, such that all sampling units have an equal chance of being selected. The distinction between statistical and non-statistical sampling is also clear, namely the random selection of elements and the use of probability to assess sample results, including measuring sampling risk. Without these two characteristics, the approach is non-statistical, regardless of what it is called. SA 530 requires auditors to establish a sample size sufficient to reduce sampling risk to an acceptable low level.
Two field techniques that are often confused
Internal audit materials based on SNI ISO 19011 used in government settings distinguish between two sampling patterns:
- Vertical slice: one sample is taken and then examined for compliance against all elements of the system
- Horizontal slice: one system element is selected and then examined against several samples
Generally, both are used together. What is incorrect is to run only one pattern and then report it as if it covers both.
What must be included in the report
Sample reporting should include the sample size, selection method, estimates made based on that sample, and the level of confidence. At the closing meeting, the auditee should also be informed that the audit evidence is based on a sample of the available information and therefore does not automatically represent the effectiveness of the entire process.
Such closing statements are not a formality. They determine the extent to which the figures in the report can be relied upon by others.
This topic is also discussed in the internal audit class based on ISO 19011. If you prefer to see the explanation directly, there is a clip on this video page.
Sources
- Indonesian Institute of Public Accountants, Audit Standards (SA), official list including SA 530 (Revised 2021) Audit Sampling and the effective date of the 2021 Audit Standards
- BPK Inspector News, Understanding Audit Standard (SA) 530 for Audit Samples
- Bogor City Inspectorate, Internal Audit based on SNI ISO 19011:2018
- ISO 9001 Auditing Practices Group, Guidance on Effective Use of ISO 19011, Edition 2 of 2020, summary of three sampling methods in Appendix A