|
Audit Sampling Methods
Sampling is a way of life for the
auditor since it is frequently unreasonable to test 100% of any given set of transactions.
Remember that auditors work to provide
reasonable, not absolute, assurance in order to ascertain whether controls are designed
and functioning properly. As such,
there are many different methods available to the auditor to pull samples in order
to perform testing and, in some cases, to make extrapolations about the testing
results. We will discuss some of these
below. We will not really go into
the mathematical explanations for the different sampling methods (if applicable),
preferring instead to give you the actual rubber-meets-the-road explanation of how
these methods work in the real world.
First let’s dispel a myth:
not all sampling is “statistical” in nature.
Statistical sampling involves actual
mathematical assumptions and methods whereas judgmental sampling is just picking
transactions based on judgment...there is no “math” involved.
In practice, there are many auditors that never use statistical sampling
because it doesn’t make sense to use it for their objectives or because they don’t
really understand how to use statistical sampling properly.
You tend to see external auditors using statistical sampling more often than
internal auditors because they must be able to make statistical inferences about
the nature of their results in order to render an opinion on the accuracy, in material
respects, of the financial statements and their related disclosures.
There are three main types of sampling
methods available to the auditor: Haphazard
Sampling, Statistical Sampling, and Judgment Sampling.
·
Haphazard Sampling – This type of sampling allows for
selection of items but is not defined by any type of statistical formula.
More or less, it is just picking items from a list at will with no defined
method. With haphazard sampling, you
cannot extrapolate your results to the larger population because your sample size
was not determined mathematically (i.e. the sample may be too small or too large)
and there is no guarantee that the distribution of your sample accurately approximates
the distribution of your population.
·
Statistical Sampling – This type of sampling involves defining
the population and related confidence intervals.
Based on these assumptions, a sample size is determined and the results of
your testing can be reasonably extrapolated to the overall population and, thus,
a quantifiable conclusion can be drawn.
(i.e. we are 95% confidence that the true value of the accounts receivable balance
is between X and Y.)
·
Judgmental Sampling – This type of sampling involves the
auditor selecting items for his/her sample based on some type of methodology in
an attempt to select items that exhibit some type feature.
This method purposefully biases the sample and, thus, the results of the
testing can not be extrapolated to the larger population.
So you see, the type of sampling
that you employ in the testing of your internal controls is largely determined by
the objectives and goals that you are seeking to achieve.
You should never use one or the other exclusively unless your goal never
changes (your audit plan is probably pretty stale if this is the case).
If you are going to use statistical
sampling, it’s a good idea to have some understanding about how statistics work
(duh, right?). You’ll appreciate this
advice when management starts asking you to explain your conclusions… chances are
they won’t know what the hell you are talking about, but you will sound much more
convincing if you have a good grasp of what you are talking about.
As there is not much explanation necessary for haphazard or judgmental sampling,
let’s talk a little more about statistical sampling.
Some basic information on statistical sampling:
Statistical sampling is rooted in
the assumption that populations have a normal distribution, represented by the all-to-familiar
bell-shaped curve (see below). Not
to say that all populations are normally distributed, as it’s possible for the population
to be positively or negatively skewed, but rather that the average of independent
repeated samples from the same distribution are approximated by the normal distribution.
This is what’s known as the central
limit theorem, which states that, regardless of the distribution of the population
from which a random sample is drawn, the shape of the sampling distribution of the
mean approaches the normal distribution as the sample size is increased.
The shape of the curve is determined
by the mean and the standard deviation of the underlying data.
The greater the number of values in the population, the flatter the curve.
The area under the curve is equal to
1. Thus, assuming that the distribution
is normal approximately 68% of the values are contained within plus or minus one
standard deviation from the mean. Approximately
95% of the values are contained within plus or minus two standard deviations from
the mean. Finally, approximately 99%
of the values are contained within plus or minus three standard deviations from
the mean. How you use these facts depends
on the type of statistical testing that you intend to conduct.
Testing of the statistical type is
based on probability theory and, as such, one must select confidence intervals (upper
and lower limits) within which the results may be placed.
Thus, setting a 95% confidence interval would mean that 95 out of every 100
items examined will reflect the population.
Confidence intervals are commonly placed at 90% or 95%.
Intervals outside of these are generally either useless (i.e. too low to
be of use for decision-making) or too costly (i.e. results in too many sample items
to physically test in light of costs).
One must also set the error rate
that is expected to exist within the population being tested.
A common error rate would be 5% but this does not necessarily have to be
so. One can conduct a pilot sample
to assess a more applicable error rate if necessary.
There are really two different types
of testing that you will be performing with statistical sampling:
Compliance tests and Substantive tests.
·
Compliance Testing – These types of tests involve testing
the existence of a particular control.
Thus, the results of your testing will be of the yes/no variety.
For example, you might test for the existence of a control.
The control is either present (“yes”) or not present (“no”).
·
Substantive
Testing
– These types of tests seek to establish
the extent to which the implications of a control weakness may be quantified. For example, you might test invoice
accuracy to find out the dollar value of invoicing errors for a given population.
Ø Variable Sampling -
This is a type of substantive testing that enables the user to take the average
results from the sample and extrapolate this to arrive at an error rate to apply
against the whole population.
Ø Monetary Unit Sampling – This type of testing assumes that the
population consists of a series of values (i.e. dollars) and in doing so you have
greater chances of selecting the item in your sample once you define the sampling
interval. The purpose of this type
of testing is to determine whether such values are overstated or understated and
to what degree so that the auditor can make an assessment about the correctness
of a particular account.
At a high-level, there are several
things that you should be aware of when you are using statistical sampling:
1.
Use statistical sampling only when necessary to satisfy your objective.
2.
You must be able to define and know the characteristics of the population in order
to effectively use statistical sampling in your testing.
3.
You must ensure that every item in your population has an equal chance of being
selected as part of your sample
4.
You must ensure that the population does not have manipulated patterns in it that
would affect the randomness of selection.
5.
Use an error rate that is reasonable
6.
If there are defined striations of data within your population, stratify it and
sample from within the striations.
In general, there are some basic
steps that are common to the statistical testing process, they are as follows:
Ø Determine the objectives of the test
Ø Define the population
Ø Define acceptable levels of sampling risk (i.e. 5%, 10%, etc)
Ø Calculate the sample size using tables, formulas, or software applications
Ø Select the sampling approach (i.e. random, MUS, Statification,
etc)
Ø Pull the actual sample and evaluate
Ø Document the sample results and approach
The size of your sample will generally
be impacted by the sample size (the larger your population the larger your sample
is likely to be), your acceptance risk (the smaller your accepted risk the larger
your sample will likely be), and the population variability (the more dispersed
or variable your population is the larger your sample will likely be).
Now, we know what you are thinking…this
seems a little complicated. You would
be correct in your thinking. You also
wouldn’t be the first to get scared away from statistical testing and relegate yourself
to a lifetime of haphazard and judgmental sampling.
DON’T DO IT! Read a statistics
book or take a business statistics course at the local community college or something.
Trust me, its well worth your effort
if you are going to be pursuing a career in internal or external auditing.
Many of the current data analysis
software packages, like Audit Command Language (ACL) or IDEA, contain some wizards
or automation that make statistical sampling a little easier.
They have pre-formatted forms and templates that you fill out in order to
generate sample selections and they also keep an audit trail of your sampling methodology.
You’ll still have to know are doing,
but they make it a little easier to execute your testing once you have identified
your sampling methodology and/or testing goals.
There are also some data analysis capabilities in Excel, such as random number
generations and such, but I’ve found that the formal data analysis software packages
are a little bit more specialized to the needs of the auditor.
|