Audits & Strategies Toolkit
Audits 'Methodology' section
Audits and Strategies home > Audits and Strategies Audits - Methodology section
Audits - 'Methodology' section
Here are some suggestions for information that you could include under the heading 'Methodology' in your audit.
Use the 'Methodology' section to give a detailed account of any procedures you used when carrying out the audit including:
You should also consider:
Details of the audit team
You should include:
details of who carried out the work associated with the audit, such as an in-house audit team, a countywide audit team, whether a LCJB representative was included or whether the work was contracted out
details of the skills and knowledge mix of the team
an explanation of the rationale behind the approach taken to conducting the audit. This may reflect available resources and expertise (or lack of it) within certain aspects of the partnership
Stages of the audit process and timescales
You should include:
a description of the key stages involved in planning the audit process and the key tasks at each stage
reference to the timescales used to produce the audit. Here is an example timetable that you can adapt.
How the data was obtained
You should include:reference to any information sharing protocols. If information sharing protocols were not used, explain how agreement and consent to use data were obtained. It is important that you take the Data Protection Act into account and do not name individuals in the recorded data. Here is more information on confidentiality and information sharing" and data protection and record retention
reference to any problems you encountered in obtaining data. It is recognised that there may still be problems in obtaining appropriate data from some organisations
a discussion of the strengths and weaknesses of any data used in the audit analysis
an explanation of how you decided what information to include in the audit such as areas/issues identified as a priority through primary research, consistent 'hot spot' areas identified through previous audits, local or central government priorities and resource or capacity limitations
Types of data used
You should include information about the types of data used:
recorded crime figures, which will form a major part of your crime and disorder audit but it is important to consider other sources of data to achieve a more rounded picture of the impact and consequences of crime. Guidance on Statutory Crime and Disorder Partnerships 1998 provides a comprehensive list of potential data sources
Here is more information about additional crime types and thematic areas such as anti-social behaviour, domestic violence, alcohol-related crime and fear of crime and links to help you gather data on these issues for your audit
Here is more information on the links to the National Drug Strategy Performance Management Framework and specific data that you might like to collect to identify the scope of drugs problems locally. It also outlines other potential local data sources on drugs.
You can usefully combine primary and secondary data when you carry out an audit.
Primary data:
Primary data is data that partnerships or agencies collect specifically for the audit process.
If you use primary data you will have to decide what data you are going to collect, how you will collect the data and how you will analyse and report the data.
Many Partnerships choose to undertake a large consultation exercise as part of their audit to obtain primary data to help identify issues within the locality. This information is useful but remember you are required by the Crime and Disorder Act 1998 to consult on the findings of your audit. For more information about consultation go to the 'Consultation' section of this toolkit. Consultation is essential to make sure that strategies are grounded in the experience of local people.
Audits can benefit from using some form of primary data, where time and budget allow.
Collecting primary data can be time consuming, therefore you must be clear about why you are undertaking primary research and what you want to achieve from it.
You must write up the results of any primary research in an appropriate format using the same principles applied to writing an audit report.
Examples of primary data include data from a local fear of crime survey, data derived from focus group consultation, from hard to reach groups or from a survey of the extent and nature of drug use amongst specific groups.
Secondary data:
Secondary data is data gathered by a particular agency for it's own purposes, which is then used for a different purpose within the audit.
Using secondary data may have limitations because it is collected for other purposes than your audit. It may not include the exact information you need for your audit and may have its own biases and inaccuracies.
Secondary data includes published crime statistics, local police data, Local Authority data, new English and Welsh Arrestee Drug Abuse Monitoring (NEW-ADAM) and data on illegal drug markets and data on risk factors such as deprivation, truancy or homelessness
Find out more about gathering and managing primary and secondary data from The NACRO Guide to Crime Audits
Find out more about sources of data for the aims of the drug strategy from Targeting drugs the cross-government national drug strategy website.
The United Nations Office on Drugs and Crime Website contains a lot of information on the nature and extent of drug abuse. It includes a number of guides on collecting and using drug statistics, which are very comprehensive. The guides start with quite basic information and progress to very complex information. Although they were designed with whole countries in mind, they may be useful for local Drug Action Teams. These guides may be particularly useful:
Reliability and validity of data
You should include information about the reliability and validity of the data used:
Reliability means consistency - the extent to which the same results could be obtained under similar circumstances.
Validity means the capacity to measure what it is intended to measure
When you carry out an audit, the reliability and validity of data are important, particularly when you are interpreting the use of data and using the data to support the analyses made, conclusions reached and potential actions.
Example:
Imagine some scales that always weigh an individual as 5 kilos too light. The results are reliable because they always weigh the individual at 5 kilos less but not valid because the scales are not measuring the true weight of the individual.
Data analysis
You should include:
an explanation of the approach taken to data analysis
a description of the key concepts that have helped to shape the development of community safety eg the Problem Analysis Triangle (PAT)
Example:
Overall recorded crime for the area was analysed. Both national and regional comparisons were made over a three-year period. Data was analysed by the Community Safety Analyst using SPSS and Geographical information systems. Data from the Criminal Justice Interventions Programme was analysed.
Analysis was undertaken on notifable offences. Where appropriate, temporal, spatial, victim/offender data and repeat victimisation analysis was also undertaken.
Local problems and issues highlighted through the consultation exercise were also analysed (e.g. road traffic incidents, domestic violence, prostitution). Where crime hotspots were identified more detailed analysis was undertaken. Demographic data along with other agency data was used.
Specific techniques were used to aid analysis such as examining issues using the Problem Analysis Triangle which promotes victim, offender and location focus for data collection and analysis.
Find out more about data analysis from:
The methodologies and approaches to data analysis can be applied to both development of misuse of drug and crime audits. Some of the tools in the Passport and Guidance are generic, but there are some that relate directly to crime.
Consultation methodology
You should include:
details of who carried out the consultation eg in-house team or external consultants.
the types of people, groups or organisations consulted.
the methods used during the consultation.
the timescales involved in the consultation.
You should also think about how you will respond to any points raised during the consultation exercise. Consultation should take place on the findings of the audit to:
make sure that the audit has identified problems correctly
check there are no important omissions
make sure that the audit is not based on misconceptions about the communities to which it relates
canvass opinion on the draft priorities
Partnerships are expected to consult with:
persons and bodies specified by the Home Secretary under the Crime and Disorder Act 1998 and the Police Reform Act 2002 and other relevant local organisations.
the general public
traditionally excluded groups: BME communities, asylum seekers and refugees, disabled people and lesbian, gay, bi-sexual and trans-gendered communities
Partnerships may also choose to consult as part of the audit in order to gain primary information to identify issues within their area. This can be useful but remember, the Crime and Disorder Act 1998 requires partnerships to consult on the findings of their audits.
Find out more about consultation mechanisms in 'Getting the Grease to Squeak: Research Lessons for Crime Prevention' (1998)
Agreed definitions
You should also consider:
clarifying key concepts and referring to agreed terms at the start of your analysis
the definitions you intend to use. Be consistent and agree definitions as a partnership because differences in definitions can be problematic when overlaying and comparing data
if you are including a number of terms that your audience might not be familiar with in your audit report, if so, it may be helpful to provide a glossary of key terms
Here are some examples of common terms and definitions
Retaining ownership
You should also consider:
how you can retain ownership of information if you use external contractors at any stage in your audit.
Using external contractors can lead to a loss of knowledge and understanding within the Partnership. It is important for Partnerships to fully understand and approve all processes.
References
You should also consider:
including references to any reports or studies cited in your audit report so that their claims can be verified
Date modified: 10 March 2004
Review date: March 2005
Originator: Crime Reduction Centre Information Services Team
Last update: Wednesday, September 06, 2006