Audio Audit Protocol

December 11, 2024| Tasmin Pritha| 0 Comment| 1:22 pm

Categories :

Research & Development

Overview

Audio recordings play a crucial role in various fields for quality assurance. An audio auditing protocol is essential to ensure the integrity, accuracy, and reliability of data. This document outlines the procedures and standards for conducting audio audits, aiming to maintain high-quality recordings and compliance with relevant regulations.

The primary objectives of this protocol

Establish Clear Guidelines: Define the steps and criteria for auditing audio recordings.
Ensure Consistency: Standardize the auditing process to achieve uniformity across different audits.
Maintain Data Integrity: Protect the authenticity and accuracy of audio data.
Enhance Accountability: Provide a transparent framework for evaluating and documenting audio quality.

By adhering to this protocol, auditors can systematically assess audio recordings, identify discrepancies, and implement corrective actions, thereby upholding the standards of excellence in audio data management.

Ethical Considerations

It is imperative to obtain explicit consent from participants specifically for audio auditing. This ensures ethical compliance and respects participants’ privacy and autonomy. Without consent, we cannot record the survey. add pic

In addition to cases involving sensitive topics such as rape victims, there are other scenarios where audio auditing and recording should not be conducted. These include:

Interviews with minors: Special care must be taken to protect the privacy and rights of children.
Medical conditions: Participants discussing sensitive health information, such as mental health issues or terminal illnesses.
Legal matters: Cases involving legal disputes or criminal activities where confidentiality is paramount.
Cultural sensitivity: Situations where cultural norms and practices prohibit recording.

Audio Recording

Methods

SurveyCTO/ODK Built-In Audio Auditing: Leveraging the built-in audio auditing features of SurveyCTO for seamless integration with survey data collection.
Device-Based Recording: Using dedicated recording devices or smartphones to capture audio

For SurveyCTO/ODK Built-In Audio Auditing these are audio recordings of part of a survey during the interview process. The audio is recorded automatically in the background by SurveyCTO/ODK — as long as you set up your form appropriately beforehand. Audio audits can serve as another measure of enumerator performance in that you can listen to how enumerators ask questions, probe, and go through the informed consent. You can also use audio audits to directly listen to the interview portion that produced an outlier or other data issue flagged by your high frequency checks. Audio audits can also be used as a random audit of a selected proportion of surveys.

Audio audits are included in your survey as a field type where you can specify how much of the survey you would like to record. The variable name will always be ‘audio_audit’ for SurveyCTO and for ODK that will ‘background-audio’

💡Tip: Do you love your job? Then make sure you have included it in the Dataflow as it is in the SurveyCTO.

Audio auditing in SurveyCTO can get very interesting. There are multiple parameters which we can use to have the audio recording. Because a survey can be very long, not every part is mandatory to record as a prerequisite of data quality management. The best way is to auditing randomly selected portions of the interview, which SurveyCTO does the best. But there can be some requisitions to audit a particular part of the interview, which is doable as well.

For Device-Based Recording we recommend a few tools which we must use and we have tested thoroughly before. First of all for any KII or FGD we do take the audio recorder from the office. (Note: Please be sure whenever you do audio recording with the recorder it is budgeted in your project budget). If we are using a mobile phone recorder, uniformity is very important. Which means the same kinda media file and everything. For that we recommend using GOM Recorder.

💡Tip: Make sure whenever you do audio recording with the recorder it is budgeted in your project budget, if you love the money that you have in your pocket.

Secondly, mobile recordings often come with lots of background noise. So for noise reduction use media.io. Recorded audio should have a uniform naming format. This naming can be different as the project demands, for example: “‘enumid’_’hhid’_’mid’_’date’”. Have a designated folder in google drive and make sure the enumerators uploaded the audio right away, Field Supervisor will be primarily responsible for this.

💡Tip: Don’t trust humans blindly. Make sure everybody in that folder has only access to contribute, only you should have the authority to delete files.

Data Storing & Management

File Naming

If the audio is recorded in the background using SurveyCTO/ODK the file naming is not something that can be done by the enumerators. In that case, we have to do the naming. The naming of the audio file must be unique so we can find out each case easily. The best way to do this after importing the data from the server using dataflow in the preparation part is to rename the audio files again. You will find the details and codes in the advanced guideline of the dataflow.

Folder Management

After using arcedfold a designated folder for media files in the raw folder will be created. All the audio files must be stored there, including the ones which may not be used.

💡Tip: Never delete anything from the raw folder.

Share And Monitor

We can never give access to the raw folder file to audio auditors. A separate folder must be created before sharing the audio files with any stakeholders.

⭐Note: Audio files are also raw data and it is as sensitive as any quantitative data because this can also contain PIIs. So we handle it carefully.

Parameters & Functions

Parameter	Function
p	In the appearance column, enter p=# where # is the percentage of submissions you wish to audit. For example, you could enter p=10 to audit 10% of your submissions (selected randomly). 10%-15% audio auditing is considered as the best practice.
s	By default, an audio audit will record just the beginning of a survey, starting when the form is first opened. You can specify an alternative starting point by using the s= parameter, Since at the very beginning in most of the surveys we do not get anything that substantial. For example s=600. Then the random portion that we want to capture from the interview will start after 10 minutes after opening the SCTO form
d	By default, an audio audit will record up to five minutes of audio. To record for a maximum duration other than 300 seconds, enter d=# into the appearance column, where # is the maximum number of seconds to record; recording will stop after the assigned number of seconds. For example d=300, the the maximum duration of the audio will be 5 minutes

⭐Note: We can make it field specific as well to do that the code in the appearance column will be, p=#;s=startfield;d=endfield.

Tip: Don’t trust machines blindly. Make sure you check the media file in an encrypted container in the ‘raw’ folder of your dataflow and confirm if you are getting all the audio files regularly. Just to remind you nobody trusts the mobile internet, right?

Responsibilities of Research Assistant

Ideally, the RA has to do the audio auditing. But if the survey is very long or has too many submissions then we hire audio auditors. Based on the survey or sample size you need to be sure about how audio auditors are needed and that should be approved beforehand.

⭐Note: Audio auditors must attend the enumerators training session & lastly they should have a different briefing session after the training session.

To hire audio auditors keep a few things in mind:

Experience with survey
Experience with audio auditing
Experience with ARCED

It is RA’s sole responsibility to ensure the quality of audio auditing, because it does add a huge contribution towards ensuring the whole data integrity.

💡Tip: Do you know auditors are human too? So you know the drill. Don’t trust them blindly too. Check at least 20% of their work randomly by yourself.

The best practice is to have a separate form and a tracker for the audio auditors. The tracker is to assign them work daily and to check their performances. Which will be essential to measure the overall health of the total audio auditing. Secondly, a form with time stamps helps a lot to have all the data from the audio auditor stored systematically. An example of tracker is given below.

⭐Note: The columns and options may vary based on the survey design.

The form of the audio auditors should include few things:

Questions about overall interview quality
Issues subject to protocol
Issues specific to questions

💡Tip: Please keep that in mind, everyone can make such a questionnaire based on the protocol and questionnaire of the main survey. But a good RA (Read ARCED’s RA) can prioritize key points of protocol and questionnaire and can design the auditing accordingly.

Do’s:

The auditor must listen to the audio carefully.
The auditor must consult with the team and confirm any confusion with the Research Analyst or the Research Associate.
Auditors must report very descriptively.

Don’ts:

The auditor must not put any value where he/she/they is not fully sure without any consultation with the team.
- If the audio is not clear use the code “-77” as value which means inaudible
- If the response is too long or relevant to any context or previous responses, auditor must discuss with the RA before putting any value.
Do not evaluate any audio or performance quality leniently.

Action Plan

Based on the audio audit report we have to take some actions.

Daily Briefing

Enumerators will receive a brief on the importance of maintaining high-quality audio auditing at the end of each day.

What if we find any mistake?

First Day: If an enumerator fails to meet the quality standards for audio auditing, they will be retrained on the protocol.

Second Day: A second failure to meet the quality standards will result in a formal warning.

Third Day: A third failure will lead to the termination of the enumerator’s contract.This structured approach ensures that all enumerators understand the importance of maintaining high-quality audio auditing and the consequences of failing to adhere to these standards.