Discussion on how to reverse one channel of an audio file by reading and writing samples with AVFoundation.

ReverseAudio

The associated Xcode project implements a SwiftUI app for macOS and iOS that presents a list of audio files included in the bundle resources subdirectory ‘Audio Files’.

Add your own audio files or use the sample set provided.

Each file in the list has an adjacent button to either play or reverse the audio.

Classes

The project is comprised of:

  1. ReverseAudioApp : The App that displays a list of audio files in the project.
  2. ReverseAudioObservable : An ObservableObject that manages the user interaction to reverse and play audio files in the list.
  3. ReverseAudio : The AVFoundation code that reads, reverses and writes audio files.

1. ReverseAudioApp

The app displays of list of audio files in the reference folder ‘Audio Files’.

You can add your own files to the list.

Files are represented by a custom File object which stores its URL location:

struct File: Identifiable {
    var url:URL
    var id = UUID()
}

The files are presented in a FileTableView using List:

List(reverseAudioObservable.files) {
    FileTableViewRowView(file: $0, reverseAudioObservable: reverseAudioObservable)
}

where each row of the table displays the audio file name and a Button to play and reverse it:

HStack {
    Text(file.url.lastPathComponent)
    
    Button("Play", action: {
        reverseAudioObservable.playAudioURL(file.url)
    })
        .buttonStyle(BorderlessButtonStyle()) // need this or tapping one invokes both actions
    
    Button("Reverse", action: {
        reverseAudioObservable.reverseAudioURL(url: file.url)
    })
        .buttonStyle(BorderlessButtonStyle())
}

Both iOS and macOS store the generated files in the Documents folder.

On the Mac the folder can be accessed using the provided Go to Documents button.

For iOS the app’s Info.plist includes an entry for Application supports iTunes file sharing so they can be accessed in the Finder of your connected device.

2. ReverseAudioObservable

This ObservableObject has a published property that stores the list of files included in the project:

@Published var files:[File]

An AVAudioPlayer is used to play the audio files:

var audioPlayer: AVAudioPlayer?
...
audioPlayer.play()

URLs of the included audio files are loaded in the init() method:

let kAudioFilesSubdirectory = "Audio Files"
...
init() {
    let fm = FileManager.default
    documentsURL = try! fm.url(for:.documentDirectory, in: .userDomainMask, appropriateFor: nil, create: true)
    
    self.files = []
    
    for audioExtension in kAudioExtensions {
        if let urls = Bundle.main.urls(forResourcesWithExtension: audioExtension, subdirectory: kAudioFilesSubdirectory) {
            for url in urls {
                self.files.append(File(url: url))
            }
        }
    }
    
    self.files.sort(by: { $0.url.lastPathComponent > $1.url.lastPathComponent })
    
}

The allowed extensions are defined:

let kAudioExtensions: [String] = ["aac", "m4a", "aiff", "aif", "wav", "mp3", "caf", "m4r", "flac"]

The action for the reverse button is implemented by:

func reverseAudioURL(url:URL)

Which invokes reverseAudio twice:

let reverseAudio = ReverseAudio()
...
func reverse(url:URL, saveTo:String, completion: @escaping (Bool, URL, String?) -> ()) {
    
    let reversedURL = documentsURL.appendingPathComponent(saveTo)
    
    let asset = AVAsset(url: url)
    
    reverseAudio.reverseAudio(asset: asset, destinationURL: reversedURL, progress: { value in
        DispatchQueue.main.async {
            self.progress = value
        }
    }) { (success, failureReason) in
        completion(success, reversedURL, failureReason)
    }
}

Once to reverse, and then reverse the reversed to verify it plays like the original as expected:

func reverseAudioURL(url:URL) {
    
    reverse(url: url, saveTo: "REVERSED.wav") { (success, reversedURL, failureReason) in
        
        if success {
           
            self.reverse(url: reversedURL, saveTo: "REVERSED-REVERSED.wav") { (success, reversedURL, failureReason) in
               
            }
        }
    } 
}

The results are written to files in Documents named REVERSED.wav and REVERSED-REVERSED.wav

The result URLs are stored in published properties:

@Published var reversedAudioURL:URL?
@Published var reversedReversedAudioURL:URL?

3. ReverseAudio

Reversing audio is performed in 3 steps using AVFoundation:

  1. Read the audio samples of a file into an Array of [Int16] and reverse it
  2. Create an array of sample buffers [CMSampleBuffer?] for the array of reversed audio samples
  3. Write the reversed sample buffers in [CMSampleBuffer?] to a file

The top level method that implements all of this, and is employed by the ReverseAudioObservable is:

func reverseAudio(asset:AVAsset, destinationURL:URL, progress: @escaping (Float) -> (), completion: @escaping (Bool, String?) -> ())

1. Read the audio samples of a file into an Array of [Int16] and reverse it

We implement a method to read the file:

func readAndReverseAudioSamples(asset:AVAsset) -> (Int, Int, [Int16])?

that returns a 3-tuple consisting of:

  1. The size of the first sample buffer, it will be used as the size of the samples buffers we write
  2. The sample rate for the audio, as the output file will have the same sample rate
  3. All the reversed audio samples as Int16 data

Audio samples are read using an AVAsset and AVAssetReader

Create an asset reader using method:

func audioReader(asset:AVAsset, outputSettings: [String : Any]?) -> (audioTrack:AVAssetTrack?, audioReader:AVAssetReader?, audioReaderOutput:AVAssetReaderTrackOutput?)

as:

let kAudioReaderSettings = [
    AVFormatIDKey: Int(kAudioFormatLinearPCM) as AnyObject,
    AVLinearPCMBitDepthKey: 16 as AnyObject,
    AVLinearPCMIsBigEndianKey: false as AnyObject,
    AVLinearPCMIsFloatKey: false as AnyObject,
    //AVNumberOfChannelsKey: 1 as AnyObject, // Set to 1 to read all channels merged into one
    AVLinearPCMIsNonInterleaved: false as AnyObject]
...
let (_, reader, readerOutput) = self.audioReader(asset:asset, outputSettings: kAudioReaderSettings)

Note that the audio reader settings keys are asking for samples to be returned with the following noteworthy specifications:

  1. Format as ‘Linear PCM’, i.e. uncompressed samples (AVFormatIDKey)
  2. 16 bit integers, Int16 (AVLinearPCMBitDepthKey)
  3. Interleaved when multiple channels (AVLinearPCMIsNonInterleaved)

Also note that if we included the additional key AVNumberOfChannelsKey set to 1 then the audio reader will read all channels merged into one. By not including the key all channels will be read separately and interleaved, and we process only the first.

Read samples:

if let sampleBuffer = audioReaderOutput.copyNextSampleBuffer(), let bufferSamples = self.extractSamples(sampleBuffer) {
...
}

We will store them in an array audioSamples of Int16:

var audioSamples:[Int16] = []

Each time we call copyNextSampleBuffer we are returned a CMSampleBuffer that contains the audio data as well as information about the data.

Most notably we can retrieve the AudioStreamBasicDescription which provides us with the following information we will need:

  1. The number of channels, channelCount, which is used to extract samples from only 1 channel via stride
  2. The bufferSize and sampleRate, which is used when we write the reversed sample buffers to a file

Since the channels are interleaved the buffer size is determined by dividing the total number of samples by channelCount because we only reverse and write one channel:

var bufferSize:Int = 0
var sampleRate:Int = 0
        
if let sampleBuffer = audioReaderOutput.copyNextSampleBuffer(), let bufferSamples = self.extractSamples(sampleBuffer) {
    
    if let audioStreamBasicDescription = CMSampleBufferGetFormatDescription(sampleBuffer)?.audioStreamBasicDescription {
        
        let channelCount = Int(audioStreamBasicDescription.mChannelsPerFrame)
        
        if bufferSize == 0 {
            bufferSize = bufferSamples.count / channelCount
            sampleRate = Int(audioStreamBasicDescription.mSampleRate)
        }
        
        ...
    }
}

Read audio samples in each CMSampleBuffer with the method:

func extractSamples(_ sampleBuffer:CMSampleBuffer) -> [Int16]?

The method extractSamples pulls the Int16 values we requested out of each CMSampleBuffer using CMSampleBufferGetAudioBufferListWithRetainedBlockBuffer into an array [Int16] named bufferSamples.

First use CMSampleBufferGetAudioBufferListWithRetainedBlockBuffer to access the data in the sampleBuffer:

var blockBuffer: CMBlockBuffer? = nil
let audioBufferList: UnsafeMutableAudioBufferListPointer = AudioBufferList.allocate(maximumBuffers: 1)

guard CMSampleBufferGetAudioBufferListWithRetainedBlockBuffer(
    sampleBuffer,
    bufferListSizeNeededOut: nil,
    bufferListOut: audioBufferList.unsafeMutablePointer,
    bufferListSize: AudioBufferList.sizeInBytes(maximumBuffers: 1),
    blockBufferAllocator: nil,
    blockBufferMemoryAllocator: nil,
    flags: kCMSampleBufferFlag_AudioBufferList_Assure16ByteAlignment,
    blockBufferOut: &blockBuffer
) == noErr else {
    return nil
}

And then move the data into an Array of [Int16]:

if let data: UnsafeMutableRawPointer = audioBufferList.unsafePointer.pointee.mBuffers.mData {
    
    let sizeofInt16 = MemoryLayout<Int16>.size
    let dataSize = audioBufferList.unsafePointer.pointee.mBuffers.mDataByteSize
    
    let dataCount = Int(dataSize) / sizeofInt16
    
    var sampleArray : [Int16] = []
    let ptr = data.bindMemory(to: Int16.self, capacity: dataCount)
    let buf = UnsafeBufferPointer(start: ptr, count: dataCount)
    sampleArray.append(contentsOf: Array(buf))
    
    return sampleArray
}

However bufferSamples now contains interleaved samples for all channels, as we requested.

Since we only want to process one channel we need to subsample the bufferSamples using stride, accumulating them in array all samples, audioSamples:

for elem in stride(from:0, to: bufferSamples.count - (channelCount-1), by: channelCount)
{
    audioSamples.append(bufferSamples[elem])
}

Note that stride does not include its end value.

Reverse the array:

audioSamples.reverse()

And return the 3-tuple consisting of:

  1. The size of the first sample buffer, it will be used as the size of the samples buffers we write
  2. The sample rate for the audio, as the output file will have the same sample rate
  3. All the reversed audio samples as Int16 data

2. Create an array of sample buffers [CMSampleBuffer?] for the array of reversed audio samples

This will be implemented by the method:

func sampleBuffersForSamples(bufferSize:Int, audioSamples:[Int16], sampleRate:Int) -> [CMSampleBuffer?]

Just as we read the data in as CMSampleBuffer it will be written out as CMSampleBuffer, where each sample buffer contains a subarray (block) of the reversed audio samples.

To facilitate that we have an extension on Array that creates an array of blocks of size bufferSize of the array returned by readAndReverseAudioSamples:

extension Array {
    func blocks(size: Int) -> [[Element]] {
        return stride(from: 0, to: count, by: size).map {
            Array(self[$0 ..< Swift.min($0 + size, count)])
        }
    }
}

Example of blocks:

let x = [4, 7, 9, 3, 5, 2]

let x_blocks_2 = x.blocks(size: 2)
let x_blocks_4 = x.blocks(size: 4)

print("x_blocks_2 = \(x_blocks_2)")
print("x_blocks_4 = \(x_blocks_4)")

Output:

x_blocks_2 = [[4, 7], [9, 3], [5, 2]]

x_blocks_4 = [[4, 7, 9, 3], [5, 2]]

In the method sampleBuffersForSamples we will pass the values previously retrieved from an AudioStreamBasicDescription for bufferSize and sampleRate, and employ that Array extension to create an array consisting of blocks of data:

let blockedAudioSamples = audioSamples.blocks(size: bufferSize)

Then for each such block of Int16 samples we create a CMSampleBuffer using the method:

func sampleBufferForSamples(audioSamples:[Int16], sampleRate:Int) -> CMSampleBuffer?

This method creates a CMSampleBuffer from the audioSamples:

It uses CMBlockBufferCreateWithMemoryBlock, an AudioStreamBasicDescription to create a CMAudioFormatDescription, and finally CMsampleBufferCreate to create a CMSampleBuffer that contains one block of the reversed audio data that will be written out to a file.

For CMSampleBufferCreate we need to prepare two of its arguments:

  1. samplesBlock - for argument dataBuffer: CMBlockBuffer?
  2. formatDesc - for argument formatDescription: CMFormatDescription?
1. samplesBlock

First create a CMBlockBuffer named samplesBlock for the dataBuffer argument using the audioSamples with CMBlockBufferCreateWithMemoryBlock.

CMBlockBufferCreateWithMemoryBlock requires an UnsafeMutableRawPointer named memoryBlock containing the audioSamples.

Allocate and initialize the memoryBlock with the audioSamples:

let bytesInt16 = MemoryLayout<Int16>.stride
let dataSize = audioSamples.count * bytesInt16

var samplesBlock:CMBlockBuffer? 

let memoryBlock:UnsafeMutableRawPointer = UnsafeMutableRawPointer.allocate(
    byteCount: dataSize,
    alignment: MemoryLayout<Int16>.alignment)

let _ = audioSamples.withUnsafeBufferPointer { buffer in
    memoryBlock.initializeMemory(as: Int16.self, from: buffer.baseAddress!, count: buffer.count)
}

Pass the memoryBlock to CMBlockBufferCreateWithMemoryBlock to create the samplesBlock, passing nil as the blockAllocator so the default allocator will release it:

CMBlockBufferCreateWithMemoryBlock(
            allocator: kCFAllocatorDefault, 
            memoryBlock: memoryBlock, 
            blockLength: dataSize, 
            blockAllocator: nil, 
            customBlockSource: nil, 
            offsetToData: 0, 
            dataLength: dataSize, 
            flags: 0, 
            blockBufferOut:&samplesBlock
        )

This is the samplesBlock - for argument dataBuffer: CMBlockBuffer?

2. formatDesc

Next we need a CMAudioFormatDescription formatDesc created from an AudioStreamBasicDescription specifying 1-channel, 16 bit, Linear PCM:

var asbd = AudioStreamBasicDescription()
asbd.mSampleRate = Float64(sampleRate)
asbd.mFormatID = kAudioFormatLinearPCM
asbd.mFormatFlags = kAudioFormatFlagIsSignedInteger | kAudioFormatFlagIsPacked
asbd.mBitsPerChannel = 16
asbd.mChannelsPerFrame = 1
asbd.mFramesPerPacket = 1
asbd.mBytesPerFrame = 2
asbd.mBytesPerPacket = 2

Recall that we previoulsy determined the channelCount, bufferSize and sampleRate of the audio we are working with using the AudioStreamBasicDescription of the first sample buffer.

Pass the asbd to CMAudioFormatDescriptionCreate:

var formatDesc: CMAudioFormatDescription?

CMAudioFormatDescriptionCreate(allocator: nil, asbd: &asbd, layoutSize: 0, layout: nil, magicCookieSize: 0, magicCookie: nil, extensions: nil, formatDescriptionOut: &formatDesc)

This is the formatDesc - for argument formatDescription: CMFormatDescription?

Finally use CMSampleBufferCreate to create the sampleBuffer that contains the block of the reversed audio data in audioSamples:

CMSampleBufferCreate(allocator: kCFAllocatorDefault, dataBuffer: samplesBlock, dataReady: true, makeDataReadyCallback: nil, refcon: nil, formatDescription: formatDesc, sampleCount: audioSamples.count, sampleTimingEntryCount: 0, sampleTimingArray: nil, sampleSizeEntryCount: 0, sampleSizeArray: nil, sampleBufferOut: &sampleBuffer)

Each CMSampleBuffer created in this way is collected into an array [CMSampleBuffer?] with sampleBuffersForSamples:

func sampleBuffersForSamples(bufferSize:Int, audioSamples:[Int16], sampleRate:Int) -> [CMSampleBuffer?] {
    
    let blockedAudioSamples = audioSamples.blocks(size: bufferSize)
    
    let sampleBuffers = blockedAudioSamples.map { audioSamples in
        sampleBufferForSamples(audioSamples: audioSamples, sampleRate: sampleRate)
    }
    
    return sampleBuffers
}

In the next section the [CMSampleBuffer?] will be written to the output file sequentially.

3. Write the reversed sample buffers in [CMSampleBuffer?] to a file

Finally implement this method to create the reversed audio file passing the array [CMSampleBuffer?]:

func saveSampleBuffersToFile(_ sampleBuffers:[CMSampleBuffer?], destinationURL:URL, progress: @escaping (Float) -> (), completion: @escaping (Bool, String?) -> ())

This method uses an asset writer to write the samples.

First create the AVAssetWriter. Since we are writing Linear PCM samples the destinationURL has extension wav and the file type is AVFileType.wav:

guard let assetWriter = try? AVAssetWriter(outputURL: destinationURL, fileType: AVFileType.wav) else {
    completion(false, "Can't create asset writer.")
    return
}

Create an AVAssetWriterInput and attach it to the asset writer. A source format hint is obtained from the first sample buffer and the output settings are set to kAudioFormatLinearPCM for Linear PCM:

let sourceFormat = CMSampleBufferGetFormatDescription(sampleBuffer)

let audioFormatSettings = [AVFormatIDKey: kAudioFormatLinearPCM] as [String : Any]

let audioWriterInput = AVAssetWriterInput(mediaType: AVMediaType.audio, outputSettings: audioFormatSettings, sourceFormatHint: sourceFormat)
...
assetWriter.add(audioWriterInput)

Then write the CMSampleBuffer’s as the asset write input is ready to receive and append them:

let serialQueue: DispatchQueue = DispatchQueue(label: kReverseAudioQueue)
...
audioWriterInput.requestMediaDataWhenReady(on: serialQueue) {
    
    while audioWriterInput.isReadyForMoreMediaData, index < nbrSamples {
        
        if let currentSampleBuffer = sampleBuffers[index] {
            audioWriterInput.append(currentSampleBuffer)
        }
        
        ...
    }
}

To conclude assemble the above pieces readAndReverseAudioSamples, sampleBuffersForSamples and saveSampleBuffersToFile into the final method reverseAudio to carry out the 3 steps outlined at the start, namely:

  1. Read the audio samples of a file into an Array of [Int16] and reverse it
  2. Create an array of sample buffers [CMSampleBuffer?] for the array of reversed audio samples
  3. Write the reversed sample buffers in [CMSampleBuffer?] to a file
func reverseAudio(asset:AVAsset, destinationURL:URL, progress: @escaping (Float) -> (), completion: @escaping (Bool, String?) -> ())  {
    
    guard let (bufferSize, sampleRate, audioSamples) = readAndReverseAudioSamples(asset: asset) else {
        completion(false, "Can't read audio samples")
        return
    }
    
    let sampleBuffers = sampleBuffersForSamples(bufferSize: bufferSize, audioSamples: audioSamples, sampleRate: sampleRate)
    
    saveSampleBuffersToFile(sampleBuffers, destinationURL: destinationURL, progress: progress, completion: completion)
}