If you work with buffered data such as Audio/Video Frame data, you have no doubt appreciated the features of Typed Arrays that came with ES2017 javascript. The ability to move, duplicate, manipulate blocks of data using object methods is achieved by 'imposing' a dataview on the data blocks. These have made buffered data processing a breeze and fast (avoid slow for-loops and extra code ). A detailed discussion of typed arrays is found here: javascript typed arrays.
The Problem:
The problem scenario where I took advantage of these features is this: Receive a stream of Audio data sampled at 48000 Hz, down-sample and produce the data at 16000 Hz. The straightforward implementation is use a FIFO Queue to buffer the incoming stream, process the data in queue to produce the output.
The problem scenario where I took advantage of these features is this: Receive a stream of Audio data sampled at 48000 Hz, down-sample and produce the data at 16000 Hz. The straightforward implementation is use a FIFO Queue to buffer the incoming stream, process the data in queue to produce the output.
A couple of immediate challenges occurred:
a. Performance: Buffers need to be processed at a rate faster than at which they arrive.
b. The size of the input buffer and the size of the output are different.
The solution:
Implement a queue for data buffers with each piece of data in the buffer having the same datatype (Float32 for web audio data). Allow for the size of the input data buffer enqueued to be different that the size of the output buffer that is dequeued. This implementation is shown below:
The Q is a large Q buffer of the correct type (Float32 in my case). A high watermark is associated with this buffer, and points to the location in the buffer where new data can be inserted during the enqueueing process. When the queue is empty, the high watermark is 0. Dequeing data off the Q is done from the 'top' (i.e. from 0) and after dequeing a block of data, the high watermark is adjusted.
The code:
Here is the code (ES2017 - JavaScript). The Q is implemented as class frameBufferQB:
Here is the code (ES2017 - JavaScript). The Q is implemented as class frameBufferQB:
class frameBufferQB {
//This class is like a queue, except that it is one large blob of float32 with a high watermark. The incoming data
//is assumed to be float32Array and is accumalated into this blob at the high watermark (called bottom in code).
//in blocks of chunksize and stored in a queue
// Creates the data store blob
constructor(blobsize,readChunkSize,dataType='Float32') {
if (dataType !='Float32')
console.log('**Error:'+ dataType + ' not implemented!');
else this.dataStore = new Float32Array(blobsize);
this.bottom=0;
this.readChunkSize=readChunkSize;
this.blobsize=blobsize
}
enqueue(chunk){
//This method adds the input chunk buffer to the bottom of the q data.
if (this.bottom+chunk.length > this.blobsize) {
console.log ('blob overflow! bottom, chunklength, blobsize '+this.bottom+', '+chunk.length+', '+this.dataStore.length)
} else {
this.dataStore.set(chunk,this.bottom) //copy the array to the bottom
this.bottom +=chunk.length
}
}
dequeue() {
// retrieves a chunk of data of size this.readChunkSize and adjusts the bottom
// If insufficient data returns false
let retval=null
if (this.bottom >= this.readChunkSize) {
retval=this.dataStore.slice(0,this.readChunkSize)
this.dataStore.copyWithin(0,this.readChunkSize,this.bottom)
this.bottom -= this.readChunkSize
}
return retval
}
isEmpty() {
return (this.bottom<this.readChunkSize)
}
}Example usage:
myQ=new frameBufferQB(100,2) //create Q with size 100 and dequeue with length of 2
inpArr = new Float32Array([10,20,30,40,50]);
myQ.enqueue(inpArr)
console.log(myQ.dequeue()) // outputs two elements [10,20]
console.log(myQ.dequeue()) //outputs next two elements [30,40]
Overriding enqueue/dequeue to Transform Data:
To perform transformations (such as downsampling audio data, the methods enqueue and dequeue can be overridden by code to perform the transformation during these operations.