audioFeatureExtractor
Streamline audio feature extraction
Description
audioFeatureExtractor encapsulates multiple audio feature
extractors into a streamlined and modular implementation.
Creation
Description
aFE = audioFeatureExtractor()
aFE = audioFeatureExtractor(Name,Value)aFE using one or more name-value
pair arguments.
Properties
Object Functions
extract | Extract audio features |
setExtractorParams | Set nondefault parameter values for individual feature extractors |
info | Output mapping and individual feature extractor parameters |
generateMATLABFunction | Create MATLAB function compatible with C/C++ code generation |
Examples
Extract Multiple Audio Features
Read in an audio signal.
[audioIn,fs] = audioread("Counting-16-44p1-mono-15secs.wav");Create an audioFeatureExtractor object that extracts the MFCC, delta MFCC, delta-delta MFCC, pitch, and spectral centroid of an audio signal. Use a 30 ms analysis window with 20 ms overlap.
aFE = audioFeatureExtractor( ... "SampleRate",fs, ... "Window",hamming(round(0.03*fs),"periodic"), ... "OverlapLength",round(0.02*fs), ... "mfcc",true, ... "mfccDelta",true, ... "mfccDeltaDelta",true, ... "pitch",true, ... "spectralCentroid",true);
Call extract to extract the audio features from the audio signal.
features = extract(aFE,audioIn);
Use info to determine which column of the feature extraction matrix corresponds to the requested pitch extraction.
idx = info(aFE)
idx = struct with fields:
mfcc: [1 2 3 4 5 6 7 8 9 10 11 12 13]
mfccDelta: [14 15 16 17 18 19 20 21 22 23 24 25 26]
mfccDeltaDelta: [27 28 29 30 31 32 33 34 35 36 37 38 39]
spectralCentroid: 40
pitch: 41
Plot the detected pitch over time.
t = linspace(0,size(audioIn,1)/fs,size(features,1)); plot(t,features(:,idx.pitch)) title('Pitch') xlabel('Time (s)') ylabel('Frequency (Hz)')
Extract Features from Dataset
Create an audio datastore that points to audio samples included with Audio Toolbox®.
folder = fullfile(matlabroot,'toolbox','audio','samples'); ads = audioDatastore(folder);
Find all files that correspond to a sample rate of 44.1 kHz and then subset the datastore.
keepFile = cellfun(@(x)contains(x,'44p1'),ads.Files);
ads = subset(ads,keepFile);Convert the data to a tall array. tall arrays are evaluated only when you request them explicitly using gather. MATLAB® automatically optimizes the queued calculations by minimizing the number of passes through the data. If you have Parallel Computing Toolbox™, you can spread the calculations across multiple machines. The audio data is represented as an M-by-1 tall cell array, where M is the number of files in the audio datastore.
adsTall = tall(ads)
Starting parallel pool (parpool) using the 'local' profile ...
Connected to the parallel pool (number of workers: 6).
adsTall =
M×1 tall cell array
{ 539648×1 double}
{ 227497×1 double}
{ 8000×1 double}
{ 685056×1 double}
{ 882688×2 double}
{1115760×2 double}
{ 505200×2 double}
{3195904×2 double}
: :
: :
Create an audioFeatureExtractor object to extract the mel spectrum, Bark spectrum, ERB spectrum, and linear spectrum from each audio file. Use the default analysis window and overlap length for the spectrum extraction.
aFE = audioFeatureExtractor('SampleRate',44.1e3, ... 'melSpectrum',true, ... 'barkSpectrum',true, ... 'erbSpectrum',true, ... 'linearSpectrum',true);
Define a cellfun function so that audio features are extracted from each cell of the tall array. Call gather to evaluate the tall array.
specsTall = cellfun(@(x)extract(aFE,x),adsTall,"UniformOutput",false);
specs = gather(specsTall);Evaluating tall expression using the Parallel Pool 'local': - Pass 1 of 1: Completed in 12 sec Evaluation completed in 12 sec
The specs variable returned from gather is a numFiles-by-1 cell array, where numFiles is the number of files in the datastore. Each element of the cell array is a numHops-by-numFeatures-by-numChannels array, where the number of hops and number of channels depends on the length and number of channels of the audio file, and the number of features is the requested number of features from the audio data.
numFiles = numel(specs)
numFiles = 12
[numHops1,numFeaturesFile1,numChanelsFile1] = size(specs{1})numHops1 = 1053
numFeaturesFile1 = 620
numChanelsFile1 = 1
[numHops2,numFeaturesFile2,numChanelsFile2] = size(specs{2})numHops2 = 443
numFeaturesFile2 = 620
numChanelsFile2 = 1
Algorithms
The audioFeatureExtractor creates a feature extraction pipeline based on
your selected features. To reduce computations, audioFeatureExtractor reuses
intermediary representations. Some intermediate representations can be output as
features:
For example, to create an object that extracts the centroid of the Bark spectrum, the flux
of the Bark spectrum, the pitch, the harmonic ratio, and the delta-delta of the MFCC, specify
the audioFeatureExtractor
as:
aFE = audioFeatureExtractor( ... "SpectralDescriptorInput","barkSpectrum", ... "spectralCentroid",true, ... "spectralFlux",true, ... "pitch",true, ... "harmonicRatio",true, ... "mfccDeltaDelta",true)
aFE =
audioFeatureExtractor with properties:
Properties
Window: [1024×1 double]
OverlapLength: 512
SampleRate: 44100
FFTLength: []
SpectralDescriptorInput: 'barkSpectrum'
Enabled Features
mfccDeltaDelta, spectralCentroid, spectralFlux, pitch, harmonicRatio
Disabled Features
linearSpectrum, melSpectrum, barkSpectrum, erbSpectrum, mfcc, mfccDelta
gtcc, gtccDelta, gtccDeltaDelta, spectralCrest, spectralDecrease, spectralEntropy
spectralFlatness, spectralKurtosis, spectralRolloffPoint, spectralSkewness, spectralSlope, spectralSpread
To extract a feature, set the corresponding property to true.
For example, obj.mfcc = true, adds mfcc to the list of enabled features.
Note
Because audioFeatureExtractor reuses intermediary representations, the
features output from audioFeatureExtractor may not correspond with the
default configuration of features output by corresponding individual feature
extractors.
Compatibility Considerations
Extended Capabilities
See Also
Audio Labeler | audioDataAugmenter | audioDatastore | Extract Audio Features | vggishFeatures