Audio Basics

The Wolfram Language provides built-in support for both programmatic and interactive audio processing, fully integrated with the Wolfram Language's powerful mathematical and algorithmic capabilities. You can create and import sound files, manipulate them with built-in functions, apply linear and nonlinear filters, and visualize them in any number of ways.

Audio Creation and Representation

An audio object can be created from numerical arrays, files, and URLs.

Audio creation functions.

The simplest way to create an audio object is to wrap the (RefLink[Audio, paclet:ref/Audio]) constructor around a vector of real values ranging from -1 to 1.

Here is a one-channel in-core audio object created from a vector of numbers:

Audio[RandomReal[{-1,1}, 44100]]

Another way is to obtain an out-of-core audio object from a file on the local file system or any accessible remote location. Out-of-core audio objects do not store samples in memory.

This creates an out-of-core audio object from the Wolfram Language documentation directory (RefLink[ExampleData, paclet:ref/ExampleData]) and displays the number of bytes used internally by the object:

a1=Audio["ExampleData/rule30.wav"]

ByteCount[a1]

You can use (RefLink[Import, paclet:ref/Import]) to create an in-core audio object from a file or URL.

This creates an in-core audio object and displays the number of bytes used internally by the object:

a2=Import["ExampleData/rule30.wav"];

ByteCount[a2]

The "Audio" collection from (RefLink[ExampleData, paclet:ref/ExampleData]) contains sample audio clips.

This lists available audio clips:

ExampleData["Audio"]//Short

This lists the properties of a given audio object:

ExampleData[{"Audio","Wind"}, "Properties"]

All types of (RefLink[Sound, paclet:ref/Sound]) objects can be converted to (RefLink[Audio, paclet:ref/Audio]).

Here is an example that converts MIDI notes to audio:

midi=Audio[]

See the waveform of the generated audio:

AudioPlot[midi]

This converts sound represented by (RefLink[SampledSoundFunction, paclet:ref/SampledSoundFunction]) to audio:

sineTone = Play[Sin[2Pi 440 t],{t,0,0.01}]

Audio[sineTone, "Real32"]//AudioPlot

Various oscillators and noises can be created using (RefLink[AudioGenerator, paclet:ref/AudioGenerator]).

This creates sinusoidal audio:

AudioGenerator["Sin", 0.02]//AudioPlot

This creates one second of the pink noise:

AudioGenerator["Pink"]//AudioPlot

Audio Properties

Useful properties of an audio object can be obtained by calling the following functions.

Audio properties.

This returns audio length, duration, number of channels, sample rate, and data type:

props = {AudioLength,Duration,AudioChannels,AudioSampleRate,AudioType};

Dataset@AssociationThread[props,Through[props[a1]]]

The array of sample values can be extracted using the function (RefLink[AudioData, paclet:ref/AudioData]). By default, the function returns real values, but you can ask for a specific type using the optional "type" argument.

This returns a fragment of the in-core audio as a vector of real values scaled to the range -1 to 1:

AudioData[AudioTrim[a1, {0.3, 0.3002}]]//MatrixForm

Here is the same fragment extracted from the out-of-core audio as a vector of integers in the range -127 to 128:

AudioData[AudioTrim[a2, {(Quantity[13230, "samples", samples, "Samples"]),(Quantity[13237, "samples", samples, "Samples"])}], "SignedInteger8"]//MatrixForm

In the case of multichannel audio, the raw sample data is represented by a list of channel values (2D array).

This creates an out-of-core multichannel audio object and extracts a fragment of that audio object as a list of channel values:

a3=Audio["ExampleData/drums.ogg"];

MatrixForm/@AudioData[AudioTrim[a3,{(Quantity[1, "samples", samples, "Samples"]),(Quantity[8, "samples", samples, "Samples"])}]]

A multichannel audio object can be split into a list of single-channel audio objects and conversely, a multichannel audio object can be created from any number of single-channel audio objects.

This splits the example stereo audio object into two mono audio objects:

AudioChannels/@AudioChannelSeparate[a3]

This creates the stereo audio object from two mono audio objects:

AudioChannelCombine[{AudioGenerator["Sin"],AudioGenerator["White"]}]//AudioChannels

Audio Visualization

Audio can be visualized in a variety of ways.

Audio visualization functions.

This plots the waveform of an audio signal:

AudioPlot[ExampleData[{"Sound","AltoFluteScale"}, "Audio"]]

This highlights silent parts of an audio signal:

audio=ExampleData[{"Audio","FluteScale"}];
silence=AudioIntervals[audio,#RMSAmplitude<0.1&];
AudioPlot[audio,Epilog->{RGBColor[1,0,0,.5],Rectangle[{#[[1]],-1},{#[[2]],1}]&/@silence}]

This plots the power spectrum of an audio signal:

Periodogram[Audio["ExampleData/rule30.wav"],300,200,HammingWindow,GridLines->Automatic]

This plots the spectrogram of an audio signal using partitions of length 512 and the offset 64:

a=AudioGenerator[Sin[2π 1633 #]+Sin[2π 1209 #]&,SampleRate->8000];Spectrogram[a,512,64]

Basic Operations

Many useful audio processing tasks require nothing more than simple arithmetic operations between two audio objects or an audio object and a constant. For example, you can change volume by multiplying an audio object by a constant factor or by adding (subtracting) a constant to (from) an audio object. For this purpose, all Wolfram Language operators and functions with attributes (RefLink[NumericFunction, paclet:ref/NumericFunction]) or (RefLink[Listable, paclet:ref/Listable]) are overloaded to work with audio objects.

Some arithmetical and statistical operations on audio objects.

This creates a linear combination of two audio objects:

0.3*AudioGenerator["Sin", 0.01]+0.7*AudioGenerator["Triangle", 0.02]//AudioPlot

This computes the mean of a list of audio objects:

Mean[{,,,,}]

This computes statistics of an audio object:

a=ExampleData[{"Audio","Bird"}];
stats={Total,Min, Max,Mean,Median,StandardDeviation};Dataset[AssociationThread[ToString/@stats ->Through[stats[a]]]]

A system option called "IndeterminateValue" is used to replace values that can result from arithmetical operation but cannot be stored in an audio object. These include (RefLink[ComplexInfinity, paclet:ref/ComplexInfinity]) and (RefLink[Indeterminate, paclet:ref/Indeterminate]).

This displays the current value of the "IndeterminateValue" option and changes it to 2:

SystemOptions["AudioProcessingOptions"->"IndeterminateValue"]

SetSystemOptions["AudioProcessingOptions"->"IndeterminateValue"->2.]

This shows how "IndeterminateValue" is used to replace unacceptable values in the result of an arithmetical operation:

data=N[{-2,-1,0,1,2}];
a=Audio[data];

data^data

AudioData[a^a]

Complex numbers in the result of an arithmetical operation are replaced by the real parts of these values:

data= {-2,-1,0,1,2};
a=Audio[data];

Sqrt[data]

AudioData[Sqrt[a]]

Re[Sqrt[data]]

Change the "IntermediateValue" back to the default zero:

SetSystemOptions["AudioProcessingOptions"->"IndeterminateValue"->0]

Consider the audio manipulation operations that change the audio duration by trimming, deleting, or padding. These operations serve a variety of useful purposes. Trimming and deleting allow you to create a new audio object from a selected portion of a larger one, while padding is typically used to extend an audio object at the ends to ensure uniform treatment of the end samples in many audio processing tasks.

Some basic audio operations.

This selects the first second of an audio object:

AudioTrim[a1,{0,1}]//Duration

This shows three different padding methods applied to the end of an audio object:

AudioPlot[AudioPad[AudioGenerator["Sawtooth",0.01],0.01,#]&/@{"Silent","Reflected","Periodic"}]

This trims the silence from the beginning and the end of an audio object:

(a=AudioPad[AudioGenerator["Sin",0.01],{0.01,0.02}])//AudioPlot

AudioTrim[a]//AudioPlot

This deletes samples between the first and the third seconds of an audio object:

AudioPlot[a=ExampleData[{"Audio","FemaleVoice"}]]

AudioDelete[a, {1,3}]//AudioPlot

This splits an audio object in half:

Duration/@AudioSplit[a, Duration[a]/2]

This partitions an audio object into non-overlapping fragments:

Row[AudioPlot[#,ImageSize->Small]&/@ AudioPartition[AudioGenerator["Triangle", 0.01],0.005]]

This partitions an audio object into overlapping fragments:

Row[AudioPlot[#,ImageSize->Small]&/@ AudioPartition[AudioGenerator["Triangle", 0.01],{0.005}, 0.004]]

Intervals that satisfy various criteria can be computed and used to extract corresponding blocks of an audio object.

This computes the intervals where the maximal value is greater than .1 and the value of the spectral centroid is smaller than 800:

intervals=AudioIntervals[a1,#Max>.1&&#SpectralCentroid<800&]

This returns audio objects corresponding to the intervals computed above:

AudioTrim[a1, intervals]//AudioPlot

It is frequently necessary to change the sample rate of an audio object by resampling, or to normalize audio samples in some manner. Functions that perform these basic tasks are readily available.

Audio resampling, normalizing, and amplifying.

This changes the sample rate of the example audio to 8000:

AudioResample[a1,8000]//AudioSampleRate

This normalizes audio samples:

AudioNormalize[a1]//AudioPlot

This increases the volume of an audio object:

AudioAmplify[a1,5]//AudioPlot

The Wolfram Language also provides functions that perform operations on the list of audio objects.

Joining, overlaying, and conforming lists of audio objects.

This joins two audio signals:

AudioJoin[{AudioGenerator["Square", 0.01],AudioGenerator["Triangle", 0.01]}]//AudioPlot

This overlays two audio signals:

AudioOverlay[{AudioGenerator["Sin", 0.01],AudioGenerator["Impulse", 0.01]}]//AudioPlot

This conforms two audio signals:

list={Import["ExampleData/rule30.wav"],Import["ExampleData/drums.ogg"]};
prop={"Channels","Duration","Length","SampleRate","Type"};

TableForm[Transpose@(AudioMeasurements[#,prop, List]&/@list),TableHeadings->{prop,None},TableDepth->2]

TableForm[Transpose@(AudioMeasurements[#,prop, List]&/@ConformAudio[list]),TableHeadings->{prop,None},TableDepth->2]