Audio Synthesis

The Wolfram Language provides extensive support for the creation, analysis, and manipulation of audio data, fully integrated with the Wolfram Language's powerful mathematical and algorithmic capabilities.

The starting point for audio synthesis is the (RefLink[AudioGenerator, paclet:ref/AudioGenerator]) function.

Audio generation.

Oscillators

The simplest way to generate a signal is to use one of the native oscillator models supported in (RefLink[AudioGenerator, paclet:ref/AudioGenerator]).

Generate a sinusoidal signal:

AudioGenerator["Sin"]

Generate a sinusoidal signal of a specific frequency:

AudioGenerator[{"Sin",1000}]

Different oscillator models have different harmonic contents:

oscillators={"Sin","Sawtooth","Square"};
Panel@TableForm[{
AudioGenerator[#,Appearance->"Minimal"],AudioPlot[AudioGenerator[#],PlotRange->.01,PlotTheme->"Minimal"],
Periodogram[AudioGenerator[#],2000,PlotRange->{{0,2000},All},Ticks->{True,False}]
}&/@oscillators,TableHeadings -> {oscillators, {"Audio", "Waveform", "Spectrum"}},TableSpacing -> {2, 2}]

All oscillator models accept time-varying parameters for frequency and phase: the input can be a (RefLink[TimeSeries, paclet:ref/TimeSeries]), another (RefLink[Audio, paclet:ref/Audio]) object, or even a pure function.

Use an audio signal to control the frequency:

vib=AudioGenerator[{"Sin",(Quantity[6, "Hz", hertz, "Hertz"]),Pi},4];
AudioGenerator[{"Sin",440+10vib}]

Use a pure function to control the frequency:

AudioGenerator[{"Sin",440+10Sin[2Pi 6#]&},4]

Use a time series to control the frequency:

vib=TimeSeries[Table[{t,Sin[2Pi 6t]},{t,0,4,1/200}]];
AudioGenerator[{"Sin",440+10vib}]

Noise Generators

In addition to oscillators, (RefLink[AudioGenerator, paclet:ref/AudioGenerator]) allows for the creation of different kinds of noise signals.

Generate one second of pink noise:

AudioGenerator["Pink"]

Different types of noise have different spectral contents:

noises={"White","PeriodicRandomNoise","Pink","Brown","Blue",{"Color",1.5`}};
Periodogram[AudioGenerator/@noises,2000,ScalingFunctions->{"Log10","dB"},PlotStyle->{,,,,,},PlotLegends->Placed[noises,Right]]

White noise is characterized by a flat spectrum. By default, sample values are picked from a uniform distribution, but it is possible to change that without affecting the spectrum.

White noise with amplitudes sampled from a normal distribution:

white=AudioGenerator[{"White",NormalDistribution[]}]//AudioNormalize

Plot the histogram and the periodogram of the signal:

histogram=Histogram[AudioData@white]

Periodogram[white,2000,PlotRange->{All,{-40,10}},ScalingFunctions->{"Log10","dB"}]

Using {"Color",α} will generate a "colored" noise signal whose spectrum follows a (f^(-α)) distribution. "Pink", "Brown", and "Blue" are instances of colored noise, where the α has values of 1, 2, and -1.

Change the "color" of the noise:

Manipulate[Periodogram[AudioGenerator[{"Color",α}],2000,ScalingFunctions->{"Log10", "dB"},PlotRange->{All,{-40,10}}],{α,-2,2}]

Processes and Time Series

(RefLink[AudioGenerator, paclet:ref/AudioGenerator]) can be used to generate a signal by sampling a random process.

Generate an audio signal from a discrete-time or continuous-time process:

AudioGenerator[FractionalBrownianMotionProcess[.5]]//AudioNormalize

(AudioGenerator[TelegraphProcess[2],5]+1).5AudioGenerator["Sin",5]

(RefLink[AudioGenerator, paclet:ref/AudioGenerator]) can also be used to create an (RefLink[Audio, paclet:ref/Audio]) object from a (RefLink[TimeSeries, paclet:ref/TimeSeries]).

Create an audio signal starting from a (RefLink[TimeSeries, paclet:ref/TimeSeries]):

windData=TimeSeriesRescale[WindSpeedData["KJFK",{DateObject[{2010,1,1}],DateObject[{2014,1,8}]}],{0,1}];
ts=QuantityMagnitude@TimeSeries[windData,MissingDataMethod->{"Interpolation",InterpolationOrder->0}];
AudioGenerator[Standardize@ts]//AudioNormalize

Advanced Synthesis

Subtractive Synthesis

By processing an audio object with one or more frequency-selective filters, it is possible to manipulate its harmonic content and therefore change its timbre. This technique is effective if the starting audio object is harmonically rich.

Filter a sawtooth wave:

LowpassFilter[AudioGenerator["Sawtooth"],(Quantity[1000, "Hz", hertz, "Hertz"])]

Use two different filters on a square waveform:

Manipulate[
lpFiltered=filter[AudioGenerator["Square",Appearance->"Minimal"],(Quantity[f, "Hz", hertz, "Hertz"]),101,Padding->"Periodic"];
TableForm[{{lpFiltered,
AudioPlot[lpFiltered,PlotRange->.01,PlotTheme->"Minimal",ImageSize->Small],
Periodogram[lpFiltered,1000,PlotRange->{{0,5000},All},Ticks->{True,False},ImageSize->Small]}}],
{{f,600, "Cutoff Frequency"},200,4000},{filter,{LowpassFilter,HighpassFilter}},ContinuousAction->False]

Additive Synthesis

In contrast to subtractive synthesis where harmonics are attenuated using filters, additive synthesis is based on the addition of harmonics. The effect is obtained with linear combination of sine waves at different frequencies, where the user has control over the coefficients of the linear combination.

Control the amplitudes of the first 8 harmonics of a 200 Hz fundamental:

Manipulate[
sineBank=Table[AudioGenerator[{"Sin",200i}],{i,8}];
amps={a1,a2,a3,a4,a5,a6,a7,a8};
as=Mean[amps sineBank];
TableForm@{{as,Periodogram[as,4000,PlotRange->{{0,2000},All},Ticks->{True,False},ImageSize->Small]}},
{{a1,.5},0,1},{{a2,.5},0,1},{{a3,.5},0,1},{{a4,.5},0,1},{{a5,.5},0,1},{{a6,.5},0,1},{{a7,.5},0,1},{{a8,.5},0,1},ContinuousAction->False,ControlPlacement->Top]

Amplitude Modulation

Amplitude modulation (AM) is a common technique to create signals with complex harmonic content.

In AM, two signals are used, called carrier and modulator. The result is computed using carrier×(1+μ×modulator), where the scaling constant μ is called modulation index.

If the modulator is a sinusoid, for each component of the carrier at a frequency f, a pair of sidebands is created at frequencies f±m, whose amplitude is controlled by the modulation index.

Amplitude modulation of a triangle wave:

carrier=AudioGenerator[{"Triangle",200},2];
modulator=AudioGenerator[{"Sin",100},2];
modIndex=.8;
am=carrier*(1+modIndex*modulator)//AudioNormalize

Plot the periodogram of the signal:

Periodogram[am,8000,PlotRange->{{0,2000},All},GridLines->{Table[f,{f,200,2000,400}],None}]

Frequency Modulation

Another popular technique, discovered in the early 1970s, uses frequency modulation (FM).

In the case of FM, the frequency of the carrier signal is controlled by the modulator and the modulation index. As the modulation index increases, sidebands occur above and below the carrier frequency at intervals of the modulating frequency.

Basic frequency modulation:

cf=500;
mf=200;
modulator=mf*AudioGenerator[{"Sin",mf}];
modIndex=1;
fm=AudioGenerator[{"Sin",cf+modIndex*modulator}]

Plot the periodogram of the signal:

Periodogram[fm,8000,ScalingFunctions->"Absolute",PlotRange->{{0,1100},{0,800}},GridLines->Automatic]

The number and intensity of these additional sidebands are related to the modulation index.

Use a higher modulation amount to increase sideband amplitude:

cf=500;
mf=100;
modIndex={1,2,3};
fm=Table[AudioGenerator[{"Sin",cf+mIndex mf Sin[2 Pi mf #]&}],{mIndex,modIndex}];
periodograms=Table[Periodogram[fm[[i]],20000,ScalingFunctions->"Absolute",ImageSize->Small,PlotRange->{{0,1000},All},GridLines->Automatic,PlotLabel->"Modulation Index: = "<>ToString[modIndex[[i]]]],{i,Length@modIndex}];
Grid[{fm,periodograms}]

The position of the sidebands is determined by the ratio between the carrier and modulator frequencies. Changing this ratio, it is possible to obtain both harmonic (typical of pitched instruments) and inharmonic (typical of percussive instruments) spectra.

Create harmonic and inharmonic signals:

cf=500;
ratios={3/2,Sqrt[2],Pi};
fm=Table[mf=cf/r;
AudioGenerator[{"Sin",cf+3.2 mf Sin[2 Pi mf #]&}],{r,ratios}];
periodograms=Table[Periodogram[fm[[i]],20000,ScalingFunctions->"Absolute",ImageSize->Small,PlotRange->{{0,1000},All},GridLines->Automatic,PlotLabel->"((Subscript[c, f])/(Subscript[m, f])) = "<>ToString[ratios[[i]],FormatType->StandardForm]],{i,Length@ratios}];
Grid[{fm,periodograms}]

If the modulation index becomes a time-varying argument, it becomes easy to generate signals whose harmonic content evolves with time.

Increase the bandwidth of the signal by incrementing the modulation index:

cf=500;
mf=cf/6;
modulator=mf AudioGenerator[{"Sin",mf},5];
modIndex=3AudioGenerator[{"Sawtooth",.1},5];
fm=AudioGenerator[{"Sin",cf+modIndex  modulator}]

Plot the spectrogram of the result:

Spectrogram[fm,4096,4096,HannWindow,PlotRange->{All,{0,1200}}]

The modulating frequency can be a time-varying parameter as well.

Change the position of the sidebands by varying the modulating frequency:

cf=500;
mf=100+400AudioGenerator[{"Sawtooth",.1},5];
modulator=mf AudioGenerator[{"Sin",mf}];
fm=AudioGenerator[{"Sin",cf+3  modulator}]

Plot the spectrogram of the result:

Spectrogram[fm,4096,4096,HannWindow,PlotRange->{All,{0,3000}}]