Encoding and Decoding Data for Neural Networks

Neural networks in the Wolfram Language can interface with many types of data, including numerical, categorical, textual, image and audio. The functions (RefLink[NetEncoder, paclet:ref/NetEncoder]) and (RefLink[NetDecoder, paclet:ref/NetDecoder]) are used to automatically and efficiently translate non-numerical data to and from net-compatible (RefLink[NumericArray, paclet:ref/NumericArray]) objects.

Encoders

(RefLink[NetEncoder, paclet:ref/NetEncoder]) — convert images, categories, etc. to net-compatible numerical arrays

Audio Encoders

(StringTypeLink["Audio", paclet:ref/netencoder/Audio]) — encode audio as a sequence of waveform amplitudes

(StringTypeLink["AudioMelSpectrogram", paclet:ref/netencoder/AudioMelSpectrogram]) — encode audio as a mel spectrogram

(StringTypeLink["AudioMFCC", paclet:ref/netencoder/AudioMFCC]) — encode audio as a sequence of MFCC vectors

(StringTypeLink["AudioSpectrogram", paclet:ref/netencoder/AudioSpectrogram]) — encode audio as a spectrogram

(StringTypeLink["AudioSTFT", paclet:ref/netencoder/AudioSTFT]) — encode audio as a sequence of Fourier transforms

Text Encoders

(StringTypeLink["BPESubwordTokens", paclet:ref/netencoder/BPESubwordTokens]) — encode tokens in a string as a sequence of integer codes

(StringTypeLink["Characters", paclet:ref/netencoder/Characters]) — encode characters in a string as a sequence of integer codes or one-hot vectors

(StringTypeLink["Tokens", paclet:ref/netencoder/Tokens]) — encode tokens in a string as a sequence of integer codes

(StringTypeLink["UTF8", paclet:ref/netencoder/UTF8]) — encode strings as their UTF8 bytes

Image Encoders

(StringTypeLink["Image", paclet:ref/netencoder/Image]) — encode a 2D image as a rank-3 array

(StringTypeLink["Image3D", paclet:ref/netencoder/Image3D]) — encode a 3D image as a rank-4 array

Other Encoders

(StringTypeLink["Boolean", paclet:ref/netencoder/Boolean]) — encode (RefLink[True, paclet:ref/True]) and (RefLink[False, paclet:ref/False]) as 1 and 0

(StringTypeLink["Class", paclet:ref/netencoder/Class]) — encode a class label as an integer code or a one-hot vector

(StringTypeLink["Function", paclet:ref/netencoder/Function]) — use a custom function to encode an input

Decoders

(RefLink[NetDecoder, paclet:ref/NetDecoder]) — interpret net-generated numerical arrays as images, probabilities, etc.

Text Decoders

(StringTypeLink["BPESubwordTokens", paclet:ref/netdecoder/BPESubwordTokens]) — decode probability vectors as a string of subword tokens

(StringTypeLink["Characters", paclet:ref/netdecoder/Characters]) — decode probability vectors as a string of characters

(StringTypeLink["Tokens", paclet:ref/netdecoder/Tokens]) — decode probability vectors as a string of tokens

Image Decoders

(StringTypeLink["Image", paclet:ref/netdecoder/Image]) — decode a rank-3 array as a 2D image

(StringTypeLink["Image3D", paclet:ref/netdecoder/Image3D]) — decode a rank-4 array as a 3D image

Other Decoders

(StringTypeLink["Boolean", paclet:ref/netdecoder/Boolean]) — decode 1 and 0 as (RefLink[True, paclet:ref/True]) and (RefLink[False, paclet:ref/False])

(StringTypeLink["Class", paclet:ref/netdecoder/Class]) — decode probability arrays as class labels

(StringTypeLink["CTCBeamSearch", paclet:ref/netdecoder/CTCBeamSearch]) — decode sequences of probability vectors trained with a (RefLink[CTCLossLayer, paclet:ref/CTCLossLayer])

(StringTypeLink["Function", paclet:ref/netdecoder/Function]) — decode using a custom function