Neural Networks

The Wolfram Language has state-of-the-art capabilities for the construction, training and deployment of neural network machine learning systems. Many standard layer types are available and are assembled symbolically into a network, which can then immediately be trained and deployed on available CPUs and GPUs.

Automated Machine Learning

(RefLink[Classify, paclet:ref/Classify]) — automatic training and classification using neural networks and other methods

(RefLink[Predict, paclet:ref/Predict]) — automatic training and data prediction

(RefLink[FeatureExtraction, paclet:ref/FeatureExtraction]) — automatic feature extraction from image, text, numeric, etc. data

LearnDistribution — automatic learning of data distribution

(RefLink[ImageIdentify, paclet:ref/ImageIdentify]) — fully trained image identification for common objects

Prebuilt Material

(RefLink[NetModel, paclet:ref/NetModel]) — complete pre-trained net models

(RefLink[ResourceData, paclet:ref/ResourceData]) — access to training data, networks, etc.

Net Representation

(RefLink[NetGraph, paclet:ref/NetGraph]) — symbolic representation of trained or untrained net graphs to be applied to data

(RefLink[NetChain, paclet:ref/NetChain]) — symbolic representation of a simple chain of net layers

(RefLink[NetPort, paclet:ref/NetPort]) — symbolic representation of a named input or output port for a layer

(RefLink[NetExtract, paclet:ref/NetExtract]) — extract properties and weights etc. from nets

(RefLink[NetInformation, paclet:ref/NetInformation]) — give summary and detailed information about any network

Net Operations

(RefLink[NetTrain, paclet:ref/NetTrain]) — train parameters in any net from examples

(RefLink[NetInitialize, paclet:ref/NetInitialize]) — randomly initialize parameters for a network

(RefLink[NetPortGradient, paclet:ref/NetPortGradient]) — differentiate a net with respect to a port

(RefLink[NetStateObject, paclet:ref/NetStateObject]) — store and reuse recurrent state in a net

(RefLink[NetTrainResultsObject, paclet:ref/NetTrainResultsObject]) — represent what happened in net training

(RefLink[NetMeasurements, paclet:ref/NetMeasurements]) — measure the performance of a net on test data

(RefLink[TrainingProgressMeasurements, paclet:ref/TrainingProgressMeasurements]) — measure performance metrics during training

Basic Layers

(RefLink[LinearLayer, paclet:ref/LinearLayer]) — trainable layer with dense connections computing w.x+b

(RefLink[ElementwiseLayer, paclet:ref/ElementwiseLayer]) — apply a specified function to each element in a tensor

(RefLink[SoftmaxLayer, paclet:ref/SoftmaxLayer]) — layer globally normalizing elements to the unit interval

Loss Layers

Elementwise Computation Layers

Structure Manipulation Layers

Array Operation Layers

(RefLink[ConstantArrayLayer, paclet:ref/ConstantArrayLayer]) — embed a learned constant array into a (RefLink[NetGraph, paclet:ref/NetGraph])

Convolutional and Filtering Layers

Recurrent Layers

Sequence-Handling Layers

(RefLink[EmbeddingLayer, paclet:ref/EmbeddingLayer]) — trainable layer for embedding integers into continuous vector spaces

(RefLink[AttentionLayer, paclet:ref/AttentionLayer]) — trainable layer for finding parts of a sequence to attend to

Training Optimization Layers

Higher-Order Network Construction

(RefLink[NetMapOperator, paclet:ref/NetMapOperator]) — define a network that maps over a sequence

(RefLink[NetMapThreadOperator, paclet:ref/NetMapThreadOperator]) — define a network that maps over multiple sequences

(RefLink[NetFoldOperator, paclet:ref/NetFoldOperator]) — define a recurrent network that folds in elements of a sequence

Network Surgery

Weight Sharing

(RefLink[NetSharedArray, paclet:ref/NetSharedArray]) — represent an array shared between several layers

(RefLink[NetInsertSharedArrays, paclet:ref/NetInsertSharedArrays]) — convert all arrays in a net into shared arrays

Encoding & Decoding

(RefLink[NetEncoder, paclet:ref/NetEncoder]) — convert images, categories, etc. to net-compatible numerical arrays

(RefLink[NetDecoder, paclet:ref/NetDecoder]) — interpret net-generated numerical arrays as images, probabilities, etc.

Activation Functions

(RefLink[Ramp, paclet:ref/Ramp]) — rectified linear (ReLU)

Importing & Exporting

(StringTypeLink["WLNet", paclet:ref/format/WLNet]) — Wolfram Language Net representation format

(StringTypeLink["MXNet", paclet:ref/format/MXNet]) — MXNet net representation format

Managing Data & Training

(RefLink[ClassifierMeasurements, paclet:ref/ClassifierMeasurements]) — measure accuracy, recall, etc. of a classifier net

(RefLink[DeleteMissing, paclet:ref/DeleteMissing]) — remove missing data before training

Reinforcement Learning Environments

(StringTypeLink["OpenAIGym", paclet:ref/device/OpenAIGym]), ... — access to video games and many other test environments