Introduction to Linear Algebra in Mathematica

Mathematica has a broad range of functions to support linear algebra operations and to integrate them into the system. It can work with vectors, matrices, and tensors that can contain machine-precision floating-point numbers, arbitrary-precision floating-point numbers, complex floating-point numbers, integers, rational numbers, and general symbolic quantities. Linear algebra operations are supported for matrices that contain all these different types of entry.

Mathematica supports both dense and sparse matrices. Several tutorials concentrate on dense matrices. Sparse matrices are documented under "Working with Sparse Arrays". In general, all the operations that work on dense matrices work on sparse matrices in an equivalent way.

Matrices are represented in Mathematica with lists. They can be entered directly with the { } notation that Mathematica provides for lists.

{{1.5,1.6},{1.7,1.8}}

They can be created programmatically with (RefLink[Table, paclet:ref/Table]).

Table[1/(i*j),{i,3},{j,3}]

mat=Table[(Subscript[m, i,j]),{i,3},{j,3}]

You can extract size information.

Dimensions[mat]

It is also possible to test if something is a matrix.

MatrixQ[ mat]

(RefLink[MatrixForm, paclet:ref/MatrixForm]) helps to make the structure of the matrix clearer.

MatrixForm[mat]

Matrices are important in many areas of computation because they are an efficient way to represent linear systems of equations. Many computer applications can work with matrices because they can work efficiently with arrays of numbers. However, Mathematica can also work directly with the systems of linear equations the matrices represent. Here, a matrix is multiplied by a vector of the unknowns and a system of equations is formed.

mat={{1.5,1.6},{1.7,1.8}};
eqns=Thread[mat.{x,y}=={2,6}]

These equations can be solved with the algebraic equation solver, (RefLink[Solve, paclet:ref/Solve]).

Solve[eqns,{x,y}]

Alternatively, the matrix representation of the equations can be solved directly with the (RefLink[LinearSolve, paclet:ref/LinearSolve]) command.

LinearSolve[mat,{2,6}]

Conversion between matrices and the linear systems they represent is often useful in order to understand the underlying principles. Using one form or the other can also be a very useful way to set up particular problems.

Tensors and Arrays

A matrix has two indices that label each entry. By convention the values of these indices label the row and column when the matrix is written in a tabular form. A generalization of matrices can have less than or more than two indices. Typically an object with no indices is called a scalar, and one with one index is called a vector. An object with more than two indices is called a tensor, although a scalar, vector, or matrix can also be called a tensor.

Mathematica works with tensors that have arbitrary numbers of indices with the same list structure that is used for matrices. The number of levels of lists represents the number of indices. For example, the following is a vector.

vec={1,2,3}

Dimensions[vec]

The following is a tensor with 3 indices, with lengths 2, 3, and 4, respectively. The number of indices required to label a tensor can be called the rank of the tensor.

tensor={ {{1,2,3,4},{4,5,6,7},{4,5,6,7}},{{1,2,3,9},{1,2,6,5},{1,2,6,5}}}

Dimensions[tensor]

It should be noted that the notion of a tensor as used in physics has additional properties beyond the labeling of elements in generalized matrices. This is discussed more in the description of tensors given in (WebLink[MathWorld, http://mathworld.wolfram.com/Tensor.html]). In Mathematica the term tensor is used to refer to generalized matrices.

Many of the Mathematica functions that operate on matrices are also generalized to work for vectors and tensors. This important principle will be demonstrated many times in this document, and is covered under "Matrix and Tensor Operations".

Matrices as Mathematica Expressions

One important feature of matrices in Mathematica is that they are Mathematica expressions. This means that all the Mathematica commands that operate on Mathematica expressions work on matrices. The same principle applies to vectors and tensors. The fact that all objects in Mathematica have a common expression structure greatly enhances the expressive nature of Mathematica programming. This principle, sometimes called the fundamental principle of Mathematica, may seem abstract, uninteresting, or obvious to people who have a background in purely numerical programming, but users who have a background in object-oriented programming will recognize the benefits of a common parent object. Some examples of this principle will be discussed in this section.

As demonstrated earlier, a matrix can be entered with a list notation.

mat={{1,2,3},{4,5,6}};

One important way to understand Mathematica expressions is through the function (RefLink[FullForm, paclet:ref/FullForm]). This shows the literal details of how the expression is arranged in Mathematica. In this case, the tree structure of the matrix is seen. There is an outer node, which is a (RefLink[List, paclet:ref/List]); in Mathematica this is called having a head of (RefLink[List, paclet:ref/List]). The outer node has two arguments, each of which has the head (RefLink[List, paclet:ref/List]); each of these has three arguments that are integers.

FullForm[mat]

The head of the matrix expression can be inspected with the function (RefLink[Head, paclet:ref/Head]).

Head[mat]

(RefLink[Length, paclet:ref/Length]) returns the number of arguments of the matrix expression.

Length[mat]

(RefLink[Part, paclet:ref/Part]) can be used to extract elements of an expression.

mat[[ 2,3]]

Mathematica operations such as (RefLink[FullForm, paclet:ref/FullForm]), (RefLink[Head, paclet:ref/Head]), (RefLink[Length, paclet:ref/Length]), and (RefLink[Part, paclet:ref/Part]) will work for any Mathematica expression. This is very convenient because it means that a common set of programming constructs will work for programming in many different applications, and not only for linear algebra. The same techniques apply to many other areas in Mathematica, for example, graphics programming, document programming, and symbolic algebra programming. These operations are often called structural operations because they work on the structure of Mathematica expressions. They can be contrasted with other operations that are specific to linear algebra. For example, the function (RefLink[MatrixQ, paclet:ref/MatrixQ]) is specific to matrix computation.

Expression Input and Output

Mathematica contains commands for I/O on expressions. It also has many commands for I/O with specially formatted data. The commands that can work with formatted data are reviewed under "Import and Export of Matrices". In this section the Mathematica commands for I/O on expressions are reviewed.

The concept of the Mathematica (RefLink[FullForm, paclet:ref/FullForm]) is completely equivalent to that of serializable objects in Java or C#. The (RefLink[FullForm, paclet:ref/FullForm]) of any Mathematica expression is a complete specification of the expression. Saving the (RefLink[FullForm, paclet:ref/FullForm]) into a file and reading it back again will re-create the expression.

You can save a Mathematica expression into a file with the command (RefLink[Put, paclet:ref/Put]), entered using >> as a shortcut notation.

mat={{1,2,3},{4,5,6}};
mat>>file.1

The contents of the file can be inspected with (RefLink[FilePrint, paclet:ref/FilePrint]).

FilePrint["file.1"]

The file can be read back into Mathematica with (RefLink[Get, paclet:ref/Get]), entered using << as a shortcut notation.

<<file.1

This restores the matrix from the file so that it can be used for further computation.

Another way to do I/O on matrices in Mathematica is to use the Wolfram Symbolic Transfer Protocol (WSTP). This is an interprocess communication mechanism that is particularly useful for communicating between different Mathematica sessions.

Using (RefLink[Put, paclet:ref/Put]) and (RefLink[Get, paclet:ref/Get]) to store matrices in files is most useful if you want to save work from Mathematica to be restored later. If you want to exchange matrices with other applications, data saved in general formats is more useful; this is described under "Import and Export of Matrices".

Design Principles of Mathematica

The support for linear algebra demonstrates a number of important Mathematica design principles.

These design principles underpin much of the design of linear algebra functions in Mathematica.