Matrix and Tensor Operations

This tutorial reviews the functions that Mathematica provides for building and working with matrices, vectors, and tensors. It focuses on functions that are specific to Mathematica, and uses matrices for many of the examples. However, all the functions are general, and they will also work for vectors and tensors.

Building Matrices

Matrices are represented in Mathematica with lists. They can be entered directly with the {  } notation that Mathematica provides for lists. An example of a matrix is shown here; by default a matrix prints with the list notation.

Mathematica provides a number of ways to build matrices.

The function (RefLink[Table, paclet:ref/Table]) is particularly important to generate matrices, when there is not a built-in function to generate your matrix. The Mathematica documentation contains more information on (RefLink[Table, paclet:ref/Table]). In this example a 3×3 matrix is built:

mat=Table[(Subscript[m, i,j]),{i,3},{j,3}]

(RefLink[MatrixForm, paclet:ref/MatrixForm]) prints the matrix in a two-dimensional form:

MatrixForm[%]

Another way to build matrices in Mathematica is with the (RefLink[Array, paclet:ref/Array]) command. Here it is used to build a 3×3 matrix:

Array[m,{3,3}]

Mathematica has a number of functions for building certain specialized matrices. This builds the identity matrix:

IdentityMatrix[5]

Note that the identity matrix that is generated is an integer matrix. If you want to compute with floating-point numbers it can be advantageous to use matrices that contain floating-point entries; this is described in more detail under "Matrix Contents". If you start with an integer matrix, such as the identity matrix generated previously, it can be converted to a floating-point matrix by using the function (RefLink[N, paclet:ref/N]). This is shown in the following:

N[IdentityMatrix[5]]

Here a diagonal matrix is constructed:

DiagonalMatrix[ {1,2,3,4}]

In this example, the elements are put on the first diagonal. (RefLink[MatrixForm, paclet:ref/MatrixForm]) is useful for showing the results:

DiagonalMatrix[{1,2,3,4},1]//MatrixForm

This makes a 5×5 matrix; each element is 2:

ConstantArray[2,{5,5}]

Here a 2×4 matrix with pseudorandom real numbers in the range 0 to 5 is built:

RandomReal[{0,5},{2,4}]

This makes a 3×4 matrix with pseudorandom integers in the range -5 to 5:

RandomInteger[{-5,5},{3,4}]

This makes a 3×4 matrix with two nonzero values filled in. (RefLink[SparseArray, paclet:ref/SparseArray]) expressions are described under "Working with Sparse Arrays":

Normal[SparseArray[{{2,3}->a,{3,2}->b},{3,4}]]

Constructing special types of matrices with (RefLink[SparseArray, paclet:ref/SparseArray]).

This sets up a general lower-triangular matrix:

SparseArray[{i_, j_}/;i>=j -> f[i, j], {3, 3}] // MatrixForm

Functions that can read in matrices from files are discussed in the section "Import and Export" of matrices.

Special Matrices

Mathematica contains definitions for a number of special matrices.

Special matrices.

Here is a 2×4 Hilbert matrix:

HilbertMatrix[{2, 4}]//MatrixForm

The elements of the Hankel matrix can be given as a list:

HankelMatrix[{w, x, y, z}]//MatrixForm

The Hankel matrix can be filled with nonzero values by giving the final row. This also allows a nonsquare matrix to be generated. Note that the last element of the first column and the first element of the final row must be identical:

HankelMatrix[{w, x, y, z}, {z, m, n}]//MatrixForm

Structural Operations

These operations are all related to the structure of matrices. Many of the techniques shown in this section can be applied to Mathematica expressions and are not specific just to matrices.

Getting Pieces of Matrices

Extracting elements, rows, or columns of a matrix is quite straightforward with the Mathematica function (RefLink[Part, paclet:ref/Part]). Typically (RefLink[Part, paclet:ref/Part]) is entered with [[ ]] notation.

Ways to get pieces of matrices.

Define the following matrix.

This gets the second element in the first row:

mat[[1,2]]

This gets the third row:

mat[[3]]

It can also obtain a column by using (RefLink[All, paclet:ref/All]) to specify all rows:

mat[[All,3]]

Negative indices are used to refer to the end of the matrix. The following gets the last element of the last row:

mat[[-1,-1]]

You can get ranges of a matrix using ;;. This gets the second through the fourth rows:

mat[[2;;4]]

This gets the second through the fourth columns:

mat[[All,2;;4]]

You can also give a step. This gets every other column:

mat[[All,1;;-1;;2]]

The function (RefLink[Tr, paclet:ref/Tr]) works on the diagonal elements of a matrix. The one-argument form adds them up:

Tr[mat]

(RefLink[Tr, paclet:ref/Tr]) can also take a function, as its second argument, to apply to the diagonal elements. If (RefLink[List, paclet:ref/List]) is used this returns the diagonal elements:

Tr[mat,List]

It should be noted that these commands for extracting parts of matrices will work for any Mathematica expression.

Getting Multiple Pieces

It is possible to extract multiple elements by using indices in lists. This is demonstrated with the following sample matrix:

mat=Table[(Subscript[m, i,j]),{i,5},{j,5}];
mat//MatrixForm

The following gets the first and third elements of the second row:

mat[[2,{1,3}]]

The following gets the second and fourth rows:

mat[[{2,4}]]

The following gets the first and third elements of the second and fourth rows:

mat[[{2,4},{1,3}]]

Setting Pieces of Matrices

Setting elements, rows, and columns so that a matrix is updated is quite straightforward using the Mathematica function (RefLink[Part, paclet:ref/Part]) on the left-hand side of an assignment.

Resetting parts of matrices.

Here is a 5×5 matrix:

mat=Table[(Subscript[m, i,j]),{i,5},{j,5}];
mat//MatrixForm

To update parts of a matrix you can use (RefLink[Part, paclet:ref/Part]) on the left-hand side of an assignment.

This sets the third element of the third row:

mat[[3,3]]=x;
mat//MatrixForm

This sets the second row to have a particular value:

mat[[2]]={1,2,3,4,5};
mat//MatrixForm

This sets the second column:

mat[[All,2]]={-1,-2,-3,-4,-5};
mat//MatrixForm

It is also possible to use negative indices to count from the end of the matrix. The following example sets the last element of the last row:

mat[[-1,-1]]=x;
mat//MatrixForm

You can also use the range syntax for setting pieces of the matrix. This sets every element in every other row to be z:

mat[[All,1;;-1;;2]]=z;
mat//MatrixForm

Setting Multiple Pieces

It is possible to set multiple elements by using indices in lists. This is demonstrated with the following sample matrix:

mat=Table[(Subscript[m, i,j]),{i,5},{j,5}];
mat//MatrixForm

The following sets the first and third elements of the second row:

mat[[2,{1,3}]]=x;
mat//MatrixForm

If the right-hand side of the assignment is a list that matches the number of elements being assigned, the assignment is done element by element. Thus, the following gives two different values for the first and third elements of the second row:

mat[[2,{1,3}]]={x,y};
mat//MatrixForm

The following sets the second and fourth rows:

mat[[{2,4}]]={1,2,3,4,5};
mat//MatrixForm

The following gives two different values for the second and fourth rows:

mat[[{2,4}]]={{1,2,3,4,5},{-1,-2,-3,-4,-5}};
mat//MatrixForm

The following sets the first and third elements of the second and fourth rows:

mat=Table[(Subscript[m, i,j]),{i,5},{j,5}];
mat[[{2,4},{1,3}]]=x;
mat//MatrixForm

The following sets the first and third elements of the second and fourth rows with different values:

mat[[{2,4},{1,3}]]={{a,b},{c,d}};
mat//MatrixForm

Extracting Submatrices

The range syntax is useful to extract a submatrix.

Extracting submatrices.

mat=Table[(Subscript[m, i,j]),{i,5},{j,5}];
mat//MatrixForm

This extracts the submatrix from (Subscript[m, 2,1]) to (Subscript[m, 3,2]):

mat[[2;;3,1;;2]]//MatrixForm

This extracts the submatrix of rows 1 to 3:

mat[[1;;3]]//MatrixForm

This extracts the submatrix of columns 2 to 4:

mat[[All,2;;4]]//MatrixForm

You can use negative indices to count from the end.

This returns the matrix with the first and last columns dropped:

mat[[All,2;;-2]]//MatrixForm

Deleting Rows and Columns

If you want to delete rows or columns you can use (RefLink[Drop, paclet:ref/Drop]).

Deleting rows and columns.

mat=Table[(Subscript[m, i,j]),{i,5},{j,5}];
mat//MatrixForm

This drops rows 2 through 4:

Drop[mat,{2,4}]//MatrixForm

This drops columns 2 through 4:

Drop[mat,{},{2,4}]//MatrixForm

In this example rows 2 and 3, and columns 1, 2, and 3 are all dropped:

Drop[mat,{2,3},{1,3}]//MatrixForm

Inserting Rows and Columns

If you want to insert a row, you can use (RefLink[Insert, paclet:ref/Insert]).

Inserting a row.

mat=Table[(Subscript[m, i,j]),{i,5},{j,5}];
mat//MatrixForm

This inserts a row before row 3:

Insert[mat,{x,x,x,x,x},3]//MatrixForm

If you want to insert a column, you must transpose the matrix, insert the column as a row, and then transpose the matrix back. This inserts a column before column 5:

Transpose[Insert[Transpose[mat],{x,x,x,x,x},5]]//MatrixForm

Extending Matrices

You can increase the size of a matrix by padding it with (RefLink[PadLeft, paclet:ref/PadLeft]) and (RefLink[PadRight, paclet:ref/PadRight]).

(RefLink[PadLeft, paclet:ref/PadLeft]) adds the elements to the beginning of the matrix:

PadLeft[mat,{4,4}]//MatrixForm

(RefLink[PadRight, paclet:ref/PadRight]) adds the elements to the end of the matrix:

PadRight[mat,{4,4}]//MatrixForm

One important use of the padding functions is to replicate and tile a matrix. In this example the input matrix is extended to have two versions in every row and three in every column:

%//MatrixForm

(RefLink[PadLeft, paclet:ref/PadLeft]) and (RefLink[PadRight, paclet:ref/PadRight]) have a number of extra features and are completely general so that they work for arbitrary rank tensors. These are described in the Mathematica documentation.

Transpose

Transposition of elements is a general matrix operation:

mat={{1,2,3},{4,5,6}};
mat//MatrixForm

(RefLink[Transpose, paclet:ref/Transpose]) swaps elements at specific indices:

Transpose[mat]

%//MatrixForm

If you wish to compute the conjugate transpose of a matrix, use the function (RefLink[ConjugateTranspose, paclet:ref/ConjugateTranspose]):

mat={{1.,-I},{I,10.}};
matCT=ConjugateTranspose[mat]

If a matrix is equal to its conjugate transpose, it is said to be Hermitian:

mat===matCT

You can also test with the function (RefLink[HermitianMatrixQ, paclet:ref/HermitianMatrixQ]):

HermitianMatrixQ[mat]

Rotating Elements

Another structural operation is to rotate elements within an index. This can be done with the functions (RefLink[RotateLeft, paclet:ref/RotateLeft]) and (RefLink[RotateRight, paclet:ref/RotateRight]):

m={{1,2,3},{4,5,6},{7,8,9}};
m//MatrixForm

This rotates left by one step in the first level, which operates on the rows:

RotateLeft[m,1]//MatrixForm

This rotates left by one step in the second level, which operates on the columns:

RotateLeft[m,{0,1}]//MatrixForm

This rotates the columns in the opposite direction:

RotateRight[m,{0,1}]//MatrixForm

Testing Matrices

Mathematica provides a number of functions for testing matrices and extracting size information.

Functions for testing the structure of vectors, matrices, and arrays.

If you want to test that an expression is a matrix, you can use the predicate (RefLink[MatrixQ, paclet:ref/MatrixQ]):

MatrixQ[{{1,2,3},{4,5,5}}]

The integer 5 is not a matrix:

MatrixQ[ 5]

A matrix has to have the same lengths in every dimension:

MatrixQ[ {{1,2,3},{4,5}}]

(RefLink[MatrixQ, paclet:ref/MatrixQ]) takes an optional second argument that specifies a test to apply to every element. In this example every element is tested to see if it is an integer:

MatrixQ[{{2,2,3},{4,5,6}},IntegerQ]

In this example every element must be an integer greater than 1:

MatrixQ[{{2,2,3},{4,5,6}},IntegerQ[#] && #>1 &]

The command (RefLink[Dimensions, paclet:ref/Dimensions]) is useful for extracting size information:

Dimensions[{{1,2,3},{4,5,6}}]

To compare if the elements of two matrices are equal, it is possible to use (RefLink[Equal, paclet:ref/Equal]), typically entered using a == shortcut notation. For example, comparing a matrix with itself returns (RefLink[True, paclet:ref/True]):

mat1={{1,2},{4,5}};
mat1==mat1

(RefLink[Equal, paclet:ref/Equal]) uses the value of numbers so it can be used to compare integer and real values:

mat1==N[mat1]

It should be noted that (RefLink[Equal, paclet:ref/Equal]) works on any Mathematica expression. If you want to compare two matrices for equality using properties of the matrix as a whole, it may be better to compare matrix norms. These are discussed in a later section.

Further Structural Operations

This section discusses some further structural operations that are useful for working with matrices.

This generates a sample matrix for demonstration:

mat={{1,2,3},{4,5,6},{7,8,9}};

This flattens the matrix into a vector:

vec=Flatten[mat]

Here the vector is partitioned back into a matrix with rows of length 4 using (RefLink[Partition, paclet:ref/Partition]):

Partition[vec,4]

Matrices can be joined together with the function (RefLink[Join, paclet:ref/Join]). This inserts the new matrix as new rows:

Join[ mat,mat]

Alternatively, you can join the new matrix as new columns:

Join[ mat,mat,2]

A new row can be inserted at the end of a matrix with (RefLink[Append, paclet:ref/Append]):

Append[mat,{x,x,x}]

It should be noted that this can also be done with (RefLink[Insert, paclet:ref/Insert]); see the section "Inserting Rows and Columns".

Element-wise Operations

If you want to operate on the elements of a matrix, you can do this easily with Mathematica. First, build a matrix of floating-point numbers:

mat=Table[Random[],{i,3},{j,3}];
mat//MatrixForm

Arithmetic operations applied to a matrix thread down to work on each element. Thus, if 5 is added to the matrix, the result is equivalent to adding 5 to each element:

mat+5

Here, every element of the matrix is doubled:

2 mat

Here, every element of the matrix is squared:

(mat^2)

If one matrix is divided by another, the division is done element by element. If the dimensions of the two matrices do not agree, there is an error:

mat^2/mat

To apply the (RefLink[Sin, paclet:ref/Sin]) function to every element, you apply (RefLink[Sin, paclet:ref/Sin]) to the entire matrix:

Sin[mat]

If both of the arguments of an operation are matrices, the operation is carried out on corresponding elements:

mat +mat

Note that the dimensions of the two matrices have to be compatible:

mat+{{1,2},{2,3}}

If one of the arguments is a matrix and the other is a vector, the operation is carried out between rows of the matrix and elements of the vector:

mat+{x,y,z}

Note that the multiplication of two matrices using the operator (RefLink[Times, paclet:ref/Times]) (typically this is entered by placing two arguments together) produces a matrix with elements that are products of the corresponding elements of the matrices. If you want to carry out matrix multiplication, this can be done with the function (RefLink[Dot, paclet:ref/Dot]) and is described in the section "Matrix Multiplication". An example of element-wise multiplication is shown in the following example:

mat1={{1,2}, {3,4}};
mat2= {{2,2}, {3,3}};
mat1  mat2

Listability

If you want to apply your own function to each element in a matrix, you may do this by giving your function the attribute (RefLink[Listable, paclet:ref/Listable]). This function squares each element and divides the result by 3:

fun[ mat]

Of course, the function will still work with symbolic matrices:

fun[{{1,2},{x,4}}]

Map

Instead of using listability you can use (RefLink[Map, paclet:ref/Map]) to apply a function to every element in a matrix.

This says that the function f should be applied to every element in the matrix:

Map[f,mat,{-1}]

Here, a function that squares its argument and divides the result by 3 is applied to every element in the matrix:

Map[#^2/3 &,mat,{-1}]

Vectors and Tensors

In addition to supporting matrices, Mathematica supports vectors and tensors. All of these are built from lists. As described in "Introduction to Linear Algebra in Mathematica", Mathematica uses the term tensor to refer to generalized matrices. All the operations for building matrices can be generalized to work for vectors and tensors. Mathematica vectors have one level of list.

Here a vector is constructed:

vec={1,2,3}

In addition, there are a number of functions that generate vectors.

Functions for generating vectors.

One very efficient way to generate a vector is to use the Mathematica function (RefLink[Range, paclet:ref/Range]). This generates a vector of integers:

Range[10]

This generates a vector of reals, starting with 1., incrementing in 0.1, and ending at 4:

Range[1.,4,0.1]

(RefLink[Table, paclet:ref/Table]) can also be used to programmatically construct a vector; this can be useful if you want to execute a function to determine the elements of the vector:

vec=Table[Sqrt[i/2.],{i,5}]

Though it is often more efficient to use listable operations:

vec=Sqrt[Range[5]/2.]

The operations and functions discussed in previous sections have equivalent versions for vectors:

vec +4

vec[[2]]

vec[[4]]=x;
vec

It should be noted that Mathematica has no concept of a row or a column vector; a vector has a single index that can be used to reference an element of the vector.

Tensors can also be built using the (RefLink[Table, paclet:ref/Table]) command. Here, three iterators are used to generate a 2×3×4 tensor:

tensor=Table[1/(i j k),{i,2},{j,3},{k,4}]

You can extract an element by using (RefLink[Part, paclet:ref/Part]) with three indices:

tensor[[1,2,1]]

Testing Vectors and Tensors

Mathematica provides a number of functions for testing vectors and tensors and extracting size information.

Functions for testing the structure of vectors, matrices, and arrays.

If you want to test that an expression is a matrix you can use the predicate (RefLink[MatrixQ, paclet:ref/MatrixQ]):

MatrixQ[{{1,2,3},{4,5,5}}]

In addition to (RefLink[MatrixQ, paclet:ref/MatrixQ]), (RefLink[VectorQ, paclet:ref/VectorQ]) and (RefLink[ArrayQ, paclet:ref/ArrayQ]) are useful for testing vectors and tensors:

VectorQ[{1,2,3}]

ArrayQ[ {{{1,2},{3,4}},{{4,5},{6,7}}}]

(RefLink[ArrayQ, paclet:ref/ArrayQ]) can also take a rank argument to test the depth of the array:

ArrayQ[ {{{1,2},{3,4}},{{4,5},{6,7}}}, 3]

(RefLink[ArrayQ, paclet:ref/ArrayQ]) also takes a third argument that tests each element. In this example the result is (RefLink[False, paclet:ref/False]) because not all the elements are (RefLink[NumberQ, paclet:ref/NumberQ]):

ArrayQ[ {{{x,2},{3,4}},{{4,5},{6,7}}}, 3,NumberQ]

The command (RefLink[Dimensions, paclet:ref/Dimensions]) is useful for extracting size information:

Dimensions[{{{1,2},{3,4}},{{4,5},{6,7}}}]

(RefLink[Dimensions, paclet:ref/Dimensions]) returns a list of length 2 when the input is a matrix, stating that two indices are used to reference any element in the matrix. Another way to test the number of indices required to reference elements is with (RefLink[ArrayDepth, paclet:ref/ArrayDepth]). In this example the result is 2:

ArrayDepth[ {{1,2,3},{4,5,6}}]

To compare if the elements of two tensors are equal it is possible to use (RefLink[Equal, paclet:ref/Equal]), typically entered using a == shortcut notation. For example, comparing a tensor with itself returns (RefLink[True, paclet:ref/True]):

ten={{{1,2},{3,4}},{{4,5},{6,7}}};
ten==ten

(RefLink[Equal, paclet:ref/Equal]) uses the value of numbers, so it can be used to compare integer and real values:

ten==N[ten]

If the structure of two objects does not match they are not equal:

ten=={1,2,3}

Visualization of Matrices

This section reviews the functions that are available for formatting and plotting matrices.

Formatting Matrices

Matrices can be formatted with the function (RefLink[MatrixForm, paclet:ref/MatrixForm]):

mat={{a,b},{c,d}};
MatrixForm[mat]

(RefLink[MatrixForm, paclet:ref/MatrixForm]) also works for vectors and higher-rank tensors; the braces can help you to understand the grouping:

MatrixForm[ Array[ m, {3,2,2}]]

Plotting Matrices

A convenient way to plot matrices is with the function (RefLink[MatrixPlot, paclet:ref/MatrixPlot]):

mat=RandomInteger[{0,1},{10,10}];
mat//MatrixForm

MatrixPlot[mat]

(RefLink[MatrixPlot, paclet:ref/MatrixPlot]) has a number of graphics options for controlling the appearance of the plot. Many of these are the typical options of Mathematica (RefLink[DensityGraphics, paclet:ref/DensityGraphics]) objects. It also has a special option, (RefLink[MaxPlotPoints, paclet:ref/MaxPlotPoints]), that controls the maximum size for the display of a matrix. Above this size the matrix is downsampled. Some of the important options are summarized in the following.

Options of (RefLink[MatrixPlot, paclet:ref/MatrixPlot]).

One useful feature of (RefLink[MatrixPlot, paclet:ref/MatrixPlot]) is that it can plot very large matrices. Here a very large sparse matrix is generated:

n=10000;
nz=2000;
sp=SparseArray[
	RandomInteger[{1,n},{nz,2}]->Table[1,{nz}],{n,n}];
sp=sp+Transpose[sp]

You can still plot this out with (RefLink[MatrixPlot, paclet:ref/MatrixPlot]). Here the number of points that are plotted is reduced to 100:

MatrixPlot[sp,MaxPlotPoints->100]

By default, no mesh is drawn around the elements; this can be enabled with the (RefLink[Mesh, paclet:ref/Mesh]) option. Typically you would not want to draw the mesh if the matrix is very large. When the mesh is drawn, it is sometimes useful to change the style of the mesh with the (RefLink[MeshStyle, paclet:ref/MeshStyle]) option:

MatrixPlot[RandomInteger[{0,1},{50,50}], Mesh->True,MeshStyle->White]

You can change the coloring with the (RefLink[ColorRules, paclet:ref/ColorRules]) option. This example plots zero as white and everything else black:

MatrixPlot[RandomInteger[{0,1},{50,50}], ColorRules->{0->White,0.->White,_->Black}]

You can also use the (RefLink[ColorFunction, paclet:ref/ColorFunction]) option to add color:

MatrixPlot[RandomInteger[{0,1},{50,50}], ColorFunction->(RGBColor[#,0,0]&)]

Import and Export of Matrices

This section reviews the functions that are available for importing and exporting matrices.

Mathematica provides a number of different tools for input/output. If you want to save your data in a file so that later you or a colleague can continue to work with it in Mathematica, you might want to use some of the functions that work with Mathematica expressions in files. These are discussed under "Expression Input and Output".

If you want to work with matrices from a source external to Mathematica using specific data formats, the functions (RefLink[Import, paclet:ref/Import]) and (RefLink[Export, paclet:ref/Export]) are useful. The (RefLink[Import, paclet:ref/Import]) function supports a variety of different formats, some of which are relevant to matrices:

$ImportFormats

The (RefLink[Table, paclet:ref/Table]) format will read tabular data into Mathematica. This format is not specific to matrices and can process different types of information such as numbers, dates, and currencies. However, this is often a simple way to read in a matrix. This example displays a sample data file:

FilePrint["LinearAlgebraExamples/Data/simplematrix.dat"]

Now the data file is read into Mathematica with the (RefLink[Import, paclet:ref/Import]) command using a format of (RefLink[Table, paclet:ref/Table]); the result is a matrix:

Import["LinearAlgebraExamples/Data/simplematrix.dat","Table"]

(RefLink[Export, paclet:ref/Export]) can be used to write out a matrix in some particular format. Here it uses the (RefLink[Table, paclet:ref/Table]) format:

Export["fileout.dat",{{1,2},{3,4}},"Table"]

FilePrint["fileout.dat"]

This example shows how to write a matrix into a CSV format. This could be read into another application such as a spreadsheet:

Export["filecsvout.dat",{{1,2},{3,4}},"CSV"]

FilePrint["filecsvout.dat"]

There are other matrix formats. For example, Harwell–Boeing, used for sparse matrices, and Matrix Market, used for both sparse and dense matrices. These are discussed under "Import and Export of Sparse Matrices". In addition the MAT matrix format and the FITS astronomical data format can also be useful as import or export formats.

Matrix Multiplication

Matrix multiplication (also called dot or inner product) is carried out in Mathematica with the function (RefLink[Dot, paclet:ref/Dot]), typically entered with a dot shorthand syntax:

mat={{1,2},{3,4}};

This demonstrates matrix multiplication of a matrix with itself:

mat.mat

This multiplies a matrix with a vector:

mat.{4,5}

The matrix product can be computed with two matrices of different sizes so long as they are compatible. For matrices this means that to multiply a (Subscript[m, 1])×(Subscript[n, 1]) matrix by a (Subscript[m, 2])×(Subscript[n, 2]) matrix, it is required that (Subscript[n, 1]) is equal to (Subscript[m, 2]). Here, a 2×3 matrix is multiplied by a 3×2 matrix.

This will result in a 2×2 matrix:

mat1.mat2

This generates a 3×3 matrix:

mat2.mat1

If the dimensions do not match, an error is generated:

mat1.{1,2}

(RefLink[Dot, paclet:ref/Dot]) can be used to multiply vectors of equal length; the result will be a scalar:

vec={1,2,3};
vec.vec

Multiplication of a matrix by a vector works equivalently. This multiples a 2×3 matrix by a length 3 vector:

mat1.vec

This multiplies the length 3 vector by a 3×2 matrix:

vec.mat2

The definition of matrix multiplication is such that the product C of two matrices A and B, where C==A.B, is given as follows.

(Subscript[c, i j])==(∑_k)(Subscript[a, i k])(Subscript[b, k j])

The definition generalizes, so that the product S of two arbitrary rank tensors T and U is as follows.

(Subscript[s, (Subscript[i , 1])(Subscript[i, 2])… (Subscript[i, m-1])(Subscript[j, 2])… (Subscript[j, m])])==(∑_k)(Subscript[t, (Subscript[i, 1])(Subscript[i, 2])… (Subscript[i, m-1]) k])(Subscript[u, k (Subscript[j, 2])… (Subscript[j, m])])

Thus applying (RefLink[Dot, paclet:ref/Dot]) to a rank m tensor and a rank n tensor results in a rank n+m-2 tensor. An example is shown next.

First, a 2×3×4 tensor is defined:

tensor1={ {{1,2,3,4},{4,5,6,7},{4,5,6,7}},{{1,2,3,9},{1,2,6,5},{1,2,6,5}}};
tensor1//MatrixForm

Now, a 4×2×1 tensor is defined:

tensor2={ {{1},{4}},{{7},{1}},{{8},{3}},{{9},{1}}};
tensor2//MatrixForm

This multiplies tensor1 by tensor2. They are compatible because the length of the innermost index of tensor1 equals the length of the outermost index of tensor2:

tensor1.tensor2//MatrixForm

The result is a 2×3×2×1 tensor:

Dimensions[%]

Outer Product

The outer product is a way to build a higher-rank tensor from those of lower rank. Mathematica provides this functionality with the function (RefLink[Outer, paclet:ref/Outer]). One use of this is to combine two vectors to form a matrix as an outer product:

vec1={a,b,c};
vec2={x,y,z};
Outer[Times,vec1,vec2]//MatrixForm

The function that is used to combine corresponding elements is given as the first argument. It can be an unknown function as in the following example:

Outer[f,{a,b,c},{ap,bp}]//MatrixForm

Visualization of the Outer Product

One way to visualize the operation of (RefLink[Outer, paclet:ref/Outer]) is demonstrated in this example. First, a list of points is created:

pts=Table[{Sin[t],Cos[t]},{t,0,2Pi,Pi/8.}];

This shows how (RefLink[Outer, paclet:ref/Outer]) joins each point to each other point:

Show[Graphics[Outer[Line[ {##}]&,pts,pts,1]],AspectRatio->Automatic]

Generalized Inner Product

Matrix multiplication is a fundamental operation of linear algebra computation. Consequently, Mathematica provides (RefLink[Dot, paclet:ref/Dot]) as a dedicated function, which is heavily optimized. However, a generalization of matrix multiplication is provided by (RefLink[Inner, paclet:ref/Inner]). This allows the two operations that are used to form the product to be specified.

Here are two vectors:

vec1={a,b,c};
vec2={x,y,z};

This is the scalar product:

vec1.vec2

This is the equivalent operation using (RefLink[Inner, paclet:ref/Inner]):

Inner[Times,vec1,vec2,Plus]

Now (RefLink[Power, paclet:ref/Power]) is used instead of (RefLink[Times, paclet:ref/Times]):

Inner[Power,vec1,vec2,Plus]

Matrix Permutations

Many matrix techniques rely on ordering a matrix in particular ways. For example, some techniques try to order the matrix to put elements on the diagonal, while others try to group certain elements into dense blocks. The Mathematica function (RefLink[Part, paclet:ref/Part]) is very well suited to applying permutations to the rows and columns of a matrix.

Applying permutations to matrices.

This generates a random matrix:

mat=Table[Random[Integer,{0,3}],{4},{4}];
mat//MatrixForm

Now the matrix will be reordered so that the rows are ordered by increasing size of 2-norm. (Norms are discussed under "Matrix Computations: Norms".)

First, the norm of each row is computed:

Map[ Norm, mat]

This computes the permutation that puts the smallest numbers first:

perm=Ordering[%,All,Less]

This applies the ordering to the rows of the matrix; the result has rows ordered by increasing size of 2-norm:

pmat=mat[[perm]];
pmat//MatrixForm

Now the inverse permutation is applied by using part assignment. Note that this modifies the matrix held by the symbol pmat. Part assignment is described previously:

pmat[[perm]]=pmat;
pmat//MatrixForm

If the inverse permutation is applied to the permuted matrix, the original matrix is restored:

pmat==mat

Permutation Matrices

Another way to work with permutations is to compute a matrix that applies the permutation by multiplication. For example, this builds a 4×4 matrix:

mat=RandomInteger[{0,3},{4,4}];
mat//MatrixForm

This computes the permutation necessary to order the rows according to ascending 2-norm:

perm=Ordering[Map[Norm,mat],All,Less]

The permutation matrix is typically a sparse matrix, which is discussed under "Working with Sparse Arrays". This input generates a sparse identity matrix:

sp=SparseArray[{i_,i_}->1,{4,4}]

If the permutation is applied to the identity matrix, a permutation matrix is generated:

permMat=sp[[perm]]

This applies the permutation. Note that the rows with smallest norms are at the top:

pmat=permMat.mat;
pmat//MatrixForm

This applies the inverse permutation:

Transpose[permMat].pmat//MatrixForm

Typically it is faster to use the Mathematica function (RefLink[Part, paclet:ref/Part]) to apply a permutation, but sometimes it is convenient to work with a permutation matrix.