Stretching and Condensing Tensors . Application to tensor and matrix equations

Since tensors are vectors, no matter their order and valencies, they can be represented in matrix form and in particular as column matrices of components with respect to their corresponding bases. This consideration allows using all the available tools for vectors and can be extremely advantageous to deal with some interesting problems in linear algebra, as solving tensor linear equations. Once, the tensors have been operated as vectors, they can be returned to their initial notation as tensors. This technique, that is specially useful for implementing computer programs, is illustrated by several examples of applications. In particular, some interesting tensor and matrix equations are solved. Several numerical examples are used to illustrate the proposed methodology. Mathematics Subject Classification: 15A24, 15A69


Introduction and Motivation
The paper describes a technique that permits to treat tensors as vectors, that is, to extend (stretch) the tensor components, as column matrices that repre-sent their vector components, solve as vectors the desired problems, and recover again the initial format of the tensor (condensation) (see (21)).This technique is described and used to solve a wide set of tensor and matrix equations.
The paper is structured as follows.In Section 2 we introduce some necessary background for the reader to understand the rest of the paper.In particular we deal with basis ordering and matrix representation of tensors.Since the technique used in this paper is based on representing tensors as vectors to take full advantage of vector theory and tools, and we stretch and condense tensors to move from vectors to tensors and vice versa, we present the corresponding extension and condensation operators, which permit moving from one of these representation to the other, and vice versa.As an initial application of these operators, Section 3 deals with some important tensor equations that are solved by the stretching-condensing technique.In Section 4 some interesting applications to matrix equations, including the generalized Sylvester equation, the tensor product commuters equation, the Schur tensor product equation, the square equation and some applications to probability and Statistics, are presented.Finally, Section 5 gives some conclusions.

Background
In this section we include some important material that is required to understand the following sections.

Basis ordering
Consider tensors of the form To work with these tensors and avoid important errors, it is very important that the different basic vectors , . . ., n 1 ; i 2 = 1, 2, . . ., n 2 ; i k = 1, 2, . . ., n k } be "ordered" by means of an agreed upon criteria.In this paper we use the following criterion: The vector e i 1 ⊗ e i 2 ⊗• • •⊗ e i k in this ordering will be ahead of vector e j 1 ⊗ e j 2 ⊗ • • • ⊗ e j k iff at least one of the following condition holds: 3. Being i s = j s for s = 1, 2, . . ., t − 1, it is i t < j t , for any t.
Then, if the indices of the ordered sequence i 1 i 2 • • • i r of basic vectors were considered as digits of a natural number, they would appear as ordered with respect to the standard ordering.This order will play an important role in this paper.

Matrix representation of a tensor
The components of a tensor can be organized in matrix form.However, there are many ways of representing a tensor in matrix form.In this paper we consider only representations such that the order of its components are kept by rows.For example, the third order tensor A ∈ V 2 ⊗ V 3 ⊗ V 4 can be written as When choosing the basis B, matrix A m,n is expressed as a linear manifold spanned by B. If a "matrix form" is adopted to notify the manifold, we get: where all the elements of matrix A mn appear "stacked" in a column matrix X according to the ordering criteria imposed by the given basis "B", and the matrix product must be understood in symbolic form and as products of blocks.
When one desires a given matrix A m,n in this form, the English speaking texts write: "obtained by stacking the elements of the rows of A m,n in sequence."However, we want to note that there is no "need" of expressing this result by "words"of a given language, but using the universal language of Linear Algebra.Given a matrix T m,n of elements of a field K, and calling σ = m • n (not a prime number) the dimension of the linear space T m,n (K σ ) of matrices, we define by "extension" the mapping: that is, the "staked" optic is replaced by "staked and extend the given matrix and write it in column form".
In other words, the extension of a matrix T is a transformation that converts its tensor representation M ({1},{2}) (T) to the column representation is the canonical basis of matrices in IR m×1 , the "stacked" column matrix T σ,1 , associated with matrix T m,n can be obtained by: where we have denoted and • is the standard product of matrices.Then, we have: Extension formula: where m = 3 and n = 4, then .
where Ω is the null matrix.Applying (1) one obtains  Similarly, given a "staked" matrix, T σ,1 we can be interested in its "condensation", that is, recover its original format T m,n as a matrix.
Since we know that σ = m • n, we define as "condensation", the mapping: Condensation formula:

Stretching and condensing Tensors
be the linear space "tensor product" of the linear spaces V n 1 1 , V n 2 2 . . .V n k k , which vectors t are called tensors, and are referred to a basis β of the given space, that is denoted with the ranges: Any tensor in this space can be written as: where all scalars t The stretching-condensation formulas can be extended to tensors.It is interesting to know the existing relation between the position n that occupies a component in the stretched vector representation of a tensor (ordered set of all of its components), and the indices i 1 , i 2 , • • • , i k of the given component in the initial representation of the tensor.
This leads to two problems:

Direct problem :
Assuming known the indices i 1 , i 2 , • • • , i k of a given component and the ranges n 1 , n 2 , • • • , n k of each index, determine the position n (position that occupies such component in the ordered set), that is given by where the first equality shows the most convenient formula for computer implementation and the second equality gives the shortest expression.

Reverse problem :
The reverse problem consists of assuming known the position n (position of a given component) and the ranges r 1 , r 2 , . . ., r k of all indices, determine the indices i 1 , i 2 , • • • , i k of the given component.
The indices i 1 , i 2 , • • • , i k can be obtained by means of the following algorithm: 1. q = n − 1 2. repeat with j from k to 2, by −1: i j = mod[q, r j ] + 1; q = quotient[q, r j ] 3. i 1 = q + 1, where quotient[q, r] and mod[q, r] are the quotient and the rest, respectively, when dividing q by r.
These two procedures define the stretching-condensation transformations for general tensors.
This extended vector can be condensated as a two dimensional matrix by rows, independently of the order of the tensor, to have a matrix representation of the tensor.The following example shows how this matrix representation has an stretching transformation that is independent on the tensor orders and ranges.However, to recover the tensor, we need to know its order and ranges. 4that can be represented in matrix form as: 8 that can be represented in matrix form as:

Example 2.2 (stretching-condensing tensors) Consider the fourth order tensor
, and the second order tensor 8 that can be represented in matrix form as: , then the stretching formula, that is, the transformation giving the column of the tensor components with respect to its ordered basis for the three tensors is the same.In other words, the order of the tensor and the dimensions of the original vector spaces are not needed.However, the condensation formula must keep record of the tensor order and the initial vector spaces dimensions.These are also required for other transformations, as basis changes, contractions and other homomorphisms, etc.

Tensor equations
In this section we deal with tensor equations, i.e., equations whose unknowns are tensors.They will be the basic material for the following section on matrix equations.Consider the following tensors where the asterisks refer to the dual spaces, it is assumed that p < r + q, and for the sake of simplicity, though this condition can be relaxed, the tensors A and B are assumed to be totally contravariant, and the tensor X totally covariant.
Consider the following tensor equation where A and B are given tensors, C is a given totally contravariant tensor of order r−p+q, X is the unknown tensor, and C (S 1 , S 2 ) refers to a contraction of the indices in the two ordered sets of cardinality p, S 2 ≡ {r +1, r +2, . . ., r +p} and S 1 ⊂ S, where S ≡ {1, 2, . . ., r} ∪ {r + p + 1, r + p + 2, . . ., r + p + q}.Note that each of the p indices of tensor X is contracted with one of the indices of either A or B; the first index r + 1 of X is contracted with the first index in S 1 , the second index r + 2 of X is contracted with the second index in S 1 , and so on.Since Equation ( 7) is equivalent to the following standard system of linear equations where are the matrices of extended or stretched forms of tensors X and C, respectively, solving Equation ( 8), the linear tensor equation ( 7) is solved.In summary, the tensor equation ( 7) can be solved using the following algorithm: Algorithm for solving Equation ( 7).
Input: Tensors A, B and C and sets of contracted indices S 1 and S 2 .

Output:
The tensor X solution to Equation (7).
Step 1. Use different index numbers I A , I X ≡ S 2 and I B , for the indices of tensors A, X and B, respectively.
Step 2. Determine the set S = I A ∪ I B .
Step 3. Calculate the tensor A ⊗ B and write it in the matrix form Step 4. Write the tensor C in matrix form E(C) = M (∅,S−S 1 ) (C).
Step 5. Solve the system of linear equations The tensor equation ( 7) can be generalized to where the S 1i , S 2i sets and the tensors A i , B i and C are assumed to be such that Equation ( 9) is well defined.
Then, Equation ( 9) is equivalent to a standard system of linear equations that solves (9).The tensor equation ( 7) can also be generalized to where the S 1i , S 2i sets and the tensors A i , B i and C are assumed to be such that Equation ( 11) is well defined, and we have I unknown tensors.Then, Equation ( 11) is equivalent to a standard system of linear equations that solves (11).
The extension of this procedure to systems of equations of the form ( 11) is obvious.The same is true if one has inequalities instead of equalities.

Example 3.1 (Tensor equation) Consider the following tensor equation
which can be written as where are given totally contravariant tensors and is the unknown totally covariant tensor.
Suppose that the tensors in matrix form are:

Then, the tensor
, so that one can write Equation (13) in its equivalent form (see (8)): , where the coefficient matrix M ({2,7,9,10},{8,1,3}) (A ⊗ B) is the convenient matrix representation of the tensor A ⊗ B and the independent term is E(C).Note that this is a simple linear system of equations that allows us solving the tensor equation ( 14) or (13).
Its solution is which is the column matrix representation of the solution tensor X.

Example 3.2 (Inverse tensors.) Consider the three fourth order tensors
where C ({k, }, {α, β}) refers to the corresponding contraction of indices, tensors A and C are considered data, and tensor B the unknown.
Using the Einstein's notation of repeated indices, equation ( 15) can be written as a which rearranged in matrix form becomes showing that any tensor equation of the form ( 16) is equivalent to a matrix equation of the form (17).
Note that the tensor A can be interpreted as a linear map that transforms vectors in Replacing ( 18) into ( 16) one gets where δ • i r • and δ • j s • are the Kronecker's deltas.If tensor A is represented in matrix form as M ({1,2},{3,4}) (A), then Expression (20) shows that where I is the identity matrix.This implies that using the adequate ordering, pseudo-inverses of tensors can be obtained using matrix tools, and the inverse tensor D of tensor A can be obtained by inverting the matrix M ({1,2},{3,4}) (A) associated with tensor A. Thus, a necessary and sufficient condition for the existence of the inverse tensor is that the matrix M ({1,2},{3,4}) (A) be regular.
Assume that the components of tensors A and C in matrix form are: that is: .
In this section we solve some important matrix equations, i.e., some equations where the unknowns are matrices, using the results of the previous section.

The generalized Sylvester matrix equations
The following matrix equation (see (15)): where the dimensions of matrices A i , X, B i and C are m × n, n × p, p × q and m × q, respectively, can be called the generalized Sylvester equation, because Sylvester (1884) considered only the particular case n = m and p = q.Equation ( 21) can be written as i.e., which shows it is a particular case of (9).Then, according to (10), Equation ( 21) is equivalent to where E(X) and E(C) are the stretched (column) matrices associated with matrices X and C, respectively, which taking into account (6) leads to (25) the system of linear equations which solves the matrix equation (21).
Another generalization of the Sylvester equation is the following matrix equation k i=1 where the dimensions of matrices A i , X i , B i and C are m × n, n × p, p × q and m × q, respectively.Note that there are k unknown matrices.Similarly, using (12), Equation ( 26) can be written in stretched form as the system of linear equations ( 27) which solves the matrix equation (26).

Example 4.1 (Generalized Sylvester equation (Case 1)) Consider the following matrix equation
where Then, Equation (25) becomes (32) which is a compatible system and has a unique solution, that after condensation leads to the solution

Example 4.2 (Generalized Sylvester equation (Case 2)) Consider the following matrix equation
where Then, Equation (25) becomes (38) which is a compatible system and has a unique solution, that after condensation leads to the solution Next, we solve the following particular cases of matrix equations.

Linear equations in a single matrix variable
Consider the matrix equation: in which the unknown is the matrix X n,m .Equation ( 40) is a particular case of Equation ( 9).To solve this equation, we proceed to write it in an equivalent form, using the "tensor product" where

Example 4.3 (Equivalent matrices) Assume that one is interested in obtaining the most general matrix C such that AC = CB, where
127 If such a matrix C exists, then we have which is a matrix equation of the form (40); thus, after stretching the matrix C, we get (see Equation ( 41)): Whence The solution of (44), is the linear subspace: , which after condensing, implies that the most general matrix C that satisfies equation AC = CB is where ρ 1 , ρ 2 and ρ 3 are arbitrary real constants.Its determinant is: and thus, the most general change of basis matrix transforming matrix B into matrix A is that given by ( 45) subject to Similarly, if the equation is the equivalent system of linear equations becomes which solves the problem.

Linear equations in several matrix variables
We start with the case of two unknown matrices and then we generalize to k unknown matrices.Consider the matrix equation: where the unknown matrices are X p,q and Y m,r .
To solve this equation we proceed to write it stretched in an equivalent form, using the "tensor product": to get which is equivalent to Equation (48) but written in the usual form.Then, the solution (X, Y) can be obtained by solving a linear system of equations.Since AX + YB = C is of the form (48), after stretching matrices X and Y, one gets (see Equation ( 50)): , and solving the resulting homogeneous system and condensing X and Y one finally gets: where ρ 1 , ρ 2 , . . ., ρ 7 ∈ IR.
Example 4.5 (One application to probability) Assume that Σ n,n is the covariance matrix of the n-dimensional random variable X, then, the covariance matrix of the n-dimensional random variable If we look for Σ n,n = I n , it must be: To obtain all changes of basis matrices C leading to this result, we initially solve the equation: which is of the form (48) and then it can be written as from which matrices X and Y can be obtained.Next, it suffices to impose the condition Y = (X) t −1 .
For the case with p = m, q = n, that is, matrices A mn and B mn that have the same number of rows and columns, we have P 1 = P t 2 and since they are orthogonal, the matrices B ⊗ A and A ⊗ B are "similar inside the orthogonal group" As a final result of the analysis of the matrices P 1 and P 2 , that appear in Formula (55), we shall propose a second and simple general expression of such matrices, be the canonical basis with n × q matrices, in the IR n×q matrix linear space.
Matrices P 1 and P 2 will be represented by blocks: Special attention must be given to the "block ordering" inside matrices P 1 and P 2 ; it is not the canonical order but the transpose.As an example, an application is given below.Example 4.7 (Commuting the tensor product) Consider again the matrices A 3,3 and B 3,3 in Example 4.6.The matrices P 1 and P 2 that solve our application (m = n = p = q = 3) have now a direct construction: ) that evidently coincides with the P obtained in Example 4.6, that was obtained after using a certainly complicated subindex relationships.

The Schur tensor product equation
Consider the following matrix equation, which allows us replacing a tensor product by a Schur product: where P and Q, the unknowns, never are square matrices.Note that this is a direct relationship among the "three matrix products".Solution matrices Q m,m 2 and P n 2 ,n are, respectively, given by Nevertheless, and following the previous criterion of having a faster formulation for matrices Q m,m 2 and P n 2 ,n in Formula (62) we propose the following block alternative: where B m 2 ≡ {E ij } is the canonical basis of the matrix linear space IR m,m , and where B n 2 = {E ij } is the canonical basis of the matrix linear space IR n,n .where Q = P t or P = Q t and Q = 1 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 1 , as it can be checked easily, using the previous formulas.
As a consequence of (62), a relation between dot and tensor products can be obtained for the particular case p = m, q = n.In fact, we know that and applying the commutative property to the left hand side member and equalling the right hand side members, we get

The square equation
Consider the matrix equation where the matrices P n,n , Q m,m and M n,m are data and the matrix Z n,m is the unknown.
Since this is a particular case of the generalized Sylvester equation, we can solve it by writing it as (66) However, there is a better alternative, that consists of realizing that the linear system (66) can be written as where N = P ⊗ I m + I n ⊗ Q t , that suggests solving Equation (67) in two steps: 1. Solve Equation Ny=m.

Conclusions
In this paper a general method for stretching-condensing tensors has been described.The technique is useful for dealing with tensors using all the already available and powerful tools for vectors.First, the tensors are written as vectors using the stretching transformation that converts the initial structure of tensors components to a column matrix of its vector components with respect to a given basis.Next, the required vector transformations are used.Finally, the vectors are transformed back to their initial tensor representation.In addition, this technique has been shown to be useful for solving important problems in algebra, as solving linear tensor and matrix equations, and to have important applications.
) where c ij are the elements of matrix C and the block representation has been used for illustrating the relation of the new matrix with matrices A and B.

Example 4 . 4 (
Matrix linear equation) Obtain the most general matrices X and Y such that AX + YB = C, where:

⎝ b 11 b 12 b 13 b 21 b 22 b 23 b 31 b 32 b 33 a 11 a 12 a 13 a 21 a 22 a 23 a 31 a 32 a 33 a 11 a 12 a 13 a 21 a 22 a 23 a 31 a 32 a 33 b 11 b 12 b 13 b 21 b 22 b 23 b 31 b 32 b 33
As one can see, the results are identical, and then P = P 1 = P 2 , where P is symmetric, involutive and orthogonal, thus, we get ⎛