De Rerum Natura

In previous posts I talked about precision error matrices as being tensors. Boy, was I wrong! This is another case of my intuition getting way ahead of my math and science. I know just enough math to shot myself in the foot with these speculations. I’ll explain.

Matrices are multi-dimensional arrays of numbers. A two-dimensional matrix $M$ needs two indices $i$ and $j$ to specify a component $M_{\mathrm{ij}}$. A three-dimensional matrix would need three indices and so on. Tensors can be thought of as matrices but the converse is not true. Not all matrices are tensors. That is where I went wrong.

Tensors are multi-dimensional geometrical objects. Yes, they can be represented by matrices but their true hallmark is that they transform correctly under coordinate transformations. The simplest example of the geometrical nature of tensors can be made with a vector. Take a vector drawn on a sheet of paper. No coordinate system has been drawn on the paper. The vector exists independent of any coordinate system. It has a length, for example, and we need no coordinate system to measure it — just a ruler. Two different coordinate systems can be put on the paper that would result in completely different components for the vector. What makes the vector a tensor is that given a coordinate transformation from one system to the next the vector transforms in such a way that both coordinate systems agree on the length of the vector.

This is the geometrical signature of tensors. Different coordinate systems (observers in the parlance of General Relativity) may have different components for the matrices they use to represent a tensor. But they agree on geometrical properties such as the length of a vector or the area of a polygon.

My claim that precision error matrices can be made into tensors may be correct, but I definitely have not proven it until I can show that the tensors I define transform properly under coordinate transformations.

My previous post on precision error tensors was misleading. We tend to think of tensors as complicated mathematical structures. Vectors are rank 1 tensors. Driving home from work today, I realized that I had already shown that precision error vectors can be calculated in our horizontal decorrelation estimation paper. So mathematically speaking, I have already shown that precision error should be treated as tensors. The precision error vector is the rank 1 tensor example. The precision error covariance matrix is the rank-2 tensor. Two examples in the usual tensor progression. At some future time I should calculate the rank-3 tensor. How would one induce representations of the Symmetric group in rank-3 tensors?

Mathematical objects have dimensions associated with them. The temperature outside my house is measured as a single number or scalar. It is a one-dimensional quantity. This fact can be observed in how mercury thermometers are built: they are a long tube or line. Thermometers are never built as squares.

The position of house in a city is an example of a two dimensional quantity. It requires two numbers to specify and is therefore two-dimensional. This fact is obvious in that maps of cities are usually printed in a sheet of paper not a very thin strip of paper. The position of the house is expressed as a vector. This vector can be expressed as an ordered series of numbers of the form $(v_1,v_2,\dots ,v_n)$. Another way to represent the vector is just with the single symbol $v_i$. You tell me the value of $i$ and I go down the list and read off the component $v_i$.

Generalizing further, we can have matrices like the precision error covariance matrix I have been going on and on about all these months. This matrix can be represented by the symbol $m_{i,j}$. You now have to tell me two numbers, $i$ ad $j$, for me to read off the correct entry in the matrix.

We can keep playing this game forever. It is possible to invent mathematical quantities of the form $m_{i,j,k}$. Three “indices” need to be specified to read off an entry. You can think of this as a cube of numbers.

Precision error covariance matrices can also be generalized to precision error tensors. Instead of just asking how are the errors between two models correlated, we can ask how are the errors of three models correlated. We can have a cube of cross-correlations between the different model errors!