De Rerum Natura

In previous posts I talked about precision error matrices as being tensors. Boy, was I wrong! This is another case of my intuition getting way ahead of my math and science. I know just enough math to shot myself in the foot with these speculations. I’ll explain.

Matrices are multi-dimensional arrays of numbers. A two-dimensional matrix $M$ needs two indices $i$ and $j$ to specify a component $M_{\mathrm{ij}}$. A three-dimensional matrix would need three indices and so on. Tensors can be thought of as matrices but the converse is not true. Not all matrices are tensors. That is where I went wrong.

Tensors are multi-dimensional geometrical objects. Yes, they can be represented by matrices but their true hallmark is that they transform correctly under coordinate transformations. The simplest example of the geometrical nature of tensors can be made with a vector. Take a vector drawn on a sheet of paper. No coordinate system has been drawn on the paper. The vector exists independent of any coordinate system. It has a length, for example, and we need no coordinate system to measure it — just a ruler. Two different coordinate systems can be put on the paper that would result in completely different components for the vector. What makes the vector a tensor is that given a coordinate transformation from one system to the next the vector transforms in such a way that both coordinate systems agree on the length of the vector.

This is the geometrical signature of tensors. Different coordinate systems (observers in the parlance of General Relativity) may have different components for the matrices they use to represent a tensor. But they agree on geometrical properties such as the length of a vector or the area of a polygon.

My claim that precision error matrices can be made into tensors may be correct, but I definitely have not proven it until I can show that the tensors I define transform properly under coordinate transformations.

I’ve been nibbling on a bunch of books for the past week. They are, in no particular order:

Mirage: Napoleon’s Scientists and the Unveiling of Egypt deals with the scientific side of Napoleon’s famous imperialistic debacle — the 1798 invasion of Egypt. We tend to think of historical knowledge as continuous in time. If we know something now, everyone in the past must have known it. This book shatters that illusion. Ancient Egypt had been lost to humanity for centuries. The savants in the expedition started the recovery of this lost civilization. One young scientist that participated in the expedition was Joseph Fourier. I have read many biographical sketches of Fourier but I do not recall ever reading that he was part of Napoleon’s Egypt “expedition”. One of the categories in this blog, “Fourier analysis”, is named after him. We can thank Fourier for many things but one that comes immediately to mind is the MP3 music file standard.

The World Without Us has been getting a lot of press. It clearly deserves it. The premise — what would happen to the world if we just disappeared overnight — forms a great hook on which to hang all sorts of scientific observations about biology, the durability of materials, the relentless march of entropy, human evolution, and much more. I highly recommend this book.

Questions in an exam are detectors of student competency. Students are detectors of the correct answers in a test. What is the variation in the student’s model of the correct exam? The precision error equations can be used to construct a covariance matrix for the students instead of the questions. What makes the difference is what is being averaged. When you want to use a test to tell you something about the questions, you average over the students. The covariance matrix is then indexed by the questions. When you want the test to tell you about the students, you average over the questions. The covariance matrix is indexed by students.

All of this suggests that it would be possible to build a completely parameter-less approach to detecting latent groups in students. This would be a different approach from that involved in topic models which use a specific probability distribution — the Dirichlet distribution. In this approach, you would assume a number of groups and arbitrarily assign students to these groups in a probabilistic fashion (60% group 1, 30% group 2, etc.). One can then see how well this group distribution predicts the observed covariance matrix by use of non-commutative harmonic analysis. Group assignment is thereby completely determined by the data — no parameters are needed.

I have started the update of MediaWiki 1.11.2 to incorporate Blahtex functionality. My goal is to have pages that validate as correct XHTML+MathML. The work is being detailed here.

The update is not trivial. The current hacked version of MediaWiki at BerliOS that incorporates blahtex is based on version 1.7. The MediaWiki installation I am playing with is at the latest version: 1.11.2. To figure out how to concentrate on the relevant parts, I am using Unix utilities diff and wc. Basically, I start by counting lines and try to figure out how many new lines are accounted for by new classes or functions added.

I have started the painful process of enabling MathML support on MediaWiki. This is crucial for my use of wiki technology in my workflow. Check out the first step I have taken: enabling texvc (which produces either HTML or .png output).

The next step is to connect it with blahtex so I produce true MathML and correct XHTML headers with MathML. This is not trivial since the blahtex hacked distribution over at BerliOS is at version 1.7 for MediaWiki but the current version is at number 1.11.2. This means that I have to go thru the list of blahtex modified files in the current version.

I have done something similar for MathML on this blog (using itex2MML). It clearly is the case that the math enabled web is still far from being the default. When I have time, and if I succeed in creating a strict XHTML+MathML MediaWiki version, I’ll document my steps somewhat similarly to the steps I took with WordPress .

One possible application of the precision error tensors framework is to use it as a criterion for selecting the number of clusters needed to describe a dataset. The number of clusters problem refers to the generic problem of deciding how many clusters describe a dataset. Many clustering algorithms exist. Deciding which one is appropriate in a particular task is up to the investigator. Suppose that one has settled on a clustering algorithm. An algorithm like k-clusters has no natural stopping criterion. You dial in how many clusters you want, i.e. you manually set the value of $k$, and the algorithm gives you the data clustered into $k$ groups.

Putting aside the correctness of using a particular algorithm for clustering a specific dataset, we can ask: what number of clusters gives me the smallest precision error? This provides an automatic algorithm for deciding on the optimal number of clusters given the chosen algorithm and the dataset to which it is applied.

Author Stewart Mader makes a convincing case in his wikipatterns book that wikis are a powerful collaboration tool. I have dabbled briefly with wikis. I’ve come to rely more and more on wikipedia to understand technical terms quickly. I just don’t practice collaboration with them.

This may change if a grant that we currently have pending with the NSF is approved. We proposed a collaboration with New Mexico scientists at the Jornada Experimental Range to develop a photogrammetric system for UAV images. Collaboration management was a mandated section of the proposal and we included a slew of tools that we currently use — revision control, emails, issue tracking — as well as the MediaWiki software in a list of tools we intend to use to facilitate managing the work related to the collaboration.

The more tools like Wikis become part of our scientific practice, the more I wondered how anybody got things done in the past. How did scientists communicate before emails? I know letters were written. I wrote a few of them back in graduate school. But it seems so strange now to think of writing a letter to someone instead of sending an email.

My previous post on precision error tensors was misleading. We tend to think of tensors as complicated mathematical structures. Vectors are rank 1 tensors. Driving home from work today, I realized that I had already shown that precision error vectors can be calculated in our horizontal decorrelation estimation paper. So mathematically speaking, I have already shown that precision error should be treated as tensors. The precision error vector is the rank 1 tensor example. The precision error covariance matrix is the rank-2 tensor. Two examples in the usual tensor progression. At some future time I should calculate the rank-3 tensor. How would one induce representations of the Symmetric group in rank-3 tensors?

Mathematical objects have dimensions associated with them. The temperature outside my house is measured as a single number or scalar. It is a one-dimensional quantity. This fact can be observed in how mercury thermometers are built: they are a long tube or line. Thermometers are never built as squares.

The position of house in a city is an example of a two dimensional quantity. It requires two numbers to specify and is therefore two-dimensional. This fact is obvious in that maps of cities are usually printed in a sheet of paper not a very thin strip of paper. The position of the house is expressed as a vector. This vector can be expressed as an ordered series of numbers of the form $(v_1,v_2,\dots ,v_n)$. Another way to represent the vector is just with the single symbol $v_i$. You tell me the value of $i$ and I go down the list and read off the component $v_i$.

Generalizing further, we can have matrices like the precision error covariance matrix I have been going on and on about all these months. This matrix can be represented by the symbol $m_{i,j}$. You now have to tell me two numbers, $i$ ad $j$, for me to read off the correct entry in the matrix.

We can keep playing this game forever. It is possible to invent mathematical quantities of the form $m_{i,j,k}$. Three “indices” need to be specified to read off an entry. You can think of this as a cube of numbers.

Precision error covariance matrices can also be generalized to precision error tensors. Instead of just asking how are the errors between two models correlated, we can ask how are the errors of three models correlated. We can have a cube of cross-correlations between the different model errors!