Let form the basis of a two-dimensional lattice. Our first goal is to replace this basis with what will be called a reduced basis.
If , then swap and , so we may assume that . Ideally, we would like to replace with a vector perpendicular to . As in the Gram-Schmidt process from linear algebra, the vector
is perpendicular to . But this vector might not lie in the lattice. Instead, let be the closest integer to (for definiteness, take to be the closest integer to , and to be closest to , etc.). Then we replace the basis with the basis
We then repeat the process with this new basis.
We say that the basis is reduced if
The above reduction process stops exactly when we obtain a reduced basis, since this means that .
In the figures, the first basis is reduced because is longer than and the projection of onto is less than half the length of . The second basis is nonreduced because the projection of onto is too long. It is easy to see that a basis is reduced when is at least as long as and lies within the dotted lines of the figures.
Let’s start with and . We have , so we do not swap the two vectors. Since
we take . The new basis is
Swap and and rename the vectors to obtain a basis
We have
so we take . This yields vectors
Swap these and name them and . We have
so . This yields, after a swap,
Since and
the basis is reduced.
A natural question is whether this process always produces a reduced basis. The answer is yes, as we prove in the following theorem. Moreover, the first vector in the reduced basis is a shortest vector for the lattice.
We summarize the discussion in the following.
Let be a basis for a two-dimensional lattice in . Perform the following algorithm:
If , swap and so that .
Let be the closest integer to .
If , stop. If , replace with and return to step 1.
The algorithm stops in finite time and yields a reduced basis of the lattice. The vector is a shortest nonzero vector for the lattice.
Proof
First we prove that the algorithm eventually stops. As in Equation 23.1, let and let . Then
Since and are orthogonal, the Pythagorean theorem yields
Also, since , and again since and are orthogonal,
Note that if then and . Otherwise, . Therefore, if , we have , which implies that
Therefore, if the process continues forever without yielding a reduced basis, then the lengths of the vectors decrease indefinitely. However, there are only finitely many vectors in the lattice that are shorter than the original basis vector . Therefore, the lengths cannot decrease forever, and a reduced basis must be found eventually.
To prove that the vector in a reduced basis is a shortest nonzero vector for the lattice, let be any nonzero vector in the lattice, where and are integers. Then
Because is reduced,
which implies that . Therefore,
since by assumption. Therefore,
But is an integer. Writing it as , we see that it is nonnegative, and it equals 0 if and only if . Since , we must have . Therefore,
so is a shortest nonzero vector.
Lattice reduction in dimensions higher than two is much more difficult. One of the most successful algorithms was invented by A. Lenstra, H. Lenstra, and L. Lovász and is called the LLL algorithm. In many problems, a short vector is needed, and it is not necessary that the vector be the shortest. The LLL algorithm takes this approach and looks for short vectors that are almost as short as possible. This modified approach makes the algorithm run very quickly (in what is known as polynomial time). The algorithm performs calculations similar to those in the two-dimensional case, but the steps are more technical, so we omit details, which can be found in [Cohen], for example. The result is the following.
Let be the -dimensional lattice generated by in . Define the determinant of the lattice to be
(This can be shown to be independent of the choice of basis. It is the volume of the parallelepiped spanned by .) Let be the length of a shortest nonzero vector in . The LLL algorithm produces a basis of satisfying
.
Statement (2) says that is close to being a shortest vector, at least when the dimension is small. Statement (3) says that the new basis vectors are in some sense close to being orthogonal. More precisely, if the vectors are orthogonal, then the volume equals the product . The fact that this product is no more than times says that the vectors are mostly close to orthogonal.
The running time of the LLL algorithm is less than a constant times , where is the dimension and is a bound on the lengths of the original basis vectors. In practice it is much faster than this bound. This estimate shows that the running time is quite good with respect to the size of the vectors, but potentially not efficient when the dimension gets large.
Let’s consider the lattice generated by (31, 59) and (37, 70), which we considered earlier when looking at the two-dimensional algorithm. The LLL algorithm yields the same result, namely and . We have and (given by , for example). The statements of the theorem are
.