Pythagoras knew how to lay out the counting numbers on an infinite grid. He probably learned this from Babylon. Each direction is a prime number, along with all powers of that prime, as shown in the diagram for 2 and 3. Altogether, there are an infinite number of directions.
To multiply 2 and 3, take the point 6 at the tip of the cone through 2 and 3. Multiplication for any number works this way, although higher dimensional cones may be necessary. To go from 4 to 6, we add 2, which is the number above the +2 arrow. This also works everywhere. So it is natural to label the directions of space by prime numbers.
The branch of mathematics that deals with whole numbers is called, wait for it ... Number Theory. Let us look at three examples of why number theory is essential to the foundations of physics.
1. The appearance of important functions in the basic gadgets of geometry.
2. The extension of whole numbers to other discrete sets with good multiplication properties.
3. The need to start with the logic of the measurement process.