As you try to make sense of Occam's Razor you find yourself bumping up against a number of related issues in the foundations of science and philosophy. Here we address some of these issues. The questions involved are big enough that our answers can hardly be considered conclusive, but they should at least help illuminate the areas of dispute.
Models vs Classifiers
As discussed in the interpretations section, one interpretation which we call "Occam's Razor proper" states that if two models make equivalent predictions, the simpler is to be preferred. This principle tells us how to choose between different models which correspond to the same classifier. It may seem counterintuitive that such models can exist. What do we mean by these words model and classifier?
In our usage, a model is a set of assumptions (aka axioms, premises) we make to explain our observations. A classifier (or decision rule) can be defined as a function which takes an input point from the feature space and assigns it to one of a number of classes (or categories).
It is important to see that the same classifier can result from different models. Consider an example. Suppose our domain is the integers from 1 to 10, and we are told that 1, 2, 3, 5, and 7 belong to class A and 4, 6, 8, 9, and 10 belong to class B. How could we explain these observations? One model might take as its axiom that numbers whose digits contain fully closed areas (as 4 has a triangle, 8 has two circles, etc) belong to class A, and numbers with no closed areas belong to class B. Another model could assume that prime numbers (including 1) belong to class A and composite numbers belong to class B. These two models make quite different assumptions, but the classifiers they produce (over the domain 1-10) are identical. A more practical example of different models producing (essentially) the same predictions is the case of Kepler's laws of planetary motion and Newton's universal law of gravitation, discussed on the interpretations page.
Models in Science
The idea of models is fundamental to science. The scientific method amounts to 1) devising models to explain observations, and then 2 )subjecting them to experimental verification. For the first, creative step, the scientist is trying to find a set of hypotheses or axioms which imply the observed phenomena. For instance, Einstein's theory of special relativity proceeded from the axioms that 1) the speed of light in a vacuum is constant and 2) the laws of physics hold equivalently regardless of which inertial frame of reference they are viewed from. These two axioms were largely sufficient to explain the strange behavior that had been observed in particles moving near the speed of light.
Epistemology vs Metaphysics
In the interpretations section, we make use of a distinction between epistemological and metaphysical interpretations of Occam's Razor. This is a reference to two of the main traditional areas of philosophical inquiry. Epistemological questions concern knowledge, how it is arrived at and justified, whereas metaphysics (as we use the term) refers to questions about reality and its fundamental properties. Clearly these are both broad areas of inquiry with substantial crossover: it is hard to make any claims about what we know without making claims about reality, and conversely we cannot say what we know about reality without making claims about how we know it.
Epistemological interpretations of Occam's Razor see it as telling us how we can conclude what to believe. Thus, what we refer to as Occam's Razor proper states that we should believe the simplest model which explains the observations. This is not a claim about reality, but about what we should believe; namely, as little as possible (but, as Einstein said, no less). Metaphysical interpretations see Occam's Razor as a general observation about reality: classification problems in the real world tend to have simple solutions. We might or might not be able to imagine worlds in which this property did not hold, but what interests the metaphysicist is that in our world it does.