That's because the pixels per inch (PPI) is different between them. The higher the PPI, the higher the quality but at a cost of (relative) smaller screen size. So the Nexus 7 has 216 pixels per inch while the Galaxy Nexus has 316 pixels per inch. The resolution isn't everything, you also have to consider the density of the pixels.
High PPI values have the advantage that it's HD but it also has a disadvantage which is a smaller screen size (at a persistent resolution).
You can read more about PPI here.
