I've found a small but annoying bug: the font-size for the page navigation menu at the bottom of the blog (the numbers) seems to be fixed when you hover, but affected by the general font size when you don't. So, with my current settings, it changes size when hovering.
It would be by design if it affected all the .wp-pagenavi, but it only affects on hover. Try this: Set the main font size to 20px, and then hover over the page numbers, you'll see what happens :)