Just like in our world, Westerners might have problems telling Asians apart, the same can be said about humans and elves.
Humans live and grow up looking at other humans. So they learn to tell them apart and categorize them much more easily. Now, this assumes that you can tell elves age from their appearance, it is just that humans don't learn that skill, because they either haven't seen an elf or haven't seen enough to create big enough sample to learn differences due to age.
Rory is over 900 years old and travels a lot. So she must have met quite a few elves in her lifetime. Also, she could have met the same elf multiple times in their lives, noting their changes in appearance.
The second thing that comes to mind is precision of her guess. Us humans are able to tell age with precision of +-5 years. It is safe to assume that elves appearance changes much slowly with age compared to humans, so the precision might be as low as +-50 years. So if Rory estimates Yao's age at 300, she could very well be between 250 to 350 years old. That is huge timespan. It is like saying an adult 40 year old man "is around age of toddler".