According to the Naruto Wiki page on Jiraiya, he
"left Konoha so he could follow Orochimaru's movements, as well as those of "Akatsuki", an organization which Orochimaru had eventually joined."
Orochimaru was at Konoha during the Chuunin Exam Arc, making his first appearance (as far as I remember) during the second phase (in the Forest of Death).
Jiraiya made his first appearance between the preliminaries and the finals (in chapter 90).
There is a fairly short period of time between these two appearances.
We quickly see that Jiraiya knows who Naruto is, as he is trying to control his chakra to walk on water (in chapter 91). He first notices something when he sees Naruto trying to walk on water:
We then have a full confirmation that he knows who he is:
In this chapter he also notices that there is some weird overlaying of seals, which he deduces was Orochimaru's work.
Now, my question is:
Did Naruto and Jiraiya meet by chance, as Jiraiya was tracking down Orochimaru's movements? (in which case Jiraiya only knew who Naruto was after seeing the seal in his stomach)
Or did he know who Naruto was before having seen him trying to walk on water, and tracked him down? (possibly to protect him from Orochimaru, and in which case he would've been watching over Naruto since his parents died)