It was Shirou.
Shirou never was in track and field, so he answers Rin's question honestly. Rin is making the assumption that only someone that does track and field would spend an entire afternoon attempting to do a high jump. But that is exactly what Shirou did, anyway. He saw the high jump left out, and decided to see for himself what he could do. He constantly failed but stubbornly kept at it for hours. At long last he triumphantly gave up, satisfied that he could, in fact, not make that jump.
Rin was also not the only person to witness this and fall in love with him over it.
Sakura is the other witness.