While exactly what Yevon did to the Al Bhed was never revealed in Final Fantasy X, this information was eventually revealed in Final Fantasy X-2.5.
Final Fantasy X-2.5 is a novel sequel to Final Fantasy X-2 written by Kazushige Nojima who had written the scenario for Final Fantasy X and Final Fantasy X-2. It was released on December 26th, 2013 to commemorate the release of Final Fantasy X/X-2 HD Remaster.
Yevon essentially blamed the Al Bhed for Sin's appearance, performed a mass execution, followed by outcasting them.
Final Fantasy X-2.5 ~Eien no Daishō~
The origins of the Al Bhed are revealed in the sequel novel to Final Fantasy X-2. There was once a mechanic called Alb who created a race called Bedohls, humans who could not use magic but excelled at wielding machina, and used them to combat a Zanarkandian mage. Their weapons were so powerful they were kept under surveillance at all times. It's speculated that their power might have been the catalyst for the Machina War. After the Yevon religion was established, the Bedohls were blamed for the emergence of Sin and executed in numbers, thereby casting them out of society. The surviving Bedohls were later renamed the Al Bhed, a corrupted mesh of their original name and their creator's name.