In an interview with Pop Sugar, the gold winner and first US-Hmong Olympian said she and a group of Asian friends were targeted while waiting for a ride.
According to her, the incident took place a few weeks ago as she was in LA to film Dancing with the Stars.
Attacks on people of Asian descent have risen amid the Covid-19 pandemic, which was first reported in China.
Lee, 18, made history by travelling to Tokyo this summer, becoming the first US Olympic athlete from the Hmong ethnic group.
The gymnast said she and her friends, all of the Asian descent, were waiting for their Uber after a night out when a car passed with people inside shouting ethnic slurs and telling them to “go back to where they came from”.
One person sprayed her in the arm with pepper spray before the vehicle sped away, she said.
“I was so mad, but there was nothing I could do or control because they skirted off,” Lee said.
“I didn’t do anything to them, and having the reputation, it’s so hard because I didn’t want to do anything that could get me into trouble.
“I just let it happen.”
Lee – whose parents immigrated to the US from Laos – comes from St Paul, Minnesota.
It comes amid a dramatic rise in attacks on Asian Americans.
In April, US karate Olympian Sakura Kokumai, who is Japanese-American, was accosted while training in a park in California.
The man, who allegedly shouted racist slurs at her, was arrested for making threats.