Mr Lemma was once a close ally of Prime Minister Abiy Ahmed and a key figure in the country’s political and economic reform over the past few years.
Prior to being appointed as a defence minister he was the head of the country’s most-populous state, Oromia.
But last November, Mr Lemma publicly criticised the prime minister’s efforts to reorganise and rebrand the Ethiopian People’s Revolutionary Democratic Front (EPRDF) – a coalition of four ethnic based parties that has been in power since the early 1990s.
The EPRDF was renamed as the Prosperity Party.
He also cast doubts on Mr Abiy’s philosophy of Medemer (coming together).
The announcement of his suspension comes after the Oromia wing of the ruling party held a two-day summit in the wake of ethnic and religious violence that claimed the lives of more than 170 people.
A spokesperson for the party accused Mr Lemma of disregarding the party and his duties.
It is not yet clear if Mr Lemma will continue holding the position of defence minister.
Teyba Hassen, who was Mr Lemma’s deputy when he was head of Oromia state, is also among those suspended for reasons that are related to the recent violence, the spokesperson said.