The donkeys, bought in neighbouring Tanzania, were given to residents of a village in Gitega province as part of a project by a local NGO to help women and children transport agricultural products, water or wood.
However, a presidential adviser described the project as “an insult to the nation”.
Gabby Bugaga, spokesman for the Senate president, also wrote on Twitter the French were “taking us for donkeys”.
“Be honest, is the donkey a symbol of a quality or a flaw,” he wrote.
Donkeys are not indigenous to Burundi.
On Sunday, Agriculture Minister Deo Guide Rurema asked a local administrator to “facilitate the immediate withdrawal of all donkeys that have been distributed … without respecting the technical procedure of the distribution of exotic animals”.
Last Thursday, the day the project was inaugurated, French Ambassador Laurent Delahousse praised “the introduction of the Land Cruiser of the animal kingdom to Burundi”.
After the controversy, Delahousse said to his knowledge “all procedures were respected”.
A European diplomat, speaking on condition of anonymity, told AFP news agency Burundi was hitting back at France over a statement it made criticising a referendum earlier this month. The vote reformed the constitution allowing President Pierre Nkurunziza to seek another two terms in office so he can potentially remain in power until 2034.
The diplomat said a similar project involving donkeys in Ruyigi province, financed by Belgium, had not been met with any problems.