The police and army were deployed to Farafanga prison when dozens of prisoners turned against guards with rocks and a stolen gun.
The police opened fire, killing 20 inmates.
The justice ministry said 19 prisoners were still at large.
Mass prison escapes are not uncommon in Madagascar. In 2016 around 40 detainees broke out from a high-security prison in Toliary in the south of the country.
Rights groups say people are often held for years in prison in Madagascar as they await trial.