The incident occurred late Wednesday in the town of Bindura, around 70 kilometres (43 miles) north of the capital Harare, said Wellington Takavarasha, head of the Zimbabwe Miners Federation.
Takavarasha told AFP the miners were working inside the disused Ran Gold Mine when a shaft caved in.
Six miners have since been pulled from the rubble and taken to hospital.
“Those who were rescued said there were about 40 people in the mine shaft at the time of the incident,” said Takavarasha, adding that rescue efforts were ongoing.
Mining is a major source of foreign currency for Zimbabwe, where gold alone accounts for 60 percent of exports.
The landlocked southern African country is home to vast gold and mineral reserves, including diamonds and platinum.
The gold sector provides jobs to nearly 10 percent of the country’s population, according to the International Crisis Group (ICG).
Small-scale miners often operate illegally to avoid selling their bullion to the state-owned buyer, Fidelity Printers and Refiners, as they are paid only 55 percent in foreign currency.
The remaining 45 percent is paid in Zimbabwean dollars, which is notorious for its weakness.
Accidents are relatively common in decommissioned mines, particularly when the ground is loosened by rain.