Most of them are small Pentecostal churches. One mosque was also closed.
A government official told the BBC that some of the more than 700 buildings shut down have already reopened after they were approved by inspectors.
According to a proposed new law, all preachers must have theological training before opening a church.
Some are massive, attracting thousands of worshippers each Sunday, but others consist of tiny structures built without planning permission.
Church leaders have at times been criticised for using loud public address systems to attract worshippers.
Government official Justus Kangwagye told the BBC’s Focus on Africa programme that they simply required the churches to meet “modest standards”.
This is one of the churches which has been closedThis is one of the churches which has been closed
Some church premises exposed worshippers to unnecessary risks and could “cause danger to those worshipping,” Mr Kangwagye told the Rwandan New Times newspaper.
The reported crackdown is being carried out by the local authorities with the support of the Rwanda Governance Board, according to the New Times.
Some 700 churches were thought to be operating illegally, the Rwanda Governance Board said in a recent statement.
The vast majority of Rwandans are Christians but many also follow traditional practices.