Details of the talks were not revealed but party leaders are to hold an emergency meeting on Monday.
Mr Zuma, who faces corruption allegations, was replaced as ANC leader by Cyril Ramaphosa in December.
Analysts say party chiefs want to avoid a power struggle that could split the ANC before elections next year.
They are expected to begin the process to remove President Zuma through a formal recall or by introducing a motion in parliament.
The six most senior figures of the governing ANC party arrived one by one on Sunday at President Zuma’s residence in Pretoria.
They remained tight-lipped as the talks ended but have called a meeting of the party’s national working committee for Monday.
Julius Malema, an opposition leader and former ANC member, said on Twitter that Mr Zuma had refused to stand down.
Mr Zuma is not due to leave office until the 2019 elections. However, the ANC has seen its popularity fall during his second term amid a weakening economy and claims of corruption.
The president has been beset by a series of corruption allegations but denies any wrongdoing.
Mr Ramaphosa, who as the ANC’s new leader is in a strong position to win the 2019 elections, has called for party unity.
But correspondents say fears are growing that he and his allies in the ANC may move against Mr Zuma if he refuses to stand aside.