This is the first time in Botswana that an ex-president has abandoned the ruling Botswana Democratic Party (BDP).
Khama told thousands of supporters in his rural home town of Serowe that he made a “mistake” in choosing Mokgweetsi Masisi as the president.
“I came here today to tell you I am parting ways with the BDP. I am throwing away my BDP membership card,” said Khama as he trashed his card.
“I don’t recognise the BDP anymore”.
Thousands of people who thronged the meeting at the town’s showgrounds also threw away their BDP membership cards.
The 57-year-old BDP has ruled Botswana uninterrupted since independence in 1966,
Kjhama said he did not have a party yet but will support the opposition Umbrella for Democratic Change (UDC) fight the BDP.
Khama and his successor have fallen out publicly, with Masisi reversing some policies introduced by his predecessor – including most recently the lifting of a ban on elephant trophy hunting.
The rift was laid bare last March when Khama accused his protege of betrayal.
Botswana has a two-term presidential limit. Masisi took over the leadership from Khama last year, after being hand-picked, but will contest his first election this year.