The President ordered a fresh vote in 90 days, adding she would no longer stand for re-election.
Malawi's electoral commission (MEC) had received multiple complaints of irregularities following the May 20 vote, including from Banda's People Party.
The MEC released preliminary results on Saturday showing opposition Democratic Progressive Party candidate Peter Mutharika leading with 42 percent of the vote, followed by Banda with 23 percent. This was based on 30 percent of the vote counted.
"I am nullifying the elections, using the powers invested in me by the Malawi constituion," Banda told a news conference in the capital Lilongwe. She cited complaints of "various irregularities".
"I want to give Malawians an opportunity to choose a candidate of their choice in a free and fair manner. When elections are to be held again, I will be stepping aside," she said.
The confusion over the results risks fanning tensions in the small country of 13 million people where Banda, southern Africa's first female head of state, has seen her popularity eroded by a corruption scandal.
Tuesday's poll has been plagued by problems from the outset, with voting materials turning up hours late and ballot papers being sent to the wrong end of the country, infuriating voters.
Organisers had to extend voting in some urban areas into a second day and initial counting was held up by a lack of lighting and generators at polling stations.
Banda enjoyed huge goodwill when she came to power two years ago, but her popularity waned after she was forced to impose austerity measures, including a sharp devaluation, to stabilise the economy.
Her administration was hit by a $15 million corruption scandal, dubbed 'Cashgate', after large amounts of cash were discovered in the car of a senior government official.