He died on Wednesday from heart complications at a hospital in Dar es Salaam, Samia Suluhu Hassan said in an address on state television.
Magufuli had not been seen in public for more than two weeks, and rumours have been circulating about his health.
Opposition politicians said last week that he had contracted Covid-19, but this has not been confirmed.
Magufuli was one of Africa's most prominent coronavirus sceptics, and called for prayers and herbal-infused steam therapy to counter the virus.
"It is with deep regret that I inform you that today... we lost our brave leader, the president of the Republic of Tanzania, John Pombe Magufuli," Vice-President Hassan said in the announcement.
She said there would be 14 days of national mourning and flags would fly at half mast.
According to Tanzania's constitution, Ms Hassan will be sworn in as the new president and should serve the remainder of Magufuli's five-year team which he began last year.
Magufuli was last seen in public on 27 February, but Prime Minister Kassim Majaliwa insisted last week that the president was "healthy and working hard".
He blamed the rumours of the president's ill-health on "hateful" Tanzanians living abroad.
But opposition leader Tundu Lissu told the BBC that his sources had told him Magufuli was being treated in hospital for coronavirus in Kenya.
Magufuli declared Tanzania "Covid-19 free" last June. He mocked the efficacy of masks, expressed doubts about testing, and teased neighbouring countries which imposed health measures to curb the virus.
Tanzania has not published details of its coronavirus cases since May, and the government has refused to purchase vaccines.
On monday, police said they had arrested four people on suspicion of spreading rumours on social media that the president was ill.
"To spread rumours that he's sick smacks of hate," Mr Majaliwa said at the time.
