Yes baby zebra finches need to be kept at a toasty 85°F They also need to be fed every 3 hours so set the alarm clock.It needs to be a hand feeding supplement and feed with a syringe just be sure to be clean about it!~Good luck with them!
You shouldn't be hand-feeding if you're asking such basic questions.
I'm hoping you are hand feeding because the parents neglected them. They will not become tame if hand fed. If you have a chance of fostering them to other birds or giving them back to their parents please do. If not than maybe find a pet shop or a breeder who will take them and hand feed them?
Hand feeding is a difficult process, the size of zebra finches makes it much harder. You have a 1 in 25 chance of keeping them alive.
But to answer your question : Yes. you most definitely need a source of heat. You need to set up a brooder.
Do my baby Zebra Finches need a heat lamp? If so what temp? P.S. I'm hand feeding the babies.