The Garden from my understanding wasn't a Garden with trees and grass and leaves and birds chirping. It portrayed a life that the saints are living right now that Adam lost. This is what I mean by allegory. It's not synonymous.
Neither did every father believe the entire Earth was incorrupt. St. Athanasius left room for belief otherwise, and the only one that/who received the grace of incorruption was man, and they were placed in God's own Paradise.
I think St. Paul also visited Paradise, and he said "man has not seen, ear not heard, nor heart can contemplate" it.
As for the body/soul idea, I honestly don't know. I don't think evolution can necessarily contradict the idea that the body and soul were formed at the same time. Evolution does not deal with matters of the soul, it only deals with matters of the flesh.