But first you have to prove that when these Fathers talked about Peter they had the Pope of Rome in mind.
what's your take on the following
When Pope Victor I (189-198) chose to excommunicate the Asian churches from the universal church and Rome for following their own tradition concerning the appropriate day to celebrate the Resurrection, a number of bishops were critical of him, but none challenged his authority to do so. St. Irenaeus urged him not “to cut off whole churches” and he relented, though he had called synods to consider the problem on his own authority. St. Irenaeus, writing his famous “Against Heresies” after 180 A.D. noted, It is possible, then, for everyone in every Church, who may wish to know the truth, to contemplate the tradition of the Apostles which has been made known throughout the whole world. And we are in a position to enumerate those who were instituted bishops by the Apostles, and their successors to our own times . . . . The blessed Apostles [Peter and Paul] having founded and built up the Church [of Rome] handed over the episcopate to Linus. Paul makes mention of this Linus in the epistle to Timothy [2 Tim 4:21] To him succeeded Anencletus; and after him in the third place, from the Apostles, Clement." These men were the first three popes.
Since, however, it would be very tedious, in such a volume as this, to reckon up the successions of all the Churches, we do put to confusion all those who, in whatever manner, whether by an evil self-pleasing, by vainglory, or by blindness and perverse opinion, assemble in unauthorized meetings; [we do this, I say, ] by indicating that tradition derived from the apostles, of the very great, the very ancient, and universally known Church founded and organized at Rome by the two most glorious apostles, Peter and Paul; as also [by pointing out] the faith preached to men, which comes down to our time by means of the successions of the bishops. For it is a matter of necessity that EVERY CHURCH SHOULD AGREE WITH THIS CHURCH, on account of its pre-eminent authority, that is, the faithful everywhere, inasmuch as the apostolical tradition has been preserved continuously by those [faithful men] who exist everywhere. (St. Irenaeus, Against Heresies, (3,3,2), 180 A.D.)