"We must hold to the Christian religion and to communication in her Church which is Catholic, and which is called Catholic not only by her members but even by all her enemies. For when heretics or the adherents of schisms talk about her, not among themselves but with strangers, willy-nilly they call her nothing else but Catholic. For they would not be understood unless they distinguish her by this name which the whole world employs in her regard."
Saint Augustine- The True Religion, 7,12, 397 A.D.
"Before His suffering the Lord Jesus Christ, as you know, chose His disciples, whom He called Apostles. Among these Apostles almost everywhere Peter alone merited to represent the whole Church. For the sake of his representing the whole Church, which he alone could do, he merited to hear, I will give you the keys of the kingdom of Heaven (Matt 16:19)."
Saint Augustine- Sermons 295,2, 391 A.D.
"Let us not listen to those who deny that the Church of GOD is able to forgive all sins. They are wretched indeed, because they do not recognize in Peter the rock and they refuse to believe that the keys of the kingdom of heaven, lost from their own hands, have been given to the Church." Saint Augustine- Christian Combat 31,33, 396 A.D.
"There is one GOD and one Christ, and one Church, and one Chair founded on Peter by the Word of the Lord. It is not possible to set up another altar or for there to be another priesthood besides that one altar and that one priesthood. Whoever has gathered elsewhere is scattering."
Letter of Cyprian to All His People, 43,40,5, 251 A.D.
"The blessed apostles, then, having founded and built up the Church, committed into the hands of Linus the office of the episcopate. Of this Linus, Paul makes mention in the Epistles to Timothy. To him succeeded Anacletus; and after him, in the third place from the apostles, Clement was allotted the bishopric. This man, as he had seen the blessed apostles, and had been conversant with them, might be said to have the preaching of the apostles still echoing [in his ears], and their traditions before his eyes. Nor was he alone [in this], for there were many still remaining who had received instructions from the apostles. In the time of this Clement, no small dissension having occurred among the brethren at Corinth, the Church in Rome despatched a most powerful letter to the Corinthians, exhorting them to peace, renewing their faith, and declaring the tradition which it had lately received from the apostles, proclaiming the one God, omnipotent, the Maker of heaven and earth, the Creator of man, who brought on the deluge, and called Abraham, who led the people from the land of Egypt, spake with Moses, set forth the law, sent the prophets, and who has prepared fire for the devil and his angels. From this document, whosoever chooses to do so, may learn that He, the Father of our Lord Jesus Christ, was preached by the Churches, and may also understand the apostolical tradition of the Church, since this Epistle is of older date than these men who are now propagating falsehood, and who conjure into existence another god beyond the Creator and the Maker of all existing things. To this Clement there succeeded Evaristus. Alexander followed Evaristus; then, sixth from the apostles, Sixtus was appointed; after him, Telephorus, who was gloriously martyred; then Hyginus; after him, Pius; then after him, Anicetus. Sorer having succeeded Anicetus, Eleutherius does now, in the twelfth place from the apostles, hold the inheritance of the episcopate. In this order, and by this succession, the ecclesiastical tradition from the apostles, and the preaching of the truth, have come down to us. And this is most abundant proof that there is one and the same vivifying faith, which has been preserved in the Church from the apostles until now, and handed down in truth."
Saint Irenaeus Adversus Haereses, Book III, chapter 3-3
"I follow no leader but Christ and join in communion with none but Your Blessedness, that is, with the chair of Peter. I know that this is the rock on which the Church has been built. Whoever eats the Lamb outside this house is profane. Anyone who is not in the ark of Noah will perish when the flood prevails."
Letter of Jerome to Pope Damasus, 374 A.D. 15,2 J1346
"He who is joined to the chair of Peter is accepted by me!"
Letter of Jerome to Pope Damasus, 374 A.D., 16,2 J1346a
"Yet, though your greatness terrifies me, your kindness attracts me. From the priest I demand the safe-keeping of the victim, from the shepherd the protection due to the sheep. Away with all that is overweening; let the state of Roman majesty withdraw. My words are spoken to the successor of the fisherman, to the disciple of the cross. As I follow no leader save Christ, so I communicate with none but your blessedness, that is with the chair of Peter. For this, I know, is the rock on which the church is built! This is the house where alone the paschal lamb can be rightly eaten. This is the ark of Noah, and he who is not found in it shall perish when the flood prevails. But since by reason of my sins I have betaken myself to this desert which lies between Syria and the uncivilized waste, I cannot, owing to the great distance between us, always ask of your sanctity the holy thing of the Lord. Consequently I here follow the Egyptian confessors who share your faith, and anchor my frail craft under the shadow of their great argosies. I know nothing of Vitalis; I reject Meletius; I have nothing to do with Paulinus. He that gathers not with you scatters; he that is not of Christ is of Antichrist."
Letter of Jerome to Pope Damasus, 376 A.D., 2
"I thank you for your reminder concerning the canons of the Church. Truly, "whom the Lord loveth he chasteneth, and scourgeth every son whom he receiveth." Still I would assure you that nothing is more my aim than to maintain the rights of Christ, to keep to the lines laid down by the fathers, and always to remember the faith of Rome; that faith which is praised by the lips of an apostle, and of which the Alexandrian church boasts to be a sharer."
Letter of Jerome to Pope Theophilus, 397 A.D., 2
"Clement, of whom the apostle Paul writing to the Philippians says "With Clement and others of my fellow-workers whose names are written in the book of life, "the fourth bishop of Rome after Peter, if indeed the second was Linus and the third Anacletus, although most of the Latins think that Clement was second after the apostle. He wrote, on the part of the church of Rome, an especially valuable Letter to the church of the Corinthians, which in some places is publicly read, and which seems to me to agree in style with the epistle to the Hebrews which passes under the name of Paul but it differs from this same epistle, not only in many of its ideas, but also in respect of the order of words, and its likeness in either respect is not very great. There is also a second Epistle under his name which is rejected by earlier writers, and a Disputation between Peter and Appion written out at length, which Eusebius in the third book of his Church history rejects. He died in the third year of Trajan and a church built at Rome preserves the memory of his name unto this day."
Saint Jerome- De viris Illustribus 15, 400 A.D.