If Iraneaus is to be trusted (and I think he is) Peter did indeed travel to rome and establish the Bishopric there with Saint Paul.
Iraneaus, Against Heresies, Book Three, Chapter Three, 2. Since, however, it would be very tedious, in such a volume as this, to reckon up the successions of all the Churches, we do put to confusion all those who, in whatever manner, whether by an evil self-pleasing, by vainglory, or by blindness and perverse opinion, assemble in unauthorized meetings; [we do this, I say,] by indicating that tradition derived from the apostles, of the very great, the very ancient, and universally known Church founded and organized at Rome by the two most glorious apostles, Peter and Paul; as also [by pointing out] the faith preached to men, which comes down to our time by means of the successions of the bishops. For it is a matter of necessity that every Church should agree with this Church, on account of its preeminent authority, that is, the faithful everywhere, inasmuch as the tradition has been preserved continuously by those [faithful men] who exist everywhere.
3. The blessed apostles, then, having founded and built up the Church, committed into the hands of Linus the office of the episcopate. Of this Linus, Paul makes mention in the Epistles to Timothy. To him succeeded Anacletus; and after him, in the third place from the apostles, Clement was allotted the bishopric. This man, as he had seen the blessed apostles, and had been conversant with them, might be said to have the preaching of the apostles still echoing [in his ears], and their traditions before his eyes. Nor was he alone [in this], for there were many still remaining who had received instructions from the apostles. In the time of this Clement, no small dissension having occurred among the brethren at Corinth, the Church in Rome dispatched a most powerful letter to the Corinthians, exhorting them to peace, renewing their faith, and declaring the tradition which it had lately received from the apostles, proclaiming the one God, omnipotent, the Maker of heaven and earth, the Creator of man, who brought on the deluge, and called Abraham, who led the people from the land of Egypt, spoke with Moses, set forth the law, sent the prophets, and who has prepared fire for the devil and his angels. From this document, whosoever chooses to do so, may learn that He, the Father of our Lord Jesus Christ, was preached by the Churches, and may also understand the tradition of the Church, since this Epistle is of older date than these men who are now propagating falsehood, and who conjure into existence another god beyond the Creator and the Maker of all existing things. To this Clement there succeeded Evaristus. Alexander followed Evaristus; then, sixth from the apostles, Sixtus was appointed; after him, Telephorus, who was gloriously martyred; then Hyginus; after him, Pius; then after him, Anicetus. Soter having succeeded Anicetus, Eleutherius does now, in the twelfth place from the apostles, hold the inheritance of the episcopate. In this order, and by this succession, the ecclesiastical tradition from the apostles, and the preaching of the truth, have come down to us. And this is most abundant proof that there is one and the same vivifying faith, which has been preserved in the Church from the apostles until now, and handed down in truth.
So there is evidence, its just a matter if whether or not the evidence is realiable.