It is the patristic teaching of St. Irenaeus, St. Leo and the council of Chalcedon that on account of Romes Secular Status of imperial center, she has received the Primacy.
St. Irenaeus says:
"as it would be very tedious to enumerate in such a work the succession of all the Churches, we will trace that of the very great and very ancient Church and known of all, which was founded and established at Rome by the two very glorious Apostles, Peter and Paul; which possesses a tradition that comes from the Apostles as much as the Faith declared to men, and which has transmitted it to us through the succession of her Bishops; by that, we confound all those who in any manner whatsoever, either through blindness or bad intention, do not gather where they should; for every Church, that is to say, the faithful who are from all places, are obliged to go toward [convenire] that Church, because of the most powerful principality. In this Church, the tradition of the Apostles has been preserved by those who are of all countries."
Apologists for the Roman papacy always invoke this passage of St Irenaeus as if it settles the matter. It's tempting, because Irenaeus was the spiritual "grandson", if you will, of the apostle St John, so such a view can claim antiquity if accurate. But what does this quote really demonstrate? It's no surprise, really, that Irenaeus, himself a bishop in the West, would look to the only Western Church with apostolic origins and foundation, the Church which is also in the most important city of the Empire, as a standard against which to compare. You might be able to argue a primacy in the West from Irenaeus based on apostolicity, but in the East? "Apostolic sees" are a dime a dozen in the East: St Thomas himself can be linked to the Churches of Persia, India, and (through the apostles SS Bartholomew and Thaddeus), the Church in Armenia, and he's just one of the Twelve.
For St Irenaeus, it is important for any local Church to be a faithful steward of the faith handed down from the apostles. To this end, it is not just the Church of Rome about which he speaks:
But Polycarp also was not only instructed by apostles, and conversed with many who had seen Christ, but was also, by apostles in Asia, appointed bishop of the Church in Smyrna, whom I also saw in my early youth, for he tarried [on earth] a very long time, and, when a very old man, gloriously and most nobly suffering martyrdom, departed this life, having always taught the things which he had learned from the apostles, and which the Church has handed down, and which alone are true. To these things all the Asiatic Churches testify, as do also those men who have succeeded Polycarp down to the present time,— a man who was of much greater weight, and a more steadfast witness of truth, than Valentinus, and Marcion, and the rest of the heretics. He it was who, coming to Rome in the time of Anicetus caused many to turn away from the aforesaid heretics to the Church of God, proclaiming that he had received this one and sole truth from the apostles—that, namely, which is handed down by the Church. There are also those who heard from him that John, the disciple of the Lord, going to bathe at Ephesus, and perceiving Cerinthus within, rushed out of the bath-house without bathing, exclaiming, Let us fly, lest even the bath-house fall down, because Cerinthus, the enemy of the truth, is within. And Polycarp himself replied to Marcion, who met him on one occasion, and said, Do you know me? I do know you, the first-born of Satan. Such was the horror which the apostles and their disciples had against holding even verbal communication with any corrupters of the truth; as Paul also says, A man that is an heretic, after the first and second admonition, reject; knowing that he that is such is subverted, and sins, being condemned of himself. Titus 3:10 There is also a very powerful Epistle of Polycarp written to the Philippians, from which those who choose to do so, and are anxious about their salvation, can learn the character of his faith, and the preaching of the truth. Then, again, the Church in Ephesus, founded by Paul, and having John remaining among them permanently until the times of Trajan, is a true witness of the tradition of the apostles.
Against Heresies, 3.3.4
And yet, there is no "Ephesine primacy".
Irenaeus is contrasting the various heretics and their "churches" with the Church of Christ, existing in many places of diverse cultures, but preserving the same faith and tradition handed down by the apostles. In the West, of course attention will be paid to Rome. But it's not so much about Peter for Irenaeus, it's about the apostles.
The faithful would convene at Rome because it was the imperial center, and as such men from all nations, and therefore every apostolic tradition, would arrive and go to the Church of Rome which retained these traditions of the apostles from every country.
How do you mean?
I won't address the rest of your post (re: Leo of Rome, Chalcedon, etc.) because I'd prefer to see the EO answer it. It is not an uncommon feeling among the OO that "papal supremacy" began in the fifth century with Leo, and got a boost from the Chalcedonian East (even if they later came to reject it). As you lay it out, it's difficult for me to see how the OO are unjustified in having such feelings. But perhaps there's more to it than what you have said.