I believe it was both Ss. Ignatius of Antioch and Cyprian of Carthage who posited that the Church and the episcopal office are inseparable--you can't have one without the other.
"Let no one do anything that pertains to the church apart from the bishop. Let that be considered a valid Eucharist which is under the bishop or one whom he has delegated. Wherever the bishop shall appear, there let the people be: just as wherever Christ Jesus may be, there is the catholic Church."
Ignatius of Antioch,
Letter to the SmyrnaeansOr as paraphrased by Nicholas Afanasiev in
The Church of the Holy Spirit, "...wherever the bishop is, there is the Church and, conversely, wherever the Church is, there is the bishop. The bishop belongs to the Church, but the Church incorporates the bishop. There can be no Church without a bishop, but there also can be no bishop without a Church."