Various forms of clerical collars go back to the sixteenth century. In almost every case (French, Dutch, British, etc.), clergy simply wore plain versions of the collars popular at the time, but, as black became the preferred color of attire, the white collar naturally stood out as a distinctive sign of the clergy. In most countries, the collar would be worn over a cassock or even robe until the the nineteenth century or even later.
The history is quite analogous to other forms of clergy attire, e.g. the cassock, which at one point was a normal, every-day piece of clothing.