I do not understand what the problem is. Whenever I became Catholic, I did not feel like I was giving everything up from my Protestant days. The fact is, even though there are definite errors within Protestantism, there is a lot good within Protestantism as well. Those things which did not directly conflict with Catholicism I tended to hold on to. An example is praying using one's own words instead of praying written prayers. I do pray a lot of written prayers of my Church such as the Act of Contrition and the Rosary. However, sometimes I just make the Sign of the Cross and then begin praying in my own words. I am not sure how common this is amongst cradle Catholics, but I know it is a lot more common in Protestant circles, especially in the evangelical/holiness movement.
I think you can still listen to Latin music and pray the Rosary without feeling like you are betraying Eastern Orthodoxy. Talk to you priest and ask him if you have any doubts. I am sure he would have no problem with it, especially if you mention to him that you are going to be using the Eastern Orthodox version of the Rosary.