As Keble pointed out, there are a number of different kinds of yoga- karma-yoga (spiritual growth through good works), hatha-yoga (spiritual growth through bodily excercises), jnana-yoga(spiritual growth through expanding of knowledge of spiritual things), bhakti-yoga(spiritual growth through devotion to a one of the deities). Yoga literally means "to yoke" or "to link". Now, in the Bhagavada-Gita, probably the most central part of the Hindu Scriptures(Vedas), Krishna presents all of these paths to his disciple Arjuna as legitimate. However, Krishna says that of all of these Bhakti-Yoga, or devotion to him, is the best path to spiritual enlightenment.
Now, I suppose when one is doing this hatha-yoga one has to see the yoga(linking process) in this context- as a means to reach a hindu god. However much we want to strip away the spiritual meaning of yoga, it still has this "linking" to a pagan god as its main goal. Stick to pilates.