I'd have to agree that your dream of learning Latin without an intermediary language is just not going to work. You can easily learn modern languages through direct immersion because when you work with a native teacher, there's a visual context and the speaker can repeat things until you understand, slightly changing the grammar or employing circumlocution if necessary. Nearly all Latin textbooks, on the other hand, assume that you are prepared to rote-memorize paradigms and frequently consult a dictionary. (There are direct immersion textbooks for Latin, but even they assume you're in a course where you'll be talking with other people.)