Is it in general practice to have a room or small chapel or prayer corner specifically for someone to come in to pray? If so, is the small room usually or ever dedicated to some purpose of prayer?
In my experience, it is not general practice in parish churches (far more common in RC and traditional Anglican churches, with their often very beautiful Lady Chapels), but I can't see why it can't be done in an Orthodox church. Orthodox practice is usually that folks simply enter the church, find a suitable place to sit or stand. Prayer before a particular icon, either hung or painted on the wall, or the iconostasis, is usually the norm.
Such a side chapel would not contain an altar, as Orthodox altars are located behind the iconostasis, unless the side chapel is a miniature nave with iconostasis like the one biro has described. Such an altar would be consecrated and used from time to time, when more than one liturgy needs to be served within one day, such as in very large parishes on great feasts, where more than one priest is available to serve.