I've had it explained to me that memory eternal does not refer to our memory of the departed person, as our memories are finite and temporal, but rather refers to God's eternal regard and lovingkindess for His departed servant. No doubt someone could say something more about this.
I would agree with you in part. An aged and venerable priest, Father Archimandrite Orestes, of blessed memory, once told me that he understood the prayerful hope expressed is both that the departed be remembered forever in the hearts, minds, and prayers of the faithful and, more especially, that God 'remember' the person in His kingdom and unto the ages of ages.
'remember' being understood in the context of the supplicative prayer of St Dismas - 'Remember me, Lord, when You come into Your Kingdom' ('Accept' me into Your Kingdom)
Addendum - and before anyone asks - 'of blessed memory' is an expression of one's belief that those of whom it is said are remembered as being faithful and devoted servants of God, whom we trust are in His presence.