I only use the die box for kids shows, but it would work for elderly.
I have everyone "raise their hand" on the side they think the die is. Then I say, "okay, most of you think it is on this side" raising my hand to indicate the obviously wrong side. I do this three times. By the then some one says, "open the other side!" Which of course I do ....
I have a duplicate underneath my linking rings, my next routine, and that's where the missing die reappears. I then drop it in the die box , which is left off to the side for the rest of the show (with all doors open).
No real story on my part, just a guessing game.