if a person likes a game for their story, lore, characters, etc enough to call themselves a fan of it, then that's the end of the conversation. shouldn't you be happy that your game is good enough that even people who can't play it can become fans of it just by watching?
The “you can be a fan of a game without playing it” argument reminds me that people don’t see games as art. It’s a medium built around personal engagement, that's what gameplay is, it’s what makes games what they are. Is it really that hard to acknowledge and respect that?