I actually do get this a lot and it's a big reason why I don't draw more often. I just can't do anything I enjoy or do it well if I feel like someone's hovering over me and it happens far more often than I'd like.
Do you ever get the feeling you're being watched?