Have you ever seen Koreans attack someone as a pedophile just for saying "I love you, baby"? No. In Korea, even though the literal translation of "baby" refers to an infant, everyone understands the English nuance. We recognize that languages have different cultural contexts.
Koreans respect these differences, so why are you labeling him a pedophile based entirely on your Western cultural lens? He is Korean. Being fluent in English doesn't change the fact that his mindset is shaped by living in Korea. It’s hypocritical to stan Korean idols while refusing to understand Korean culture. If you're going to consume K-pop, the least you can do is educate yourself on the language and culture first.