As someone who speaks Japanese barely well enough to watch without subs (I prefer subs so I can make sure my translations are correct, but I often ignore them), I don’t have the awkwardness issue that you might have when watching, the voices and stuff just make sense. However, I still think subs are a good thing. They only really detract from the show if you’re fully fluent most likely, being that I’m still learning, I like having them, and think they add to my experience in that I can keep track of whats going on even if I don’t understand a word or two, or if they speak too fast. The few times I have watched without subs I’ve spent a lot more time missing what is going on because I’m trying to remember what was said than I would with subs, because again, I don’t understand every single word. (Although a lot of the words used in Anime are pretty easy to understand, as they’re straight forward dialogue generally, but many plot devices and stuff aren’t, just regular dialogue.)
So yeah, it depends on how well the person speaks the language most likely.
But on another note, sometimes translations aren’t 100% accurate, or lose some of the flavor, so I feel like with subs even I get more out of the dialogue than someone who doesn’t speak the language, because I can understand things that are intentionally not translated. Like in your case, you’re watching One Piece as you said, and in One Piece they use the word “Nakama” untranslated sometimes (or at least the translation team I watched did) and in the show it can take on many different meanings, from crewmate, to friend, to ally, this is something that isn’t clear to non-speakers or even non-native speakers. That said, I don’t always know which type of nakama would be the best in these cases, as it’s a word that has a lot of meanings to it in context.
But again, I am not a be all end all source on the language or culture, as I only speak it at an intermediate level, rather than being brand new or fluent. Finding both of those perspectives might be harder, but possibly more valuable on this topic.