I think it's feasible to imagine some people out there in the world have been born without chakra during the time period where Naruto is alive. I say this because chakra as we learn in the anime was originally given to Hagoromo Ōtsutsuki during a time period in the history of Naruto where no one had chakra, after which she passed it on to her children. Therefore it's presumable that of the other people alive at the time some of them could have continued to have children without ever mixing bloodlines with those who had chakra all the way until Naruto's generation.
Now despite this I can see how considering how many generations it has been there's not likely to be many left who haven't attained chakra, but do you think they are there, and why were they never a plot point for the show?