Yep, definitely ableist. It's something I've often responded with in the past. It's still true, but now I'm more tempted to respond with, "Is it any of your business how someone chooses to read a book?", because people will often say, "Okay, it's good for disabled people, but no one else should use it," which is still annoying to me.