Many times they miss b/c they're just slapping at it. They'll come up and turn RIGHT before grabbing it. It looks like the bait goes under but that's just b/c of the commotion they make. Frustrating!
Sometimes they'll actually just bump the bait also. This happens more with frogs. I've even had a day when I didn't get a single hook up but they would hit my frog up to 3 feet in the air. It was crazy.