After being a pastor for close to 10 years, I've noticed a funny thing.
People don't like sermons that are too "watered down" and they don't like sermons that are "heretical" which means that they don't like a message that is too much what they already know but also not one that is too much what they don't know.
I don't think this is a specific thing to pastors or religion or church... it's everywhere in most of life. I also don't find it upsetting or frustrating. We all feel the same way - I feel the same way. Of course, I'm responsible for some of that. I need to work on being someone who can be pushed without reacting but it's also about a chemistry between the listener and the speaker: the student and the teacher.
When you find a mentor, teacher, speaker, guru, instructor, etc... that can ride that middle ground, hold on tight. When you find the person that can push you just enough without making you run off in terror - well you've found something special. Keep listening. Find more.
And if you can't find any human who does that to you, well... it might not be them.