You've got to honor your relationship with your audience - that they sit down because they want to be entertained. And that doesn't mean you can't provoke them and antagonize them and challenge them in the course of the entertainment as long as you keep the entertainment part of the equation alive.
Frustration although quite painful at times is a very positive and essential part of success.