vex

(v)  to bring trouble, distress, or agitation to 
The math wizard was obviously vexed when he was finally stumped by a challenging problem.