gaining some fat while bulking is almost inevitable, but how much you're willing to tolerate is completely up to you. A few things worth considering:
1. Most obvious one is appearance. If you don't like what you see in the mirror, it's time to cut.
2. Athletic performance. Carrying extra weight around can affect your performance in sports (e.g. can't run as fast or as far) and to a slightly lesser extent, weight lifting (exercises involving body weight actually get harder).
3. Even day to day to day tasks can be impacted. Even if you are a lot stronger, carrying an extra 50lbs of muscle/fat is a lot of extra work for your heart, legs, lungs, etc.
4. Health reasons. Too much fat can be an indicator of poor health, negatively affecting your cardiovascular system.
In your case, probably only #1 really applies, so if you're starting to worry about your looks, consider cutting. Many people use a zig-zag pattern when lifting: alternating periods of bulking with periods of cutting every 8 weeks or so. I would recommend gradually easing into a cut, rather than a sudden lifestyle change. Monitor what you eat a little more, slowly cut back the calories/carbs a bit, keep the protein intake high. That alone may curb the extra fat without having to do a more hard core diet. Just keep in mind that during a cut, it's significantly harder to progress in your lifts. However, as long as you keep lifting heavy, you can at least prevent yourself from losing any muscle.