If he's kind of annoying--BUT you have sexual fantasies about him nonetheless--that person would be a good candidate for a casual-sex partner. He himself will be a constant reminder about why the relationship could never work out. The minute he opens his mouth, the reason will be clear.