The reason why it is acceptable to say you have a difficult time dealing with people like this is because everyone will have a hard time dealing with this type of person. So it doesn't make you look bad because you can't get along with this person, however, this is a good answer because it shows taking initiative to correct the relationship.