We know them, what they look like, how they dress, and what they say. They're stereotypical bad guys, often called criminals or perps. They could be serial killers, extortionists, rapists or drug dealers, but we know them because we've been conditioned to recognize the bad guy in pop culture.
Being a gray area kind of creator myself, I like to imagine the bright side of the bad guys, and the dark side of the good people. That bad guy could be your neighbor, friend, or relative. You've liked him or her all these years but never knew about that dark secret. Or the bodies. Or the secret bank account.
To make any character convincing, you have to allow a connection between the the reader or viewer and the character. It follows that making a sinister character convincing means you have to create this same connection: something the reader can identify with, sympathize with, or generally have feelings for. While perusing this topic, I came across this snippet from Roger Ebert. The point was made for movies in this case, but the point applies, and all the examples listed are from books originally:
There are so many bad horror movies. A good one is incredibly hard to make. It has to feel a fundamental sympathy for its monster, as movies as different as "Frankenstein," "Carrie" and "The Silence of the Lambs" did. It has to see that they suffer, too. The crimes of too many horror monsters seem to be for their own entertainment, or ours. In the best horror movies, the crimes are inescapable, and the monsters are driven toward them by the merciless urgency of their natures.