Is this the Foreigner series by C.J. Cherryh...?
It's a lengthy series, spanning over twenty books. The human protagonist is Bren Cameron. The aliens are called the atevi and are depicted on the cover as being a couple of feet taller than a human.
Jusging by this review of the first book, Foreigner (1994), the plot aligns closely to your description:
Far-future alien-contact yarn from the author of Chanur's Legacy, The Goblin Mirror, etc., where, in a stuttering, episodic liftoff, we learn that a human colony ship, lost in space, luckily comes near a planet inhabited by humanoid "atevi." Later, the two species fight a war in which the humans' technological superiority barely compensates for their physical inferiority and lack of numbers. So the humans are confined to the island of Mospheira, and only their paidhi, or translator/technical liaison, is permitted to enter atevi society—the latter a complex of warring factions, loyalty codes, and assassination. The paidhi, Bren Cameron, is caught up in a factional struggle and, after narrowly avoiding assassination, is spirited away by Tabini, leader of the pro-human faction, to an isolated estate. Here, surrounded by potential enemies, again threatened with assassination, Bren ponders the loyalties of his hosts as an antihuman faction comes close to provoking another war—a process exacerbated by the humans' own factional split into the planetbound and the spacefaring. Things improve after that bumpy start, though the frequent dull interior monologues don't help. These matters aside: a seriously probing, thoughtful, intelligent piece of work, with more insight in half a dozen pages than most authors manage in half a thousand.
You can read a preview of the first book here.