If you had to pick a flaw that lead to his downfall, something that makes him make *stupid* decisions when he's otherwise very smart, what would that be?
Pride and greed that made him crave power to destroy his enemies (and thus summon Sebastian and make a pact with him)

This absolutely does detract from his life quality and happiness, yes.