Christopher Beha’s latest novel follows the trajectory of one New York family’s failures and how the irrational undermines even the best and brightest.