You don't need to be violent to be brutal. Few of Walter's actions were as out-and-out ugly as the way he bullied himself back into the house, and lives, of his estranged wife and son. Breaking into his former home through the crawlspace, a place where he'll end up spending an increasing amount of time, he responds to Skyler's threat to call the police and tell them everything by calling her bluff, knowing her life would be nearly as ruined as his own if she turned him in. Worse, he uses her inability to be honest about the situation to cast himself as the good guy in Walt Jr.'s eyes, in much the same way that he used to play the martyr by pretending he had no idea what Skyler meant when she demanded to know what was really going on. If you can resist the urge to punch Walt right in his self-righteous face when he condescendingly tells the kid not to see his mother as the bad guy, you've got the steely self-control of Gus Fring.