Lydia had nothing to do with the Aryans taking Walt's money.
She did nothing wrong that Walt was aware of.....other than guilt by association.
He retrieved and planted the ricin in the Stevia even before he spoke a single word to her.....so before he even got to town, he had it in his mind to kill her.