Never confuse Stake Authority with Withdraw Authority. When you delegate, you give the validator permission to vote with your stake weight. You always keep the Withdraw Authority, which means you alone can move or withdraw the funds. Validators can never access your SOL.