- Preheat the oven to 350 degrees. Grease a 9 by 9 by 2-inch baking dish.
- Mix together granulated sugar, eggs, and milk in a bowl; add vanilla. Pour over cubed bread and let sit for 30 minutes.
- In another bowl, mix and crumble together brown sugar, butter, and pecans.
- Pour bread mixture into prepared pan. Sprinkle brown sugar mixture over the top and bake for 35 to 45 minutes, or until set. Remove from oven.
For the sauce:
- Mix together the granulated sugar, butter, egg, and vanilla in a saucepan over medium heat. Stir together until the sugar is melted on a low simmer, whisking often. Mixture will darken and thicken some while cooking. Do not overcook.
- Add the brandy (if using) stirring well.
- Pour over bread pudding, if desired, or serve separately with each slice. Serve warm.