Add milk, egg yolks, and vanilla extract to a saucepan and whisk until thoroughly combined.
In a separate bowl, combine brown sugar, cornstarch and salt. Whisk into milk mixture. Heat over medium-high heat until mixtures comes to a gentle boil, stirring frequently. Reduce heat to medium and continue cooking until thick and creamy. Remove from heat and whisk in butter.
Place saucepan in a large bowl filled with ice and whisk pudding periodically until chilled.
Divide between 4 serving dishes, top with toasted pecans, and serve.