Fill a large bowl with ice to use later as an ice bath. Place an empty bowl on top of the ice. Set aside.
In a heavy bottom sauce pan combine milk, heavy cream, and vanilla. Cook over medium heat until mixture is steaming and small bubbles start to form around the edge of the pot.
Meanwhile, whisk egg yolks, sugar, and cocoa powder together in a medium sized mixing bowl. Once the milk and cream is steaming, spoon a ladle full of the hot mixture over the eggs and quickly whisk together. Repeat process once more.
Add egg mixture to the saucepan, and whisk to combine. Continue to cook, stirring constantly, until the ice cream base reaches 180°F on an instant read thermometer.
Place a fine mesh strainer over the empty bowl on top of the ice bath. Stain ice cream base. Chill ice cream base until it has stopped steaming. Cover and refrigerate overnight.
An hour before churning the ice cream, remove the ganache from the refrigerator to allow to come to room temperature.
Pour the ice cream base into an ice cream maker and process for about 20-25 minutes, or according to the manufacturer's directions.
Place a third of the ice cream in a plastic storage container, swirl in ganache and sprinkle pieces of Reese's cups. Repeat with remaining two layers. The ice cream will be on the soft side but can be eaten right away or put into the freezer to "cure" or harden up for a couple of hours. Store in an airtight container in the freezer for up to two weeks.