In a medium bowl, sift together flour, cocoa powder, baking powder, and salt, and set aside.

In the bowl of an electrical mixer fitted with the paddle attachment, cream butter, vanilla, and sugar. Add eggs and milk, and blend until combined. Add reserved flour mixture, and blend on low speed until incorporated, scraping the sides of the bowl at least once. Divide the dough in half, and shape each half into a flat disk. Wrap each disk in plastic wrap, and chill until firm, about 1 hour.

Preheat the oven to 350°F. Roll dough out on a floured surface, sliding an offset spatula under the dough to release it every few turns of the rolling pin. Roll dough to 1/8-inch thick. Cut dough using a number of heart-shape cookie cutters 2 1/2 to 3 inches in diameter, ensuring each cookie has a match to make a sandwich. Place hearts on a parchment-lined baking sheet; chill until firm, about 30 minutes. Remove from fridge; use a fork to prick the cookies all over with holes. Bake until just firm, 12 to 15 minutes. Let cool on baking sheet; transfer to a wire rack.

With the underside of half the cookies face up, spoon softened strawberry ice cream about 1/2-inch thick to cover one side. Place matching cookie on top of ice cream, top side facing out. Transfer instantly to freezer to harden; repeat with remaining cookies and ice cream. Serve directly from the freezer. Sandwiches can be kept stored in an airtight container in the freezer for 3 to 4 days.