Place the flour, sugar, baking powder, softened butter, milk, and vanilla extract into a bowl.
Using your hands, mix the ingredients together to form a dough. Be sure to thoroughly combine.
Make a flattened disc of dough and wrap in plastic. Refrigerate to chill the dough while the oven preheats.
Roll the dough 1/4-inch thick between two pieces of lightly floured parchment paper. The flour will ensure that the cut outs are easy to move.
Cut out the dough with a cookie cutter and transfer to the prepared baking sheets. Bake for 7-9 minutes until lightly browned around the edges.
Make the icing. Stir the ingredients together in a small bowl. Add a drop of food coloring if desired.
Let the cookies cool on a cooking rack. Use a small butter knife or paintbrush to spread on the icing or pipe on as desired. If adding sprinkles, add them right away!