Preheat your oven to 350 degrees F. Line muffin tin cups with cupcake liners. Set aside.
In a medium mixing bowl, stir together the flour, pumpkin pie spice, baking powder, and salt. Set aside.
In a large mixing bowl, beat together the sugar and oil until combined. Add the eggs and vanilla; beat to combine. Stir in the pumpkin puree and mix until smooth and combined.
Add half the dry ingredients to the wet ingredients and mix until just combined. Pour in the milk and mix until completely incorporated. Add the remaining dry ingredients and mix until just combined, the batter will be thick. Do NOT overmix. Set aside.
In a medium mixing bowl, beat together the cream cheese, sugar, and vanilla until smooth.
Use a tablespoon measuring spoon or small cookie scoop to portion the pumpkin batter into the prepared muffin tin cups so they are 1/2 to 2/3 full. Add 1/2 tablespoon of the cream cheese mixture on top of the pumpkin batter. Use a metal skewer or knife to swirl the cream cheese into the pumpkin, if desired.
Bake for 30 to 35 minutes or until a toothpick inserted in the center of the muffin comes out clean.
Carefully remove from the oven. Allow to cool for 5 minutes in the pan before removing the muffins to a wire rack to cool completely. Serve warm or at room temperature. Store in an airtight container.