Preheat oven to 350°F.
In a small saucepan cook the butter over low heat until the butter is caramel colored and smells toasty. Strain the butter into the bowl of a stand mixer. Leave it to cool slightly.
In a medium bowl whisk together the flour, baking powder, cinnamon, ginger, and salt. Set aside.
Using the paddle attachment, beat together the browned butter, brown sugar and vanilla until it is cooled off slightly. Add the eggs one at a time, mixing well after each. Alternate adding the flour mixture and buttermilk, starting and ending with the dry ingredients. Add the fresh ginger and mix just until it is evenly distributed.
Place the batter evenly over the peaches and spread smooth with a spatula. Bake for about 30 minutes or until the cake is set and a tester comes out clean. Let sit until the pan is cool enough to handle and invert onto a serving platter.
I serve the cake with the ginger coins and vanilla still in place, but you can remove them before if you want. The ginger can be eaten and has a delightful heat to it, the vanilla is just for drama, but can't be eaten.