Any process that modulates the frequency, rate or extent of phytoalexin metabolism, the chemical reactions and pathways involving phytoalexins, any of a range of substances produced by plants as part of their defense response.