Any process that modulates the establishment or extent of the mitochondrial membrane potential, the electric potential existing across the mitochondrial membrane arising from charges in the membrane itself and from the charges present in the media on either side of the membrane.