The process in which a relatively unspecialized cell acquires specialized features of a mammary gland mesenchymal cell. Mammary gland mesenchymal cells form a loosely connected network of cells that surround the mammary ducts.