The multiplication or reproduction of brown fat cells by cell division, resulting in the expansion of their population. A brown fat cell is a fat cell found the thermogenic form of adipose tissue found in newborns of many species.