The multiplication or reproduction of glial cells by cell division, resulting in the expansion of their population. Glial cells exist throughout the nervous system, and include Schwann cells, astrocytes, and oligodendrocytes among others.