Any process that modulates the rate, frequency or extent of organ formation. Organ formation is the process pertaining to the initial formation of an organ from unspecified parts. The process begins with the specific processes that contribute to the appearance of the discrete structure, such as inductive events, and ends when the structural rudiment of the organ is recognizable, such as a condensation of mesenchymal cells into the organ rudiment.