Any process that stops, prevents, or reduces the frequency, rate or extent of conidium formation, a processes leading to the production of a type of asexual non-motile reproductive spore of fungi via mitosis at the tip or side of hyphae or on special spore-producing structures called conidiophores.