Any process in which an organism stops, prevents, or reduces the frequency, rate or extent of callose deposition performed by the host as part of its defense response. The host is defined as the larger of the organisms involved in a symbiotic interaction.