Any process in which an organism stops, prevents or reduces the frequency, rate or extent of the activity of a high voltage-gated calcium channel in another organism. A high voltage-gated channel is a channel whose open state is dependent on high voltage across the membrane in which it is embedded.