Any process in which a virus stops, prevents, or reduces the activity of a host TRAF (tumor necrosis factor receptor-associated factor) protein. TRAFs are intracellular signal transducers that lie downstream of receptors including RIG-I, MDA-5 and Toll-like receptors (TLR) and transfer the signal to other intracellular signaling components.