Interacting selectively and non-covalently with scavenger receptors, a family of proteins that are expressed on myeloid cells and are involved in the uptake of effete cellular components and foreign particles.