Catalysis of the transfer of oligopeptide from one side of the membrane to the other. In high affinity transport the transporter is able to bind the solute even if it is only present at very low concentrations. Oligopeptides are molecules that contain a small number (2 to 20) of amino-acid residues connected by peptide linkages.