Interacting selectively and non-covalently with a specific DNA sequence in an RNA polymerase II (Pol II) core promoter, the region composed of the transcription start site and binding sites for transcription factors of the Pol II basal transcription machinery, in order to modulate transcription by Pol II. The transcription factor may or may not also interact selectively with a protein or macromolecular complex.