The process in which relatively unspecialized cells acquire specialized structural and/or functional features that characterize the renal vesicle progenitor cells of the kidney as it progresses from its formation to the mature state. A renal vesicle progenitor cell is a cell that will give rise to terminally differentiated cells of the renal vesicle without self-renewing.