Any process that modulates the rate, frequency or extent of chylomicron remnant clearance. Chylomicron clearance is the process in which a chylomicron remnant is removed from the blood via receptor-mediated endocytosis into liver cells and its constituent parts degraded.