The chemical reactions and pathways involving pyrimidine nucleoside diphosphate, a compound consisting of a pyrimidine base linked to a ribose or deoxyribose sugar esterified with diphosphate on the sugar.