Any process that decreases the rate, frequency, or extent the controlled release of molecules that form the extracellular matrix, including carbohydrates and glycoproteins by a cell or a group of cells.