The process whose specific outcome is the progression of any organ involved in hematopoiesis (also known as hemopoiesis) or lymphoid cell activation over time, from its formation to the mature structure. Such development includes differentiation of resident cell types (stromal cells) and of migratory cell types dependent on the unique microenvironment afforded by the organ for their proper differentiation.