The aggregation, arrangement and bonding together of a set of components to form the spindle during meiosis II of a meiotic cell cycle in females. An example of this is found in Drosophila melanogaster.