Slice loaves into small slices. Brush each slice with olive oil. Place on center rack of preheated oven and bake for 10 minutes.
Mix water, salt, cinnamon, cloves, and allspice together. Bring to a boil over medium-high heat.
Add cranberries to water and spice combination. Continue to boil until cranberries pop and create a fluid sauce.
Slice goat cheese thinly and place one piece on each bread slice. Spread cranberry sauce on top of cheese. Zest both oranges and place a pinch of zest on each crostini.
