One of the most beautiful and at the same time most horny fishes in the ocean is the mandarinfish, Synchiropus splendidus. This fish lives in the tropical Indo-Pacific, supposedly north to the Ryukyu islands, even though I have never seen one there during many dives.
I have had the chance to see and photograph them in Yap, Micronesia, and Bohol & Malapascua, Philippines. These really pretty fish usually live in more secluded, low-surge bays, and often dwell along dead finger corals, not too deep (< 12 m). Around sunset, the male mandarin fish, slightly larger than the females, cruise between the tips of the coral fingers, looking for females. When two of them decide to mate, they quickly raise into the water column, and release gametes. Often I have seen small cardinal fish go for the newly fertilized eggs right away, negating the reproductive effort of the mandarins.
Mandarin fish belong to the fish family of the dragonets, some of which we also have here in Sydney.