Summon Boat Carpe Diem

I’ve been stuck on the island with the Castle on Carpe Diem after going through the one way portal. My strongest hero is there and I have the summon boat spell. It keeps saying there’s no boats to summon.

However, some enemy hero just came out of the one way portal and immediately summoned a boat next to me. How can he summon a boat and I can’t?