From my experience this only happens when you leave the “fighting area” like going into the water and killing the boss