Abstract: Aiming at the time-varying formation control problem of unmanned surface vessel swarms, this paper proposes a time-varying formation control algorithm based on collaborative-exploration deep reinforcement learning. The algorithm formulates time-varying formation construction strategies and local negotiation strategies for the swarm, as well as the motion control method of unmanned surface vessels. It solves the problems including formation transformation, formation navigation, internal position conflict and inter-vessel collision avoidance, and realizes accurate formation control of the swarm. Simulation experiments in open water and constrained channel scenarios verify the effectiveness of the proposed algorithm.