To solve the problems of existing video synopsis methods such as incomplete tracks and difficulty to retain interactive behaviors in complex scenes, a segmentation-adaptive video synopsis method is proposed. Firstly, a video segmentation module is proposed, which detects the crowding degree of each frame of the input video, and divides the video into sparse and crowded segmentation using self-adaption threshold, and links the interrupted track to form extending crowded segmentation. Secondly, an interactive behavior judgment module is designed, which combines spatial distance and video self-adaption threshold to comprehensively judge and retain interactive behavior between objects. Finally, a segmentation-adaptive rearrangement module is proposed, which combines collision constraints, space proportion constraint, interactive constraints and temporal constraints to generate the optimal time tag, and fuses the background to generate synopsis video. Experimental results on public dataset VISOR, BEHAVE and CAVIAR show that compared with the current mainstream methods, the proposed method reduces 0.136 and 0.011 frame compression rate and collision rate respectively, and the time cost is reduced by 120.03 s.