This article introduces a method for detecting ice floes in the Yellow River based on unmanned aerial vehicle (UAV) remote sensing images and deep learning technology. First, the article establishes a semantic segmentation dataset for Yellow River ice floes, filling the current data gap in this field. It designs a data augmentation scheme to improve the algorithm’s overfitting phenomenon. By analyzing the features of ice jam UAV remote sensing images and experimental verification, this article employs the U-Net algorithm as the foundation. By introducing the ConvNeXt-U feature extraction network, Squeeze-and-Excitation Network attention mechanism, and designing a hybrid loss function, the algorithm’s feature extraction ability is enhanced, improving the detection effect of small targets and overall improving the detection metrics of various semantic segmentation algorithms. Experimental results show that the proposed ConvNeXt-SE-U-Net increases accuracy by 16.05%, precision by 38.89%, and mean intersection over union by 30.35%. This method can accomplish high-precision ice floe semantic segmentation tasks in UAV remote sensing images, which is of great significance for Yellow River management.