The grain filling stage is a critical growth phase of rice. To segment the panicle accurately during filling stage and explore the relationship between its characteristics and plant maturation, a method of segmentation and characteristics analysis is proposed based on neural architecture search (NAS). Based on the DeepLabV3Plus network model, the backbone network is automatically designed using NAS, and the semantic segmentation network Rice-DeepLab is built by modifying atrous spatial pyramid pooling (ASPP). The area ratios, dispersion, average curvature, and color characteristics of the panicles of four rice varieties are calculated and analyzed after segmentation by Rice-DeepLab. The experimental results show that the improved Rice-DeepLab network has a mean intersection over union (mIoU) of 85.74% and accuracy (Acc) of 92.61%, which is 6.5% and 2.97% higher than that of the original model, respectively. According to the panicles' area ratios, dispersion, average curvature, and color characteristics recorded in the image, it can be roughly distinguished whether the panicles are sparse or dense, whether grain filling is complete, and whether the color is green, golden, or gray. This study suggests that field cameras can be easily used to monitor rice in the filling stage preliminarily to estimate maturation and crop size by panicle segmentation and characteristics analysis, thus providing support for field management.