Pixel-based processing method mainly extracts spectral information from hyperspectral remote sensing images, but site specific management zone (SSMZ) delineation and crop yield estimation with images need to take spatiotemporal heterogeneity into account. As the spatial resolution of remote sensing data increases, the so-called “salt-and-pepper” problem of pixel-based classification becomes more serious. The spatiotemporal heterogeneity of soil properties and crop biophysical parameters are mainly delineated with grid sampling and geostatistics interpolation, but the widely used method has some problems: time consuming and high cost. Satellite imageries are introduced to delineate SSMZ, but there are also problems needed to be resolved: (1) single date imagery is used to map SSMZ which is difficult to determine the optimal date for SSMZ delineation; (2) only few SSMZs were mapped, which limited application of site specific fertilizing and management; (3) pixel-based method for SSMZ delineation didn't concern the spatial relationship between pixels and site specific management does not implement at pixel level, but at SSMZ level. To improve the accuracy of crop yield estimation, a time-series of hyperspectral airborne images with high spatial resolution (1 m) of a cotton field, which is located in San Joaquin Valley, California US, were acquired and classified by using object-oriented segmentation, then yield predicting models were built, and the accuracy and stability of yield models were validated with determining coefficients R2 and the root mean square error (RMSE). Results are as follows: (1) object-oriented SSMZ delineating method combines spectral, spatial and temporal information, reduces noises in images and yield data, improves the accuracy of yield prediction; (2) for same SSMZ number, first derivative predicting model is more accurate; (3) for same spectral input, models with fewer SSMZs show higher accuracy, which is due to spatial errors of airborne images and yield data. The results will improve monitoring methods for crop growth and yield while accelerate the application of UAV remote sensing in precision agriculture.