Abstract: In order to improve the matching speed and the robustness to initial positioning error and noise of the scene matching system, the scene matching model was set by analyzing the relationship of gray level between real-time image and referenced image. An improved least-squares scene matching algorithm was proposed. The generalized cost function in the algorithm was constructed by adding an auxiliary constraint term to the sum of the squared errors. The auxiliary constraint term involved the requirement for the smoothness of measurement input to improve the stability of the algorithm. The recursive equations of the algorithm were derived using Newton iterative algorithm without any simplification. By using the first order and second order derivative information of the generalized cost function, the algorithm had high convergence speed. Simulation results demonstrate the effectiveness of the algorithm.