Markov random field (MRF) approaches are able to implement better SAR image segmentation by combing the image intensity and structure information. However, this kind of approaches often obtains a global solution at the cost of computational burden. A fast simulated annealing method was presented by combing the prior knowledge of SAR image distribution and applied to the SAR image segmentation based MRF. In the process of segmentation based MRF, the fast method first searched the neighborhood of each pixel to find out whether there was a predominant marker. If yes, the pixel would be marked with this predominant marker in the updated segmentation field; if not, the pixel would be marked randomly as the traditional simulated annealing algorithm does. Because a prior judgment based on Gibbs distribution was introduced, the proposed method is able to obtain a global best solution very quickly. In the end, experiments were carried out on real SAR images and the results validate the feasibility and efficiency of the proposed method.