Discriminative correlation filter-based (DCF) object-tracking methods exhibit excellent and efficient performance in short-term tracking. However, these methods do not evaluate the credibility of the feature channels and spatial features. In addition, they regard the candidates with the largest response in each frame as the training sample without evaluating the confidence of these samples. As a result, DCF may be misled by distractors and fails in the occlusion scene. We propose a visual attention learning and anti-occlusion-based correlation filter tracking method to solve the abovementioned limitations. First, we propose a correlation filtering framework for simultaneous estimation of position and scale in which we sample the multi-scale detection candidates from the target’s center in the previous frame and calculate the spatial attention of each candidate according to the spatial prior and color likelihood. Then, we design a channel attention calculation algorithm to evaluate the importance of each channel according to the response in the learning and detection stages. Finally, a historical multi-template pool is proposed to further evaluate the confidence of the best candidate in each frame, achieving robust tracking in occlusion scenarios. A series of experiments are conducted to verify the proposed method, indicating that the proposed method achieves excellent results on the OTB50, OTB100, and UAV20L datasets and outperforms many state-of-the-art methods.