In image quality assessment (IQA), the quality of an image is essentially dependent on both semantic contents and image distortions. Conventional nonreference IQA (NR-IQA) methods either encode the semantic contents only or combine them in a straightforward manner without exploring their intrinsic correlation. A unified deep multiview learning network (DMvLNet) is proposed to combine both semantic contents and image distortions for NR-IQA. Using ResNet50 as the backbone, we design IQA-aware semantic features instead of directly using a recognition-specific CNN model for IQA, as in many existing methods. To this end, a high-level feature refining module built on deep layers is developed for generating refined high-level feature maps. Meanwhile, shallow layers are employed for low-level descriptions. Thus, both the refined high-level and low-level features are used for characterizing semantic contents and image distortions, respectively. Furthermore, the refined semantic features and distortion features are simultaneously integrated into DMvLNet by exploring their mutual correlation, leading to the projected multiview embedding passing through the subsequent transformer network for image quality prediction. Extensive experiments conducted on both our assembled large-scale dataset and seven public benchmark datasets demonstrate the superiority of DMvLNet over the state-of-the-art methods.