In recent years, convolutional neural networks have excelled in image Moiré pattern removal, yet their high memory consumption poses challenges for resource-constrained devices. To address this, we propose the lightweight multi-scale network (LMSNet). Designing lightweight multi-scale feature extraction blocks and efficient adaptive channel fusion modules, we extend the receptive field of feature extraction and introduce lightweight convolutional decomposition. LMSNet achieves a balance between parameter numbers and reconstruction performance. Extensive experiments demonstrate that our LMSNet, with 0.77 million parameters, achieves Moiré pattern removal performance comparable to full high definition demoiréing network (FHDe2Net) with 13.57 million parameters.