Acute pancreatitis （AP） is one of the most common digestive disease， while the analysis based on medical images of AP still depends on simple manual features with low efficiency and accuracy， which is not commensurate with AP’s harmfulness. Due to the anatomical variation of pancreas and complications of AP， AP has complex imaging manifestations and large appearance pattern variation of lesions that exist among patients and lesion kinds. It is challenging for diagnosis of acute pancreatitis based on CT images. To address these issues， we propose a dual-attention network for acute pancreatitis diagnosis. Specifically， the dual-attention network utilizes the global feature to generate local attention feature for each local feature on different stages， and final classification is facilitated by the fusion of multi-scale attention features focusing on lesions of different scales. Meanwhile， channel-domain attention is used to produce attention features based on the dependencies between each channel to improve the model’s feature representation ability. We evaluate the proposed method on the collected real acute pancreatitis dataset. Results show that the proposed network achieve superior performance in acute pancreatitis diagnosis compared with several competing methods， with the sensitivity improved by 3.4%. And the improvement of area under the curve （AUC） the proposed network brings to ResNet is 2.7% higher than other attention model such as SENet.