In text classification tasks, effectively extracting text features while improving computational efficiency is a critical challenge. However, traditional methods often struggle to balance feature richness and computational efficiency. To address this issue, this paper proposes a novel text classification model, Linear Attention Text Classification by Combining Text Features and Word Frequency Implicit Factors (LTTW), which incorporates word frequency implicit factors and textual features, and introduces a linear attention mechanism to capture key features in the text. Specifically, the model leverages Non-negative Matrix Factorization (NMF) to extract word frequency implicit factors from the term frequency matrix, capturing latent semantic information. Simultaneously, it utilizes pre-trained models to extract semantic features of the text, which are then fused with the word frequency implicit factors to construct a richer text representation. Based on this representation, the linear attention mechanism is applied to effectively capture global dependencies and enhance the processing efficiency of long text sequences. Experiments conducted on public datasets demonstrate that the proposed model outperforms mainstream methods in terms of both accuracy and computational efficiency, with particularly significant efficiency advantages when handling long sequences. The study highlights that the integration of word frequency implicit factors complements the shortcomings of pre-trained models in semantic feature extraction, while the linear attention mechanism effectively captures key textual features and improves sequence processing efficiency. Together, these contributions significantly enhance the performance and efficiency of text classification.