Terrain adaptive ability is the basis for the stable movement of agents under complex terrain conditions. Due to the complexity of the dynamical systems of these agents， such as humanoid robots， it is usually difficult for traditional inverse dynamics methods to have such ability. Recent research has used the advantages of reinforcement learning in solving sequential decision-making problems to train agents to adapt to terrain. However， these single-task learning methods cannot effectively learn the correlation in various terrains. In fact， complex terrain adaptive tasks can be considered as a multi-task problem， and the relationship between sub-tasks can be measured by different terrain factors. And then， the problem of incomplete acquisition of data distribution information can be solved by mutual learning of sub-task models. Therefore， this paper proposes a multi-task reinforcement learning method. It contains an execution layer which is consist of pre-trained subtask models and a decision layer based on reinforcement learning method. Moreover， the decision layer uses soft constraints to fuse models of the execution layer. Experiments on LeggedGym terrain simulator prove that the agent trained by the method in this paper is more stable in movement and has fewer falls down on complex terrains， showing better generalization performance.