Matrix decomposition was used in the early collaborative filtering algorithms in order to solve the problem of data sparsity. But it performed poorly in handling serious sparsity problem and cannot meet the application requirements. Then, transfer learning was introduced into collaboration filtering to deal with the data sparsity in the target domain by utilizing common users’ information in the auxiliary and target domains.Although the introduced auxiliary information would prompt knowledge acquisition in the target domain, these methods only use shallow features to measure the users’ similarity. As a result, these methods could not capture the potential features when the users have only a few common items and would result in poor performance in similarity measurement. In order to address these problems, this paper proposes a collaborative filtering recommendation method based on transfer learning and joint matrix decomposition. In this method, the information of common users and items in the two domains is mapped into a potential semantic space with the information of users as anchors; the user item joint rating matrix of two domains is decomposed with the user information as the constrain. The experiment was performed to validate the proposed method and the method showed superior performance over the state of the art migration learning methods based on similarity calculation on benchmark data set, proving its effectiveness.