String matching method and coarse grained sentiment analysis method are often used in sensitive information detection,the accuracy, however, is rather low. To mitigate this issue, this paper proposed a method based on phrase level sentiment analysis. This method takes advantage of a rule set which is used to extract phrases concerning the sensitive word, by analyzing the sentiment orientation of relative phrases, it is possible to determine the sentiment orientation of the expresser towards the sensitive word, thus determine the sensitivity of the information. This method takes the context of the sensitive word into consideration and is able to extract sentiment orientation towards the sensitive word instead of the whole text, which is missing from string matching method and coarse grained method respectively. Experimental results suggest that the accuracy is considerably increased compared to string matching and coarse grained sentiment analysis method.