Abstract: The Doppler spectrum of sea surface echo from High Frequency Surface Wave Radar (HFSWR) contains very rich wave information. In order to solve the problem of inadequate use of echo spectrum information in empirical models, a method of wave height inversion using convolutional neural network is proposed in this paper. In this paper, based on the radar backscattering cross section equation and the parametric empirical model of Significant Wave Height (SWH) inversion, combined with the analysis of measured data, 11 kinds of echo spectral characteristic parameters related to SHW are selected, including the ratio of second-order to first-order spectral energy, the sum of second-order spectral energy, the ratio of second-order to first-order peak power, the undirected peak power at the Bragg frequency, the second-order spectral energy, the second-order peak power, the left and right first-order peak power and their ratios, the peak power at the 1/ \sqrt2 Bragg frequency, and the first-order spectral energy. Secondly, in order to further explain the role of first and second order information on SWH, the 11 characteristic parameters are divided into 4 combinations, and multi-layer deep convolutional neural networks are built to retrieve the SWH. Then, the inversion results of the multi-layer deep convolutional neural network model and the empirical model are compared and analyzed in high and low sea states. The results show that the inversion accuracy of SWH based on 11 characteristic parameters is higher. In the high and low sea state test sets, the correlation coefficients (R) of SWH retrieved by radar and by buoy are 0.92 and 0.78, respectively, and the Root Mean Square Error (RMSE) is 0.32 m and 0.21 m, respectively. The Mean Absolute Error (MAE) is 0.25 m and 0.16 m, and the Mean Relative Error (MRE) is 0.12 and 0.27, respectively. The comprehensive use of these 11 characteristic parameters can improve the inversion accuracy of significant wave height under complex sea conditions.