Abstract: The Gaussian-based conditional restricted Boltzmann machine(GCRBM)time series model could efficiently predict for single type of gait time series data, but the model could not make accurate recognition and prediction for multi-category gait time series data. To solve the problem above, an ensemble/integrated method with convolutional neural network(CNN)and deep belief network(DBN)for gait recognition and simulation was proposed. Multiple CNNs models with different structures were trained by all the gait data. Multiple DBNs models corresponding to the multi-category data were trained to study low dimensional features, and corresponding to train multiple GCRBMs models through the low dimensional features. In the step of recognition and simulation, model will identify the class of gait data with all CNNs classifiers by the “minority-obeying” voting strategy, then the low-dimensional feature of the DBNs model corresponding to the identified class was used as the input of the corresponding GCRBMs model to predict the late timing low-dimensional feature of the target data. The gait images could be reconstructed by the corresponding DBNs model. Compared with the method of support vector machine(SVM), integrated DBN and CNN, the proposed method’s gait recognition rate was improved based on CASIA gait datasets. Moreover, the predicting result could be simulated to the true gait sequences by the proposed method, which demonstrated the validity of the model.

Key words: gait recognition and simulation, convolutional neural network(CNN), deep belief networks(DBN), time series model, Gaussian-based conditional restricted Boltzmann machine(GCRBM)