Table of Content

    20 April 2021
    Volume 51 Issue 2
    Machine Learning & Data Mining
    Eye tracking in human-computer interaction control
    Hui HE,Junhao HUANG
    Journal of Shandong University(Engineering Science). 2021, 51(2):  1-8.  doi:10.6040/j.issn.1672-3961.0.2020.346
    Abstract ( 439 )   HTML ( 46 )   PDF (4362KB) ( 189 )   Save
    Figures and Tables | References | Related Articles | Metrics

    To actualize the simple and low-cost eye-tracking based human-computer interaction, an exact interaction method based on the visual directions estimation and eye tracking with webcam videos was proposed. A simple and fast convolution neural network model was used to roughly estimate the user′s viewpoints on the screen. And then an accurate human-computer interaction method was proposed on the basis of the eye movements recognition and sight line tracking results. To verify the effectiveness of the method, the key operations of eye mouse and eye typing were developed. The test results show that the proposed method enabled users to achieve eye tracking and to actualize most precise human-computer interactions with only one common monocular camera, which was expected to completely replace the mouse and keyboard hardwares.

    MIRGAN: a medical image report generation model based on GAN
    Junsan ZHANG,Qiaoqiao CHENG,Yao WAN,Jie ZHU,Shidong ZHANG
    Journal of Shandong University(Engineering Science). 2021, 51(2):  9-18.  doi:10.6040/j.issn.1672-3961.0.2020.227
    Abstract ( 321 )   HTML ( 41 )   PDF (2295KB) ( 108 )   Save
    Figures and Tables | References | Related Articles | Metrics

    The medical image report generation task based on image understanding became a widely concerned issue. Compared with the traditional image understanding task, medical image report generation was a more challenging task. We proposed a medical image report generative adversarial network (MIRGAN) model for this task. A co-attention mechanism was adopted to synthesize the visual and semantic features of multiple feature areas and generate descriptions corresponding to these areas. Combining the generative adversarial networks (GAN) and reinforcement learning (RL) optimized the performance of the generative model to output higher quality reports. The experiment results demonstrated the effectiveness of our proposed MIRGAN model.

    Real-time semantic segmentation of high-resolution remote sensing image based on multi-level feature cascade
    Chunhong CAO,Hongxuan DUAN,Ling CAO,Lele ZHANG,Kai HU,Fen XIAO
    Journal of Shandong University(Engineering Science). 2021, 51(2):  19-25.  doi:10.6040/j.issn.1672-3961.0.2020.225
    Abstract ( 291 )   HTML ( 31 )   PDF (4639KB) ( 129 )   Save
    Figures and Tables | References | Related Articles | Metrics

    Aiming at the problems of long segmentation time and inaccurate segmentation of small targets in remote sensing image semantic segmentation, a fast semantic segmentation model of high-resolution remote sensing image based on multi-level feature cascade network (MFCNet) was proposed. The model was mainly composed of feature encoding, feature fusion and target refinement. Feature encoding extracted the input images feature of different resolutions and used different backbone networks. Due to the lower resolution of low-resolution images, heavy-weight backbone networks were used to obtain rich semantic information with fewer parameters. For medium and high-resolution images, lightweight backbone network was used to reduce the amount of parameters and obtain global information. While medium and low-resolution encoding used the way of weights and calculation sharing to further reduce model parameters and computational complexity. The feature fusion section fused features from different branches to obtain information at different scales. The target refinement used residual to correction the fused features and the features of the coded part to restore the spatial detail information of the image, making the segmentation more accurate. And the entire model worked efficiently in an end-to-end manner. The experimental verified the validity of the model in semantic segmentation of remote sensing images, and achieved a good balance between model complexity and accuracy.

    Construction of knowledge graph of relationship between LncRNA and diseases
    GONG Lejun, YANG Lu, GAO Zhihong, LI Huakang
    Journal of Shandong University(Engineering Science). 2021, 51(2):  26-33.  doi:10.6040/j.issn.1672-3961.0.2020.212
    Abstract ( 228 )   PDF (4656KB) ( 119 )   Save
    References | Related Articles | Metrics
    Based on the analysis of the relationship between LncRNA and diseases, the concept of knowledge modeling of LncRNA and diseases was proposed, and an effective method of knowledge mapping of the relationship between LncRNA and diseases was proposed. Protege was used to construct the ontology structure and the concept layer, integrate the structured and unstructured data from two different sources to the data layer, and describe the data and the corresponding relationship through RDF/OWL technology. The production rules based on forward reasoning were used to carry out the corresponding knowledge reasoning. The inference effect of knowledge query was demonstrated by SPARQL query language and visualization technology. This study provided further reference value for the study of the relationship between LncRNA and diseases. Moreover, it also promoted the development of this field.
    Global and local multi-view multi-label learning with active three-way clustering
    ZHU Changming, YUE Wen, WANG Panhong, SHEN Zhenyu, ZHOU Rigui
    Journal of Shandong University(Engineering Science). 2021, 51(2):  34-46.  doi:10.6040/j.issn.1672-3961.0.2020.234
    Abstract ( 177 )   PDF (1030KB) ( 94 )   Save
    References | Related Articles | Metrics
    In order to consider the uncertain belongingness relationship between instances and clusters and then extend the application scopes of global and local multi-view multi-label learning, an algorithm of global and local multi-view multi-label learning machine with active three-way clustering(GLMVML-ATC)was proposed. With the usage of active three-way clustering strategy, the belongingness of instances to a cluster depended on the probabilities of uncertain instances belonging to core regions. This made local label correlations more authentic, which enhanced the performances of multi-view multi-label learning machines further and accelerated their development. Experimental results validated that GLMVML-ATC improved the classification performances with 3% at least, while the added training time less than 7%. It was superior to the classical multi-view learning machines and multi-label learning machines.
    Adaptive harmony search algorithm based on global optimization
    ZHOU Kaiqing, LI Hangcheng, MO Liping
    Journal of Shandong University(Engineering Science). 2021, 51(2):  47-56.  doi:10.6040/j.issn.1672-3961.0.2020.395
    Abstract ( 167 )   PDF (7849KB) ( 91 )   Save
    References | Related Articles | Metrics
    An adaptive harmony search algorithm utilizing global optimal mechanism (AGOHS) was proposed to overcome the drawbacks of harmony search (HS) algorithm, such as slow convergence speed and low search accuracy. The modifications of AGOHS was classified into the following aspects. In the improvisation phase, the bandwidth (BW) was represented by the difference between the optimal harmony variable and the worst harmony variable in the current harmony, so that the BW had the ability to adapt to specific situations, and saved a random harmony variable in the optimal harmony every time. A novel harmony variable was generated randomly by using the intrapopulation difference while the obtained random number was greater than the reconciliation probability of harmony memory storage. To improve the search ability and the robustness, a novel harmony was randomly generated from the minimum value to the maximum value of harmony in the current population. The best harmony with the smallest error among the gained harmonies in this phase was selected and used to update the harmony memory. The proposed algorithm was compared with three improved harmony search algorithms on 13 test functions, experimental results revealed that the AGOHS had better global search capability and convergence speed.
    Community detection using nonnegative matrix factorization with structure extension
    LIN Xiaowei, CHEN Lifei
    Journal of Shandong University(Engineering Science). 2021, 51(2):  57-64.  doi:10.6040/j.issn.1672-3961.0.2020.231
    Abstract ( 178 )   PDF (3332KB) ( 100 )   Save
    References | Related Articles | Metrics
    Nonnegative matrix factorization with structure extension(NMF-SE)was proposed to enhance the structural similarity of adjacent nodes and increase the density of connections between nodes, which improved the performance of nonnegative matrix factorization in community detection. In the process of structure extension, nodes transmitted their own structure to the surrounding nodes in a certain proportion, so that adjacent nodes could get the topology information from each other. This process constructed a new feature matrix, which made nonnegative matrix factorization(NMF)more suitable for community detection. Meanwhile, in the semi-supervised task with graph regularization, the prior information could be better incorporated. The experimental results on synthetic network and real network showed that NMF-SE algorithm effectively improved the accuracy of community detection in complex network.
    Active learning of pairwise constraints in block diagonal subspace clustering
    XIE Ziqi, WANG Lihong, LI Man
    Journal of Shandong University(Engineering Science). 2021, 51(2):  65-73.  doi:10.6040/j.issn.1672-3961.0.2020.182
    Abstract ( 219 )   PDF (2258KB) ( 105 )   Save
    References | Related Articles | Metrics
    Focusing on the poor performance of subspace clustering by block diagonal representation(BDR)on high-dimensional data with overlapped subspaces, an active learning strategy was designed to obtain partial pairwise information among a few data points. A pairwise constrained block diagonal representation algorithm(CBDR)was proposed to improve the performance of the BDR algorithm. The objective function and solution process of the CBDR were given. The experimental results on the test datasets showed that the CBDR algorithm reduced the clustering error by more than 5% with less than 5‰ constraint information in terms of clustering error and normalized mutual information, which significantly outperformed the compared algorithms, i.e., BDR, SBDR(structured block diagonal representation)with random selection of pairwise constraints.
    Application of Gossip authentication algorithm based on punishment in IOV
    HUANG Qimeng, LIU Zhaowei, DU Zhenbin
    Journal of Shandong University(Engineering Science). 2021, 51(2):  74-82.  doi:10.6040/j.issn.1672-3961.0.2020.250
    Abstract ( 168 )   PDF (2309KB) ( 100 )   Save
    References | Related Articles | Metrics
    Combining blockchain technology and internet of vehicles technology, a Gossip authentication algorithm based on a penalty mechanism was proposed. This method adopted a window protection mechanism to control the number of nodes in the network and avoid the destruction of consensus information due to network channel blockage. The Gossip protocol was used to ensure the efficient dissemination of information, and a penalty mechanism was proposed to reduce the number of malicious nodes in the consensus process. Algorithm analysis and experimental results showed that this method could improve the consensus efficiency of nodes while ensuring communication security, and effectively compensated the defects of identity authentication in the internet of vehicles.
    Aspect-level sentiment classification combined with syntactic dependency information
    ZHANG Qinyang, LI Xu, YAO Chunlong, LI Changwu
    Journal of Shandong University(Engineering Science). 2021, 51(2):  83-89.  doi:10.6040/j.issn.1672-3961.0.2020.246
    Abstract ( 207 )   PDF (1130KB) ( 104 )   Save
    References | Related Articles | Metrics
    Considering introducing syntactic dependency information into the original aspect terms, a new aspect term representation method was proposed. First Glove word vector was used to represent the words and dependency relationship between words, and the dependency adjacency matrix and the representation of dependency relationship matrix including syntactic dependency information was constructed. Then graph convolution neural network and multi-head attention mechanism were used to integrate syntactic dependency information into aspect terms, so that aspect terms were highly related to context structure. The models generalization ability were effectively improved by replacing the existing models with the improved aspect term expression. Through comparative experiments and analysis, effectiveness and generalization of the method were proved.
    Model and application of short-term electricity consumption forecast based on C-LSTM
    LIAO Jinping, MO Yuchang, YAN Ke
    Journal of Shandong University(Engineering Science). 2021, 51(2):  90-97.  doi:10.6040/j.issn.1672-3961.0.2020.226
    Abstract ( 242 )   PDF (3154KB) ( 96 )   Save
    References | Related Articles | Metrics
    Research on short-term household electricity consumption prediction based on long-term short-term memory recurrent neural network under deep learning. This research introduced a hybrid deep neural network model C-LSTM that combines convolutional neural network(CNN)and long short term memory(LSTM)models, and proposed a multi-step prediction strategy based on this model. According to the research on the daily electricity consumption data set of 5 real households, C-LSTM realized household electricity demand forecasting in 5 min. Through continuous modification of model parameters and improvement of the model, from the analysis of the three error indicators provided in this study, the prediction accuracy of C-LSTM was higher than the autoregressive integrated moving average model, support vector regression model and LSTM model. The main basis for the evaluation of the model prediction effect in this study was the average absolute percentage error value. From the test results, it could be obtained that the C-LSTM model's household electricity demand forecast in 5 minutes was 4.63% higher than the support vector regression model, 22.8% higher than the LSTM, and 34.74% higher than the autoregressive integrated moving average model. Therefore, the C-LSTM model provided a guarantee for the smart grid's accurate and timely prediction of household-level electricity demand, and had an important impact on promoting the widespread popularity of personalized electricity packages and reducing energy waste.
    Shadow occlusion diagnosis of distributed photovoltaic power station based on random forest and expert system
    LIU Xinfeng, ZHANG YiNi, XU Huisan, SONG Ling, CHEN Mengya
    Journal of Shandong University(Engineering Science). 2021, 51(2):  98-104.  doi:10.6040/j.issn.1672-3961.0.2020.404
    Abstract ( 170 )   PDF (1515KB) ( 68 )   Save
    References | Related Articles | Metrics
    A human-machine collaborative discriminant method based on the random forest algorithm was proposed to diagnose distributed photovoltaic shadow occlusion. Key characteristic parameters, such as the current dispersion rate on the direct current side of the string, solar altitude angle, solar azimuth angle, and instantaneous power generation level of the power station, were constructed based on the analysis of the shadow occlusion mechanism and the conversion of inverter telemetry parameters. The random forest shadow occlusion diagnosis model was subsequently established. The parameters were optimized based on the grid search method and the K-fold cross-validation method, and the splitting method based on information gain was determined by comparing the accuracy with other machine learning algorithms, such as support vector machine, logistic regression, and decision tree. The random forest algorithm had obvious advantages in shadow occlusion diagnosis scenes. An expert system was combined to obtain the diagnosis position, and then the effectiveness of the method using the random forest algorithm based on information gain and an expert system was verified on site.
    Dynamic prediction of spatiotemporal big data based on relationship transfer and reinforcement learning
    ZHENG Zijun, FENG Xiang, YU Huiqun, LI Xiuquan
    Journal of Shandong University(Engineering Science). 2021, 51(2):  105-114.  doi:10.6040/j.issn.1672-3961.0.2020.233
    Abstract ( 251 )   PDF (4402KB) ( 45 )   Save
    References | Related Articles | Metrics
    A dynamic prediction algorithm based on relationship transfer and reinforcement learning was proposed to alleviate the problem that the dynamic prediction in a large spatiotemporal range fails to obtain an accurate solution. The algorithm adopted a crowd intelligence computing manner with complex workflow models to solve the spatiotemporal data optimization problem. A relationship transfer block was designed to learn the probability of relationship transfer by extracting features from spatiotemporal data. A prediction reinforcement learning block was established along with the time series to process the transition relationship probability in parallel and prioritize the spatiotemporal data according to feature preferences that predict the problem status trend. A deep multi-step iterative strategy optimization was adopted to obtain a reasonable solution. Theoretical analysis and discussion of the convergence and convergence rate of the proposed algorithm were conducted. Experimental results on patent transfer data verified this approach's strengths and demonstrated that the ranking accuracy could be significantly improved by applying the relationship transfer block and prediction reinforcement learning block.
    Script identification of Central Asian document images based on LTP and HOG texture feature fusion
    WU Zhengjian, MUTALLIP Mamut, HORNISA Mamat, ALIM Aysa, KURBAN Ubul
    Journal of Shandong University(Engineering Science). 2021, 51(2):  115-121.  doi:10.6040/j.issn.1672-3961.0.2020.348
    Abstract ( 161 )   PDF (4746KB) ( 52 )   Save
    References | Related Articles | Metrics
    Due to the existence of a number of scripts with high similarity in Central Asia, a document image script identification method based on the cross-fusion of a unified local ternary pattern(riu2-LTP)with rotational invariance and histogram of oriented gradients(HOG)features was proposed. An SVM classifier was used to perform experiments on a database containing a total of 10 000 images of 10 scripts. In order to improve multi-script identification, Bayesian optimized SVM hyperparameters were used. The method first extracted riu2-LTP with a radius of and a sampling 8 points for the document images; HOG was extracted from the database again; the cross-fusion method was to incorporate the 20-dimensional riu2-LTP features and 36-dimensional HOG features sequentially into the new feature set, respectively. The experiments showed that the average recognition rate of this method reached 99%, which was better than the single LTP, riu2-LTP, and HOG methods.
    Incremental high utility pattern mining method based on compact utility list
    ZHANG Chunyan, HAN Meng, SUN Rui, DU Shiyu, SHEN Mingyao
    Journal of Shandong University(Engineering Science). 2021, 51(2):  122-128.  doi:10.6040/j.issn.1672-3961.0.2020.228
    Abstract ( 110 )   PDF (2530KB) ( 80 )   Save
    References | Related Articles | Metrics
    Aiming at the problem of large amounts of redundant data, a compact incremental high utility mining algorithm was proposed. The HUI-trie structure and a compact utility list were used. The former was used to update the utility of the high utility itemsets, and the latter was used to store information without generating any candidates. These two structures enabled the algorithm to reflect the increased data into the previous analysis results without reanalyzing the entire data set, and processed incremental data sets more effectively. The test results showed that the algorithm had an average increase of 38% in running time and an average reduction in memory of 32% on various data sets, and it had certain scalability.