# Notebook from HUANGZHIHAO1994/Social-network-analysis
Path: similar/similar_results/ensemble-Copy1.ipynb

<code>
import os

path = os.getcwd()
fileList = os.listdir(path)
try:
    fileList.remove("ensemble.ipynb")
except:
    pass
try:
    fileList.remove('.ipynb_checkpoints')
except:
    pass
print(fileList)
['glove_spearman.txt', 'glove_apsynp.txt', 'word2vec_kendall.txt', 'word2vec_apsyn.txt', 'ensemble-Copy1.ipynb', 'word2vec_pearson.txt', 'word2vec_apsynp.txt', 'glove_avg_cosine.txt', 'word2vec_avg_cosine.txt', 'word2vec_spearman.txt', 'glove_kendall.txt', 'glove_pearson.txt', 'glove_apsyn.txt']
set_list = []
for txt in fileList:
    s = set()
    with open(txt, encoding="utf-8") as f:
        conclusions = f.read().split('\n')[:-1]
    print(conclusions[0:5])
    for conclusion in conclusions[:50]:
        try:
            s.add(conclusion.split('\t')[1])
        except:
            pass
#     print(len(s))
    set_list.append(s)

            ['1.\tCompany disclosures greatly aid in the process of financial decision-making; therefore, they are consulted by financial investors and automated traders before exercising ownership in stocks. While humans are usually able to correctly interpret the content, the same is rarely true of computerized decision support systems, which struggle with the complexity and ambiguity of natural language. A possible remedy is represented by deep learning, which overcomes several shortcomings of traditional methods of text mining. For instance, recurrent neural networks, such as long short-term memories, employ hierarchical structures, together with a large number of hidden layers, to automatically extract features from ordered sequences of words and capture highly non-linear relationships such as context-dependent meanings. However, deep learning has only recently started to receive traction, possibly because its performance is largely untested. Hence, this paper studies the use of deep neural networks for financial decision support. We additionally experiment with transfer learning, in which we pre-train the network on a different corpus with a length of 139.1 million words. Our results reveal a higher directional accuracy as compared to traditional machine learning when predicting stock price movements in response to financial disclosures. Our work thereby helps to highlight the business value of deep learning and provides recommendations to practitioners and executives. (C) 2017 Elsevier B.V. All rights reserved.\t1.0', "2.\tPredicting the final closing price of a stock is a challenging task and even modest improvements in predictive outcome can be very profitable. Many computer-aided techniques based on either machine learning or statistical models have been adopted to estimate price changes in the stock market. One of the major challenges with traditional machine learning models is the feature extraction process. Indeed, extracting relevant features from data and identifying hidden nonlinear relationships without relying on econometric assumptions and human expertise is extremely complex and makes deep learning particularly attractive. In this paper, we propose a deep neural network-based approach to predict if the stock price will increase by 25% for the following year, same quarter or not. We also compare our deep learning method against 'shallow' approaches, random forest and gradient boosted machines. To test the proposed methods, KIS-VALUE database consisting of the Korea Composite Stock Price Index (KOSPI) of companies for the period 2007 to 2015 was considered. All the methods yielded satisfactory performance, namely, deep neural network achieved an AUC of 0.806. 'Shallow' approaches, random forest and gradient boosted machines have been used for comparisons.\t0.8960437338192647", "3.\tIn today's financial markets, where most trades are performed in their entirety by electronic means and the largest fraction of them is completely automated, an opportunity has risen from analyzing this vast amount of transactions. Since all the transactions are recorded in great detail, investors can analyze all the generated data and detect repeated patterns of the price movements. Being able to detect them in advance, allows them to take profitable positions or avoid anomalous events in the financial markets. In this work we proposed a deep learning methodology, based on Convolutional Neural Networks (CNNs), that predicts the price movements of stocks, using as input large-scale, high-frequency time-series derived from the order book of financial exchanges. The dataset that we use contains more than 4 million limit order events and our comparison with other methods, like Multilayer Neural Networks and Support Vector Machines, shows that CNNs are better suited for this kind of task.\t0.8958628429204768", "4.\tNeural networks (NN) are considered as black boxes due to the lack of explainability and transparency of their decisions. This significantly hampers their deployment in environments where explainability is essential along with the accuracy of the system. Recently, significant efforts have been made for the interpretability of these deep networks with the aim to open up the black box. However, most of these approaches are specifically developed for visual modalities. In addition, the interpretations provided by these systems require expert knowledge and understanding for intelligibility. This indicates a vital gap between the explainability provided by the systems and the novice user. To bridge this gap, we present a novel framework i.e. Time-Series eXplanation (TSXplain) system which produces a natural language based explanation of the decision taken by a NN. It uses the extracted statistical features to describe the decision of a NN, merging the deep learning world with that of statistics. The two-level explanation provides ample description of the decision made by the network to aid an expert as well as a novice user alike. Our survey and reliability assessment test confirm that the generated explanations are meaningful and correct. We believe that generating natural language based descriptions of the network's decisions is a big step towards opening up the black box.\t0.8927241413793485", '5.\tFinancial markets forecasting represents a challenging task for a series of reasons, such as the irregularity, high fluctuation, noise of the involved data, and the peculiar high unpredictability of the financial domain. Moreover, literature does not offer a proper methodology to systematically identify intrinsic and hyper-parameters, input features, and base algorithms of a forecasting strategy in order to automatically adapt itself to the chosen market. To tackle these issues, this paper introduces a fully automated optimized ensemble approach, where an optimized feature selection process has been combined with an automatic ensemble machine learning strategy, created by a set of classifiers with intrinsic and hyper-parameters learned in each marked under consideration. A series of experiments performed on different real-world futures markets demonstrate the effectiveness of such an approach with regard to both to the Buy and Hold baseline strategy and to several canonical state-of-the-art solutions.\t0.8906383404260048']
['1.\tCompany disclosures greatly aid in the process of financial decision-making; therefore, they are consulted by financial investors and automated traders before exercising ownership in stocks. While humans are usually able to correctly interpret the content, the same is rarely true of computerized decision support systems, which struggle with the complexity and ambiguity of natural language. A possible remedy is represented by deep learning, which overcomes several shortcomings of traditional methods of text mining. For instance, recurrent neural networks, such as long short-term memories, employ hierarchical structures, together with a large number of hidden layers, to automatically extract features from ordered sequences of words and capture highly non-linear relationships such as context-dependent meanings. However, deep learning has only recently started to receive traction, possibly because its performance is largely untested. Hence, this paper studies the use of deep neural networks for financial decision support. We additionally experiment with transfer learning, in which we pre-train the network on a different corpus with a length of 139.1 million words. Our results reveal a higher directional accuracy as compared to traditional machine learning when predicting stock price movements in response to financial disclosures. Our work thereby helps to highlight the business value of deep learning and provides recommendations to practitioners and executives. (C) 2017 Elsevier B.V. All rights reserved.\t188.11679275757453', "2.\tIn today's financial markets, where most trades are performed in their entirety by electronic means and the largest fraction of them is completely automated, an opportunity has risen from analyzing this vast amount of transactions. Since all the transactions are recorded in great detail, investors can analyze all the generated data and detect repeated patterns of the price movements. Being able to detect them in advance, allows them to take profitable positions or avoid anomalous events in the financial markets. In this work we proposed a deep learning methodology, based on Convolutional Neural Networks (CNNs), that predicts the price movements of stocks, using as input large-scale, high-frequency time-series derived from the order book of financial exchanges. The dataset that we use contains more than 4 million limit order events and our comparison with other methods, like Multilayer Neural Networks and Support Vector Machines, shows that CNNs are better suited for this kind of task.\t188.04374245235118", "3.\tPredicting the final closing price of a stock is a challenging task and even modest improvements in predictive outcome can be very profitable. Many computer-aided techniques based on either machine learning or statistical models have been adopted to estimate price changes in the stock market. One of the major challenges with traditional machine learning models is the feature extraction process. Indeed, extracting relevant features from data and identifying hidden nonlinear relationships without relying on econometric assumptions and human expertise is extremely complex and makes deep learning particularly attractive. In this paper, we propose a deep neural network-based approach to predict if the stock price will increase by 25% for the following year, same quarter or not. We also compare our deep learning method against 'shallow' approaches, random forest and gradient boosted machines. To test the proposed methods, KIS-VALUE database consisting of the Korea Composite Stock Price Index (KOSPI) of companies for the period 2007 to 2015 was considered. All the methods yielded satisfactory performance, namely, deep neural network achieved an AUC of 0.806. 'Shallow' approaches, random forest and gradient boosted machines have been used for comparisons.\t188.04236307691187", '4.\tEmotions widely affect human decision-making. This fact is taken into account by affective computing with the goal of tailoring decision support to the emotional states of individuals. However, the accurate recognition of emotions within narrative documents presents a challenging undertaking due to the complexity and ambiguity of language. Performance improvements can be achieved through deep learning; yet, as demonstrated in this paper, the specific nature of this task requires the customization of recurrent neural networks with regard to bidirectional processing, dropout layers as a means of regularization, and weighted loss functions. In addition, we propose sent2affect, a tailored form of transfer learning for affective computing: here the network is pre-trained for a different task (i.e. sentiment analysis), while the output layer is subsequently tuned to the task of emotion recognition. The resulting performance is evaluated in a holistic setting across 6 benchmark datasets, where we find that both recurrent neural networks and transfer learning consistently outperform traditional machine learning. Altogether, the findings have considerable implications for the use of affective computing.\t188.03981522267694', '5.\tDue to the extensive practical value of time series prediction, many excellent algorithms have been proposed. Most of these methods are developed assuming that massive labeled training data are available. However, this assumption might be invalid in some actual situations. To address this limitation, a transfer learning framework with deep architectures is proposed. Since convolutional neural network (CNN) owns favorable feature extraction capability and can implement parallelization more easily, we propose a deep transfer learning method resorting to the architecture of CNN, termed as DTr-CNN for short. It can effectively alleviate the available labeled data absence and leverage useful knowledge to the current prediction. Notably, in our method, transfer learning process is implemented across different datasets. For a given target domain, in real-world scenarios, relativity of truly available potential source datasets may not be obvious, which is challenging and rarely referred to in most existing time series prediction methods. Aiming at this problem, the incorporation of Dynamic Time Warping (DTW) and Jensen-Shannon (JS) divergence is adopted for the selection of the appropriate source domain. Effectiveness of the proposed method is empirically underpinned by the experiments conducted on one group of synthetic and two groups of practical datasets. Besides, an additional experiment on NN5 dataset is conducted. (C) 2020 Elsevier Ltd. All rights reserved.\t188.0385407755412']
['1.\tCompany disclosures greatly aid in the process of financial decision-making; therefore, they are consulted by financial investors and automated traders before exercising ownership in stocks. While humans are usually able to correctly interpret the content, the same is rarely true of computerized decision support systems, which struggle with the complexity and ambiguity of natural language. A possible remedy is represented by deep learning, which overcomes several shortcomings of traditional methods of text mining. For instance, recurrent neural networks, such as long short-term memories, employ hierarchical structures, together with a large number of hidden layers, to automatically extract features from ordered sequences of words and capture highly non-linear relationships such as context-dependent meanings. However, deep learning has only recently started to receive traction, possibly because its performance is largely untested. Hence, this paper studies the use of deep neural networks for financial decision support. We additionally experiment with transfer learning, in which we pre-train the network on a different corpus with a length of 139.1 million words. Our results reveal a higher directional accuracy as compared to traditional machine learning when predicting stock price movements in response to financial disclosures. Our work thereby helps to highlight the business value of deep learning and provides recommendations to practitioners and executives. (C) 2017 Elsevier B.V. All rights reserved.\t1.0', '2.\tVision-based person re-identification aims to match a persons identity across multiple images, which is a fundamental task in multimedia content analysis and retrieval. Deep neural networks have recently manifested great potential in this task. However, a major bottleneck of existing supervised deep networks is their reliance on a large amount of annotated training data. Manual labeling for person identities in large-scale surveillance camera systems is quite challenging and incurs significant costs. Some recent studies adopt generative model outputs as training data augmentation. To more effectively use these synthetic data for an improved feature learning and re-identification performance, this paper proposes a novel feature affinity-based pseudo labeling method with two possible label encodings. To the best of our knowledge, this is the first study that employs pseudo-labeling by measuring the affinity of unlabeled samples with the underlying clusters of labeled data samples using the intermediate feature representations from deep networks. We propose training the network with the joint supervision of cross-entropy loss together with a center regularization term, which not only ensures discriminative feature representation learning but also simultaneously predicts pseudo-labels for unlabeled data. We show that both label encodings can be learned in a unified manner and help improve the overall performance. Our extensive experiments on three person re-identification datasets: Market-1501, DukeMTMC-reID, and CUHK03, demonstrate significant performance boost over the state-of-the-art person re-identification approaches.\t0.786711259754738', '3.\tGradient descent optimization of learning has become a paradigm for training deep convolutional neural networks (DCNN). However, utilizing other learning strategies in the training process of the DCNN has rarely been explored by the deep learning (DL) community. This serves as the motivation to introduce a non-iterative learning strategy to retrain neurons at the top dense or fully connected (FC) layers of DCNN, resulting in, higher performance. The proposed method exploits the Moore-Penrose Inverse to pull back the current residual error to each FC layer, generating well-generalized features. Further, the weights of each FC layers are recomputed according to the Moore-Penrose Inverse. We evaluate the proposed approach on six most widely accepted object recognition benchmark datasets: Scene-15, CIFAR-10, CIFAR-100, SUN-397, Places365, and ImageNet. The experimental results show that the proposed method obtains improvements over 30 state-of-the-art methods. Interestingly, it also indicates that any DCNN with the proposed method can provide better performance than the same network with its original Backpropagation (BP)-based training.\t0.7866666666666667', "4.\tNeural networks (NN) are considered as black boxes due to the lack of explainability and transparency of their decisions. This significantly hampers their deployment in environments where explainability is essential along with the accuracy of the system. Recently, significant efforts have been made for the interpretability of these deep networks with the aim to open up the black box. However, most of these approaches are specifically developed for visual modalities. In addition, the interpretations provided by these systems require expert knowledge and understanding for intelligibility. This indicates a vital gap between the explainability provided by the systems and the novice user. To bridge this gap, we present a novel framework i.e. Time-Series eXplanation (TSXplain) system which produces a natural language based explanation of the decision taken by a NN. It uses the extracted statistical features to describe the decision of a NN, merging the deep learning world with that of statistics. The two-level explanation provides ample description of the decision made by the network to aid an expert as well as a novice user alike. Our survey and reliability assessment test confirm that the generated explanations are meaningful and correct. We believe that generating natural language based descriptions of the network's decisions is a big step towards opening up the black box.\t0.7758305462653289", "5.\tPredicting the final closing price of a stock is a challenging task and even modest improvements in predictive outcome can be very profitable. Many computer-aided techniques based on either machine learning or statistical models have been adopted to estimate price changes in the stock market. One of the major challenges with traditional machine learning models is the feature extraction process. Indeed, extracting relevant features from data and identifying hidden nonlinear relationships without relying on econometric assumptions and human expertise is extremely complex and makes deep learning particularly attractive. In this paper, we propose a deep neural network-based approach to predict if the stock price will increase by 25% for the following year, same quarter or not. We also compare our deep learning method against 'shallow' approaches, random forest and gradient boosted machines. To test the proposed methods, KIS-VALUE database consisting of the Korea Composite Stock Price Index (KOSPI) of companies for the period 2007 to 2015 was considered. All the methods yielded satisfactory performance, namely, deep neural network achieved an AUC of 0.806. 'Shallow' approaches, random forest and gradient boosted machines have been used for comparisons.\t0.7744035674470457"]
['1.\tCompany disclosures greatly aid in the process of financial decision-making; therefore, they are consulted by financial investors and automated traders before exercising ownership in stocks. While humans are usually able to correctly interpret the content, the same is rarely true of computerized decision support systems, which struggle with the complexity and ambiguity of natural language. A possible remedy is represented by deep learning, which overcomes several shortcomings of traditional methods of text mining. For instance, recurrent neural networks, such as long short-term memories, employ hierarchical structures, together with a large number of hidden layers, to automatically extract features from ordered sequences of words and capture highly non-linear relationships such as context-dependent meanings. However, deep learning has only recently started to receive traction, possibly because its performance is largely untested. Hence, this paper studies the use of deep neural networks for financial decision support. We additionally experiment with transfer learning, in which we pre-train the network on a different corpus with a length of 139.1 million words. Our results reveal a higher directional accuracy as compared to traditional machine learning when predicting stock price movements in response to financial disclosures. Our work thereby helps to highlight the business value of deep learning and provides recommendations to practitioners and executives. (C) 2017 Elsevier B.V. All rights reserved.\t6.282663880299503', '2.\tDistributed deep learning systems (DDLS) train deep neural network models by utilizing the distributed resources of a cluster. Developers of DDLS are required to make many decisions to process their particular workloads in their chosen environment efficiently. The advent of GPU-based deep learning, the ever-increasing size of datasets, and deep neural network models, in combination with the bandwidth constraints that exist in cluster environments require developers of DDLS to be innovative in order to train high-quality models quickly. Comparing DDLS side-by-side is difficult due to their extensive feature lists and architectural deviations. We aim to shine some light on the fundamental principles that are at work when training deep neural networks in a cluster of independent machines by analyzing the general properties associated with training deep learning models and how such workloads can be distributed in a cluster to achieve collaborative model training. Thereby we provide an overview of the different techniques that are used by contemporary DDLS and discuss their influence and implications on the training process. To conceptualize and compare DDLS, we group different techniques into categories, thus establishing a taxonomy of distributed deep learning systems.\t5.903099328519827', '3.\tJust like its remarkable achievements in many computer vision tasks, the convolutional neural networks (CNN) provide an end-to-end solution in handwritten Chinese character recognition (HCCR) with great success. However, the process of learning discriminative features for image recognition is difficult in cases where little data is available. In this paper, we propose a matching network which builds a connection between template characters and handwritten characters inspired by the human learning process of writing Chinese characters. The matching network replaces the parameters in the softmax regression layer with the features extracted from the template character images. After the training process has been finished, the powerful discriminative features help us to generalize the predictive power not just to new data, but to entire new Chinese characters that never appear in the training set before. Experiments performed on the ICDAR-2013 offline HCCR datasets have shown that the proposed method achieves a comparable performance to current CNN-based classifiers. Besides, the matching network has a very promising generalization ability to new Chinese characters that never appear in the existing training set. (C) 2020 Elsevier Ltd. All rights reserved.\t5.839917023828722', '4.\tCompute is a term coined from the etymology of French and Latin wordscomputerandcomputarerespectively, so is computing. This field of computing has grown enormously over the years. From the simple, traditional Turing machine invented in 1936 by Alan Turing to the current neural network (NN) computing. NNs, a field of artificial intelligence (AI) was exhilarated from the structure and inner workings of the brain. Just as the brain is, that is, an interconnection of neurons, so is the NN which is an interconnection of basic structures known as the perceptron. They do not differ much in structure. Their only difference is that one is artificial while the other is entirely biological. The hierarchical intricacies of the NN can be represented in three layers: the perceptron, artificial NN (ANN), and deep NN (DNN). With the influx of mental and behavioral disorders, basic surveillance, and the urgency to improve the mental health of people, studying the behavioral dynamics of people is requisite. CCTV and street cameras can only do so much, thus the need to employ the field of NN which makes use of supervised learning in training the models to perfect and automate surveillance. The results of this retrospective research indicate that the use of the NN model surpasses those of traditional methods in terms of efficiency and reliability.\t5.814875019246909', '5.\tCooperative wind farm control is a complex problem due to wake effect, and it is hard to find the proper model. Reinforcement learning can find the optimal policy in a dynamic environment using "trial and error," but may damage the machine and cause high cost during the learning process. In order to address this challenge, this article proposes the knowledge-assisted reinforcement learning framework by combining the low-fidelity analytical model with a reinforcement learning framework. Moreover, the knowledge-assisted deep deterministic policy gradient (KA-DDPG) algorithm and three kinds of knowledge-assisted learning methods are proposed based on the framework. The proposed methods are tested in nine different scenarios of WFSim. The simulation results show that the KA-DDPG algorithm can reach the maximum power output and ensure safety during learning. In addition, the learning cost is reduced by accelerating the learning process.\t5.791570190741011']
['{', ' "cells": [', '  {', '   "cell_type": "code",', '   "execution_count": 1,']
['1.\tCompany disclosures greatly aid in the process of financial decision-making; therefore, they are consulted by financial investors and automated traders before exercising ownership in stocks. While humans are usually able to correctly interpret the content, the same is rarely true of computerized decision support systems, which struggle with the complexity and ambiguity of natural language. A possible remedy is represented by deep learning, which overcomes several shortcomings of traditional methods of text mining. For instance, recurrent neural networks, such as long short-term memories, employ hierarchical structures, together with a large number of hidden layers, to automatically extract features from ordered sequences of words and capture highly non-linear relationships such as context-dependent meanings. However, deep learning has only recently started to receive traction, possibly because its performance is largely untested. Hence, this paper studies the use of deep neural networks for financial decision support. We additionally experiment with transfer learning, in which we pre-train the network on a different corpus with a length of 139.1 million words. Our results reveal a higher directional accuracy as compared to traditional machine learning when predicting stock price movements in response to financial disclosures. Our work thereby helps to highlight the business value of deep learning and provides recommendations to practitioners and executives. (C) 2017 Elsevier B.V. All rights reserved.\t1.0', '2.\tGradient descent optimization of learning has become a paradigm for training deep convolutional neural networks (DCNN). However, utilizing other learning strategies in the training process of the DCNN has rarely been explored by the deep learning (DL) community. This serves as the motivation to introduce a non-iterative learning strategy to retrain neurons at the top dense or fully connected (FC) layers of DCNN, resulting in, higher performance. The proposed method exploits the Moore-Penrose Inverse to pull back the current residual error to each FC layer, generating well-generalized features. Further, the weights of each FC layers are recomputed according to the Moore-Penrose Inverse. We evaluate the proposed approach on six most widely accepted object recognition benchmark datasets: Scene-15, CIFAR-10, CIFAR-100, SUN-397, Places365, and ImageNet. The experimental results show that the proposed method obtains improvements over 30 state-of-the-art methods. Interestingly, it also indicates that any DCNN with the proposed method can provide better performance than the same network with its original Backpropagation (BP)-based training.\t0.938835776333012', '3.\tVision-based person re-identification aims to match a persons identity across multiple images, which is a fundamental task in multimedia content analysis and retrieval. Deep neural networks have recently manifested great potential in this task. However, a major bottleneck of existing supervised deep networks is their reliance on a large amount of annotated training data. Manual labeling for person identities in large-scale surveillance camera systems is quite challenging and incurs significant costs. Some recent studies adopt generative model outputs as training data augmentation. To more effectively use these synthetic data for an improved feature learning and re-identification performance, this paper proposes a novel feature affinity-based pseudo labeling method with two possible label encodings. To the best of our knowledge, this is the first study that employs pseudo-labeling by measuring the affinity of unlabeled samples with the underlying clusters of labeled data samples using the intermediate feature representations from deep networks. We propose training the network with the joint supervision of cross-entropy loss together with a center regularization term, which not only ensures discriminative feature representation learning but also simultaneously predicts pseudo-labels for unlabeled data. We show that both label encodings can be learned in a unified manner and help improve the overall performance. Our extensive experiments on three person re-identification datasets: Market-1501, DukeMTMC-reID, and CUHK03, demonstrate significant performance boost over the state-of-the-art person re-identification approaches.\t0.9387909038681558', "4.\tNeural networks (NN) are considered as black boxes due to the lack of explainability and transparency of their decisions. This significantly hampers their deployment in environments where explainability is essential along with the accuracy of the system. Recently, significant efforts have been made for the interpretability of these deep networks with the aim to open up the black box. However, most of these approaches are specifically developed for visual modalities. In addition, the interpretations provided by these systems require expert knowledge and understanding for intelligibility. This indicates a vital gap between the explainability provided by the systems and the novice user. To bridge this gap, we present a novel framework i.e. Time-Series eXplanation (TSXplain) system which produces a natural language based explanation of the decision taken by a NN. It uses the extracted statistical features to describe the decision of a NN, merging the deep learning world with that of statistics. The two-level explanation provides ample description of the decision made by the network to aid an expert as well as a novice user alike. Our survey and reliability assessment test confirm that the generated explanations are meaningful and correct. We believe that generating natural language based descriptions of the network's decisions is a big step towards opening up the black box.\t0.9362242415987034", '5.\tThe decision-making ability of deep reinforcement learning has been proved successfully in a variety of fields. Here, we use this method for precise character detection by making tight bounding boxes around the Chinese characters in historical documents. An agent is trained to learn the control policy of fine-tuning a bounding box step-by-step through a Markov Decision Process. We introduce a novel fully convolutional network with position-sensitive Region-of-Interest (RoI) pooling (FCPN). The network receives character patches as input without fixed size, and it can fuse position information into the features of actions. Besides, we propose a dense reward function (DRF) that provides excellent rewards according to different actions and environment states, improving the decision-making ability of the agent. Our approach is designed as a universal method that can be applied to the output of all character-level or word-level text detectors to obtain more precise detection results. Application to the Tripitaka Koreana in Han (TKH) and Multiple Tripitaka in Han (MTH) datasets confirm the very promising performance of this method. In particular, our approach yields a significant improvement under a large Intersection over Union (IoU) of 0.8. The robustness and generality are also proved by experiments on the scene text datasets ICDAR2013 and ICDAR2015. (C) 2020 Elsevier Ltd. All rights reserved.\t0.935284240112779']
['1.\tCompany disclosures greatly aid in the process of financial decision-making; therefore, they are consulted by financial investors and automated traders before exercising ownership in stocks. While humans are usually able to correctly interpret the content, the same is rarely true of computerized decision support systems, which struggle with the complexity and ambiguity of natural language. A possible remedy is represented by deep learning, which overcomes several shortcomings of traditional methods of text mining. For instance, recurrent neural networks, such as long short-term memories, employ hierarchical structures, together with a large number of hidden layers, to automatically extract features from ordered sequences of words and capture highly non-linear relationships such as context-dependent meanings. However, deep learning has only recently started to receive traction, possibly because its performance is largely untested. Hence, this paper studies the use of deep neural networks for financial decision support. We additionally experiment with transfer learning, in which we pre-train the network on a different corpus with a length of 139.1 million words. Our results reveal a higher directional accuracy as compared to traditional machine learning when predicting stock price movements in response to financial disclosures. Our work thereby helps to highlight the business value of deep learning and provides recommendations to practitioners and executives. (C) 2017 Elsevier B.V. All rights reserved.\t188.1167927575745', '2.\tJust like its remarkable achievements in many computer vision tasks, the convolutional neural networks (CNN) provide an end-to-end solution in handwritten Chinese character recognition (HCCR) with great success. However, the process of learning discriminative features for image recognition is difficult in cases where little data is available. In this paper, we propose a matching network which builds a connection between template characters and handwritten characters inspired by the human learning process of writing Chinese characters. The matching network replaces the parameters in the softmax regression layer with the features extracted from the template character images. After the training process has been finished, the powerful discriminative features help us to generalize the predictive power not just to new data, but to entire new Chinese characters that never appear in the training set before. Experiments performed on the ICDAR-2013 offline HCCR datasets have shown that the proposed method achieves a comparable performance to current CNN-based classifiers. Besides, the matching network has a very promising generalization ability to new Chinese characters that never appear in the existing training set. (C) 2020 Elsevier Ltd. All rights reserved.\t188.04162166605124', '3.\tDistributed deep learning systems (DDLS) train deep neural network models by utilizing the distributed resources of a cluster. Developers of DDLS are required to make many decisions to process their particular workloads in their chosen environment efficiently. The advent of GPU-based deep learning, the ever-increasing size of datasets, and deep neural network models, in combination with the bandwidth constraints that exist in cluster environments require developers of DDLS to be innovative in order to train high-quality models quickly. Comparing DDLS side-by-side is difficult due to their extensive feature lists and architectural deviations. We aim to shine some light on the fundamental principles that are at work when training deep neural networks in a cluster of independent machines by analyzing the general properties associated with training deep learning models and how such workloads can be distributed in a cluster to achieve collaborative model training. Thereby we provide an overview of the different techniques that are used by contemporary DDLS and discuss their influence and implications on the training process. To conceptualize and compare DDLS, we group different techniques into categories, thus establishing a taxonomy of distributed deep learning systems.\t188.03805152943764', "4.\tPredicting the final closing price of a stock is a challenging task and even modest improvements in predictive outcome can be very profitable. Many computer-aided techniques based on either machine learning or statistical models have been adopted to estimate price changes in the stock market. One of the major challenges with traditional machine learning models is the feature extraction process. Indeed, extracting relevant features from data and identifying hidden nonlinear relationships without relying on econometric assumptions and human expertise is extremely complex and makes deep learning particularly attractive. In this paper, we propose a deep neural network-based approach to predict if the stock price will increase by 25% for the following year, same quarter or not. We also compare our deep learning method against 'shallow' approaches, random forest and gradient boosted machines. To test the proposed methods, KIS-VALUE database consisting of the Korea Composite Stock Price Index (KOSPI) of companies for the period 2007 to 2015 was considered. All the methods yielded satisfactory performance, namely, deep neural network achieved an AUC of 0.806. 'Shallow' approaches, random forest and gradient boosted machines have been used for comparisons.\t188.03338334392498", '5.\tRecent developments in applying deep learning techniques to train end-to-end communication systems have shown great promise in improving the overall performance of the system. However, most of the current methods for applying deep learning to train physical-layer characteristics assume the availability of the explicit channel model. Training a neural network requires the availability of the functional form all the layers in the network to calculate gradients for optimization. The unavailability of gradients in a physical channel forced previous works to adopt simulation-based strategies to train the network and then fine tune only the receiver part with the actual channel. In this letter, we present a practical method to train an end-to-end communication system without relying on explicit channel models. By utilizing stochastic perturbation techniques, we show that the proposed method can train a deep learning-based communication system in real channel without any assumption on channel models.\t188.02996132672507']
['1.\tCompany disclosures greatly aid in the process of financial decision-making; therefore, they are consulted by financial investors and automated traders before exercising ownership in stocks. While humans are usually able to correctly interpret the content, the same is rarely true of computerized decision support systems, which struggle with the complexity and ambiguity of natural language. A possible remedy is represented by deep learning, which overcomes several shortcomings of traditional methods of text mining. For instance, recurrent neural networks, such as long short-term memories, employ hierarchical structures, together with a large number of hidden layers, to automatically extract features from ordered sequences of words and capture highly non-linear relationships such as context-dependent meanings. However, deep learning has only recently started to receive traction, possibly because its performance is largely untested. Hence, this paper studies the use of deep neural networks for financial decision support. We additionally experiment with transfer learning, in which we pre-train the network on a different corpus with a length of 139.1 million words. Our results reveal a higher directional accuracy as compared to traditional machine learning when predicting stock price movements in response to financial disclosures. Our work thereby helps to highlight the business value of deep learning and provides recommendations to practitioners and executives. (C) 2017 Elsevier B.V. All rights reserved.\t1.0', '2.\tEmotions widely affect human decision-making. This fact is taken into account by affective computing with the goal of tailoring decision support to the emotional states of individuals. However, the accurate recognition of emotions within narrative documents presents a challenging undertaking due to the complexity and ambiguity of language. Performance improvements can be achieved through deep learning; yet, as demonstrated in this paper, the specific nature of this task requires the customization of recurrent neural networks with regard to bidirectional processing, dropout layers as a means of regularization, and weighted loss functions. In addition, we propose sent2affect, a tailored form of transfer learning for affective computing: here the network is pre-trained for a different task (i.e. sentiment analysis), while the output layer is subsequently tuned to the task of emotion recognition. The resulting performance is evaluated in a holistic setting across 6 benchmark datasets, where we find that both recurrent neural networks and transfer learning consistently outperform traditional machine learning. Altogether, the findings have considerable implications for the use of affective computing.\t0.9818780271871737', "3.\tNeural networks (NN) are considered as black boxes due to the lack of explainability and transparency of their decisions. This significantly hampers their deployment in environments where explainability is essential along with the accuracy of the system. Recently, significant efforts have been made for the interpretability of these deep networks with the aim to open up the black box. However, most of these approaches are specifically developed for visual modalities. In addition, the interpretations provided by these systems require expert knowledge and understanding for intelligibility. This indicates a vital gap between the explainability provided by the systems and the novice user. To bridge this gap, we present a novel framework i.e. Time-Series eXplanation (TSXplain) system which produces a natural language based explanation of the decision taken by a NN. It uses the extracted statistical features to describe the decision of a NN, merging the deep learning world with that of statistics. The two-level explanation provides ample description of the decision made by the network to aid an expert as well as a novice user alike. Our survey and reliability assessment test confirm that the generated explanations are meaningful and correct. We believe that generating natural language based descriptions of the network's decisions is a big step towards opening up the black box.\t0.981663698990191", '4.\tIt is quite challenging to correctly identify entities such as disease names, symptoms, and drugs from Chinese online medical inquiry texts. On the one hand, traditional natural language related methods cannot be directly applied to the field of online medical inquiry. Although supervised or unsupervised learning algorithms provide an entity recognition strategy for medical inquiries on online platforms, these methods either rely extensively on specific knowledge sources or artificially-designed features, or have a strong self-adaptivity, can barely obtain fairly good entity recognition outcomes, and consequently have weak generalization. On the other hand, Chinese online medical inquiry data is characterized by a large volume and excessively rich unstructured data, medical entities are indeed distributed in a sparse way, and the Chinese characters and words are quite complicated. It is difficult to establish a robust model for the recognition of entities in Chinese online medical inquiry texts. Therefore, establishing a new deep neural network (OMINer-CE) is the attempt of this paper. First, in order to form a proper feature strategy, the basic features of other tasks are introduced, while extended features, such as continuous bag of word cluster (CBOWC) feature, are constructed. Second, the feature vectors of Chinese character and word fusion are introduced to reserve all the Chinese character information of original sequences while introducing Chinese word-based semantic information. Third, a context encoding layer and a label decoding layer are introduced; On the basis of the recurrent neural network model BiLSTM, a convolutional neural network (CNN) is added to learn more key local features of Chinese word context, and the attention mechanism is used to obtain the long-distance dependency of the Chinese words to form a OMINer model. Finally, the basic and extended feature vectors are integrated into the OMINer model, and the grammatical and semantic information contained in the labeled texts is obtained from different perspectives. Therefore, it considers the feature strategies of Chinese online medical platforms, and realizes a strong self-adaptivity by utilizing the deep neural network. It is showed by the experiments that preferably good performances can be realized by combining the CE basic and extended feature vectors in the BiLSTM of the OMINer model, suggesting that the OMINer-CE model improves the performance of recognizing entities in Chinese online medical inquiry texts. (C) 2020 Published by Elsevier B.V.\t0.9815964797730828', "5.\tInfodemiology is the process of mining unstructured and textual data so as to provide public health officials and policymakers with valuable information regarding public health. The appearance of this new data source, which was previously unimaginable, has opened up a new way in which to improve public health systems, resulting in better communication policies and better detection systems. However, the unstructured nature of the Internet, along with the complexity of the infectious disease domain, prevents the information extracted from being easily understood. Moreover, when dealing with languages other than English, for which some of the most common Natural Language Processing resources are not available, the correct exploitation of this data becomes even more difficult. We intend to fill these gaps proposing an ontology-driven aspect-based sentiment analysis with which to measure the general public's opinions as regards infectious diseases when expressed in Spanish by employing a case study of tweets concerning the Zika, Dengue and Chikungunya viruses in Latin America. Our proposal is based on two technologies. We first use ontologies in order to model the infectious disease domain with concepts such as risks, symptoms, transmission methods or drugs, among other concepts. We then measure the relationship between these concepts in order to determine the degree to which one concept influences other concepts. This new information is subsequently applied in order to build an aspect-based sentiment analysis model based on statistical and linguistic features. This is done by applying deep-learning models. Our proposal is available on a web platform, where users can see the sentiment for each concept at a glance and analyse how each concept influences the sentiment of the others. (c) 2020 Elsevier B.V. All rights reserved.\t0.9805257637007774"]
['1.\tCompany disclosures greatly aid in the process of financial decision-making; therefore, they are consulted by financial investors and automated traders before exercising ownership in stocks. While humans are usually able to correctly interpret the content, the same is rarely true of computerized decision support systems, which struggle with the complexity and ambiguity of natural language. A possible remedy is represented by deep learning, which overcomes several shortcomings of traditional methods of text mining. For instance, recurrent neural networks, such as long short-term memories, employ hierarchical structures, together with a large number of hidden layers, to automatically extract features from ordered sequences of words and capture highly non-linear relationships such as context-dependent meanings. However, deep learning has only recently started to receive traction, possibly because its performance is largely untested. Hence, this paper studies the use of deep neural networks for financial decision support. We additionally experiment with transfer learning, in which we pre-train the network on a different corpus with a length of 139.1 million words. Our results reveal a higher directional accuracy as compared to traditional machine learning when predicting stock price movements in response to financial disclosures. Our work thereby helps to highlight the business value of deep learning and provides recommendations to practitioners and executives. (C) 2017 Elsevier B.V. All rights reserved.\t1.0000000000000002', '2.\tGradient descent optimization of learning has become a paradigm for training deep convolutional neural networks (DCNN). However, utilizing other learning strategies in the training process of the DCNN has rarely been explored by the deep learning (DL) community. This serves as the motivation to introduce a non-iterative learning strategy to retrain neurons at the top dense or fully connected (FC) layers of DCNN, resulting in, higher performance. The proposed method exploits the Moore-Penrose Inverse to pull back the current residual error to each FC layer, generating well-generalized features. Further, the weights of each FC layers are recomputed according to the Moore-Penrose Inverse. We evaluate the proposed approach on six most widely accepted object recognition benchmark datasets: Scene-15, CIFAR-10, CIFAR-100, SUN-397, Places365, and ImageNet. The experimental results show that the proposed method obtains improvements over 30 state-of-the-art methods. Interestingly, it also indicates that any DCNN with the proposed method can provide better performance than the same network with its original Backpropagation (BP)-based training.\t0.9391395924197377', '3.\tVision-based person re-identification aims to match a persons identity across multiple images, which is a fundamental task in multimedia content analysis and retrieval. Deep neural networks have recently manifested great potential in this task. However, a major bottleneck of existing supervised deep networks is their reliance on a large amount of annotated training data. Manual labeling for person identities in large-scale surveillance camera systems is quite challenging and incurs significant costs. Some recent studies adopt generative model outputs as training data augmentation. To more effectively use these synthetic data for an improved feature learning and re-identification performance, this paper proposes a novel feature affinity-based pseudo labeling method with two possible label encodings. To the best of our knowledge, this is the first study that employs pseudo-labeling by measuring the affinity of unlabeled samples with the underlying clusters of labeled data samples using the intermediate feature representations from deep networks. We propose training the network with the joint supervision of cross-entropy loss together with a center regularization term, which not only ensures discriminative feature representation learning but also simultaneously predicts pseudo-labels for unlabeled data. We show that both label encodings can be learned in a unified manner and help improve the overall performance. Our extensive experiments on three person re-identification datasets: Market-1501, DukeMTMC-reID, and CUHK03, demonstrate significant performance boost over the state-of-the-art person re-identification approaches.\t0.939063147693186', "4.\tNeural networks (NN) are considered as black boxes due to the lack of explainability and transparency of their decisions. This significantly hampers their deployment in environments where explainability is essential along with the accuracy of the system. Recently, significant efforts have been made for the interpretability of these deep networks with the aim to open up the black box. However, most of these approaches are specifically developed for visual modalities. In addition, the interpretations provided by these systems require expert knowledge and understanding for intelligibility. This indicates a vital gap between the explainability provided by the systems and the novice user. To bridge this gap, we present a novel framework i.e. Time-Series eXplanation (TSXplain) system which produces a natural language based explanation of the decision taken by a NN. It uses the extracted statistical features to describe the decision of a NN, merging the deep learning world with that of statistics. The two-level explanation provides ample description of the decision made by the network to aid an expert as well as a novice user alike. Our survey and reliability assessment test confirm that the generated explanations are meaningful and correct. We believe that generating natural language based descriptions of the network's decisions is a big step towards opening up the black box.\t0.9364504089199812", '5.\tThe decision-making ability of deep reinforcement learning has been proved successfully in a variety of fields. Here, we use this method for precise character detection by making tight bounding boxes around the Chinese characters in historical documents. An agent is trained to learn the control policy of fine-tuning a bounding box step-by-step through a Markov Decision Process. We introduce a novel fully convolutional network with position-sensitive Region-of-Interest (RoI) pooling (FCPN). The network receives character patches as input without fixed size, and it can fuse position information into the features of actions. Besides, we propose a dense reward function (DRF) that provides excellent rewards according to different actions and environment states, improving the decision-making ability of the agent. Our approach is designed as a universal method that can be applied to the output of all character-level or word-level text detectors to obtain more precise detection results. Application to the Tripitaka Koreana in Han (TKH) and Multiple Tripitaka in Han (MTH) datasets confirm the very promising performance of this method. In particular, our approach yields a significant improvement under a large Intersection over Union (IoU) of 0.8. The robustness and generality are also proved by experiments on the scene text datasets ICDAR2013 and ICDAR2015. (C) 2020 Elsevier Ltd. All rights reserved.\t0.9355215524414866']
['1.\tCompany disclosures greatly aid in the process of financial decision-making; therefore, they are consulted by financial investors and automated traders before exercising ownership in stocks. While humans are usually able to correctly interpret the content, the same is rarely true of computerized decision support systems, which struggle with the complexity and ambiguity of natural language. A possible remedy is represented by deep learning, which overcomes several shortcomings of traditional methods of text mining. For instance, recurrent neural networks, such as long short-term memories, employ hierarchical structures, together with a large number of hidden layers, to automatically extract features from ordered sequences of words and capture highly non-linear relationships such as context-dependent meanings. However, deep learning has only recently started to receive traction, possibly because its performance is largely untested. Hence, this paper studies the use of deep neural networks for financial decision support. We additionally experiment with transfer learning, in which we pre-train the network on a different corpus with a length of 139.1 million words. Our results reveal a higher directional accuracy as compared to traditional machine learning when predicting stock price movements in response to financial disclosures. Our work thereby helps to highlight the business value of deep learning and provides recommendations to practitioners and executives. (C) 2017 Elsevier B.V. All rights reserved.\t1.0', '2.\tGradient descent optimization of learning has become a paradigm for training deep convolutional neural networks (DCNN). However, utilizing other learning strategies in the training process of the DCNN has rarely been explored by the deep learning (DL) community. This serves as the motivation to introduce a non-iterative learning strategy to retrain neurons at the top dense or fully connected (FC) layers of DCNN, resulting in, higher performance. The proposed method exploits the Moore-Penrose Inverse to pull back the current residual error to each FC layer, generating well-generalized features. Further, the weights of each FC layers are recomputed according to the Moore-Penrose Inverse. We evaluate the proposed approach on six most widely accepted object recognition benchmark datasets: Scene-15, CIFAR-10, CIFAR-100, SUN-397, Places365, and ImageNet. The experimental results show that the proposed method obtains improvements over 30 state-of-the-art methods. Interestingly, it also indicates that any DCNN with the proposed method can provide better performance than the same network with its original Backpropagation (BP)-based training.\t0.9391868798542206', '3.\tVision-based person re-identification aims to match a persons identity across multiple images, which is a fundamental task in multimedia content analysis and retrieval. Deep neural networks have recently manifested great potential in this task. However, a major bottleneck of existing supervised deep networks is their reliance on a large amount of annotated training data. Manual labeling for person identities in large-scale surveillance camera systems is quite challenging and incurs significant costs. Some recent studies adopt generative model outputs as training data augmentation. To more effectively use these synthetic data for an improved feature learning and re-identification performance, this paper proposes a novel feature affinity-based pseudo labeling method with two possible label encodings. To the best of our knowledge, this is the first study that employs pseudo-labeling by measuring the affinity of unlabeled samples with the underlying clusters of labeled data samples using the intermediate feature representations from deep networks. We propose training the network with the joint supervision of cross-entropy loss together with a center regularization term, which not only ensures discriminative feature representation learning but also simultaneously predicts pseudo-labels for unlabeled data. We show that both label encodings can be learned in a unified manner and help improve the overall performance. Our extensive experiments on three person re-identification datasets: Market-1501, DukeMTMC-reID, and CUHK03, demonstrate significant performance boost over the state-of-the-art person re-identification approaches.\t0.9372486360959565', '4.\tThis paper considers the scenario that multiple data owners wish to apply a machine learning method over the combined dataset of all owners to obtain the best possible learning output but do not want to share the local datasets owing to privacy concerns. We design systems for the scenario that the stochastic gradient descent (SGD) algorithm is used as the machine learning method, because SGD (or its variants) is at the heart of recent deep learning techniques over neural networks. Our systems differ from the existing systems in the following features: 1) any activation function can be used, meaning that no privacy-preserving-friendly approximation is required; 2) gradients computed by SGD are not shared but the weight parameters are shared instead; and 3) robustness against colluding parties even in the extreme case that only one honest party exists. One of our systems requires a shared symmetric key among the data owners (trainers) to ensure the secrecy of the weight parameters against a central server. We prove that our systems, while privacy preserving, achieve the same learning accuracy as SGD and, hence, retain the merit of deep learning with respect to accuracy. Finally, we conduct several experiments using benchmark datasets and show that our systems outperform the previous system in terms of learning accuracies.\t0.932889920999122', "5.\tNeural networks (NN) are considered as black boxes due to the lack of explainability and transparency of their decisions. This significantly hampers their deployment in environments where explainability is essential along with the accuracy of the system. Recently, significant efforts have been made for the interpretability of these deep networks with the aim to open up the black box. However, most of these approaches are specifically developed for visual modalities. In addition, the interpretations provided by these systems require expert knowledge and understanding for intelligibility. This indicates a vital gap between the explainability provided by the systems and the novice user. To bridge this gap, we present a novel framework i.e. Time-Series eXplanation (TSXplain) system which produces a natural language based explanation of the decision taken by a NN. It uses the extracted statistical features to describe the decision of a NN, merging the deep learning world with that of statistics. The two-level explanation provides ample description of the decision made by the network to aid an expert as well as a novice user alike. Our survey and reliability assessment test confirm that the generated explanations are meaningful and correct. We believe that generating natural language based descriptions of the network's decisions is a big step towards opening up the black box.\t0.9322592473249702"]
['1.\tCompany disclosures greatly aid in the process of financial decision-making; therefore, they are consulted by financial investors and automated traders before exercising ownership in stocks. While humans are usually able to correctly interpret the content, the same is rarely true of computerized decision support systems, which struggle with the complexity and ambiguity of natural language. A possible remedy is represented by deep learning, which overcomes several shortcomings of traditional methods of text mining. For instance, recurrent neural networks, such as long short-term memories, employ hierarchical structures, together with a large number of hidden layers, to automatically extract features from ordered sequences of words and capture highly non-linear relationships such as context-dependent meanings. However, deep learning has only recently started to receive traction, possibly because its performance is largely untested. Hence, this paper studies the use of deep neural networks for financial decision support. We additionally experiment with transfer learning, in which we pre-train the network on a different corpus with a length of 139.1 million words. Our results reveal a higher directional accuracy as compared to traditional machine learning when predicting stock price movements in response to financial disclosures. Our work thereby helps to highlight the business value of deep learning and provides recommendations to practitioners and executives. (C) 2017 Elsevier B.V. All rights reserved.\t1.0', "2.\tIn today's financial markets, where most trades are performed in their entirety by electronic means and the largest fraction of them is completely automated, an opportunity has risen from analyzing this vast amount of transactions. Since all the transactions are recorded in great detail, investors can analyze all the generated data and detect repeated patterns of the price movements. Being able to detect them in advance, allows them to take profitable positions or avoid anomalous events in the financial markets. In this work we proposed a deep learning methodology, based on Convolutional Neural Networks (CNNs), that predicts the price movements of stocks, using as input large-scale, high-frequency time-series derived from the order book of financial exchanges. The dataset that we use contains more than 4 million limit order events and our comparison with other methods, like Multilayer Neural Networks and Support Vector Machines, shows that CNNs are better suited for this kind of task.\t0.72958751393534", "3.\tPredicting the final closing price of a stock is a challenging task and even modest improvements in predictive outcome can be very profitable. Many computer-aided techniques based on either machine learning or statistical models have been adopted to estimate price changes in the stock market. One of the major challenges with traditional machine learning models is the feature extraction process. Indeed, extracting relevant features from data and identifying hidden nonlinear relationships without relying on econometric assumptions and human expertise is extremely complex and makes deep learning particularly attractive. In this paper, we propose a deep neural network-based approach to predict if the stock price will increase by 25% for the following year, same quarter or not. We also compare our deep learning method against 'shallow' approaches, random forest and gradient boosted machines. To test the proposed methods, KIS-VALUE database consisting of the Korea Composite Stock Price Index (KOSPI) of companies for the period 2007 to 2015 was considered. All the methods yielded satisfactory performance, namely, deep neural network achieved an AUC of 0.806. 'Shallow' approaches, random forest and gradient boosted machines have been used for comparisons.\t0.7241025641025641", "4.\tNeural networks (NN) are considered as black boxes due to the lack of explainability and transparency of their decisions. This significantly hampers their deployment in environments where explainability is essential along with the accuracy of the system. Recently, significant efforts have been made for the interpretability of these deep networks with the aim to open up the black box. However, most of these approaches are specifically developed for visual modalities. In addition, the interpretations provided by these systems require expert knowledge and understanding for intelligibility. This indicates a vital gap between the explainability provided by the systems and the novice user. To bridge this gap, we present a novel framework i.e. Time-Series eXplanation (TSXplain) system which produces a natural language based explanation of the decision taken by a NN. It uses the extracted statistical features to describe the decision of a NN, merging the deep learning world with that of statistics. The two-level explanation provides ample description of the decision made by the network to aid an expert as well as a novice user alike. Our survey and reliability assessment test confirm that the generated explanations are meaningful and correct. We believe that generating natural language based descriptions of the network's decisions is a big step towards opening up the black box.\t0.717948717948718", '5.\tDue to the extensive practical value of time series prediction, many excellent algorithms have been proposed. Most of these methods are developed assuming that massive labeled training data are available. However, this assumption might be invalid in some actual situations. To address this limitation, a transfer learning framework with deep architectures is proposed. Since convolutional neural network (CNN) owns favorable feature extraction capability and can implement parallelization more easily, we propose a deep transfer learning method resorting to the architecture of CNN, termed as DTr-CNN for short. It can effectively alleviate the available labeled data absence and leverage useful knowledge to the current prediction. Notably, in our method, transfer learning process is implemented across different datasets. For a given target domain, in real-world scenarios, relativity of truly available potential source datasets may not be obvious, which is challenging and rarely referred to in most existing time series prediction methods. Aiming at this problem, the incorporation of Dynamic Time Warping (DTW) and Jensen-Shannon (JS) divergence is adopted for the selection of the appropriate source domain. Effectiveness of the proposed method is empirically underpinned by the experiments conducted on one group of synthetic and two groups of practical datasets. Besides, an additional experiment on NN5 dataset is conducted. (C) 2020 Elsevier Ltd. All rights reserved.\t0.7143812709030101']
['1.\tCompany disclosures greatly aid in the process of financial decision-making; therefore, they are consulted by financial investors and automated traders before exercising ownership in stocks. While humans are usually able to correctly interpret the content, the same is rarely true of computerized decision support systems, which struggle with the complexity and ambiguity of natural language. A possible remedy is represented by deep learning, which overcomes several shortcomings of traditional methods of text mining. For instance, recurrent neural networks, such as long short-term memories, employ hierarchical structures, together with a large number of hidden layers, to automatically extract features from ordered sequences of words and capture highly non-linear relationships such as context-dependent meanings. However, deep learning has only recently started to receive traction, possibly because its performance is largely untested. Hence, this paper studies the use of deep neural networks for financial decision support. We additionally experiment with transfer learning, in which we pre-train the network on a different corpus with a length of 139.1 million words. Our results reveal a higher directional accuracy as compared to traditional machine learning when predicting stock price movements in response to financial disclosures. Our work thereby helps to highlight the business value of deep learning and provides recommendations to practitioners and executives. (C) 2017 Elsevier B.V. All rights reserved.\t1.0', '2.\tEmotions widely affect human decision-making. This fact is taken into account by affective computing with the goal of tailoring decision support to the emotional states of individuals. However, the accurate recognition of emotions within narrative documents presents a challenging undertaking due to the complexity and ambiguity of language. Performance improvements can be achieved through deep learning; yet, as demonstrated in this paper, the specific nature of this task requires the customization of recurrent neural networks with regard to bidirectional processing, dropout layers as a means of regularization, and weighted loss functions. In addition, we propose sent2affect, a tailored form of transfer learning for affective computing: here the network is pre-trained for a different task (i.e. sentiment analysis), while the output layer is subsequently tuned to the task of emotion recognition. The resulting performance is evaluated in a holistic setting across 6 benchmark datasets, where we find that both recurrent neural networks and transfer learning consistently outperform traditional machine learning. Altogether, the findings have considerable implications for the use of affective computing.\t0.9818824889541503', "3.\tNeural networks (NN) are considered as black boxes due to the lack of explainability and transparency of their decisions. This significantly hampers their deployment in environments where explainability is essential along with the accuracy of the system. Recently, significant efforts have been made for the interpretability of these deep networks with the aim to open up the black box. However, most of these approaches are specifically developed for visual modalities. In addition, the interpretations provided by these systems require expert knowledge and understanding for intelligibility. This indicates a vital gap between the explainability provided by the systems and the novice user. To bridge this gap, we present a novel framework i.e. Time-Series eXplanation (TSXplain) system which produces a natural language based explanation of the decision taken by a NN. It uses the extracted statistical features to describe the decision of a NN, merging the deep learning world with that of statistics. The two-level explanation provides ample description of the decision made by the network to aid an expert as well as a novice user alike. Our survey and reliability assessment test confirm that the generated explanations are meaningful and correct. We believe that generating natural language based descriptions of the network's decisions is a big step towards opening up the black box.\t0.9818318374774253", '4.\tIt is quite challenging to correctly identify entities such as disease names, symptoms, and drugs from Chinese online medical inquiry texts. On the one hand, traditional natural language related methods cannot be directly applied to the field of online medical inquiry. Although supervised or unsupervised learning algorithms provide an entity recognition strategy for medical inquiries on online platforms, these methods either rely extensively on specific knowledge sources or artificially-designed features, or have a strong self-adaptivity, can barely obtain fairly good entity recognition outcomes, and consequently have weak generalization. On the other hand, Chinese online medical inquiry data is characterized by a large volume and excessively rich unstructured data, medical entities are indeed distributed in a sparse way, and the Chinese characters and words are quite complicated. It is difficult to establish a robust model for the recognition of entities in Chinese online medical inquiry texts. Therefore, establishing a new deep neural network (OMINer-CE) is the attempt of this paper. First, in order to form a proper feature strategy, the basic features of other tasks are introduced, while extended features, such as continuous bag of word cluster (CBOWC) feature, are constructed. Second, the feature vectors of Chinese character and word fusion are introduced to reserve all the Chinese character information of original sequences while introducing Chinese word-based semantic information. Third, a context encoding layer and a label decoding layer are introduced; On the basis of the recurrent neural network model BiLSTM, a convolutional neural network (CNN) is added to learn more key local features of Chinese word context, and the attention mechanism is used to obtain the long-distance dependency of the Chinese words to form a OMINer model. Finally, the basic and extended feature vectors are integrated into the OMINer model, and the grammatical and semantic information contained in the labeled texts is obtained from different perspectives. Therefore, it considers the feature strategies of Chinese online medical platforms, and realizes a strong self-adaptivity by utilizing the deep neural network. It is showed by the experiments that preferably good performances can be realized by combining the CE basic and extended feature vectors in the BiLSTM of the OMINer model, suggesting that the OMINer-CE model improves the performance of recognizing entities in Chinese online medical inquiry texts. (C) 2020 Published by Elsevier B.V.\t0.9816048260685091', "5.\tInfodemiology is the process of mining unstructured and textual data so as to provide public health officials and policymakers with valuable information regarding public health. The appearance of this new data source, which was previously unimaginable, has opened up a new way in which to improve public health systems, resulting in better communication policies and better detection systems. However, the unstructured nature of the Internet, along with the complexity of the infectious disease domain, prevents the information extracted from being easily understood. Moreover, when dealing with languages other than English, for which some of the most common Natural Language Processing resources are not available, the correct exploitation of this data becomes even more difficult. We intend to fill these gaps proposing an ontology-driven aspect-based sentiment analysis with which to measure the general public's opinions as regards infectious diseases when expressed in Spanish by employing a case study of tweets concerning the Zika, Dengue and Chikungunya viruses in Latin America. Our proposal is based on two technologies. We first use ontologies in order to model the infectious disease domain with concepts such as risks, symptoms, transmission methods or drugs, among other concepts. We then measure the relationship between these concepts in order to determine the degree to which one concept influences other concepts. This new information is subsequently applied in order to build an aspect-based sentiment analysis model based on statistical and linguistic features. This is done by applying deep-learning models. Our proposal is available on a web platform, where users can see the sentiment for each concept at a glance and analyse how each concept influences the sentiment of the others. (c) 2020 Elsevier B.V. All rights reserved.\t0.98053874457699"]
for i in range(1, len(set_list)):
    set_list[0].intersection_update(set_list[i])
    print(len(set_list[0]))
    break37
print(set_list[0]){"Sentiment analysis requires a lot of information coming from different sources and about different topics to be retrieved and fused. For this reason, one of the most important subtasks of sentiment analysis is subjectivity detection, i.e., the removal of 'factual' or 'neutral' comments that lack sentiment. It is possibly the most essential subtask of sentiment analysis as sentiment classifiers are often optimized to categorize text as either negative or positive and, hence, forcefully fit unopinionated sentences into one of these two categories. This article reviews hand-crafted and automatic models for subjectivity detection in the literature. It highlights the key assumptions these models make, the results they obtain, and the issues that still need to be explored to further our understanding of subjective sentences. Lastly, the advantages and limitations of each approach are compared. The methods can be broadly categorized as hand-crafted, automatic, and multi-modal. Hand-crafted templates work well on strong sentiments, however they are unable to identify weakly subjective sentences. Automatic methods such as deep learning provide a meta-level feature representation that generalizes well on new domains and languages. Multi-modal methods can combine the abundant audio and video forms of social data with text using multiple kernels. We conclude that the high-dimensionality of n-gram features and temporal nature of sentiments in long product reviews are the major challenges in sentiment mining from text.", 'In this paper, we propose a novel approach for efficient training of deep neural networks in a bottom-up fashion using a layered structure. Our algorithm, which we refer to as deep cascade learning, is motivated by the cascade correlation approach of Fahlman and Lebiere, who introduced it in the context of perceptrons. We demonstrate our algorithm on networks of convolutional layers, though its applicability is more general. Such training of deep networks in a cascade directly circumvents the well-known vanishing gradient problem by ensuring that the output is always adjacent to the layer being trained. We present empirical evaluations comparing our deep cascade training with standard end-end training using back propagation of two convolutional neural network architectures on benchmark image classification tasks (CIFAR-10 and CIFAR-100). We then investigate the features learned by the approach and find that better, domain-specific, representations are learned in early layers when compared to what is learned in end-end training. This is partially attributable to the vanishing gradient problem that inhibits early layer filters to change significantly from their initial settings. While both networks perform similarly overall, recognition accuracy increases progressively with each added layer, with discriminative features learned in every stage of the network, whereas in end-end training, no such systematic feature representation was observed. We also show that such cascade training has significant computational and memory advantages over end-end training, and can be used as a pretraining algorithm to obtain a better performance.', 'Review sentiment influences purchase decisions and indicates user satisfaction. Inferring the sentiment from reviews is an essential task in Natural Language Processing and has managerial implications for improving customer satisfaction and item quality. Traditional approaches to polarity classification use bag-of-words techniques and lexicons combined with machine learning. These approaches suffer from an inability to capture semantics and context. We propose a Deep Learning solution called OSLCFit (Organic Simultaneous LSTM and CNN Fit). In our architecture, we include all the components of a CNN until but not including the final fully connected layer and do the same in case of a bi-directional LSTM. The final fully connected layer in our architecture consists of fixed length features from the CNN, and features for both variable length and temporal dependencies from the bi-directional LSTM. The solution fine-tunes Language Model embeddings for the specific task of polarity classification using transfer learning, enabling the capture of semantics and context. The key contribution of this paper is the combination of features from both a CNN and a bi-directional LSTM into a single architecture with a single optimizer. This combination forms an organic combination and uses embeddings fine-tuned to the reviews for the specific purpose of sentiment polarity classification. The solution is benchmarked on six different datasets such as SMS Spam, YouTube Spam, Large Movie Review Corpus, Stanford Sentiment Treebank, Amazon Cellphone & Accessories and Yelp, where it beats existing benchmarks and scales to large datasets. The source code is available for the purposes of reproducible research on GitHub. (C) 2020 Elsevier Ltd. All rights reserved.', 'Introduction: Increase in computing power and the deeper usage of the robust computing systems in the financial system is propelling the business growth, improving the operational efficiency of the financial institutions, and increasing the effectiveness of the transaction processing solutions used by the organizations. Problem: Despite that the financial institutions are relying on the credit scoring patterns for analyzing the credit worthiness of the clients, still there are many factors that are imminent for improvement in the credit score evaluation patterns. There is need for improving the pattern to enhance the quality of analysis. Objective: Machine learning is offering immense potential in Fintech space and determining a personal credit score. Organizations by applying deep learning and machine learning techniques can tap individuals who are not being serviced by traditional financial institutions. Methodology: One of the major insights into the system is that the traditional models of banking intelligence solutions are predominantly the programmed models that can align with the information and banking systems that are used by the banks. But in the case of the machine-learning models that rely on algorithmic systems require more integral computation which is intrinsic. Hence, it can be advocated that the models usually need to have some decision lines wherein the dynamic calibration model must be streamlined. Such structure demands the dynamic calibration to have a decision tree system to empower with more integrated model changes. Results: The test analysis of the proposed machine learning model indicates effective and enhanced analysis process compared to the non-machine learning solutions. The model in terms of using various classifiers indicate potential ways in which the solution can be significant. Conclusion: If the systems can be developed to align with more pragmatic terms for analysis, it can help in improving the process conditions of customer profile analysis, wherein the process models have to be developed for comprehensive analysis and the ones that can make a sustainable solution for the credit system management. Originality: The proposed solution is effective and the one conceptualized to improve the credit scoring system patterns. If the model can be improved with more effective parameters and learning metrics, it can be sustainable outcome. Limitations: The model is tested in isolation and not in comparison to any of the existing credit scoring patterns. Only the inputs in terms of shortcomings from the existing models are taken in to account and accordingly the proposed solution is developed.', 'Image segmentation is the most fundamental part of computer vision, which is the foundation of all other methods of image processing. The quality of image segmentation technology will affect the subsequent processing considerably. Comparing with traditional image segmentation algorithms, image segmentation algorithm based on deep learning is constantly proposed, with high performance and efficiency. But there is also a lot of room for improvement. For example, key parts such as fastening bolt are usually small in size, polluted and covered, and do not have enough characteristic information, so it is difficult to obtain satisfactory results. These factors affect the accuracy of the test, which is easy to cause serious accidents. As traditional methods sometimes cannot meet the requirement of high-accuracy result, deep learning play a particularly important role in facing those problems. To solve the problem that traditional object recognition methods are not robust enough to extract image features, parts recognition accuracy is low, and segmentation is not possible, we have made some modifications based on Mask R-CNN. In this method, convolutional neural network is used to extract features from part images. Then we use some annotated images from dataset to fine-tuned Mask R-CNN network to guarantee the accuracy. At the same time, data enhancement and k-folding cross-validation are carried out to improve the robustness of the model. Finally, the result of part recognition and segmentation by building the experimental platform proves the significance of the method.', "The application of deep neural networks to finance has received a great deal of attention from researchers because no assumption about a suitable mathematical model has to be made prior to forecasting and they are capable of extracting useful information from large sets of data, which is required to describe nonlinear input-output relations of financial time series. The paper presents a new deep neural network model where single layered autoencoder and 4 layered neural network are serially coupled for stock price forecasting. The autoencoder extracts deep features, which are fed into multi-layer neural networks to predict the next day's stock closing prices. The proposed deep neural network is progressively learned layer by layer ahead of the final learning of the total network. The proposed model to predict daily close prices of KOrea composite Stock Price Index (KOSPI) is built, and its performance is demonstrated.", 'Deep learning (DL) is affecting each and every sphere of public and private lives and becoming a tool for daily use. The power of DL lies in the fact that it tries to imitate the activities of neurons in the neocortex of human brain where the thought process takes place. Therefore, like the brain, it tries to learn and recognize patterns in the form of digital images. This power is built on the depth of many layers of computing neurons backed by high power processors and graphics processing units (GPUs) easily available today. In the current scenario, we have provided detailed survey of various types of DL systems available today, and specifically, we have concentrated our efforts on current applications of DL in medical imaging. We have also focused our efforts on explaining the readers the rapid transition of technology from machine learning to DL and have tried our best in reasoning this paradigm shift. Further, a detailed analysis of complexities involved in this shift and possible benefits accrued by the users and developers.', 'Financial markets forecasting represents a challenging task for a series of reasons, such as the irregularity, high fluctuation, noise of the involved data, and the peculiar high unpredictability of the financial domain. Moreover, literature does not offer a proper methodology to systematically identify intrinsic and hyper-parameters, input features, and base algorithms of a forecasting strategy in order to automatically adapt itself to the chosen market. To tackle these issues, this paper introduces a fully automated optimized ensemble approach, where an optimized feature selection process has been combined with an automatic ensemble machine learning strategy, created by a set of classifiers with intrinsic and hyper-parameters learned in each marked under consideration. A series of experiments performed on different real-world futures markets demonstrate the effectiveness of such an approach with regard to both to the Buy and Hold baseline strategy and to several canonical state-of-the-art solutions.', "Deep Learning (DL) networks are recent revolutionary developments in artificial intelligence research. Typical networks are stacked by groups of layers that are further composed of many convolutional kernels or neurons. In network design, many hyper-parameters need to be defined heuristically before training in order to achieve high cross-validation accuracies. However, accuracy evaluation from the output layer alone is not sufficient to specify the roles of the hidden units in associated networks. This results in a significant knowledge gap between DL's wider applications and its limited theoretical understanding. To narrow the knowledge gap, our study explores visualization techniques to illustrate the mutual information (MI) in DL networks. The MI is a theoretical measurement, reflecting the relationship between two sets of random variables even if their relationship is highly non-linear and hidden in high-dimensional data. Our study aims to understand the roles of DL units in classification performance of the networks. Via a series of experiments using several popular DL networks, it shows that the visualization of MI and its change patterns between the input/output with the hidden layers and basic units can facilitate a better understanding of these DL units' roles. Our investigation on network convergence suggests a more objective manner to potentially evaluate DL networks. Furthermore, the visualization provides a useful tool to gain insights into the network performance, and thus to potentially facilitate the design of better network architectures by identifying redundancy and less-effective network units.", 'In recent years, Community Question Answering (CQA) has emerged as a popular platform for knowledge curation and archival. An interesting aspect of question answering is that it combines aspects from natural language processing, information retrieval, and machine learning. In this paper, we have explored how the depth of the neural network influences the accuracy of prediction of deleted questions in question-answering forums. We have used different shallow and deep models for prediction and analyzed the relationships between number of hidden layers, accuracy, and computational time. The results suggest that while deep networks perform better than shallow networks in modeling complex non-linear functions, increasing the depth may not always produce desired results. We observe that the performance of the deep neural network suffers significantly due to vanishing gradients when large number of hidden layers are present. Constantly increasing the depth of the model increases accuracy initially, after which the accuracy plateaus, and finally drops. Adding each layer is also expensive in terms of the time required to train the model. This research is situated in the domain of neural information retrieval and contributes towards building a theory on how deep neural networks can be efficiently and accurately used for predicting question deletion. We predict deleted questions with more than 90% accuracy using two to ten hidden layers, with less accurate results for shallower and deeper architectures.', 'Face recognition "in the wild" has been revolutionized by the deployment of deep learning-based approaches. In fact, it has been extensively demonstrated that deep convolutional neural networks (DCNNs) are powerful enough to overcome most of the limits that affected face recognition algorithms based on hand-crafted features. These include variations in illumination, pose, expression, and occlusion, to mention some. The DCNNs discriminative power comes from the fact that low- and high-level representations are learned directly from the raw image data. As a consequence, we expect the performance of a DCNN to be influenced by the characteristics of the image/video data that are fed to the network, and their preprocessing. In this paper, we present a thorough analysis of several aspects that impact on the use of DCNN for face recognition. The evaluation has been carried out from two main perspectives: the network architecture and the similarity measures used to compare deeply learned features; and the data (source and quality) and their pre-processing (bounding box and alignment). The results obtained on the IARPA Janus Benchmark-A, MegaFace, UMDFaces, and YouTube Faces data sets indicate viable hints for designing, training, and testing DCNNs. Considering the outcomes of the experimental evaluation, we show how competitive performance with respect to the state of the art can be reached even with standard DCNN architectures and pipeline.', 'Company disclosures greatly aid in the process of financial decision-making; therefore, they are consulted by financial investors and automated traders before exercising ownership in stocks. While humans are usually able to correctly interpret the content, the same is rarely true of computerized decision support systems, which struggle with the complexity and ambiguity of natural language. A possible remedy is represented by deep learning, which overcomes several shortcomings of traditional methods of text mining. For instance, recurrent neural networks, such as long short-term memories, employ hierarchical structures, together with a large number of hidden layers, to automatically extract features from ordered sequences of words and capture highly non-linear relationships such as context-dependent meanings. However, deep learning has only recently started to receive traction, possibly because its performance is largely untested. Hence, this paper studies the use of deep neural networks for financial decision support. We additionally experiment with transfer learning, in which we pre-train the network on a different corpus with a length of 139.1 million words. Our results reveal a higher directional accuracy as compared to traditional machine learning when predicting stock price movements in response to financial disclosures. Our work thereby helps to highlight the business value of deep learning and provides recommendations to practitioners and executives. (C) 2017 Elsevier B.V. All rights reserved.', 'Convolutional Neural Networks (CNNs), also known as deep learners have seen much success in the last few years due to the availability of large amounts of data and high-performance computational resources. A CNN can be trained effectively if large amounts of data are available as it enables a CNN to find the optimal set of features and weights that can achieve the highest generalization performance. However, due to the requirement of large data size, CNNs require a lot of resources for example running time and computational resources to achieve a reasonable performance. Additionally, unbalanced data makes it difficult to train a CNN effectively that can achieve good generalization performance. In order to alleviate these limitations, in this paper, we propose a novel ensemble of deep learners that learns by combining multiple deep learners trained on small strongly class associated input data effectively. We propose a novel methodology of generating random subspace through clustering input data and propose a measure which can classify each cluster as a strong data cluster and a balanced data cluster. A methodology is also proposed that balances all strong data clusters in the pool so that an architecturally simple CNN can be trained on all balanced data clusters simultaneously. Classification decisions on all trained CNNs are then fused through majority voting to generate class decisions of the ensemble. The performance of the proposed ensemble approach is evaluated on UCI benchmark datasets, and results are compared with existing state-of-the-art ensemble approaches. Significance testing was conducted to further validate the efficacy of the results and a significance test analysis is presented. (C) 2020 Elsevier Ltd. All rights reserved.', "In today's financial markets, where most trades are performed in their entirety by electronic means and the largest fraction of them is completely automated, an opportunity has risen from analyzing this vast amount of transactions. Since all the transactions are recorded in great detail, investors can analyze all the generated data and detect repeated patterns of the price movements. Being able to detect them in advance, allows them to take profitable positions or avoid anomalous events in the financial markets. In this work we proposed a deep learning methodology, based on Convolutional Neural Networks (CNNs), that predicts the price movements of stocks, using as input large-scale, high-frequency time-series derived from the order book of financial exchanges. The dataset that we use contains more than 4 million limit order events and our comparison with other methods, like Multilayer Neural Networks and Support Vector Machines, shows that CNNs are better suited for this kind of task.", 'Pointly-supervised learning is an important topic for scene parsing, as dense annotation is extremely expensive and hard to scale. The state-of-the-art method harvests pseudo labels by applying thresholds upon softmax outputs (logits). There are two issues with this practice: (1) Softmax output does not necessarily reflect the confidence of the network output. (2) There is no principled way to decide on the optimal threshold. Tuning thresholds can be time-consuming for deep neural networks. Our method, by contrast, builds upon uncertainty measures instead of logits and is free of threshold tuning. We motivate the method with a large-scale analysis of the distribution of uncertainty measures, using strong models and challenging databases. This analysis leads to the discovery of a statistical phenomenon called uncertainty mixture. Specifically speaking, for each independent category, the distribution of uncertainty measures for unlabeled points is a mixture of two components (certain v.s. uncertain samples). The phenomenon of uncertainty mixture is surprisingly ubiquitous in real-world datasets like PascalContext and ADE20k. Inspired by this discovery, we propose to decompose the distribution of uncertainty measures with a Gamma mixture model, leading to a principled method to harvest reliable pseudo labels. Beyond that, we assume the uncertainty measures for labeled points are always drawn from the certain component. This amounts to a regularized Gamma mixture model. We provide a thorough theoretical analysis of this model, showing that it can be solved with an EM-style algorithm with convergence guarantee. Our method is also empirically successful. On PascalContext and ADE20k, we achieve clear margins over the baseline, notably with no threshold tuning in the pseudo label generation procedure. On the absolute scale, since our method collaborates well with strong baselines, we reach new state-of-the-art performance on both datasets.', "The success of neural networks is typically attributed to their ability to closely mimic relationships between features and labels observed in the training dataset. This, however, is only part of the answer: in addition to being fit to data, neural networks have been shown to be useful priors on the conditional distribution of labels given features and can be used as such even in the absence of trustworthy training labels. This feature of neural networks can be harnessed to train high quality models on low quality training data in tasks for which large high-quality ground truth datasets don't exist. One of these problems is assertion classification in biomedical texts: discriminating between positive, negative and speculative statements about certain pathologies a patient may have. We present an assertion classification methodology based on recurrent neural networks, attention mechanism and two flavours of transfer learning (language modelling and heuristic annotation) that achieves state of the art results on MIMIC-CXR radiology reports.", 'This paper considers the scenario that multiple data owners wish to apply a machine learning method over the combined dataset of all owners to obtain the best possible learning output but do not want to share the local datasets owing to privacy concerns. We design systems for the scenario that the stochastic gradient descent (SGD) algorithm is used as the machine learning method, because SGD (or its variants) is at the heart of recent deep learning techniques over neural networks. Our systems differ from the existing systems in the following features: 1) any activation function can be used, meaning that no privacy-preserving-friendly approximation is required; 2) gradients computed by SGD are not shared but the weight parameters are shared instead; and 3) robustness against colluding parties even in the extreme case that only one honest party exists. One of our systems requires a shared symmetric key among the data owners (trainers) to ensure the secrecy of the weight parameters against a central server. We prove that our systems, while privacy preserving, achieve the same learning accuracy as SGD and, hence, retain the merit of deep learning with respect to accuracy. Finally, we conduct several experiments using benchmark datasets and show that our systems outperform the previous system in terms of learning accuracies.', "Infodemiology is the process of mining unstructured and textual data so as to provide public health officials and policymakers with valuable information regarding public health. The appearance of this new data source, which was previously unimaginable, has opened up a new way in which to improve public health systems, resulting in better communication policies and better detection systems. However, the unstructured nature of the Internet, along with the complexity of the infectious disease domain, prevents the information extracted from being easily understood. Moreover, when dealing with languages other than English, for which some of the most common Natural Language Processing resources are not available, the correct exploitation of this data becomes even more difficult. We intend to fill these gaps proposing an ontology-driven aspect-based sentiment analysis with which to measure the general public's opinions as regards infectious diseases when expressed in Spanish by employing a case study of tweets concerning the Zika, Dengue and Chikungunya viruses in Latin America. Our proposal is based on two technologies. We first use ontologies in order to model the infectious disease domain with concepts such as risks, symptoms, transmission methods or drugs, among other concepts. We then measure the relationship between these concepts in order to determine the degree to which one concept influences other concepts. This new information is subsequently applied in order to build an aspect-based sentiment analysis model based on statistical and linguistic features. This is done by applying deep-learning models. Our proposal is available on a web platform, where users can see the sentiment for each concept at a glance and analyse how each concept influences the sentiment of the others. (c) 2020 Elsevier B.V. All rights reserved.", 'Automatic dynamic sign language recognition is even more challenging than gesture recognition due to the fact that the vocabularies are large and signs are context dependent. Previous works in this direction tend to build classifiers based on complex hand-crafted features computed from the raw inputs. As a type of deep learning model, convolutional neural networks (CNNs) have significantly advanced the accuracy of human gesture classification. However, such methods are currently used to treat video frames as 2D images and recognize gestures at the individual frame level. In this paper, we present a data driven system in which 3D-CNNs are applied to extract spatial and temporal features from video streams, and the motion information is captured by noting the variation in depth between each pair of consecutive frames. To further boost the performance, multi-modal of video streams, including infrared, contour and skeleton are used as input for the architecture and the prediction results estimated from the different sub-networks were fused together. In order to validate our method, we introduce a new challenging multi-modal dynamic sign language dataset captured with Kinect sensors. We evaluate the proposed approach on the collected dataset and achieve superior performance. Moreover, our method achieves a mean Jaccard Index score of 0.836 on the ChaLearn Looking at People Gesture datasets.', 'Features learned by deep Convolutional Neural Networks (CNNs) have been recognized to be more robust and expressive than hand-crafted ones. They have been successfully used in different computer vision tasks such as object detection, pattern recognition and image understanding. Given a CNN architecture and a training procedure, the efficacy of the learned features depends on the domain-representativeness of the training examples. In this paper we investigate the use of CNN-based features for the purpose of food recognition and retrieval. To this end, we first introduce the Food-475 database, that is the largest publicly available food database with 475 food classes and 247,636 images obtained by merging four publicly available food databases. We then define the food-domain representativeness of different food databases in terms of the total number of images, number of classes of the domain and number of examples for class. Different features are then extracted from a CNN based on the Residual Network with 50 layers architecture and trained on food databases with diverse food-domain representativeness. We evaluate these features for the tasks of food classification and retrieval. Results demonstrate that the features extracted from the Food-475 database outperform the other ones showing that we need larger food databases in order to tackle the challenges in food recognition, and that the created database is a step forward toward this end.', "Many existing learning algorithms suffer from limited architectural depth and the locality of estimators, making it difficult to generalize from the test set and providing inefficient and biased estimators. Deep architectures have been shown to appropriately learn correlation structures in time series data. This paper compares the effectiveness of a deep feedforward Neural Network (DNN) and shallow architectures (e.g., Support Vector Machine (SVM) and one-layer NN) when predicting a broad cross-section of stock price indices in both developed and emerging markets. An extensive evaluation is undertaken, using daily, hourly, minute and tick level data related to thirty-four financial indices from 32 countries across six years. Our evaluation results show a considerable advantage from training deep (cf. shallow) architectures, using a rectifier linear (RELU) activation function, across all thirty-four markets when 'minute' data is used. However, the predictive performance of DNN was not significantly better than that of shallower architectures when using tick level data. This result suggests that when training a DNN algorithm, the predictive accuracy peaks, regardless of training size. We also examine which activation function works best for stock price index data. Our results demonstrate that the RELU activation function performs better than TANH across all markets and time horizons when using DNN to predict stock price indices. (C) 2019 Elsevier Ltd. All rights reserved.", 'Due to the extensive practical value of time series prediction, many excellent algorithms have been proposed. Most of these methods are developed assuming that massive labeled training data are available. However, this assumption might be invalid in some actual situations. To address this limitation, a transfer learning framework with deep architectures is proposed. Since convolutional neural network (CNN) owns favorable feature extraction capability and can implement parallelization more easily, we propose a deep transfer learning method resorting to the architecture of CNN, termed as DTr-CNN for short. It can effectively alleviate the available labeled data absence and leverage useful knowledge to the current prediction. Notably, in our method, transfer learning process is implemented across different datasets. For a given target domain, in real-world scenarios, relativity of truly available potential source datasets may not be obvious, which is challenging and rarely referred to in most existing time series prediction methods. Aiming at this problem, the incorporation of Dynamic Time Warping (DTW) and Jensen-Shannon (JS) divergence is adopted for the selection of the appropriate source domain. Effectiveness of the proposed method is empirically underpinned by the experiments conducted on one group of synthetic and two groups of practical datasets. Besides, an additional experiment on NN5 dataset is conducted. (C) 2020 Elsevier Ltd. All rights reserved.', 'Neural networks have become very popular in recent years, because of the astonishing success of deep learning in various domains such as image and speech recognition. In many of these domains, specific architectures of neural networks, such as convolutional networks, seem to fit the particular structure of the problem domain very well and can therefore perform in an astonishingly effective way. However, the success of neural networks is not universal across all domains. Indeed, for learning problems without any special structure, or in cases where the data are somewhat limited, neural networks are known not to perform well with respect to traditional machine-learning methods such as random forests. In this article, we show that a carefully designed neural network with random forest structure can have better generalization ability. In fact, this architecture is more powerful than random forests, because the back-propagation algorithm reduces to a more powerful and generalized way of constructing a decision tree. Furthermore, the approach is efficient to train and requires a small constant factor of the number of training examples. This efficiency allows the training of multiple neural networks to improve the generalization accuracy. Experimental results on real-world benchmark datasets demonstrate the effectiveness of the proposed enhancements for classification and regression.', 'Currently, breast tissue images are primarily classified by pathologists, which is time-consuming and subjective. Deep learning, however, can perform this task with the utmost precision. In order to achieve an improved performance, a large number of annotated datasets are required to train the network, which is a challenging task in the medical field. In this paper, we propose an intelligent system, based on generative adversarial networks (GANs) and a convolution neural network (CNN) for the automatic classification of breast cancer, using optical coherence tomography (OCT) images. In this network, the GAN is used to generate synthetic datasets and to further utilize these synthetic datasets to increase the quantity of information, so as to improve the classification performance of the CNN. Our method is demonstrated by means of a limited set of OCT images of breast tissue. The classification performance of our method, using only the classic data increase, yielded a sensitivity level of 93.6%, with 90.8% specificity and 91.7% accuracy, based on the test datasets. By adding the synthetic data increase, the accuracy of the training datasets increased to 93.7% from 92.0%. We believe that this approach will help radiologists and pathologists to improve their diagnotic capability.', 'It is quite challenging to correctly identify entities such as disease names, symptoms, and drugs from Chinese online medical inquiry texts. On the one hand, traditional natural language related methods cannot be directly applied to the field of online medical inquiry. Although supervised or unsupervised learning algorithms provide an entity recognition strategy for medical inquiries on online platforms, these methods either rely extensively on specific knowledge sources or artificially-designed features, or have a strong self-adaptivity, can barely obtain fairly good entity recognition outcomes, and consequently have weak generalization. On the other hand, Chinese online medical inquiry data is characterized by a large volume and excessively rich unstructured data, medical entities are indeed distributed in a sparse way, and the Chinese characters and words are quite complicated. It is difficult to establish a robust model for the recognition of entities in Chinese online medical inquiry texts. Therefore, establishing a new deep neural network (OMINer-CE) is the attempt of this paper. First, in order to form a proper feature strategy, the basic features of other tasks are introduced, while extended features, such as continuous bag of word cluster (CBOWC) feature, are constructed. Second, the feature vectors of Chinese character and word fusion are introduced to reserve all the Chinese character information of original sequences while introducing Chinese word-based semantic information. Third, a context encoding layer and a label decoding layer are introduced; On the basis of the recurrent neural network model BiLSTM, a convolutional neural network (CNN) is added to learn more key local features of Chinese word context, and the attention mechanism is used to obtain the long-distance dependency of the Chinese words to form a OMINer model. Finally, the basic and extended feature vectors are integrated into the OMINer model, and the grammatical and semantic information contained in the labeled texts is obtained from different perspectives. Therefore, it considers the feature strategies of Chinese online medical platforms, and realizes a strong self-adaptivity by utilizing the deep neural network. It is showed by the experiments that preferably good performances can be realized by combining the CE basic and extended feature vectors in the BiLSTM of the OMINer model, suggesting that the OMINer-CE model improves the performance of recognizing entities in Chinese online medical inquiry texts. (C) 2020 Published by Elsevier B.V.', 'Small, imperceptible perturbations of the input data can lead to DNNs making egregious errors during inference, such as misclassifying an image of a dog as a cat with high probability. Thus, defending against adversarial examples for deep neural networks (DNNs) is of great interest to sensing technologies and the machine learning community to ensure the security of practical systems where DNNs are used. Whereas many approaches have been explored for defending against adversarial attacks, few have made use of the full state of the entire network, opting instead to only consider the output layer and gradient information. We develop several motivated techniques that make use of the full network state, improving adversarial robustness. We provide principled motivation of our techniques via analysis of attractor dynamics, shown to occur in the highly recurrent human brain, and validate our improvements via empirical results on standard datasets and white-box attacks.', 'The increase of network users has led to a large number of commentary languages on various network platforms. Traditional manual processing is time-consuming and labor-intensive. We need a mechanized way to process these commentary corpora and quickly uncover the emotional tendencies. A method of sentimental orientation analysis of comment text based on deep learning is proposed. First, we used GloVe model to train the word vector. Then, Give the different weight on word vector by using TF-IDF. Finally, the processed word vectors would be classificated by TextCNN. Experiments were carried out on the six categories of commodity review data crawled by Jingdong. This method can effectively identify the emotional tendency of the review text, which is more accurate than the traditional deep learning method.', 'Deep learning techniques are commonly used to process large amounts of data, and good results are obtained in many applications. Those methods, however, can lead to long training times. An alternative to simultaneously tune all parameters of a large network is to stack smaller modules, improving the model efficiency. However, methods such as Deep Stacked Network (DSN) have some problems that increase its training time and memory usage. To deal with these problems, Fast DSN (FDSN) was proposed, where the modules are trained using an Extreme Learning Machine (ELM) variant. Nonetheless, to speed-up the FDSN training, the ELM random feature mapping is shared among the modules, which can impact the network performance if the weights are not properly chosen. In this paper, we focus on the weight initialization of FDSN in order to improve its performance. We also propose FKDSN, a kernel-based variant of FDSN, besides discussing the theoretical complexity of the methods. We evaluate three different initialization approaches on ELM-trained neural networks over 50 public real-world regression datasets. Our experiments show that FDSN when combined with a more complex initialization method achieves similar results to ELM algorithms applied to large SLFNs, besides having a shorter training time and memory usage, implying that it can be suitable to be used on systems with restrict resources, such as Internet of Things devices. FKDSN also obtained similar results and training time to the large SLFNs, requiring less memory.', 'How to design deep neural networks (DNNs) for the representation and analysis of high dimensional but small sample size data is still a big challenge. One solution is to construct a sparse network. At present, there exist many approaches to achieve sparsity for DNNs by regularization, but most of them are carried out only in the pre-training process due to the difficulty in the derivation of explicit formulae in the fine-tuning process. In this paper, a log-sum function is used as the regularization terms for both the responses of hidden neurons and the network connections in the loss function of the fine-tuning process. It provides a better approximation to the L-0-norm than several often used norms. Based on the gradient formula of the loss function, the fine-tuning process can be executed more efficiently. Specifically, the commonly used gradient calculation in many deep learning research platforms, such as PyTorch or TensorFlow, can be accelerated. Given the analytic formula for calculating gradients used in any layer of DNN, the error accumulated from successive numerical approximations in the differentiation process can be avoided. With the proposed log-sum enhanced sparse deep neural network (LSES-DNN), the spar-sity of the responses and the connections can be well controlled to improve the adaptivity of DNNs. The proposed model is applied to MRI data for both the diagnosis of schizophrenia and the study of brain developments. Numerical experiments demonstrate its superior performance among several classical classifiers tested. (C) 2020 Elsevier B.V. All rights reserved.', "Predicting the final closing price of a stock is a challenging task and even modest improvements in predictive outcome can be very profitable. Many computer-aided techniques based on either machine learning or statistical models have been adopted to estimate price changes in the stock market. One of the major challenges with traditional machine learning models is the feature extraction process. Indeed, extracting relevant features from data and identifying hidden nonlinear relationships without relying on econometric assumptions and human expertise is extremely complex and makes deep learning particularly attractive. In this paper, we propose a deep neural network-based approach to predict if the stock price will increase by 25% for the following year, same quarter or not. We also compare our deep learning method against 'shallow' approaches, random forest and gradient boosted machines. To test the proposed methods, KIS-VALUE database consisting of the Korea Composite Stock Price Index (KOSPI) of companies for the period 2007 to 2015 was considered. All the methods yielded satisfactory performance, namely, deep neural network achieved an AUC of 0.806. 'Shallow' approaches, random forest and gradient boosted machines have been used for comparisons.", "Neural networks (NN) are considered as black boxes due to the lack of explainability and transparency of their decisions. This significantly hampers their deployment in environments where explainability is essential along with the accuracy of the system. Recently, significant efforts have been made for the interpretability of these deep networks with the aim to open up the black box. However, most of these approaches are specifically developed for visual modalities. In addition, the interpretations provided by these systems require expert knowledge and understanding for intelligibility. This indicates a vital gap between the explainability provided by the systems and the novice user. To bridge this gap, we present a novel framework i.e. Time-Series eXplanation (TSXplain) system which produces a natural language based explanation of the decision taken by a NN. It uses the extracted statistical features to describe the decision of a NN, merging the deep learning world with that of statistics. The two-level explanation provides ample description of the decision made by the network to aid an expert as well as a novice user alike. Our survey and reliability assessment test confirm that the generated explanations are meaningful and correct. We believe that generating natural language based descriptions of the network's decisions is a big step towards opening up the black box.", 'Cybersecurity data remains a challenge for the machine learning community as the high volume of traffic makes it difficult to properly disambiguate anomalous from normal behaviour. That decision is the core of an intelligent Intrusion Detection System (IDS), a component responsible for raising alerts whenever a potential threat is detected. However, with high volume data in contemporary systems, these IDSs generate numerous alerts, too large for human operators to exhaustively investigate. Moreover, simply reporting a single possible threat is often not sufficient, since the security analyst has to investigate the alert without any further clues of the underlying cause. In order to combat these issues, we empirically compare popular deep neural learning architectures for the problem of intrusion detection in sequential data streams. Contrary to a majority of research studies, we do not take a classification-based approach that requires labeled examples of hostile attacks. Instead, we adopt an unsupervised anomaly detection approach that aims to model a benign sequential data distribution against which new test instances are compared to. We also examine one additional deep network in the form of an attention model capable of providing explanations in addition to its predictions; such information is of crucial importance to network operators since it provides additional guidance to resolve potential threats. For our experiments, we evaluate the models against a variety of data sets of different complexities, ranging from simple unidimensional (synthetic and Yahoo!) to more complex multi-source (CICIDS2017 and small real-world enterprise network) data streams. In order to facilitate end-user needs, we focus on ranking-based metrics for comparing different deep neural architectures. This evaluation is especially important for security analysts to prioritize their anomaly investigations. Overall, our experiments demonstrate that a variant of a recurrent neural network generally outperforms a popular non-sequential deep autoencoder commonly used for unsupervised anomaly detection. The attentional model did not provide sufficiently good performance and explanations that we discuss in our analysis. Nonetheless, given that the global financial outlays for cybersecurity are calculated in trillions of dollars, our evaluation and identification of the top-performing RNN architectures for anomaly detection in sequential data streams can lead to improved intelligent IDS design, while our contributions of attentional explanation will hopefully lay the foundations for future improvements to the explanatory capability of these intelligent learning-based IDSs. (C) 2020 Elsevier Ltd. All rights reserved.', 'In order to conduct an in-depth study on financial transactions of block chain, the classical back propagation (BP) neural network based on the artificial neural network (ANN) model is selected, and its propagation mode, weight change, and learning process are analyzed. For the problem of slow convergence speed and local minimum value of BP neural network, based on the idea of deep learning, the initial value and training step are changed by auto-encoder and restricted Boltzmann machine, and the theory is analyzed. Taking the stock index futures trading in the block chain financial trading as an example, the stock price trading of stock index futures is studied using the two deep learning neural network models to predict the price changes. The results show that the auto-encoder, as an unsupervised learning system, performs better than the restricted Boltzmann machine in setting the initial weights and thresholds, with fewer iterations, faster convergence rate, and smaller convergence error. The results obtained by the auto-encoder can be used as initialization settings and data analysis. The prediction accuracy of the whole model is around 59%. When the transaction cost is not considered, the transaction can be conducted based on the prediction signal of the deep learning model. Therefore, deep learning neural network model can be applied to block chain financial transactions as a reference for financial transactions, which has a good practical significance for the development of this field. (C) 2020 Elsevier B.V. All rights reserved.', "Text classification is of importance in natural language processing, as the massive text information containing huge amounts of value needs to be classified into different categories for further use. In order to better classify text, our paper tries to build a deep learning model which achieves better classification results in Chinese text than those of other researchers' models. After comparing different methods, long short-term memory (LSTM) and convolutional neural network (CNN) methods were selected as deep learning methods to classify Chinese text. LSTM is a special kind of recurrent neural network (RNN), which is capable of processing serialized information through its recurrent structure. By contrast, CNN has shown its ability to extract features from visual imagery. Therefore, two layers of LSTM and one layer of CNN were integrated to our new model: the BLSTM-C model (BLSTM stands for bi-directional long short-term memory while C stands for CNN.) LSTM was responsible for obtaining a sequence output based on past and future contexts, which was then input to the convolutional layer for extracting features. In our experiments, the proposed BLSTM-C model was evaluated in several ways. In the results, the model exhibited remarkable performance in text classification, especially in Chinese texts.", 'Emotions widely affect human decision-making. This fact is taken into account by affective computing with the goal of tailoring decision support to the emotional states of individuals. However, the accurate recognition of emotions within narrative documents presents a challenging undertaking due to the complexity and ambiguity of language. Performance improvements can be achieved through deep learning; yet, as demonstrated in this paper, the specific nature of this task requires the customization of recurrent neural networks with regard to bidirectional processing, dropout layers as a means of regularization, and weighted loss functions. In addition, we propose sent2affect, a tailored form of transfer learning for affective computing: here the network is pre-trained for a different task (i.e. sentiment analysis), while the output layer is subsequently tuned to the task of emotion recognition. The resulting performance is evaluated in a holistic setting across 6 benchmark datasets, where we find that both recurrent neural networks and transfer learning consistently outperform traditional machine learning. Altogether, the findings have considerable implications for the use of affective computing.', "An increasing number of the renowned company's investors are turning attention to stock prediction in the search for new efficient ways of hypothesizing about markets through the application of behavioral finance. Accordingly, research on stock prediction is becoming a popular direction in academia and industry. In this study, the goal is to establish a model for predicting stock price movement through knowledge graph from the financial news of the renowned companies. In contrast to traditional methods of stock prediction, our approach considers the effects of event tuple characteristics on stocks on the basis of knowledge graph and deep learning. The proposed model and other feature selection models were used to perform feature extraction on the websites of Thomson Reuters and Cable News Network. Numerous experiments were conducted to derive evidence of the effectiveness of knowledge graph embedding for classification tasks in stock prediction. A comparison of the average accuracy with which the same feature combinations were extracted over six stocks indicated that the proposed method achieves better performance than that exhibited by an approach that uses only stock data, a bag-of-words method, and convolutional neural network. Our work highlights the usefulness of knowledge graph in implementing business activities and helping practitioners and managers make business decisions.", 'Control of underactuated dynamical systems has been studied for decades in robotics, and is now emerging in other fields such as neuroscience. Most of the advances have been in model based control theory, which has limitations when the system under study is very complex and it is not possible to construct a model. This calls for data driven control methods like machine learning, which has spread to many fields in the recent years including control theory. However, the success of such algorithms has been dependent on availability of large datasets. Moreover, due to their black box nature, it is challenging to analyze how such algorithms work, which may be crucial in applications where failure is very costly. In this paper, we develop two related novel supervised learning algorithms. The algorithms are powerful enough to control a wide variety of complex underactuated dynamical systems, and yet have a simple and intelligent structure that allows them to work with a sparse data set even in the presence of noise. Our algorithms output a bang-bang (binary) control input by taking in feedback of the state of the dynamical system. The algorithms learn this control input by maximizing a reward function in both short and long time horizons. We demonstrate the versatility of our algorithms by applying them to a diverse range of applications including: switching between bistable states, changing the phase of an oscillator, desynchronizing a population of synchronized coupled oscillators, and stabilizing an unstable fixed point. For most of these applications we are able to reason why our algorithms work by using traditional dynamical systems and control theory. We also compare our learning algorithms with some traditional control algorithms, and reason why our algorithms work better. (C) 2020 Elsevier B.V. All rights reserved.'}

</code>