/BiLSTM-Attention_CN_Text_Clf_Pytorch

BiLSTM 加普通Attention中文文本多分类Pytorch实现

Primary LanguagePython

<script type="text/javascript" src="http://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script> <script type="text/x-mathjax-config"> MathJax.Hub.Config({ tex2jax: {inlineMath: [['$', '$']]}, messageStyle: "none" }); </script>

#Neural Networks Introduction ##So what are neural networks?? ###Associationism

  • Humans learn through association ###Connectionism
  • The information is in the connections ####Bain’s Idea 1: Neural Groupings
  • Neurons excite and stimulate each other
  • Different combinations of inputs can result in different outputs
  • Different intensities of activation of A lead to the differences in when X and Y are activated

####Bain’s Idea 2: Making Memories

###Connectionist Machines

Neurons connect to other neurons.The processing/capacity of the brain is a function of these connections All world knowledge is stored in the connections between the elements

  • Neural networks are connectionist machines
  • The machine has many non-linear processing units
  • Connections may also define memory

###The Universal Model Multi-layer Perceptron

  • Neural networks began as computational models of the brain

  • Neural network models are connectionist machines ~ The comprise networks of neural units

  • Multi-layer perceptrons can model arbitrarily complex Boolean functions

  • Multi-layer perceptrons are connectionist computational models – Individual perceptrons are computational equivalent of neurons – The MLP is a layered composition of many perceptrons

  • MLPs can model Boolean functions – Individual perceptrons can act as Boolean gates – Networks of perceptrons are Boolean functions

  • MLPs are Boolean machines – They represent Boolean functions over linear boundaries – They can represent arbitrary decision boundaries – They can be used to classify data

  • MLPs are classification engines – They can identify classes in the data – Individual perceptrons are feature detectors – The network will fire if the combination of the detected basic features matches an “acceptable” pattern for a desired class of signal

  • MLP can also model continuous valued functions

  • MLP can Represent probability distributions – Over integer, real and complex-valued domains – MLPs can model both a posteriori and a priori distributions of data • A posteriori conditioned on other variables – MLPs can generate data from complicated,or even unknown distributions

  • The network is a function

  • Interesting AI tasks are functions that can be modelled by the network