question: how do you configure and determine the layer with dense against the vocab?
kenken64 opened this issue · 2 comments
model.add(tf.layers.dense({ units: 2, inputShape: [vocabulary.length] }));
model.add(tf.layers.dense({ units: 1, inputShape: [2] }));
if i add new vocab how can i reconfigure the dense layer?
Simple explanation of how the TF model & the vocab are configured
The model is configured to work with vocabulary shaped like this:
[ string, string, string, string... n - string ]
Then when you're training or trying to get prediction there is a fitData method, which receives string as input and returns sequence of numbers which tensorflow can understand. For example:
Input: "This app is really bad and the design ugly. It crashes and shows me a lot of ads. Just bad."
Vocabulary: [ "bad", "ugly", "crashes", "slow", "ads", "good", "beautiful", "fast", "worth" ]
Output of the fitData method: [ 2, 1, 1, 0, 1, 0, 0, 0, 0 ]
The output is an array, similar to the vocab but contains numbers - how many times a word from the vocab is used. Then a model is trained like this:
x layer: [ [ fitData output ], [ ... ], [ ... ], ... ]
y layer: [ [ number ], [ number ], ... ]
The y layer is a 2d tensor, with a single integer for each value. In my case I use zeros and ones.
0 - negative, 1 - positive
But you can also set it up with more complex values to get more than 2 results. That's helpful when you try to label something.
Your question, @kenken64
The first layer (x) is initialized by tf.layers.dense({ units: 2, inputShape: [vocabulary.length] }), with inputShape equial to vocabulary.length, because it contains int value for each vocab item.
if i add new vocab how can i reconfigure the dense layer?
- If you wanna change the vocab items but keep the current shape, you won't need any changes on the model or its layers.
[ string, string, ... n strings ] // The length is not in significant importance, because the model automatically gets it via vocab.length- If you think that the current shape won't fit your need, you should rethink the model. I can help you, but in order to keep the issues specific and helpful to others you can chat me on my e-mail rsg.group.here@gmail.com or join my Slackroom - sign up here http://rsg-slack.herokuapp.com/ and explaining your specific use case and ideas.
Resources
Few sources which might be helpful to you and anyone else:
- https://www.quora.com/What-is-the-bag-of-words-algorithm
- https://en.wikipedia.org/wiki/Bag-of-words_model
- https://machinelearningmastery.com/gentle-introduction-bag-words-model/
- https://www.youtube.com/watch?v=hgQTIMrGwtg
- https://cloud.google.com/blog/products/gcp/intro-to-text-classification-with-keras-automatically-tagging-stack-overflow-posts