Current Challenges
- Transfer learning: Unable to transfer representation to most reasonably related domains except in specialized formulations.
- Understanding: Lacks “reasoning” or ability to truly derive “understanding” as previously defined on anything but specialized problem formulations. (Definition used: Ability to turn complex information to into simple, useful information.)
- Requires big data: inefficient at learning from data
- Requires supervised data: costly to annotate real-world data
- Not fully automated: Needs hyperparameter tuning for training: learning rate, loss function, mini-batch size, training iterations, momentum, optimizer selection, etc.
- Reward: Defining a good reward function is difficult.
- Transparency: Neural networks are for the most part black boxes (for realworld applications) even with tools that visualize various aspects of their operation.
- Edge cases: Deep learning is not good at dealing with edge cases.