In this Jupyter Notebook, we will explore different aspects of NLP (Natural Language Processing). We will use well report summaries which are published by the NPD (Norwegian Petroleum Directorate), by requesting via an API. We will go through the different stages of text cleaning and formatting before moving on to NLP processing: tokenization, lemmatization and finally topic modeling.
Please check out notebook here:
/notebooks/NLP_transform21_v01_work.ipynb
Link to presentation video: https://youtu.be/Sh_zgiptTqw
- Get request to NPD wellbore tables via API, we are interested in:
- wells with coordinates (with-coordinates) and
- wells with report summary (with-history)
- Reformating using pandas dataframes - merging of dataframe with coordinates together with dataframe with report summaries
- Visualisation on maps for QC and evaluation of how the data is distributed geographically (North Sea, Norwegian Sea, Barents)
- Cleaning of summaries (removing tags, etc)
- Pre processing for NLP (removing stopwords, tokenisation, lemmatization)
- Exploring wells with WordClouds
- Further NLP analysis: topic modeling and visualisation with pyLDAvis
- Conclusions, what to do next?
