rovertm/eVTOL-Aircraft-Data-Extraction

Python web scraping tools for extracting eVTOL aircraft data and maintaining a dataframe.

eVTOL Aircraft Data Extraction and Analysis

Web scraping tools for eVTOL aircraft data extraction maintaining a dataframe

Table of Contents

1. About
2. Use Cases
3. Installation
4. File Descriptions
5. Licensing, Authors, and Acknowledgements
6. View the notebooks

About

This repository contains tools for scraping eVTOL aircraft data from two websites:

1. https://evtol.news/
2. https://transportup.com/

Each set of scraper functions, contained in individual packages, follows an identical process architecture show below.

Use Cases

Source data for:

• Market research
• Competitive intelligence

Installation

• BeautifulSoup
• Standard libraries across Python 3* Anaconda distribution.

File Descriptions

• Jupyter Notebook for analysis:

  • evtolnews_scrape.ipynb
  • transportup_scrape.ipynb

• Python files for custom functions

  • transportup_funcs.py
  • evtolnews_scrapefuncs.py

• .csv files for read and writing to directory and results dataframes

  • tu_directory_{date}.csv --> dataframe of aircraft links (to scrape)

  • tu_results_df_{date}.csv --> dataframe of scraped and cleaned aircraft data

  • evtolnews_directory_{date}.csv --> dataframe of aircraft links (to scrape)

  • evtolnews_results_df_{date}.csv --> dataframe of scraped and cleaned aircraft data

Licensing, Authors, Acknowledgements

All code is open for any and all usage.

View the notebooks

View the evtol.news scraper at Jupyter's NBviewer site, click here.

View the TransportUP scraper at Jupyter's NBviewer site, click here.