/freelance-job-scraper

This is a freelance job scraper built using Ruby by following OOP. It takes keywords from the user input, scrapes 3 job sites and exports them to 3 separate CSV files.

Primary LanguageRubyMIT LicenseMIT

Run on Repl.it

Freelance Job Scraper

Ruby Capstone Project of Microverse, which students have to complete a real-world-like project within 72 hours according to this project specifications

This web scraper is a freelance job scraper that scrapes freelance jobs from these three websites: freelancer.com, guru.com, peopleperhour.com with keywords taken from the user input and exports them to 3 separate CSV files.

demo-image

Built With

  • Ruby
  • Nokogiri gem
  • HTTParty gem

Project Structure

├── README.md
├── bin
│   └── main.rb
└── lib
    └── scraper.rb
    └── csv_exporter.rb
└── rspec
    └── scraper_spec.rb
    └── csv_exporter_spec.rb
    └── spec_helper.rb

Video Presentation

Feel free to check out this link for a 5min video walkthrough :)

Deployment

  1. Git clone this repo and cd the to the freelance-job-scraper directory.
  2. Run bundle install in command line to install Nokogiri and HTTParty Gem.
  3. Run ruby bin/main.rb.
  4. Input keywords in separate lines. Press enter key on a new line to begim the search.
  5. Tada! 'freelance.com-jobs.csv', 'guru.com-jobs.csv', and 'peopleperhour.com-jobs.csv' would be created at the root directory respectively :)

Run tests

  1. Git clone this repo and cd the to the freelance-job-scraper directory.
  2. Install rspec with gem install rspec.
  3. Run rspec in Command Line.
  4. 15 examples, 0 failures will be shown on the screen.

Future releases

  • User can select which sites they want to scrape
  • Add more sites for scraping
  • User can specify how many jobs he/she wants to scrape

Author

👤 Tirthajyoti Ghosh

🤝 Contributing

Contributions, issues and feature requests are welcome!

Feel free to check the issues page.

Show your support

Give a ⭐️ if you like this project!

Acknowledgments

  • Microverse
  • Nokogiri gem
  • HTTParty Parser
  • Freelancer.com
  • Guru.com
  • Peopleperhour.com

📝 License

This project is MIT licensed.