sandkoan/yc_company_scraper

Barebones scraper for listing all YC companies

yc_company_scraper by akshaybhalotia

Barebones scraper for listing all YC companies
View Data · Report Bug · Request Feature

Table of Contents

• About the Project
  • Built With
• Getting Started
  • Dependencies
  • Installing
  • Usage
    • Run locally
  • Help
• Versioning
  • Version History
• Roadmap
• Contributing
• Support
• License
• Acknowledgments
• About Authors

About The Project

This is an attempt to get publicly available data for all Y-Combinator (fondly known as "YC") backed companies, as available at the official "startup directory".

Built With

• jq
• Ruby
• VS Code
• 
• ❤️

Getting Started

You can clone or download the repo, and run the refresh_data.sh to refresh the data. You may also look at the other scripts if you want to run individual tasks or learn more about the project.

If you just want to view the data, you can look into the data directory.

Dependencies

To run these scripts locally, you would need

• Ruby language (if not pre-installed, visit https://www.ruby-lang.org/en/documentation/installation/)

  ruby -v

• jq (download using OS and distribution specific version from https://stedolan.github.io/jq/download/)

Installing

Clone the repo

git clone https://github.com/akshaybhalotia/yc_company_scraper.git

Usage

Run project locally

sh refresh_data.sh

Help

You might encounter 3 kinds of issues:

1. ruby or jq related: Something might change with the future (or past) versions of the Ruby lang or jq tool, causing the scripts to break. This script has been tested on ruby v2.7.x and v3.0.x, and jq v1.6. Your best bet is to read the official changelogs to figure out what changed and how to fix it. Submit a PR too! (Or you could ping me, I'll look at it in my free time - no promises)

2. OS related: I built this script on macOS v12.1, but is mostly generic enough to be run on all UNIX-derived, UNIX-like or Linux-like systems. If you still run into errors, please look at OS specific instructions to run the Ruby lang and jq properly.

3. Data related: Y-Combinator or Algolia may decide to change their APIs or data formats, which is always a risk for scrapers if something at the data source changes. If you can figure out what changed, you might want to submit a PR. (Or, again, you could ping me but no promises)

Versioning

We use SemVer for versioning. For the versions available, see the tags on this repository.

Version History

See CHANGELOG.

Roadmap

See the open issues for a list of proposed features (and known issues).

Contributing

Contributions are what make the open source community such an amazing place to be learn, inspire, and create. Any contributions you make are greatly appreciated. Sincere thanks to all our contributors. Thank you, contributors!

You are requested to follow the contribution guidelines specified in CONTRIBUTING.md and code of conduct at CODE_OF_CONDUCT.md while contributing to the project 😄.

Support

Contributions, issues, and feature requests are welcome! Give a ⭐️ if you like this project. Donate $$ at https://paypal.me/akshaybhalotia if you like this work.

License

This project is licensed under the MIT License - see the LICENSE file for details.

Acknowledgments

This project is inspired by:

• yclist
• yclist code

When I didn't find a source to download the data for all YC companies, and came across this repo with outdated data - I decided to build my own. I hope to have enough time at some point to update the data at yclist repo.

About Authors

yc_company_scraper is created & maintained by Akshay Bhalotia. You can find me on Twitter - @akshay_bhalotia or write to me at opensource [at] axy.one.