/github-scraping

抓取github star 的项目

Primary LanguageTypeScript

TODO

  1. github star 有很多没有分类的都遗漏了，应该直接抓取所有的数据。

  2. 不仅应该抓取所有摘要数据，应该再抓取每个项目的 readme

  3. 数据集是否可以多语言翻译，再喂给llm

Turborepo starter

This is an official starter Turborepo.

Using this example

Run the following command:

npx create-turbo@latest

What's inside?

This Turborepo includes the following packages/apps:

Apps and Packages

  • docs: a Next.js app
  • web: another Next.js app
  • @repo/ui: a stub React component library shared by both web and docs applications
  • @repo/eslint-config: eslint configurations (includes eslint-config-next and eslint-config-prettier)
  • @repo/typescript-config: tsconfig.jsons used throughout the monorepo

Each package/app is 100% TypeScript.

Utilities

This Turborepo has some additional tools already setup for you:

Build

To build all apps and packages, run the following command:

cd my-turborepo
pnpm build

Develop

To develop all apps and packages, run the following command:

cd my-turborepo
pnpm dev

Remote Caching

Turborepo can use a technique known as Remote Caching to share cache artifacts across machines, enabling you to share build caches with your team and CI/CD pipelines.

By default, Turborepo will cache locally. To enable Remote Caching you will need an account with Vercel. If you don't have an account you can create one, then enter the following commands:

cd my-turborepo
npx turbo login

This will authenticate the Turborepo CLI with your Vercel account.

Next, you can link your Turborepo to your Remote Cache by running the following command from the root of your Turborepo:

npx turbo link

Useful Links

Learn more about the power of Turborepo: