How to run Firecrawl convert web to Markdown

1. Clone the project and install dependencies

git clone https://github.com/JYC0413/firecrawl-integration.git
cd firecrawl-integration
npm install

2. Modify environment variables to add your FireCrawl key

You can get this key from FireCrawl API keys page.

If on MacOS/Linux

export FIRECRAWL_KEY="your_api_key_here"

as a temporary environment variable

If on Windows, when using CMD

set FIRECRAWL_KEY=your_api_key_here

When using PowerShell

$env:FIRECRAWL_KEY="your_api_key_here"

as a permanent environment variable

Use CMD in windows, enter the following command and open a new window to take effect

setx FIRECRAWL_KEY=your_api_key_here

Add .env file in the current directory

FIRECRAWL_KEY=your_api_key_here

3. Run crawlWebToMd.js

node crawlWebToMd.js

Follow the prompts to enter the URL you want to convert and the quantity limit.

If you provide a URL that does not end with /, we will assume that you only want to convert this page to Markdown

Note: If you want to crawl the entire website, your usage may be consumed quickly or may not be sufficient. I recommend setting a very small value, such as 10, if you just want to try out this feature and do not have an actual need.

After that, you will get the result.