/durablefunctions-apiscraping-nodejs

Build an Azure Durable Functions that will scrape GitHub for opened issues and store them on Azure Storage.

Primary LanguageJavaScriptMIT LicenseMIT

page_type products languages name urlFragment description
sample
azure
azure-functions
azure-storage
javascript
nodejs
Retrieve opened issue count on GitHub with Azure Durable Functions (Node)
retrieve-opened-issue-count-on-github-with-azure-durable-functions
Build an Azure Durable Functions that will scrape GitHub for opened issues and store them on Azure Storage.

Retrieve opened issue count on GitHub with Azure Durable Functions

Build

The project requires the latest version of the Azure Functions CLI.

It can be installed by running the following code:

npm i -g azure-functions-core-tools@core --unsafe-perm true

More installation options are also available.

Running the Sample

Pre-requisite

Locally

Change the GitHubToken value in local.settings.json to match your GitHub Personal Access Token created previously in the pre-requisite.

To ensure that everything is configured properly run the following commands within the FanOutFanInCrawler folder.

func extensions install
npm install

Once the dependencies has been installed, you can either run it from VS Code by running code . and hitting F5 on your keyboard. If you want to run it from the command line, you can run it with the following command within the FanOutFanInCrawler folder.

func host start

On Azure

To deploy components on Azure you will require to install and login with the Azure CLI.

First, you will need to provision the service. Look into the provision.ps1 file provided to familiarize yourself with the resources we are going to create.

Then you can execute the file with the previously generated GitHub.

.\provision.ps1 -githubToken <TOKEN> -resourceGroup <ResourceGroupName> -storageName <StorageAccountName> -functionName <FunctionName>

If you do not have PowerShell installed, you can simply take the commands within the file and run it manually.

Contribute

You can contribute to this sample by following the guidelines.

Resources