forge-bim360search

Description

Demonstrate how to index metadata into a Elasticsearch database (AWS Service) and search on it. This sample recursively iterate through all projects of the selected BIM 360 Account (Hub), then into the Project Files folder, then subfolders, then the last version of an item, get all the metadata and send to Elasticsearch as a text (not JSON). When a new file is uploaded, a webhook triggers and update the index. Once the index is done, one can search and redirect to the BIM 360 file.

Thumbnail

Setup

Prerequisites

Forge Account: Learn how to create a Forge Account, activate subscription and create an app at this tutorial.
Visual Studio: Either Community (Windows) or Code (Windows, MacOS).
.NET Core basic knowledge with C#
ngrok: Routing tool, download here
MongoDB: noSQL database, learn more. Or use a online version via mLab (this is used on this sample)
AWS Account with Elasticsearch: create or use your AWS Account, learn more.

Running locally

Clone this project or download it. It's recommended to install GitHub desktop. To clone it via command line, use the following (Terminal on MacOSX/Linux, Git Shell on Windows):

git clone https://github.com/autodesk-forge/forge-bim360search

Amazon Elasticsearch

Create or use your AWS Account. Go to Elasticseach dashboard and Create new domain.

Step 1: you may consider using Development and testing:Development and testing.
Step 2: type yourElasticsearch domain name. Other settings as default or adjust as needed.
Step 3: depending on your Network configuration, or select Public access. Select the appropriate Access policy, this sample uses Allow or deny access to one or more IAM users. Select the appropriate user. This step requires a IAM user.
Step 4: review and confirm.

The creating may take a few minutes, after that take note of the Endpoint.

MongoDB

MongoDB is a no-SQL database based on "documents", which stores JSON-like data. For testing purpouses, you can either use local or live. For cloud environment, try MongoDB Atlas (offers a free tier). With MongoDB Atlas you can set up an account for free and create clustered instances, intructions:

Create a account on MongoDB Atlas.
Under "Collections", create a new database (e.g. named bim360search) with a collection (e.g. named users).
Under "Command Line Tools", whitelist the IP address to access the database, see this tutorial. If the sample is running on Heroku, you'll need to open to all (IP 0.0.0.0/0). Create a new user to access the database.

At this point the connection string should be in the form of mongodb+srv://<username>:<password>@clusterX-a1b2c4.mongodb.net/inventor2revit?retryWrites=true. Learn more here

There are several tools to view your database, Robo 3T (formerly Robomongo) is a free lightweight GUI that can be used. When it opens, follow instructions here to connect to MongoDB Atlas.

Visual Studio (Windows):

Right-click on the project, then go to Debug. Adjust the settings as shown below. For environment variable, define the following:

ASPNETCORE_ENVIRONMENT: Development
FORGE_CLIENT_ID: your id here
FORGE_CLIENT_SECRET: your secret here
FORGE_CALLBACK_URL: http://localhost:3000/api/forge/callback/oauth
FORGE_WEBHOOK_URL: http://YOUR HOST OR NGROK ADDRESS
FORGE_PARALLEL_JOBS: Number of parallels jobs to get metadata, this affects the memory usage
OAUTH_DATABASE: mongodb+srv://<username>:<password>@clusterX-a1b2c4.mongodb.net/bim360search?retryWrites=true
ELASTIC_SEARCH_SERVER: https://yourSearchEndpoint.region.es.amazonaws.com
AWS_REGION: where the Elastic Search is running, e.g. us-east-1
AWS_ACCESS_KEY: your AWS access key here
AWS_SECRET_KEY: your AWS secret key here

Visual Sutdio Code (Windows, MacOS):

Open the folder, at the bottom-right, select Yes and Restore. This restores the packages (e.g. Autodesk.Forge) and creates the launch.json file. See Tips & Tricks for .NET Core on MacOS.

At the .vscode\launch.json, find the env vars and add your Forge Client ID, Secret and callback URL. Also define the ASPNETCORE_URLS variable. The end result should be as shown below:

"env": {
    "ASPNETCORE_ENVIRONMENT": "Development",
    "ASPNETCORE_URLS": "http://localhost:3000",
    "FORGE_CLIENT_ID": "your id here",
    "FORGE_CLIENT_SECRET": "your secret here",
    "FORGE_CALLBACK_URL": "http://localhost:3000/api/forge/callback/oauth",
    "FORGE_WEBHOOK_URL": "http://YOUR HOST OR NGROK ADDRESS",
    "FORGE_PARALLEL_JOBS": "20",
    "OAUTH_DATABASE": "mongodb+srv://<username>:<password>@clusterX-a1b2c4.mongodb.net/bim360search?retryWrites=true",
    "ELASTIC_SEARCH_SERVER": "https://search-yourElasticSearch.region.es.amazonaws.com",
    "AWS_REGION": "where the Elastic Search is running, e.g. us-east-1",
    "AWS_ACCESS_KEY": "your AWS access key here",
    "AWS_SECRET_KEY": "your AWS secret key here"
},

Open http://localhost:3000 to start the app. Select Index my BIM 360 Account before using (this process may take a while). Check the http://localhost:3000/dashboard to see the jobs running (Hangfire dashboard).

Deployment

Due to memory usage, it's likely that this sample will crash on Heroku free & hobby tier (limited to 512Mb of memory). It may work if drawing do not have much metadata.

To deploy this application to Heroku, the Callback URL for Forge must use your .herokuapp.com address. After clicking on the button below, at the Heroku Create New App page, set your Client ID, Secret and Callback URL for Forge.

Autodesk-Forge/forge-bim360search

forge-bim360search

Description

Thumbnail

Setup

Prerequisites

Running locally

Deployment

Further Reading

Tips & Tricks

Troubleshooting

License

Written by