/NeedleAndARjs

Primary LanguageTypeScript

First test of Needle Engine with AR.js

This is just a quick test of how to integrate AR.js into the Needle Engine (https://github.com/needle-tools/needle-engine-support). A working prototype can be found here - https://palm-steep-nectarine.glitch.me/ Make sure to have a hiro marker available to test the tracking (https://github.com/AR-js-org/AR.js/blob/master/data/images/HIRO.jpg).

The sample is based on the default three.js marker tracking sample (https://github.com/AR-js-org/AR.js/blob/master/three.js/examples/basic.html), with a few adjustments. The adjusted script can be found at ./Projects/IntegrationOfARjs/src/scripts/ARjsMarkerTracking.ts

Pls note that this is just a POC. I won't provide actual support for this, apart from my personal development interests. For any questions about Needle Engine reach out to https://needle.tools/ and join their Discord. The community is lively and really helpful! Great team there :)

Steps for creating this POC

  • Install Unity and integrate Needle-Engine (https://github.com/needle-tools/needle-engine-support/blob/main/documentation/getting-started.md)
  • Create a new scene with the Needle-Minimal template and make sure your VSCode project (generated by Needle Engine) is working and syncing correctly.
  • Make sure that the Camera GameObject is at position 0,0,0 and rotation 0,180,0 to successfully show the tracked content.
  • Add AR.js via npm to your project (https://www.npmjs.com/package/@ar-js-org/ar.js)
  • Implementation of the simple AR.js Marker-Tracking sample (https://github.com/AR-js-org/AR.js/blob/master/three.js/examples/basic.html) with three.js in a Unity-like style, to make it available as Unity component - ARjsMarkerTracking.ts
  • Add an empty GameObject to the SceneRoot, set position and rotation to 0,0,0 and scale to 1,1,1 and name it ARjsMarkerTracking.
  • Add the Unity component to an empty GameObject in your scene.
  • Create a new Sphere as Trackable and link it to your ARjsMarkerTracking component.
  • Make sure that the Sphere is located at 0,0,0 has rotation 0,0,0 and scale 1,1,1.
  • Open your project in VSCode via die VS Workspace button on the Export GameObject.
  • In VSCode open the IntegrationOfARjs folder, select the style.css and change the background-color property of #main from black to transparent! (It's already changed in this repo) Otherwise your camera image will not be visible. This may not be necessary with a newer version of needle tools. Please keep that in mind!
  • Wait for Unity to rebuild and launch it on your local machine to test. :)

Where is the AR.js stuff located?

When following the path Projects/IntegrationOfARjs/src/scripts/ARjsMarkerTracking.ts you'll find the Typescript component for handling all the AR.js stuff. It's pretty straight forward, implementing a very basic sample and contains example code to add objects via three.js (from line 88 to 96). Feel free to add e.g. a normals-material cube, ...

Issues

  • AR.js was added via npm (https://www.npmjs.com/package/@ar-js-org/ar.js).
    • There are no typescript types yet, so you'll have to know how to handle three.js.
    • After opening the Unity project, make sure to rebuild everything once, otherwise AR.js will not be started. You can to that, by opening the VSCode project, change one line, save and wait for rebuilding to complete.
  • When on mobile, tracking works best in Portrait mode and in Chrome. When changing to Landscape the tracking and camera image will behave strange. This is likely an issue of AR.js, issue is already opened.
  • The tracking will have a small offset, due to some resizing issues. Make sure to not resize your browser window while using this prototype.
  • Works with notebook webcam too, but the tracking is squishy. Make sure to have a high contrast hiro marker (https://github.com/AR-js-org/AR.js/blob/master/data/images/HIRO.jpg) available.