AR-js-org/AR.js

ES6 feature in master and beyond: AR.js next

kalwalt opened this issue · 7 comments

kalwalt commented

ES6 feature in master and beyond: AR.js next

I merged the ES6 feature 2 days ago into master, i think this is a great step for AR.js. Tough this means that is not assured to be bug free. We converted all the code to the ES6 standard but maybe we missed something, please if you find some inconsistencies or issue file here or open a new one describing the "glitch" you found.
Anyway there are a couple of things to point out:

  • Now AR.js not rely anymore on jsartoolkit5 but on the new one artoolkit5-js with ES6 support. It can be imported as a module and i will improve it as i can.
  • Aframe and three.js examples should works but i can't test on many devices, i haven't them...
  • As in the previuos version all the classes are preserved, no real changes in this sense. Rather we tried to improve them and fix some bugs we found.
  • For Pattern and Barcode marker you need to provide the orientation of the screen see intiContext:

    AR.js/three.js/examples/basic.html

    Lines 101 to 130 in 4b5d84f

    function initARContext() { // create atToolkitContext
    arToolkitContext = new THREEx.ArToolkitContext({
    cameraParametersUrl: THREEx.ArToolkitContext.baseURL + '../data/data/camera_para.dat',
    detectionMode: 'mono'
    })
    // initialize it
    arToolkitContext.init(() => { // copy projection matrix to camera
    camera.projectionMatrix.copy(arToolkitContext.getProjectionMatrix());
    arToolkitContext.arController.orientation = getSourceOrientation();
    arToolkitContext.arController.options.orientation = getSourceOrientation();
    console.log('arToolkitContext', arToolkitContext);
    window.arToolkitContext = arToolkitContext;
    })
    // MARKER
    arMarkerControls = new THREEx.ArMarkerControls(arToolkitContext, camera, {
    type: 'pattern',
    patternUrl: THREEx.ArToolkitContext.baseURL + '../data/data/patt.hiro',
    // patternUrl : THREEx.ArToolkitContext.baseURL + '../data/data/patt.kanji',
    // as we controls the camera, set changeMatrixMode: 'cameraTransformMatrix'
    changeMatrixMode: 'cameraTransformMatrix'
    })
    scene.visible = false
    console.log('ArMarkerControls', arMarkerControls);
    window.arMarkerControls = arMarkerControls;
    }

    and few lines after getSourceOrientation:

    AR.js/three.js/examples/basic.html

    Lines 133 to 150 in 4b5d84f

    function getSourceOrientation() {
    if (!arToolkitSource) {
    return null;
    }
    console.log(
    'actual source dimensions',
    arToolkitSource.domElement.videoWidth,
    arToolkitSource.domElement.videoHeight
    );
    if (arToolkitSource.domElement.videoWidth > arToolkitSource.domElement.videoHeight) {
    console.log('source orientation', 'landscape');
    return 'landscape';
    } else {
    console.log('source orientation', 'portrait');
    return 'portrait';
    }

    or create a new instance of ARjs-session as in the arjs-session example. The routine is implemented in its core you don't need to write nothing:

    AR.js/three.js/src/new-api/arjs-session.js

    Lines 91 to 119 in 4b5d84f

    window.addEventListener('arjs-video-loaded', function () {
    arContext.init(() => {
    arContext.arController.orientation = getSourceOrientation();
    arContext.arController.options.orientation = getSourceOrientation();
    })
    })
    function getSourceOrientation() {
    console.log(_this);
    if (!_this) {
    return null;
    }
    console.log(
    'actual source dimensions',
    arSource.domElement.clientWidth,
    arSource.domElement.clientHeight
    );
    if (arSource.domElement.clientWidth > arSource.domElement.clientHeight) {
    console.log('source orientation', 'landscape');
    return 'landscape';
    } else {
    console.log('source orientation', 'portrait');
    return 'portrait';
    }
    }
  • Image tracking has no improvements, sorry we will do in another step.
  • Three.js has also a new location based module, this is still experimental so, of course, to be improved.
  • Multimarkers: it should works if you load a predefined set but we hadn't worked on learner and player, and honestly i will not work on. If you need them we are open to PR.
  • test: this will done in another time ( i think they are outdated). We can setup our CI with github action or travis (actual badge point to the old repository!)

Future steps

I released 3.4.0-alpha-rc1 because i think the code is not still stable. It needs more testing and feedback from devs. As written in te Readme you can install as a npm package:

npm install @ar-js-org/ar.js

or with yarn:

yarn add @ar-js-org/ar.js

I created a simple example with react: react-threex-example but i think need some work yet. You can find other resources in this issue #234
Coming back to the version: we should do in little steps as we find bugs and try to solve them and release a stable one in anear future.
So what are the next steps in the near future in my opinion?

  • First, reduce the size of the package and repository. We should try to remove the double libs nft and no-nft, this was caused by a old bug inside jsartoolkit5, but we should have solved it with artoolkit5-js. I haven't tested, so i'm not 100% sure. This will reduce the size a lot.
  • Improve image tracking adding arjs-nft-init-data event listener as in #114
  • CI tests and improving existing tests.
  • Maybe move the threex classes externally in a separate package? If you works only with them does not make sense to download the whole package. With the new standard we should promote modularity as we can.
  • Improve onResize: adding the feature to override it and/or to skip it and customize it.
  • Type definitions?

In the not near future:

  • Developing AR.js-next with a complete rewrite to make the project more flexible and near to recent standard and techs.
kalwalt commented

Aframe nft examples does not works, it load partially but the worker is not created. Can't understand why didn't happened before. I will fix this as soon as i will found the reason fo this bug.

kalwalt commented

New release 3.4.0-alpha-rc2 we fixed some bugs and made some improvements:
First, aframe image tracking (NFT) was not loading the nft marker, the issue was in the AddEventListener (arjs-session) see #407 for more. This is not ideal and we should investigate why it block the nft loading, for now i simply create a separate file.
We improved three.js location based code thanks to @nickw1, fixed some small bug and added new sweeties.
I implemented prettier as code formatter and husky to lint while making a commit to prevent code bugs. As a consequence an action github script check the code linting quality, this is a first step to develop a build script that will build all the libs and deploy an automatic release.

  • 👍1
rafelis1997 commented

I think on resize may be causing the models to tilt in the marker based tracking. It tends to happen mostly when in landscape mode ou when the user causes a rotation on the device when the scene is loaded on the marker. Maybe a rebuild scene function on the device orientation changes could fix the issue

  • 👍2
kalwalt commented

I plan to add Typescript support to AR.js and to artoolkit5-js, see this issue AR-js-org/artoolkit5-js#20. I created also this other issue AR-js-org/artoolkit5-js#19 mentioning SIMD feature. I have no idea when this can be done, but this is on my tick list now!

  • 👍1
kalwalt commented

I started to develop the AR.js-core package see this issue AR-js-org/AR.js-core#2 this will be a first step to the new AR.js-next generation library. Please tell us your opinion about this idea. This however it will be done in the next year, due to lack of free time in this part of the year.

  • 🚀1
nickw1 commented

@kalwalt looks good to me, definitely makes sense to have the core more rendering-framework agnostic.

  • 👍1