AccessibleReality: A Swift repository from aheze

My WWDC21 Swift Student Challenge submission

I made a playground book that teaches you the basics of ARKit through interactive lessons. It covers positioning, hit-testing, and how to calculate relationships between objects. In the end, it combines all these concepts into an app: A navigation aid for those who are visually impaired.

Status: Accepted LETS GOOOOOOOOOOOOO!!!!

Try it out: Download link

See other 2021 submissions! wwdc/2021 congrats everyone!

Screenshots

Intro	Positioning	Distances	Angles	Completed App

Tips and Tricks

Want to make your own playground book? Here's some tips and tricks that I picked up while making mine.

Development

Instead of dragging files into the project, create them locally (Command + N), then copy over the contents. Whenever I dragged in a file, I got a "There was a problem running this page" error.
Use ARKit instead of RealityKit. The camera preview works, but I was never able to see my objects in the scene.
Pre-compile ML models if possible. See this article for details.
To change the name of the playground, edit the BuildSettings.xcconfig file
You might need to add required capabilities in the Manifest.plist file, for example arkit. See here for the possible values.

Storyboards

Every time you make changes in the storyboard, clean the project (Shift + Command + K), otherwise your changes won't show
The Assistant editor doesn't work, so you need to manually add a new editor using the button
For any views or view controllers that you put in the storyboard, make sure that the class has the @objc(BookCore_YOURCLASSNAME) tag. For example:

@objc(BookCore_SceneViewWrapper)
class SceneViewWrapper: UIView { }

Then, in the storyboard, for the Class put BookCore_YOURCLASSNAME. Make sure Module is set to None and Inherit Module From Target is unchecked. Your custom class should look like this:

Debugging

To avoid the "There was a problem running this page" error, turn on Authoring Debug Mode in the Settings app. This gives you the full stack trace! If only I knew this earlier... I was commenting out lines of code until the error went away, AirDropping to my iPad every time. Go upvote their answer on Stack Overflow now.
You can get the playground page's current text, including the user-modified code, with PlaygroundPage.current.text. Put the below code in viewDidLoad to see what it is (for testing purposes).

DispatchQueue.main.asyncAfter(deadline: .now() + 1.0) {
    let text = PlaygroundPage.current.text
    let label = UILabel()
    self.view.addSubview(label)
    label.translatesAutoresizingMaskIntoConstraints = false
    NSLayoutConstraint.activate([
        label.topAnchor.constraint(equalTo: self.view.topAnchor),
        label.rightAnchor.constraint(equalTo: self.view.rightAnchor),
        label.bottomAnchor.constraint(equalTo: self.view.bottomAnchor),
        label.leftAnchor.constraint(equalTo: self.view.leftAnchor)
    ])
    label.numberOfLines = 0
    label.text = text /// show the text
}

Get Help

I probably wouldn't have finished if not for these resources.

License

MIT License

Copyright (c) 2021 A. Zheng

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

aheze/AccessibleReality