/quickpose-ios-sdk

Quickly add MediaPipe Pose Estimation and Detection to your iOS app. Enable powerful features in your app powered by the body or hand.

Primary LanguageSwiftApache License 2.0Apache-2.0

quickpose-ios-sdk

Swift Package Manager

QuickPose provides developer-oriented cutting edge ML features of MediaPipe and BlazePose, with easy integration and production ready code, which dramatically improves the speed of implementation of pose estimation, skeleton tracking and fitness counting features into mobile applications.

See our Features below or checkout our full documentation on our website docs.quickpose.ai/docs/MobileSDK

Range Of Motion Example Leg Raises Counter Example
health-shoulder-right-rom fitness-leg-raises

Register an SDK Key

Get your free SDK key on https://dev.quickpose.ai, usage limits may apply. SDK Keys are linked to your bundle ID, please check Key before distributing to the App Store.

How it works

QuickPose process a video frame and makes it easy for developers to perform complex AI features to the image, such as overlaying markings to the output image to highlight the user's pose.

+----------+          +-------------+          +-----------------+
|          |          |             |          |  Overlay Image  |
|  Camera  |--------->|  QuickPose  |--------->|        +        |
|          |          |             |          |     Results     |
+----------+          +-------------+          +-----------------+

Features

Feature Example Supported
Joint Positions MediaPipe Landmarks v0.1

Overlays

Whole Body

Upper Body

Lower Body

Shoulder

Left Arm

Right Arm

Left Leg

Right Leg

Whole Body Overlay v0.1

Health - Range Of Motion

Left Shoulder

Right Shoulder

Left Hip

Right Hip

Left Knee

Right Knee

Back

Neck

health-shoulder-right-rom v0.3

Distance Independent Measuring Line beta

Measurement relative to body height

Measurement in CM (using body height as input)

feature-scale-independent-ruler v1.1.0

Fitness - Exercise Detection & Counter

Squat

Sumo Squat

Jumping Jack

Push Up

Plank

Cobra Wings

Left Leg Lunge

Right Leg Lunge

SitUps

CobraWings

Plank

Leg Raises

Glute Bridge

Overhead Dumbbell Press

vUps

Lateral Raises

Front Raises

Hip Abduction Standing Left

Hip Abduction Standing Right

Side Lunges Left

Side Lunges Right

Biceps Curls

fitness-pushup v0.8 - v1.1.0

Input

Raised Finger Detection & Count

finger-counter-left v0.5

Input

Thumbs Up Detection, Thumbs up and down detection

thumbs-up v0.6

Meta Features

Feature Example Supported
Stacked Feature Styling shoulder-conditional-image
Bike Side View Video by Tariq Ali
v0.4
Conditional Styling knee-conditional-image v0.4

Fitness - Feedback

Body Position

Named Leg or Arm not visible

fitness-body-feedback v0.8
Orientation Switching landscape v1.0.0

Supported Platforms

iOS Device Silicon Mac (M1, M2, etc) iOS Simulator x86_64 iOS Simulator arm64
✅ Runs ✅ Runs ⚙ Compiles ⚙ Compiles

Requirements

  • iOS 14.0+
  • Xcode 10.0+

Installing the SDK

Swift Package Manager

Step 1: Click on Xcode project file

Step 2: Click on Swift Packages and click on the plus to add a package

Step 3: Enter the following repository url https://github.com/quickpose/quickpose-ios-sdk.git and click next

Import Package

Step 4: Choose all modules and click add package.

Module Description
QuickPoseCore Core SDK (required)
QuickPoseMP Mediapipe Library with all models (one QuickPoseMP variant is required)
QuickPoseMP-lite Mediapipe Lite Library
QuickPoseMP-full Mediapipe Full Library
QuickPoseMP-heavy Mediapipe Heavy Library
QuickPoseCamera Utility Class for Integration (optional, recommended)
QuickPoseSwiftUI Utility Classes for SwiftUI Integration (optional, recommended)

CocoaPods

Step 1: Open your project's Podfile

Step 2: Add your pod file dependencies:

pod 'QuickPoseCore', :git => 'https://github.com/quickpose/quickpose-ios-sdk.git'
pod 'QuickPoseCamera', :git => 'https://github.com/quickpose/quickpose-ios-sdk.git'
pod 'QuickPoseSwiftUI', :git => 'https://github.com/quickpose/quickpose-ios-sdk.git'
Module Description
QuickPoseCore Includes Core SDK and Mediapipe Library (required)
QuickPoseCamera Utility Class for Integration (optional, recommended)
QuickPoseSwiftUI Utility Classes for SwiftUI Integration (optional, recommended)

Step 3: Run pod update from the command line

Getting Started

See code examples below or download our Sample Apps.

Getting Started with Newer Macs M1/M2

Step 1: Download/Clone Repo

Step 2: Open Basic Demo

Step 3: Choose Build Target "My Mac (Designed For iPad/iPhone)"

Step 4: Run

Getting Started With Mac Picture

Step 5: Explore the features and returned results

quickPose.start(features: [.overlay(.upperBody)], onFrame: { status, image, features, feedback, landmarks in
    if case .success(_) = status {
        overlayImage = image
    }
})

Getting Started with Older Intel Macs

Step 1: Download/Clone Repo

Step 2: Open Basic Demo

Step 3: Choose Build Target as your physical device

Step 5: You will need to change the bundleid and register with apple if you haven't already.

Step 5: Run

Step 6: Explore the features and returned results

quickPose.start(features: [.overlay(.upperBody)], onFrame: { status, image, features,  feedback, landmarks in
    if case .success(_) = status {
        overlayImage = image
    }
})

SwiftUI Example

import SwiftUI
import QuickPoseCore
import QuickPoseSwiftUI

....

struct QuickPoseBasicView: View {
    
    private var quickPose = QuickPose(sdkKey: "YOUR SDK KEY HERE") // register for your free key at https://dev.quickpose.ai
    @State private var overlayImage: UIImage?
    
    var body: some View {
        GeometryReader { geometry in
            ZStack(alignment: .top) {
                QuickPoseCameraView(useFrontCamera: true, delegate: quickPose)
                QuickPoseOverlayView(overlayImage: $overlayImage)
            }
            .frame(width: geometry.size.width)
            .edgesIgnoringSafeArea(.all)
            .onAppear {
                quickPose.start(features: [.overlay(.userLeftArm)], onFrame: { status, image, features,  feedback, landmarks in
                    if case .success(_) = status {
                        overlayImage = image
                    }
                })
            }.onDisappear {
                quickPose.stop()
            }
            
        }
    }
}

Documentation

Checkout our full documentation at https://docs.quickpose.ai/docs/MobileSDK including more SwiftUI and UIKit examples.

Troubleshooting

No Such Module

Xcode reports error no such module QuickPoseCore or no such module QuickPoseSwiftUI

This happens when the linker cannot find the provided XCFrameworks. These needs to be added to your build Target.

xcode troubleshooting no such module error

xcode troubleshooting no such module guide