technophilist/Lookup

A fully AI powered landmarks recognition app built with Gemini Client SDK / OpenAI-API and tensorflow-lite.

Lookup - A fully AI powered Landmarks Recognition App

Lookup is a fully AI powered Android app that lets you explore the world's wonders in a whole new way! Simply take a photo of a famous monument, and the app will instantly recognize it and generate fascinating descriptions, answer your questions, and even write unique articles with your preferred writing tone about the place.

Table of Contents

1. Demo
2. Screenshots
3. Tech Stack
4. Remote API's / Client SDK's
5. Nptable Features
6. Source code, Architecture, & Testing
7. Underlying Model
8. Building and running the app

Demo

Lookup.-.Demo.mp4

Screenshots

Tech Stack

• Entirely written in Kotlin.
• CameraX for accessing and processing the images captured from the camera.
• TensorFlow Lite for image recognition.
• Hilt for dependency injection.
• Jetpack Compose for UI and navigation.
• Coil compose for image loading and caching.
• Coil-gif for loading and displaying gif's.
• Lottie compose for displaying animations.
• Kotlin Coroutines for threading.
• Kotlin Flows for creating reactive streams.
• Work Manager for persistent long-running background tasks.
• Retrofit for communicating with the OpenAI API.
• Room for database.
• Moshi + Moshi Kotlin CodeGen for deserializing responses from the API.
• Uses Kotlin Gradle DSL.

Underlying Model

Under the hood, the app uses the classifier-north-america-v1 image classifier built by Google. So, the app will be able to recognize all the famous landmarks in the north america region that the classifier was trained to identify.

Remote API's / Client SDK's

• Gemini Client SDK
• OpenAI API
• Unsplash API

Notable features

Haptic Feedback 🪄

The app subtly improves the user-experience by leveraging the on-device vibration actuators to provide a haptic feedback when certain events occur in the app such as clicking the shutter button or displaying the bottom sheet once the app has recognized the landmark.

Dynamic Colors & Light/Dark Themes 🎨

The app not only supports dynamic colors on devices that support it, but also switches to separate defined light and dark themes (based on whether the device is in dark / light mode) helping it to provide a more cohesive user experience.

Themed App Icons (Android 13+) 🌈

The app also supports the "Themed Icons" feature available on Android 13 and later. If the user has opted in for the feature on a device that is running Android 13 and higher, the app's icon will be tinted to inherit the coloring of the user’s chosen wallpaper and other themes.

Intelligent Caching 🔮

The app caches the results of many API calls helping to not only improve the average response time, but also to reduce the number of calls being made to the underlying RESTful services. Moreover, the app also leverages WorkManger to prefetch the different articles associated with a specific bookmarked location before the user even navigates to the detail screen where the article(s) are displayed. This helps in reducing the average Time To Initial Display (TTID) of the associated screen.

Meta Prompting 🤖

The app uses the technique of meta-prompting to display possible questions (as a suggestion chip) that a user might have about a specific identified location.

Source code, Architecture, & Testing

• Uses multi-repository pattern.
• Commit messages follow the Conventional Commits specification.
• Consists of unit tests that predominantly test the data layer.

Building and running the app

1. Create an unsplash developer account and get the api token from https://unsplash.com/documentation.
2. In the local.properties file of your project add the api key as shown below.

UNSPLASH_API_ACCESS_KEY = PASTE-YOUR-TOKEN-HERE

3. The app can be made to either use Gemini or Chat-GPT for it's contextual text generation features. To use either of the two, make sure to get the access / api key for the chosen LLM service, and paste it in the local.properties file as follows. For the service that you've chosen not to use, make sure to enter a random string as the access key since the existence of both the keys are required for the app to build.

UNSPLASH_API_ACCESS_KEY = PASTE-YOUR-TOKEN-HERE
GOOGLE_GEMINI_API_KEY = PASTE-YOUR-TOKEN-HERE

5. By default, the app uses Gemini for text generation. If you want to use Chat-GPT, then replace the '@GeminiClient' di qualifier with the @OpenAiClient qualifier in all places where an instance of TextGeneratorClient is injected.
6. Build the app and run it.