This project uses a TensorFlow model created by the TensorFlow burglar alert model sub project
This repository contains a Kafka Streams application that can process webcam images uploaded into Apache Kafka. It uses TensorFlow for Java to detect potential burglars and will send burglar alerts using the Telegram connector for Kafka Connect.
However, not all detected burglar alerts are sent to the Telegram connector. Depending on the state of a Nuki smartlock, alerts are filtered and discarded. The state of the smartlock is retrieved using the Nuki Web API and loaded into Kafka using the REST Connector for Kafka Connect.
The idea behind this is that sending burglar alerts only makes sense when you're either not at home or (hopefully) fast asleep. And those situations tend to coincide with a given lock state (locked).
Before the images can be processed by the Kafka Streams application, they must be uploaded into Kafka. When your camera supports FTP, such as those from Foscam, you can use the Landoop Kafka Connect FTP converter to achieve this.
However, there are a few important issues to consider, to facilitate the image detection by TensorFlow for Java.
- Pictures taken with sufficient light will be in full colour, but those taken at night are made with infrared light and thus in grayscale. Therefore it's better to convert all images to grayscale.
- Full HD cameras have resolutions that lead to impractical image sizes. It's better to scale the image down to a smaller size, such as a width of 1024. That will also help to prevent exceeding the maximum message size in Kafka.
- If your camera records areas that are irrelevant for burglar detection, such as the street in front of your house, you can crop parts of the image. This may also help preventing false alerts to be sent by the application.
These issues can be handled by a custom Kafka Connect value converter. You can simply plug this into the Kafka Connect FTP converter. See the JSON configuration file for the FTP converter for more details.
- Synology NAS - with Docker feature supported
- Outdoor camera that supports motion detection and FTPS upload, e.g Foscam FI9901 EP
- Nuki smart locks
- Kafka Streams
- Kafka Connect
- FTP connector, used to load images into Kafka
- REST connector, used to retrieve the smartlock state
- Telegram connector, used to send alerts from Kafka
- Kafka Connect value converter, to crop, resize and convert images to grayscale
- TensorFlow for Java