/vkCaptchaBreaker

VKontakte captcha bypass with pseudoCRNN model running as a chrome extension, Python, JS, 2020

Primary LanguageJupyter Notebook

vkCaptchaBreaker

Use this VKontakte captcha breaker with 91% accuracy right in browser with chrome extension, Python, JS, 2020

title

HTML Live Demo from above picture: 🔥DEMO🔥

Code CodeSandbox Sandbox for HTML with ONNX.js+ONNXruntimeWeb: vk-captcha-breaker

Images in Demo or Sandbox are broken? Solution

Extension can be downloaded here:

Chrome Web Store Firefox add-ons GitHub Releases

Recognition speed comparison on different devices with different CPU specs (in milliseconds):

Recognition speed comparison on different devices


Table of Contents


Features

Currently supported actions (on mobile domain m.vk.com only):

  • messages ✉️
  • likes ❤️
  • comments 💬
  • adding to friends 🧑‍🤝‍🧑

Please, note that captcha on login page and promocode page is not supported!

Installation

Update v1.6: 30% faster, CORS is no longer a problem

Update v1.5: recognition in background, Firefox addon

Update v1.4: added support for other captcha scenarios

Update v1.3: now it works for mobile domain only (m.vk.com)

Google Chrome Extension and Firefox Addon for VKontakte which utilizes trained AI models to predict captcha from image and automatically enters it.

Currently only captcha from messages are supported (no likes or comments for you, my bad)

You can download latest version from: VKCaptchaBreaker.crx

Windows/Linux/MacOS

To install the extension, simply follow this tutorial:

Finally, activate it:

title

Android

To install the extension, simply follow this tutorial:

qwikfone.medium.com/how-to-install-chrome-extension-on-android-3cd89f3a771c

Installation and activation example:

title

If something won't work, just reactivate extension.


Description

vkcaptchaformat

Keras implementation of Convolutional Recurrent Neural Network with ONNX.js support. To achieve better accuracy, CRNN utilizes CTC loss and Knowledge distillation with Label smoothing. CRNN comes in two versions: the one BIG with recurrent layers (RNNs) and the other small one without them.

The reason for doing so is that if we want to use trained models on end devices, such as Mobile devices/Client-side browsers, we want to get a model with a minimum number of parameters.

For the BIG version, Google's Inception-like CNN encoder is used with Bidirectional GRUs as RNN decoder. For the small version, the ShuffleNetV2 encoder is used with 1d convs instead of RNN cells.


DataMining

To parse captcha you can use any URL that Vkontakte provided, e.g. https://vk.com/captcha.php?sid=625944628258&s=1 Here 625944628258 is a random seed. It's important to mention that some seeds will generate old captchas with Cyrillic symbols. Using URLs with different seeds over 1.5 million images was parsed.

At first 5000 captcha's was manually labelled (that process take approximately 1 year). Thus, an initial dataset was created, let's call it Dataset5k. Next, on this dataset, BIG CRNN was trained and achieved 80% accuracy on unseen data. After that using Vkontakte account without phone verification more images were automatically labelled online. Thus, a new dataset was created, and the process continues.

After achieving 99% accuracy with BIG CRNN model, it was used for labelling the rest of 1.5 million images offline. Datasets with 850k images and 1500k images were constructed this way. This huge "filthy" datasets (implying that there are mistakes in predictions) were used for training small pseudoCRNN model.


ONNXTroubles

onnxlogo

Unfortunately, in the case of using models on Client-side, e.g. with JavaScript, existing frameworks (Open Neural Network Exchange (ONNX.js)) does not support RNNs. The only one which supports them is Tensorflow.js, but it is slower. It's important to note that RNNs also are quite slow due to its nature.

To mitigate this problem this project introduces a small pseudoCRNN architecture without RNN blocks. Instead of bidirectional Gated Recurrent Units (GRU) as in the BIG version, it uses one-dimensional convolutions. Other troubles:

  • Can't use dynamic reshaping, e.g Reshape(None, -1, 32) will crash
  • Can't use 1d Convolutions, instead, the following trick was used:

(None, 32, 128) --Reshape--> (None, 32, 1, 128) --Conv2d--> (None, 32, 1, filters) --Reshape--> (None, 32, filters)

  • Can't use Droupout, it's important to reconstruct model without this layer
  • Can't use Casting operation in ONNX.js, e.g. INT32 -> FLOAT32 conversion
  • Can't use Ragged Tensors in ONNX.js, e.g. for CTC algorithm
  • Can't use tf.where, because Nonzero is not implemented in ONNX.js

You can check the list of supported operators in ONNX.js HERE

ONNX is a great thing, but its JavaScript Framework needs more attention from Microsoft developers.

As a result, in the next project, Tensorflow.js should be used instead of ONNX.js


Results

After training on 1.5 Million filthy labeled captcha for 300k steps, the student model achieved ~90% accuracy (while teacher accuracy was 99%)

training on filthy data

Comparision:

Model Dataset Size Steps Knowledge Distillation, % Accuracy, % Size, MB Images/sec
BIG CRNN 5k 70k - 80 66 ~284
BIG CRNN 16k 70k - 93 66 ~284
BIG CRNN 40k 70k - 98.9 66 ~284
small pseudoCRNN 40k 70k - 64 4 ~2155
small pseudoCRNN 850k* 70k - 82 4 ~2155
small pseudoCRNN 850k* 70k 50 83 4 ~2155
small pseudoCRNN 1.5M* 140k 50 86.7 4 ~2155
small pseudoCRNN 1.5M* 280k 30 91.2 4** ~2155

* - means that dataset was made with filthy labels from the teacher

** - after conversion the final size of the model is 1 MB

Thus, you can achieve up to 10 times faster inference time with the small version.

Optimized pseudoCRNN student architecture after convertion to ONNX format:

architecture


Sources

This project was inspired by number-plate recognition work and uses its modified version of CRNN implementation, @RIA.com.

ONNX.js reference example, created by @elliotwaite.

For small pseudoCRNN CNN encoder, this system uses a modified version of ShuffleNetV2, implemented by @opconty.

Tensorflow Dataset class with caching implementation is the modified version of @krasserm.

For the model's graph visualization was used the Netron application, by @lutzroeder