This project provides a collection of Real-ESRGAN Compact ONNX upscaling models, along with a custom build of mpv video player. The video player (currently Windows only), enables real-time upscaling of 1080p content to 4K by running these models using TensorRT (NVIDIA only). While the default configuration upscales using the 2x_AnimeJaNai V2 models, it can be easily customized to utilize any Real-ESRGAN Compact ONNX models.
Samples: https://slow.pics/c/V3GkBJyR
Join the AnimeJaNai Discord server to get the latest news, download pre-release and experimental models, get support and ask questions, share your screenshots (use the s
key in mpv), or share your feedback. 日本語も大丈夫です。
Ensure your NVIDIA graphics drivers are up to date. Download and extract the latest release archive of mpv-upscale-2x_animejanai. Open the video player at mpvnet.exe
to use mpv.net or mpv.exe
to use plain mpv depending on your preference.
When playing a video for the first time, a TensorRT engine file will be created for the selected ONNX model. Playback will be paused and a command prompt box will open. Please make sure to wait while the engine is created. Engine creation only needs to happen once per model. Playback will resume on its own when finished.
The player is preconfigured to upscale with 2x_AnimeJaNai_V2, and makes 6 upscaling profiles available by default. The available models and their respective profiles are described in more detail below. Any of these profiles can be selected on the fly using the keybinding listed below.
Model | Description | Profile | Keybinding | Minimum recommended GPU for upscaling 1080p to 4k |
---|---|---|---|---|
Compact | Highest quality model | upscale-on-compact4x |
Shift+1 |
RTX 4090 |
upscale-on-compact2x |
Shift+4 |
|||
UltraCompact | High quality model which trades slight quality for major performance gains | upscale-on-ultracompact4x |
Shift+2 |
RTX 3080 |
upscale-on-ultracompact2x |
Shift+5 |
|||
SuperUltraCompact | Fastest performance model which sacrifices a bit more quality | upscale-on-superultracompact4x |
Shift+3 |
RTX 3060? |
upscale-on-superultracompact2x |
Shift+6 |
Comparisons between Compact, UltraCompact, and SuperUltraCompact models: https://slow.pics/c/V3GkBJyR
The 2x and 4x profiles behave the same on HD videos, but the 4x profiles will run the models twice on SD videos and produce a sharper result.
The default upscaling profile is set up to use the UltraCompact model with the profile name upscale-on-ultracompact4x
. The default upscaling profile is specified in mpv-upscale-2x_animejanai/portable_config/mpv.conf
. To change the default profile, edit the mpv.conf
file and change the profile=upscale-on-ultracompact4x
line to a profile name from the above table based on your hardware requirements and preferences.
The upscaling can be further customized using the configuration file for AnimeJaNai which is located at mpv-upscale-2x_animejanai/portable_config/shaders/animejanai_v2.conf
. The configuration file allows the setup of up to 9 custom slots and also the use of custom chains, conditional settings based on video resolution and framerate, downscaling to improve performance, and more. All available settings are described in more detail in the config file. More information on custom configurations will be available on the wiki soon. The custom slots can be activated with keybindings Ctrl+1
through Ctrl+9
. To use one of these custom slots as the default upscaler, set the appropriate profile name corresponding to the desired slot in mpv.conf, such as profile=upscale-on-1
.
All keybindings can be customized by editing lines near the bottom of the mpv-upscale-2x_animejanai/portable_config/input.conf
file. By default, AnimeJaNai upscaling can be turned off using the Ctrl+0
keybinding.
All other mpv settings can be configured by editing mpv-upscale-2x_animejanai/portable_config/mpv.conf
(see the mpv manual for all options) for mpv options or mpv-upscale-2x_animejanai/portable_config/input.conf
for mpv keybindings.
By default, screenshots can be taken with the s
key and are stored in mpv-upscale-2x_animejanai/portable_config/screenshots
.
The 2x_AnimeJaNai V2 models are a collection of real-time 2x Real-ESRGAN Compact, UltraCompact, and SuperUltraCompact models designed specifically for upscaling 1080p anime to 4K resolution. These models prioritize correcting the inherent blurriness often found in anime while preserving essential details and colors. Although trained on 1080p anime and optimized for upscaling from 1080p to 4K, the models can still produce worthwhile results when upscaling some lower-resolution anime. SD anime can be upscaled to HD, or the model can be run twice to upscale SD content to UHD.
Most HD anime are not produced in native 1080p resolution but rather have a production resolution between 720p and 1080p. When the anime is distributed to consumers via TV broadcast, web streaming, or home video, the video is scaled up to 1080p, leading to scaling artifacts and a loss of image clarity in the source video. The aim of these models is to address these scaling and blur-related issues while upscaling to deliver a result that appears as if the anime was originally mastered in 4K resolution.
The development of the V2 models spanned over four months, during which over 200 release candidate models were trained and meticulously refined. The V2 models introduce several notable improvements compared to their V1 counterparts, including:
- More accurate "native-res aware" sharpening, so the model works just as well on blurry native 720p sources, sharper native 1080p sources, and everything in between, without oversharpening artifacts
- More accurate colors including line colors
- Improved artifact handling
- Better preservation and enhancement of background details and grain.
Overall, the V2 models yield significantly more natural and faithful results compared to the V1 models.
Benchmarks for various hardware configurations tested against various upscaling configurations are available on the wiki.
Any media player which supports external DirectShow filters should be able to run these models, by using avisynth_filter to get VapourSynth running in the video player.
The 2x_AnimeJaNai_V2 ONNX models can be used on a PC with any graphics card to render upscaled videos, even when using graphics cards not fast enough for realtime playback. Please see the file shaders/animejanai_v2_encode_single.bat
or shaders/animejanai_v2_encode_batch.bat
included in the release package for more details on how to set this up. Alternatively, any program that supports ONNX models can be used, such as chaiNNer or VSGAN-tensorrt-docker.
For chaiNNer, the TensorRT backend is recommended for NVIDIA users for fastest rendering performance. AMD users should use the NCNN backend instead. Templates for chaiNNer are available for NVIDIA and AMD users. Simply download and open the appropriate chn
file in chaiNNer, and select the ONNX model file and the input video file to upscale.
- MangaJaNai: Upscale manga with ESRGAN models
- Upscale Wiki and associated Discord server
- 4x-AnimeSharp by Kim2091
- 1x_HurrDeblur_SuperUltraCompact by Zarxrax
- SaiyaJin DeJpeg by Twittman
- 422415 for significant assistance in dataset preparation and continuous feedback during development of V2 models
- Community feedback on V1 models
- MPV_lazy and vs-mlrt
- traiNNer-redux
- Dataset Destroyer
- Real-ESRGAN
- OpenModelDB
- getnative and anibin