Convert books downloaded with licenserequest

Question

Convert books downloaded with licenserequest

mkb79 opened this issue 5 years ago · 42 comments

Books downloaded with

license, _ = client.post(
   "content/{asin}/licenserequest",
    body={
        "drm_type": "Adrm",
        "consumption_type": "Download",
        "quality":"Extreme"
    }
)
content_url = license['content_license']['content_metadata']['content_url']['offline_url']

can’t converted with ffmpeg -activation_bytes ... at this moment.

Read here

Audible uses a new aaxc format when downloading via audible app on android/iOS or the api. But with this client you can get the response of a licenserequest. Example shorten on many places with ... :

{
    "content_license": {
        "acr": "CR!...",
        "asin": "B07DDKJH33",
        "content_metadata": {
            "content_url": {
                "offline_url": "https://dyrrggeck87jc.cloudfront.net/.../bk_adko_003749ade_lc_128_44100_2.aax?voucherId=cdn:...&Policy=...&Signature=...&Key-Pair-Id=..."
            }
        },
        "drm_type": "Adrm",
        "license_response": "...==",
        "message": "Eligibility details:[GrantRightsReason [reasonCode=MEMBERSHIP, reason=No need to verify active offline license when request is not for offline consumption. OWNERSHIP-user does not have expected ownership rights over the parent title asin. OWNERSHIP-title does not qualify for long title child part ownership or customer does not own child part. OWNERSHIP-user has ownership rights. GEO_RIGHTS-user has geo-rights[DE] over the title[B07DDKJH33] in marketplaceId[AN7V1F1VY261K]. TITLE_ATTRIBUTES-title does not have rodizio plan association. GEO_RIGHTS-user has geo-rights[DE] over the title[B07DDKJH33] in marketplaceId[AN7V1F1VY261K]. No need to verify active offline license when request is not for offline consumption. BENEFIT-user has valid Radio benefit associated[RadioStub]. TITLE_ATTRIBUTES-title does not have radio plan association. Client Asin Mapping validation skipped since client id is null. AAA Client with id: ApolloEnv:AudibleApiExternalRouterService/EU/Prod, does not have access to asin: B07DDKJH33. Client does not have plans that support asin benefits.]]. Licensing details:[ADRM license granted]",
        "request_id": "...",
        "status_code": "Granted",
        "voucher_id": "cdn:..."
    },
    "response_groups": [
        "always-returned"
    ]
}

Maybe someone has the know-how to use this informations to decode the new aaxc format.

Meanwhile you can grab the audible web page for the download urls. You can use cookies to authenticate like so:

import audible
import requests

auth = audible.FileAuthenticator(...)
cookies = auth.login_cookies

r = requests.get("https://www.audible.com/...", cookies=cookies)

Edit 2022-01-11:
Since ffmpeg 4.4 decryption of aaxc files should be build in. You have to use ffmpeg with the --audible_iv and --audible_key options and the correct iv/key pair from the decrypted licenserequest response!

Answer 1 · 2019-08-04T23:01:42.000Z

You can also use:

asin = "some_asin"
codec = "some_codec" # Desired quality from /1.0/library/{asin}?response_groups=product_attrs,relationships

auth = audible.FileAuthenticator(...)
client = audible.AudibleAPI(auth)

content_url = f"https://cde-ta-g7g.amazon.com/FionaCDEServiceEngine/FSDownloadContent"
params = {
    'type': 'AUDI',
    'currentTransportMethod': 'WIFI',
    'key': asin,
    'codec': codec
}

signed_headers = client._sign_request('GET', content_url, params, {})
headers = client.headers.copy()
for item in signed_headers:
    headers[item] = signed_headers[item]

r = client.session.request('GET', content_url, headers=headers, params=params, json={}, allow_redirects=False)
print(r.headers['Location'])

To get files in .aa (Format4) or .aax (Enhanced) format.

Answer 2 · 2019-08-05T04:07:03.000Z

I can’t download any books that way. It gives me a download link. But if I want to download the file it gives me an Unspecified database error.. And I‘m not the only one with that problem.

I wrote about this multiple times on reddit to you.

Answer 3 · 2019-08-05T06:13:26.000Z

With help from omarroth here is a workaround to solve the problem:

import audible


asin = "some_asin"
codec = "some_codec"  # Desired quality from /1.0/library/{asin}?response_groups=product_attrs,relationships

auth = audible.FileAuthenticator(...)
client = audible.AudibleAPI(auth)

content_url = f"https://cde-ta-g7g.amazon.com/FionaCDEServiceEngine/FSDownloadContent"
params = {
    'type': 'AUDI',
    'currentTransportMethod': 'WIFI',
    'key': asin,
    'codec': codec
}

signed_headers = client._sign_request('GET', content_url, params, {})
headers = client.headers.copy()
for item in signed_headers:
    headers[item] = signed_headers[item]

r = client.session.request('GET', content_url, headers=headers, params=params, json={}, allow_redirects=False)
link = r.headers['Location']

# prepare link to fit correct tld
tld = auth.locale.audible_api.split("api.audible.")[1]
new_link = link.replace("cds.audible.com", f"cds.audible.{tld}")

print(new_link)

To get files in best quality codec is to be set to LC_128_44100_stereo

Answer 4 · 2020-10-08T00:29:53.000Z

Make a request to /1.0/content/#{asin}/licenserequest, with this body:

{
  "quality: "Extreme",
  "consumption_type": "Download",
  "drm_type": "Adrm"
}

The expected body should provide an offline_url and license_response. Downloading the offline_url produces a .aax (aaxc) file.

Following information from this patch to ffmpeg, get the audible_key and audible_iv by decrypting license_response with a hash produced from customer_id, device_type, device_serial, and asin.

In the patch notes Vesselin provides this code:

def decrypt_voucher(deviceSerialNumber, customerId, deviceType, asin, voucher):
    buf = (deviceType + deviceSerialNumber + customerId + asin).encode("ascii")
    digest = hashlib.sha256(buf).digest()
    key = digest[0:16]
    iv = digest[16:]

    # decrypt "voucher" using AES in CBC mode with no padding
    cipher = AES.new(key, AES.MODE_CBC, iv)
    plaintext = cipher.decrypt(voucher).rstrip(b"\x00")
    return json.loads(plaintext)

deviceSerialNumber and deviceType are the same as those submitted to /auth/register. For example A2CZJZGLK2JJVM and AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA. The customerId is returned from /auth/register and looks like amzn1.account.AAAAAAAAAAAAAAAAAAAAAAAAAAAA. voucher is the base64-decoded license_response data.

Successfully decrypting license_response should produce a json object:

{
    "key": "ffffffffffffffffffffffffffffffff",
    "iv": "ffffffffffffffffffffffffffffffff",
    "refreshDate": "2020-10-28T00:15:10Z",
    "removalOnExpirationDate": "2020-12-07T00:15:10Z",
    "rules": [
        {
            "parameters": [
                {
                    "expireDate": "2020-11-07T00:15:10Z",
                    "type": "EXPIRES"
                }
            ],
            "name": "DefaultExpiresRule"
        },
        {
            "parameters": [
                {
                    "directedIds": [
                        "amzn1.account.AAAAAAAAAAAAAAAAAAAAAAAAAAAA"
                    ],
                    "type": "DIRECTED_IDS"
                }
            ],
            "name": "AllowedUsersRule"
        }
    ]
}

The key and iv can then be passed to a version of ffmpeg with Vesselin's patch applied to decrypt the desired aaxc file.

Any Audible Plus book should be able to be downloaded without purchase. I tested B084C68ZJK and B00URXOQ1E. The same method works for regular books as well. I would consider it a better alternative to the FSDownloadContent method.

Answer 5 · 2020-10-08T04:15:59.000Z

Hi omarroth,

Thank you and Vesselin for your very important information and hard work. I will look at this as fast as possible. It‘s definitely a better option than FSDownloadContent.

Answer 6 · 2020-10-08T06:06:18.000Z

I can verify that it works for me too. I got the needed key and iv from voucher. I will implement them in my project.

Thanks again for your work!

Answer 7 · 2020-10-08T08:06:35.000Z

I‘ve open a pr with function to decrypt voucher gained from licenserequest and an example how to download aaxc files and decrypt there voucher to retrieve the iv and key for further purposes.

This is running for me with Pythonista 3. Can someone test if it running on other systems?

Answer 8 · 2020-10-08T12:41:05.000Z

@omarroth you have a patched ffmpeg binary for testing purposes?

Answer 9 · 2020-10-08T15:05:43.000Z

https://omar.yt/27c9ffdd33130cf7511f8605bcde868ce08b92ba/ffmpeg

Statically built with some patches to alpine's APKBUILD.

Should be able to build with:

$ git clone https://git.ffmpeg.org/ffmpeg.git
$ cd ffmpeg
$ curl -o https://patchwork.ffmpeg.org/project/ffmpeg/patch/17559601585196510@sas2-2fa759678732.qloud-c.yandex.net/raw/
$ git apply aaxc.patch
$ ./configure
$ make
$ ...

Use with:

./ffmpeg -audible_key ... -audible_iv ... -i in.aax out.mp3

Answer 10 · 2020-10-08T15:15:07.000Z

Thank you for the binary. Have you applied the patch? When I execute this I got the error Unrecognized option 'audible_key'.

Answer 11 · 2020-10-08T15:18:40.000Z

Yes. Make sure you're running the correct binary:

$ ./ffmpeg
ffmpeg version 4.3.1 Copyright (c) 2000-2020 the FFmpeg developers
  built with gcc 10.2.0 (Alpine 10.2.0)
  configuration: --prefix=/usr --pkg-config-flags=--static --extra-ldexeflags=-static --disable-gnutls --enable-gpl --enable-nonfree --enable-libmp3lame --enable-libx264 --enable-pic --enable-pthreads --disable-stripping --enable-static --enable-libopus --disable-debug
...

Answer 12 · 2020-10-08T15:21:55.000Z

Ah, my mistake. I‘m executed the wrong version. Thanks for your hind and your work!!!

Answer 13 · 2020-10-10T18:29:12.000Z

@omarroth , @mkb79 -- the above patched binary appears to be linux only. Is there a link to the patched windows version?

Answer 14 · 2020-10-10T20:05:38.000Z

@rmcrackan

You can take a look on this page. You only have to find out how you can apply the aaxc patch!

Answer 15 · 2020-10-11T20:34:06.000Z

@rmcrackan

I've tested something to compile ffmpeg on windows. And I got a working and patched ffmpeg bin. But the bin is very huge, because I compiled it with most of the features. You can compile it according to your wish very easy. Only follow these steps:

Download this package and extract it.
Download ffmpeg_extra.zip and extract the zip file into the build folder of the new dir.
Execute media-autobuild_suite.bat in the main dir and answer the questions to build your custom ffmpeg.
Drink a coffee. And drink a coffee again. Do this for the next hours until your compilation is finished.

Answer 16 · 2020-12-19T23:32:00.000Z

@mkb79 would it be possible to give me instructions on how to build a version of ffmpeg with this new commit? I just built ffmpeg straight from the git repo and it didn't have the option -audible_key. The patch also fails.

Answer 17 · 2020-12-20T07:06:21.000Z

@oaksisfolks

Which OS you are running? If you have Windows, you can try the guide above.

Answer 18 · 2021-01-15T02:46:29.000Z

I can't seem to build ffmpeg with the patch on Windows, even with you guide @mkb79
Is there an already built version of ffmpeg with said patch applied?

Answer 19 · 2021-01-15T08:00:24.000Z

I build ffmpeg for windows as I wrote the guide above. ffmpeg was running smooth. But I build ffmpeg with most of features, so the binary was very huge. But I doesn’t have it anymore (I use ffmpeg on my linux machine).

You can try the ffmpeg bin build for alpine above and run it with wsl2 under windows. Or you can take a look here. Maybe these bins are compiled with the patch. Otherwise you can search for a guide to build ffmpeg for windows and apply the patch yourself before compiling.

Answer 20 · 2021-01-15T19:39:01.000Z

Thanks a lot, I tried the git full binaries built from gyan.dev and they already have the patch applied! This could also be interesting for you @oaksisfolks

Answer 21 · 2021-01-15T20:01:26.000Z

Thank you for this very important information @KlaraStoff .

Answer 22 · 2021-06-03T13:10:01.000Z

I've been following this and I've two questions

So I know htey have to be download in .aaxc format, but is that script speific or Audible being Audible?
Given th e./ffmpeg command, is there a way to cut the book up into chapters or a way to enable continue from where I left off?

Answer 23 · 2021-06-03T13:20:05.000Z

I don’t understand your question. You can use the licenserequest endpoint to download books in aaxc format. The same endpoint is used by the Audible app for Android or iOS.
The aaxc file contains chapter information. With this information you can split and convert the file with ffmpeg. Using the metadata or licenserequest API endpoint gives you also the last heard position (if you are using the right response group)

Answer 24 · 2021-06-03T14:03:17.000Z

I understand I can get the metadata from the .aaxc file, but I'm struggling on how to reinsert it. Using the above .ffmpeg command as the example, what would I add to it? When I try it just gives me a message was repeated error and doesn't add the metadata when I've extracted the metadata into the metadata file

Answer 25 · 2021-06-03T15:45:28.000Z

Maybe you can take a look here. This is a command for my audible-cli package which uses ffmpeg to replace chapter metadata from aaxc file with the one obtained from licenserequest API endpoint.

Answer 26 · 2021-06-03T16:01:02.000Z

That'd do what I'm asking. What's the simplest way to make that working with the pip version since that's how I've gone the pip3 route to get it installed and working.

Answer 27 · 2021-06-03T16:09:23.000Z

Simple use audible-cli and copy the command file to the plug in dir. Please check the README for audible-cli.

Answer 28 · 2021-06-03T18:55:43.000Z

Sorry for all the stupid questions but do I need to ask @omarroth for advice on building the ffmpeg mentioned in the thread? I tried and got nowhere with it so I'm not sure if I need to spin that off somewhere more appropriate however

Answer 29 · 2021-06-03T19:49:26.000Z

You can try the last build from here. The aaxc patch should be included in this build.

Answer 30 · 2021-07-17T19:31:53.000Z

@mkb79 Question: is there a reason to use the newer aaxc format over aax? E.g. are there books no longer available in aax format, and/or is the quality of aaxc better? Or does it not matter for now.

Answer 31 · 2021-07-18T06:42:40.000Z

AAXC-files use a different encryption than AAXC-files. To be clear, AAXC and AAX are only container which contains the audio file in aac-codec, the cover art and some metadata. So booth have the same quality.

Audible Plus titles and some other are only delivered as AAXC. So if you want them, you have to support AAXC.

Answer 32 · 2021-07-18T15:40:09.000Z

@mkb79 Thanks so much for the explanation, and of course for this software! As an Audible UK customer I don't have access to Audible Plus, but this is good to know. Luckily none of the titles in my library failed to download as AAX but I will be on the lookout for errors.

Answer 33 · 2021-09-25T03:51:28.000Z

For those looking to work with AAXC files without ffmpeg, let me shamelessly promote my new C# library.

https://github.com/Mbucari/AAXClean

And for the record, they're actually encrypted in the same way as AAX files. The only difference is that AAX files contain key derivation information inside a custom mpeg4 box named Adrm. Keys are derived with activation bytes + Adrm data, and after you derive the key the decryption is exactly the same as with AAXC.

Answer 34 · 2021-09-25T17:00:50.000Z

@Mbucari -- I didn't know you wanted to promote AAXClean. I frequently refer people to this page that I curate for audiobook software. I'll be glad to add you. What kind of description would you like me to include?

Answer 35 · 2021-09-25T17:03:55.000Z

@rmcrackan

Maybe you can add audible-cli to this page?

Answer 36 · 2021-09-25T17:13:01.000Z

Maybe you can add audible-cli to this page?

Added your cli and AAXClean. Let me know if you'd like any wording changed

Answer 37 · 2021-09-25T17:14:51.000Z

Thank you very much!

Answer 38 · 2022-01-11T00:24:09.000Z

@mkb79 I know this is closed but is this still the recommended way to convert AACX files with ffmpeg? Getting down the git and applying the patch was recommended 2 years ago. Is there a better way or is this still the solution?

Answer 39 · 2022-01-11T00:30:51.000Z

@normal1ze The patch should be bundled with any reasonably recent build of FFmpeg. So unless your source for FFmpeg builds is extremely out of date, there should be no need for patching and compiling anything manually. There are other options, such as that described by @Mbucari in #3 (comment). There is also, I believe, an optional plug-in for audible-cli, but AFAIK it uses FFmpeg underneath.

Answer 40 · 2022-01-11T00:36:06.000Z

@jscholes thanks for the feedback mate. I have just installed the latest FFmpeg so I'll give it a go with vanilla install. Cheers.

Answer 41 · 2022-01-11T05:53:15.000Z

@jscholes Thanks from me too!

@normal1ze Since ffmpeg v4.4 this function should be build in.

Answer 42 · 2022-10-11T10:13:25.000Z

The Ubuntu APT repositories currently have an outdated version of ffmpeg which doesn't have that option.

This 3rd-party APT repository has an up-to-date version and I can confirm that it solves the issue for me on Ubuntu.

Here are the current instructions to install it:

sudo add-apt-repository ppa:savoury1/ffmpeg4
sudo add-apt-repository ppa:savoury1/ffmpeg5
sudo apt-get update
sudo apt-get upgrade && sudo apt-get dist-upgrade
sudo apt-get install ffmpeg

Alternatively you can download the latest statically-linked ffmpeg from here: https://johnvansickle.com/ffmpeg/