Amazon DRM Update 3rd January 2023

[elfiflos]

Question / bug report

Amazon updated its DRM software on the 3rd of January 2023 and since then, all books published after this date cannot be freed anymore.

Has there been or will there be any updates for that?

Which version of Calibre are you running?

6.2.1

Which version of the DeDRM plugin are you running?

V10.0.3

If applicable, which version of the Kindle software are you running?

No response

Log output

Paste log output here.

[ElleKayEm]

Known workarounds are in this post: https://www.mobileread.com/forums/showthread.php?t=283371

[elfiflos]

Nope, doesn't work. Books downloaded from Amazon directly no longer get freed by Caliber and the DRM plugin.

See this: https://auresnotes.com/how-to-remove-amazon-kindles-drm-latest-update/

[ElleKayEm]

Read the link I gave before more carefully. For most books, you can still get a copy with the older DRM.

[Satsuoni]

I have found that if one adds

def scramble(st,magic):
  ret=bytearray(len(st))
  padlen=len(st)
  for counter in range(len(st)):
    ivar2=(padlen//2)-2*(counter%magic)+magic+counter-1
    ret[ivar2%padlen]=st[counter]
  return ret
  
def obfuscate2(secret, version):
    if version == 1:  # v1 does not use obfuscation
        return secret
    magic, word = OBFUSCATION_TABLE["V%d" % version]
    # extend secret so that its length is divisible by the magic number
    if len(secret) % magic != 0:
        secret = secret + b'\x00' * (magic - len(secret) % magic)
    obfuscated = bytearray(len(secret))
    wordhash = bytearray(hashlib.sha256(word).digest()[16:])
    shuffled=bytearray(scramble(secret,magic))
    for i in range(0, len(secret)):
        obfuscated[i] = shuffled[i] ^ wordhash[i % 16]
    return obfuscated

to ion.py, and then uses both obfuscation functions in decryptvoucher

        sharedsecrets = [obfuscate(shared, self.version),obfuscate2(shared, self.version)]
        decrypted=False
        ex=None
        for sharedsecret in sharedsecrets:
          key = hmac.new(sharedsecret, b"PIDv3", digestmod=hashlib.sha256).digest()
          aes = AES.new(key[:32], AES.MODE_CBC, self.cipheriv[:16])
          try:
            b = aes.decrypt(self.ciphertext)
            b = pkcs7unpad(b, 16)
            decrypted=True 
            print("Decryption succeeded")
            break
          except Exception as ex:
            print("Decryption failed, trying next fallback ")
        if not decrypted:
          raise ex

it helps with some books downloaded using kindle 1.40. Give it a try?

[Krynh]

Known workarounds are in this post: https://www.mobileread.com/forums/showthread.php?t=283371

Only method 3 works and that requires buying a kindle e-ink

[ElleKayEm]

Only method 3 works and that requires buying a kindle e-ink

Method 6 also works.

[Krynh]

Only method 3 works and that requires buying a kindle e-ink

Method 6 also works.

Oh I didn't see method 6. Thanks 👍

[noDRM]

Oh, interesting. I didn't see this patch hidden in this bug report. Thanks a lot! I added it to the repo marking you as the author and I will take a closer look and see if it helps with newer books.

[ElleKayEm]

People are reporting over at MobileRead that it is working for most KFX books. See toward the bottom of this page: https://www.mobileread.com/forums/showthread.php?t=351285&page=142

[ZolaLa9]

Oh, interesting. I didn't see this patch hidden in this bug report. Thanks a lot! I added it to the repo marking you as the author and I will take a closer look and see if it helps with newer books.

@noDRM, you might want to look at @Satsuoni's fork at https://github.com/Satsuoni/DeDRM_tools. I believe Satsuoni is the author of the patch. They released their fork as 10.0.4s. FWIW, this release is working for me, and it is working for those who have tested it on MobileRead: https://www.mobileread.com/forums/showthread.php?p=4333103#post4333103

[noDRM]

Yeah, that patch was what I (also) added to my repo yesterday.

Looking at my changelog of not-yet-released changes I guess it's time for another release asap so people can easily use this before Amazon patches it again ..

EDIT: I don't currently have Amazon books with the newer DRM, would you mind testing if it also works with my newest version? The Amazon code is identical to the one in these forks: https://github.com/noDRM/DeDRM_tools/suites/13824666316/artifacts/767310613

[Satsuoni]

Could you please wait with release until I am done fixing issues raised in the thread? I expect to put out another experimental release in about... next 8 hours, hopefully. It will be a bit bloated due to lookup tables.

[noDRM]

Sure, no problem.

[ElleKayEm]

@noDRM
Tested the plugin you linked above and it worked on a KFX format book published this year and just downloaded to my Kindle Oasis.

[Satsuoni]

I have added a commit/release with more scramble methods. They use lookup tables dumped from memory, about a 1.2 Mb of them. I recommend testing/cleaning it, for my code, is, as ever "research grade", and I have my own formatting "preferences". It also fixed incorrect string for V12 and maybe some others. It should work for books with "advanced" encryption. If somebody has a book that is tied to PC (that is, actually uses information bound to account), that would be nice to test as well.

[noDRM]

Wow, that's quite a bit of data. Thanks a lot for all your work. Do you know if this (= the lookup in these large tables) increases processing time by a noticeable amount?

I'll take a look and see how to best add all that to the plugin.

[Satsuoni]

Lookup by itself is fast (though it is lookup in a loop), but I did add them in a rather wasteful manner (it calculates all scramble functions and then checks every one) so it might add some latency when adding an encrypted book? In theory, every version maps to a single scramble, but they did change that during the update (kindle 1.17 -> to 1.40 was just shifting v10(?) and v11 to other scramble), so I did not bother. It is calculated once for the book, as far as I can tell. And personally, I don't notice the difference when importing book (compared to length of import itself), so I am not sure. As far as I can tell, all that data seems necessary and is baked into executable instead of being generated...

[ElleKayEm]

I don't know if this matters but...Kindle for PC/Mac 1.17 doesn't download KFX format at all. DeDRM 10.0.3 works for KFX downloaded with Kindle 1.19 to 1.26 (which of course is now only possible for books published before 2023).

[Satsuoni]

My mistake. It would be more correct to say "change from KindleFor PC version that worked to the one that did not work". I think the last version that worked till recently was... 1.26? Not sure. But they did not add any new versions in update, just shifted how some old ones worked. I think? Should not matter.

[Satsuoni]

Also just pointed out to me, I should not have updated version number inside plugin, since it needs to be integer... Will fix.

[Satsuoni]

Added commit fixing false positive for decryption (moved check into the loop). No release for that.

[j-howell]

I have tested Satsuoni's changes and as far as I can tell they correctly handle the KFX DRM currently in use and very likely all of the methods that Amazon has implemented and held in reserve.

I do wonder how Amazon will react. Is there some better hidden encryption algorithm waiting in the wings to be enabled? Will they rush out new Kindle releases with added DRM? Will they accept that DRM removal is a thing and let it go? Time will tell.

[ZolaLa9]

EDIT: I don't currently have Amazon books with the newer DRM, would you mind testing if it also works with my newest version? The Amazon code is identical to the one in these forks: https://github.com/noDRM/DeDRM_tools/suites/13824666316/artifacts/767310613

@noDRM: Your newest version works with latest Kindle for Mac.

[noDRM]

@ElleKayEm @ZolaLa9 Thanks for confirming the new version works.

@Satsuoni Thanks for the update, I'll try to get it added to the plugin in the next couple days.

@j-howell I highly doubt Amazon is going to accept that DRM removal is a thing now. They wouldn't have done all the changes they did recently (block old K4PC versions, block USB downloads for some books, KFX-only books, ...) if they were just like "Oops, got circumvented again, so be it".

Assuming that Satsuoni did manage to find and circumvent all the methods they currently supported, I'd guess that Amazon is already working on new device firmware and new K4PC and K4Mac versions that implement newer DRM, and then at some point in the future they'll do another switch like "Books published after X need K4PC version XYZ". But hopefully that will take a while (as they don't want to force customers to update over and over again), same as it did with the old version.

[j-howell]

@j-howell I highly doubt Amazon is going to accept that DRM removal is a thing now.

I agree. That last option was more wishful thinking than anything serious.

[Satsuoni]

@Satsuoni Thanks for the update, I'll try to get it added to the plugin in the next couple days.

Good, thank you! I'll leave it to you then. I should stop messing with it, I think. Current version should be covered, and later versions... Maybe if I need the plugin myself again.

@j-howell I highly doubt Amazon is going to accept that DRM removal is a thing now. They wouldn't have done all the changes they did recently (block old K4PC versions, block USB downloads for some books, KFX-only books, ...) if they were just like "Oops, got circumvented again, so be it".

I assume they'll put in something nastier, that amateur like me cannot pull out within reasonable time frame. Sadly, there is no dearth of such techniques. Hopefully that will take a few months though, as you say.

[j-howell]

I assume they'll put in something nastier, that amateur like me cannot pull out within reasonable time frame.

You are not giving yourself enough credit. Thank you for the excellent work.

[Satsuoni]

You are not giving yourself enough credit. Thank you for the excellent work.
Thank you! Still, if it were anywhere close to widevine, I would have given up.
... Now I am kind of curious as to what they'll do. I know what Google did, and it was more of the same, just... much more.

[noDRM]

@Satsuoni It looks like EmulatorHelper in your code is undefined, though it seems to only be used in the (unused) function memhex. I assume that that was just something you added for debugging and I can remove it?

All the code is now merged into this repo (plugin can be downloaded from https://github.com/noDRM/DeDRM_tools/suites/13846866293/artifacts/769072163 ); I'm now going to try and test some books from a newer K4PC to test this.

[Satsuoni]

@Satsuoni It looks like EmulatorHelper in your code is undefined, though it seems to only be used in the (unused) function memhex. I assume that that was just something you added for debugging and I can remove it?

Ahh, yes, that was to compare memory dumps, i think. a2hex and memhex should be deleted

[rmzg]

If it helps I just confirmed that 10.0.5(s) from Satsuoni works on a .azw (kfx zip) file I downloaded today that nodrm/10.0.3 does not work on. Windows, Calibre 6.22.

[ElleKayEm]

noDRM's master code here now contains Satsuoni's additions.

[RudeBoyEEEE]

noDRM's master code here now contains Satsuoni's additions.

Hi. Where can I try this out?

[ElleKayEm]

Link in this comment above: #315 (comment)

[RudeBoyEEEE]

Crap, I think I misunderstood. I thought it'd work with new Amazon kindle books. Any workaround for those found yet? Besides the "buying a regular kindle as well" method.

[rmzg]

I just bought and downloaded a kindle book using windows kindle 1.40 and it worked on the azw file.

[ElleKayEm]

It does work on the new drm. You also need the kfx input plugin.

[RudeBoyEEEE]

Ah! I see. Got it now, thanks! Any known issues with converting the KFX to EPUB?

[ElleKayEm]

Should work fine once the drm is gone. Generally speaking the azw3 format is somewhat better than kfx for conversion to epub because azw3 and epub are quite similar. But kfx to epub works well enough.

[realtofuine]

I added my kindle's serial key to the plugin, but it still says that the book contains DRM. Is there something else I need to do? I also have the kfx input plugin.

[ElleKayEm]

@realtofuine
Did you get the book via download & transfer from the website? Is it an E-ink Kindle? Did you add the book after entering the serial number and re-starting calibre?

[realtofuine]

Yes, I got the book using the download feature and have the file in .azw format. However, the kindle I am using is a Kindle Fire. Will that work or do I need a different type of kindle?

[ElleKayEm]

Needs to be E-ink, not a Fire, for that method. See this post for other methods: https://www.mobileread.com/forums/showthread.php?t=283371

[realtofuine]

Oh, ok thank you! I will try the other method!

[yourealwaysbe]

A hint for people using the Calibre plugin via the Calibre Flatpak. With the KFX input plugin i found that importing the .azw file failed to remove the DRM.

I think this is because the KFX input plugin tries to bundle all files in the directory containing the .azw file into one .kfx-zip file during import. Particularly, the amzn1.drm-voucher... file is needed for the DRM (and maybe others in the book directory). However, through Flatpak, it might be that Calibre only has permission to read the .azw file you explicitly chose, so can't get the other files needed to remove DRM.

My solution was to zip the files in that directory up into a file called book.kfx-zip (all files at top level), then import the .kfx-zip file. Possibly granting Calibre access to your Amazon books directly would also work.

[wosym]

Is there a way to check if my file is impacted by this? I bought an eBook on a non-Amazon website, but I know the same book exists on Amazon.

I am able to open it with Calibre (by using the DeACSM plugin), but can not DeDRM is.

[ElleKayEm]

That would be Adobe DRM. Nothing whatsoever to do with Amazon's DRM changes. DeDRM should work. If you post a log, we can take a look: https://github.com/noDRM/DeDRM_tools/blob/master/FAQs.md#i-cannot-solve-my-problem-with-the-dedrm-plugin-and-now-i-need-to-post-a-log-how-do-i-do-that