Chromium v53 (v11 cookies and libsecret backend on Ubuntu)
oltodosel opened this issue · 29 comments
The latest version of chromium has new cookies that start with 'v11' and wouldn't decrypt.
Please provide your operating system and version info.
Linux 4.7.2-1 x86_64
Chromium 52.0.2743.85-20 - has encrypted cookies with v10 at the beginning and it works fine.
Chromium 53.0.2785.92-1 - has encrypted cookies with v11 at the beginning and decryption doesn't work.
Looks like linux still uses the same implementation (v10 / saltysalt / peanuts) unless you're using a third party password-based system.
Still the old version:
- https://chromium.googlesource.com/chromium/src/+/53.0.2785.100/components/os_crypt/os_crypt_posix.cc
- https://chromium.googlesource.com/chromium/src/+/53.0.2785.100/components/os_crypt/os_crypt_mac.mm
Comments on new version with option for v11:
I don't know much about libsecret, I'll have to see if I can figure out how to retrieve the new password for these systems.
I've noticed that for some values, the encrypted key does not start with either v10 or v11. Examples include __ar_v4 from .hub.docker.com/, ref from www.intel.com/. But it's still encrypted in some way...
@djtm Hmm, I am only seeing v10 for Docker.com currently. Do you think this is related to this issue?
I'm not 100% certain it's the same issue. I'll do more testing in the future, next week if all goes well. :)
I am also seeing cookies with a v11 prefix.
Here is an example (from running select * ... in sqlite): v11�I�̫P3ζ�VnG@��[u�}�Ҵ����X-�vfw�kOt"p|��uw6;
Here is the same example base64 encoded: djEx+EmEzKtQM862HVZuR0C3+1t1r33b0rSv6KAWWC0UdmZ3o2tPPwh0InB87dR1dzY7
I am running Google chrome 54.0.2840.71 on Linux (ubuntu)
Huh, still working for me on OS X, all v10 cookies, and my Linux box broke. Reinstalling Arch, will be able to test soon. Would appreciate help from anyone knowledgeable about libsecret as mentioned above -- hopefully it may be as simple as getting the Chrome pw from keychain on OSX.
I think starting Chrome with --password-store=basic might be a workaround.
It looks like it's implemented here: https://github.com/bertrandom/chrome-cookies-secure/blob/master/index.js
I'm guessing /usr/bin/google-chrome --use-mock-keychain might also be the right way to avoid the issue.
Affecting me too.. I've been trying to get the passwords out of keyring with no luck. --password-store=basic and --use-mock-keychain didn't work for me
Okay so I managed to extract the passwords from libsecret, they are in a service called "Chrome Safe Storage"
import gi
gi.require_version('Secret', '1')
from gi.repository import Secret
def get_cookie_password():
service = Secret.Service.get_sync(Secret.ServiceFlags.LOAD_COLLECTIONS)
default_keyring = service.get_collections()[0]
default_keyring = service.unlock_sync([default_keyring])[1][0]
for item in (default_keyring.get_items()):
if item.get_label() == "Chrome Safe Storage":
item.load_secret_sync()
return item.get_secret().get_text()
return None
Hit me up if that doesn't work for you
You can also use the command line:
$ secret-tool search application chrome
Sweet, thanks. I tried and couldn't replicate the issue a couple days ago, but I forgot to check what version of Chromium I was running (couldn't be too old, a fresh Arch installation from scratch within the last month or two). When I get back to my Arch box I'll see if I can figure out how to reproduce, and if so how to incorporate this.
Thanks again for the updates.
@trideceth12: Could you send a pull request? Makes it easier to try this out. I'd be happy to try it.
$ secret-tool search application chromium works for me. Chrome didn't.
I can't replicate the issue on Chromium 56.0.2924.87-1 on Arch Linux, cookies still have v10 prefix, all tests pass. I wonder why we're seeing a difference here.
Which desktop environment do you use @n8henrie ? Depending on your environment, chrome might not find a keychain to store it's encryption key in. I think you get this in recent Ubuntu and Kubuntu.
That could be it. I'm on wayland / sway.
Hi! I'm new to GitHub and Python so please be nice :)
I couldn't check in a Branch to create a Pull Request so here's what I did to make it work.
-
Install additional Python libraries
sudo apt-get install libsecret-1-dev
sudo apt-get install python-gi -
Merging the code from @trideceth12 into pycookiecheat.py
added after import sys:
import platform
added before def chrome_cookies
# used for reading from keychain
if sys.platform == "linux2" and "Ubuntu" in platform.platform():
try:
import gi
gi.require_version('Secret', '1')
from gi.repository import Secret
except ImportError:
raise Exception("Error: please install libsecret-1-dev and/or python-gi using apt-get")
def get_cookie_password():
service = Secret.Service.get_sync(Secret.ServiceFlags.LOAD_COLLECTIONS)
default_keyring = service.get_collections()[0]
default_keyring = service.unlock_sync([default_keyring])[1][0]
for item in (default_keyring.get_items()):
if item.get_label() == "Chrome Safe Storage":
item.load_secret_sync()
return item.get_secret().get_text()
return None
added after my_pass = 'peanuts'.encode('utf8')
my_pass_v11 = get_cookie_password()
added after kdf = PBKDF2HMAC...
kdf_v11 = PBKDF2HMAC(
algorithm=SHA1(),
length=length,
salt=salt,
iterations=iterations,
backend=default_backend(),
)
key_v11 = kdf_v11.derive(bytes(my_pass_v11))
changed last decryption block to:
for k, v, ev in conn.execute(sql, (host_key,)):
# if there is a not encrypted value or if the encrypted value
# doesn't start with the 'v10'/'v11' prefix, return v
if v or (ev[:3] != b'v10' and ev[:3] != b'v11'):
cookies_list.append((k, v))
elif (ev[:3] == b'v11'):
decrypted_tuple = (k, chrome_decrypt(ev, key=key_v11))
cookies_list.append(decrypted_tuple)
else:
decrypted_tuple = (k, chrome_decrypt(ev, key=key))
cookies_list.append(decrypted_tuple)
Just needs more testing to better handle exceptions and other platforms.
@n8henrie - Issue occurred on Ubuntu 16.04.2 Desktop 64 bit with Google Chrome 57.0.2987.133 installed from Google.
Let me know if you need more information!
Great, thanks for getting me the version numbers. I'll see if I can spin up a VM and reproduce, if so will try to merge in that code.
It would probably be better to check the desktop environment since that is the default for --password-store:
--password-store=<basic|gnome|kwallet>
Set the password store to use. The default is to automatically detect based on the desktop environment. basic selects the built in, unencrypted password store. gnome selects Gnome keyring. kwallet selects (KDE)
Many thanks to @stat1c1c3au @trideceth12 -- been plugging away on Virtualbox / Ubuntu all day and have incorporated your code in a way that seems to work without breaking anything.
https://github.com/n8henrie/pycookiecheat/tree/56ccaf345e1dcf74ff0db1af2dbd9a5eebb43721
NB: If using in a virtualenv, you apparently have to use the --system-site-packages flag in order to get access to the necessary gi libraries, no way to install them directly.
Don't know if anybody has had time to test, but I'm going to go ahead and merge into master. I would really like to get better automated testing ideally using something like Selenium (especially since I'm not primarily on Linux, and when I am it's Arch, which is part of the reason I had such a hard time replicating this issue). I've started a docker-selenium branch to see if that might be a good solution, but I've never used either of these technologies, so would be happy to have help if anyone has more experience here. Specifically, would be nice to have CI testing on a Ubuntu 16.04 distro that uses the commits from this thread.
Hey @n8henrie. Sorry, only just had a chance to try this out and have a couple of issues:
1/ Ubuntu
It seems that it's not quite pulling the password out of the keyring. my_pass is still being set to 'peanuts'
Debugging, I tracked it down to this code:
gnome_keyring = service.get_collections()
unlocked_keyring = service.unlock_sync(gnome_keyring).unlocked[0]
and had to change it to this to make it work:
gnome_keyring = service.get_collections()[0]
unlocked_keyring = service.unlock_sync([gnome_keyring]).unlocked[0]
2/ macOS Sierra 10.12.3
I couldn't upgrade using sudo pip install --upgrade pycookiecheat
Had to do this: sudo pip install --ignore-installed six --upgrade pycookiecheat
Ran into an issue when pip tried to uninstall the "six" package:
Installing collected packages: six, pyparsing, packaging, asn1crypto, appdirs, setuptools, cryptography, keyring, pycookiecheat
Found existing installation: six 1.4.1
DEPRECATION: Uninstalling a distutils installed project (six) has been deprecated and will be removed in a future version. This is due to the fact that uninstalling a distutils project will only partially uninstall the project.
Uninstalling six-1.4.1:
Exception:
Traceback (most recent call last):
File "/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/basecommand.py", line 215, in main
status = self.run(options, args)
File "/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/commands/install.py", line 342, in run
prefix=options.prefix_path,
File "/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/req/req_set.py", line 778, in install
requirement.uninstall(auto_confirm=True)
File "/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/req/req_install.py", line 754, in uninstall
paths_to_remove.remove(auto_confirm)
File "/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/req/req_uninstall.py", line 115, in remove
renames(path, new_path)
File "/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/utils/__init__.py", line 267, in renames
shutil.move(old, new)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 302, in move
copy2(src, real_dst)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 131, in copy2
copystat(src, dst)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 103, in copystat
os.chflags(dst, st.st_flags)
OSError: [Errno 1] Operation not permitted: '/tmp/pip-QRL7Xx-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/six-1.4.1-py2.7.egg-info'
Is this normal?
Thanks!
Hmmm...
Debugging, I tracked it down to this code:
gnome_keyring = service.get_collections() unlocked_keyring = service.unlock_sync(gnome_keyring).unlocked[0]and had to change it to this to make it work:
gnome_keyring = service.get_collections()[0] unlocked_keyring = service.unlock_sync([gnome_keyring]).unlocked[0]
Unfortunately, I changed that line because the way it's working for you wasn't working for me. We'll have to track this down.
gnome_keyring = service.get_collections()
unlocked_keyring = service.unlock_sync(gnome_keyring).unlocked[0]vs
gnome_keyring = service.get_collections()[0]`
unlocked_keyring = service.unlock_sync([gnome_keyring]).unlocked[0]The way I have it, service.get_collections() returns a list of all collections. I assumed the reason that service.get_collections()[0] wasn't working for me is that it only returns the first collection, and perhaps my setup was using a different one and should therefore return all collections.
That said, the service.unlock_sync(gnome_keyring).unlocked[0] will only return the first unlocked one, so it probably doesn't matter if I return all or just one. I'll mess around with this a little more today.
2/ macOS Sierra 10.12.3
I couldn't upgrade using sudo pip install --upgrade pycookiecheat
Had to do this: sudo pip install --ignore-installed six --upgrade pycookiecheat
I'd really rather you not install into the system site-packages. Please use a virtualenv.
I think to find the source of the error you'd have to go through the dependencies and find which one / where they were trying to upgrade six -- I think this may be an upstream issue from one of the dependencies. The quick answer is that you should generally be using virtualenvs, especially for debugging or development purposes, and that should fix this issue.
@stat1c1c3au Please test my latest dev commit: c55c9c7
$ pip install git+https://github.com/n8henrie/pycookiecheat@c55c9c77652fe2fa5f498f11b98cc80dcf36bdfa#egg=pycookiecheat
@n8henrie latest dev commit is good 👍
and thanks for the tip on the virtualenv
v0.3.5 is out on PyPI. Thanks for helping sort this out.