pbkdf2_hmac() function is unimplemented on this machine

Question

pbkdf2_hmac() function is unimplemented on this machine

alx opened this issue 12 years ago · 28 comments

Hi,

I've updated to 1.0 and I'm trying to convert to the new file format, and got this error:

~$ pws resave --in 0.9 --out 1.0
ACCESS GRANTED
/Users/alx/.rvm/gems/ruby-1.9.2-p290/gems/pws-1.0.0/lib/pws/format/1.0.rb:127:in `pbkdf2_hmac': pbkdf2_hmac() function is unimplemented on this machine (NotImplementedError)
    from /Users/alx/.rvm/gems/ruby-1.9.2-p290/gems/pws-1.0.0/lib/pws/format/1.0.rb:127:in `kdf'
    from /Users/alx/.rvm/gems/ruby-1.9.2-p290/gems/pws-1.0.0/lib/pws/format/1.0.rb:35:in `encrypt'
    from /Users/alx/.rvm/gems/ruby-1.9.2-p290/gems/pws-1.0.0/lib/pws/format/1.0.rb:20:in `write'
    from /Users/alx/.rvm/gems/ruby-1.9.2-p290/gems/pws-1.0.0/lib/pws/format.rb:52:in `write'
    from /Users/alx/.rvm/gems/ruby-1.9.2-p290/gems/pws-1.0.0/lib/pws.rb:244:in `write_safe'
    from /Users/alx/.rvm/gems/ruby-1.9.2-p290/gems/pws-1.0.0/lib/pws.rb:196:in `resave'
    from /Users/alx/.rvm/gems/ruby-1.9.2-p290/gems/pws-1.0.0/lib/pws/runner.rb:135:in `public_send'
    from /Users/alx/.rvm/gems/ruby-1.9.2-p290/gems/pws-1.0.0/lib/pws/runner.rb:135:in `run'
    from /Users/alx/.rvm/gems/ruby-1.9.2-p290/gems/pws-1.0.0/bin/pws:4:in `<top (required)>'
    from /Users/alx/.rvm/gems/ruby-1.9.2-p290/bin/pws:19:in `load'
    from /Users/alx/.rvm/gems/ruby-1.9.2-p290/bin/pws:19:in `<main>'

My machine: macbook air 13" 2011, osx Lion 10.7.4, ruby 1.9.2-p290

Answer 1 · 2012-05-30T14:45:06.000Z

👍 I'm getting the exact same issue, except I'm not upgrading. I just now installed it.

Answer 2 · 2012-05-30T14:57:03.000Z

Same error after upgrading ruby to 1.9.3-p194

Answer 3 · 2012-05-30T15:06:22.000Z

tried to brew install openssl with no effect

Answer 4 · 2012-05-30T15:39:57.000Z

Same here with a fresh install

Answer 5 · 2012-05-30T18:35:16.000Z

hi @alx maybe you should upgrade your brew , you may have old formulas in your system, this function requires at least openssl 0.9.9. I upgraded brew, installed openssl 1.0.1 and re-built my ruby with openssl pointing to the openssl folder under brew like this:

rvm install 1.9.3-p125 --with-openssl-dir=/usr/local/Cellar/openssl/1.0.1c/

And then pws started working

Answer 6 · 2012-05-30T19:16:42.000Z

I've tried without success:

$ rvm reinstall 1.9.3-p194 --with-openssl-dir=/usr/local/Cellar/openssl/1.0.1c/
Removing /Users/alx/.rvm/src/ruby-1.9.3-p194...
Removing /Users/alx/.rvm/rubies/ruby-1.9.3-p194...
Removing default ruby interpreter
Removing ruby-1.9.3-p194 aliases...
Removing ruby-1.9.3-p194 wrappers...
Removing ruby-1.9.3-p194 environments...
Removing ruby-1.9.3-p194 binaries...
Installing Ruby from source to: /Users/alx/.rvm/rubies/ruby-1.9.3-p194, this may take a while depending on your cpu(s)...

ruby-1.9.3-p194 - #fetching 
ruby-1.9.3-p194 - #extracting ruby-1.9.3-p194 to /Users/alx/.rvm/src/ruby-1.9.3-p194
ruby-1.9.3-p194 - #extracted to /Users/alx/.rvm/src/ruby-1.9.3-p194
ruby-1.9.3-p194 - #configuring 
ruby-1.9.3-p194 - #compiling 
ruby-1.9.3-p194 - #installing 
Removing old Rubygems files...
Installing rubygems-1.8.24 for ruby-1.9.3-p194 ...
Installation of rubygems completed successfully.
ruby-1.9.3-p194 - adjusting #shebangs for (gem irb erb ri rdoc testrb rake).
ruby-1.9.3-p194 - #importing default gemsets (/Users/alx/.rvm/gemsets/)
Install of ruby-1.9.3-p194 - #complete 
$ pws resave --in 0.9 --out 1.0
ACCESS GRANTED
/Users/alx/.rvm/gems/ruby-1.9.3-p194/gems/pws-1.0.0/lib/pws/format/1.0.rb:127:in `pbkdf2_hmac': pbkdf2_hmac() function is unimplemented on this machine (NotImplementedError)
    from /Users/alx/.rvm/gems/ruby-1.9.3-p194/gems/pws-1.0.0/lib/pws/format/1.0.rb:127:in `kdf'
    from /Users/alx/.rvm/gems/ruby-1.9.3-p194/gems/pws-1.0.0/lib/pws/format/1.0.rb:35:in `encrypt'
    from /Users/alx/.rvm/gems/ruby-1.9.3-p194/gems/pws-1.0.0/lib/pws/format/1.0.rb:20:in `write'
    from /Users/alx/.rvm/gems/ruby-1.9.3-p194/gems/pws-1.0.0/lib/pws/format.rb:52:in `write'
    from /Users/alx/.rvm/gems/ruby-1.9.3-p194/gems/pws-1.0.0/lib/pws.rb:244:in `write_safe'
    from /Users/alx/.rvm/gems/ruby-1.9.3-p194/gems/pws-1.0.0/lib/pws.rb:196:in `resave'
    from /Users/alx/.rvm/gems/ruby-1.9.3-p194/gems/pws-1.0.0/lib/pws/runner.rb:135:in `public_send'
    from /Users/alx/.rvm/gems/ruby-1.9.3-p194/gems/pws-1.0.0/lib/pws/runner.rb:135:in `run'
    from /Users/alx/.rvm/gems/ruby-1.9.3-p194/gems/pws-1.0.0/bin/pws:4:in `<top (required)>'
    from /Users/alx/.rvm/gems/ruby-1.9.3-p194/bin/pws:19:in `load'
    from /Users/alx/.rvm/gems/ruby-1.9.3-p194/bin/pws:19:in `<main>'
    from /Users/alx/.rvm/gems/ruby-1.9.3-p194/bin/ruby_noexec_wrapper:14:in `eval'
    from /Users/alx/.rvm/gems/ruby-1.9.3-p194/bin/ruby_noexec_wrapper:14:in `<main>'

Answer 7 · 2012-05-30T19:25:43.000Z

just in case, more info about my openssl brew:

$ brew info openssl
openssl 1.0.1c
http://openssl.org

This formula is keg-only.
Mac OS X already provides this program and installing another version in
parallel can cause all kinds of trouble.

The OpenSSL provided by OS X is too old for some software.

/usr/local/Cellar/openssl/1.0.1c (428 files, 15M)
https://github.com/mxcl/homebrew/commits/master/Library/Formula/openssl.rb

Answer 8 · 2012-05-30T19:35:34.000Z

Just in case, I've tried with 1.9.3-p125, same error

Answer 9 · 2012-05-30T20:32:52.000Z

hi @alx, i don't know why it didn't work for you. What if you remove and install back the pws gem ?

Answer 10 · 2012-05-30T20:47:25.000Z

Same error:

$ which pws
/Users/alx/.rvm/gems/ruby-1.9.3-p125/bin/pws
$ gem uninstall pws
Remove executables:
    pws

in addition to the gem? [Yn]  Y
Removing pws
Successfully uninstalled pws-1.0.0
$ gem install pws
Fetching: pws-1.0.0.gem (100%)
       ┌── info ─────────────────────────┐
 J-_-L │ https://github.com/janlelis/pws │
       ├── usage ────────────────────────┤
       │ pws --help                      │
       └─────────────────────────────────┘
Successfully installed pws-1.0.0
1 gem installed
$ pws resave --in 0.9 --out 1.0
ACCESS GRANTED
/Users/alx/.rvm/gems/ruby-1.9.3-p125/gems/pws-1.0.0/lib/pws/format/1.0.rb:127:in `pbkdf2_hmac': pbkdf2_hmac() function is unimplemented on this machine (NotImplementedError)

Answer 11 · 2012-05-30T21:56:35.000Z

Same error on this gist, I've tried install pbkdf2 gem, but still no success: https://gist.github.com/1749005#comments

Answer 12 · 2012-05-30T22:38:33.000Z

Hi alx, thanks for the bug report.

This error doesn't look nice and I cannot reproduce it, because I don't have a macbook available. I could use the pbkdf2 gem, but it is significantly slower, so it should only be a fallback option. Does the fix by thiagobrandam work for someone else?

Answer 13 · 2012-05-31T06:52:32.000Z

thiagobrandam's fix didn't work for me, either (OS X Lion 10.7.4, ruby 1.9.3-p194 via rvm (fresh install)), same error:

gems/pws-1.0.0/lib/pws/format/1.0.rb:127:in `pbkdf2_hmac': pbkdf2_hmac() function is unimplemented on this machine (NotImplementedError)

Answer 14 · 2012-05-31T08:53:38.000Z

Released 1.0.1 with a fallback to the pbkdf2 gem. Can you please confirm it's working now? I will nevertheless let this issue open till someone has got an idea, how to solve the original problem.

Answer 15 · 2012-05-31T09:00:05.000Z

Thanks, it's working better - and slower, but at least it's working :) - here are the logs:

$ gem install pws
Fetching: pws-1.0.1.gem (100%)
       ┌── info ─────────────────────────┐
 J-_-L │ https://github.com/janlelis/pws │
       ├── usage ────────────────────────┤
       │ pws --help                      │
       └─────────────────────────────────┘
Successfully installed pws-1.0.1
1 gem installed
$ pws resave --in 0.9 --out 1.0
ACCESS GRANTED
The password file has been resaved
$ pws list
ACCESS GRANTED
Entries in /Users/alx/.pws

Answer 16 · 2012-05-31T09:13:31.000Z

Jan, I have the same Issue on my Ubuntu 12.04 machine.

Linux freeside 3.2.0-24-generic #39-Ubuntu SMP Mon May 21 16:52:17 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux
ruby 1.9.3p194 (2012-04-20 revision 35410) [x86_64-linux]

pws 1.0.1 which you published while I got to this page, does not have the above error.

Thanks for the timely fix :)

Answer 17 · 2012-06-06T14:01:41.000Z

Any update or idea to solve this issue?

Ruby fix is nice but the app is really slugish to use with it, approx 25s. to load a password in the clipboard.

Answer 18 · 2012-06-11T21:13:34.000Z

@alx Can you please paste the output of: rvm 1.9.3-p194 && cd ~/.rvm/src/ruby-1.9.3-p194/ext/openssl && ruby extconf.rb

Answer 19 · 2012-06-12T06:18:37.000Z

here it is: https://gist.github.com/2915564

Answer 20 · 2012-06-12T10:10:33.000Z

@alx Thank you.

I don't know why, but your openssl library somehow misses the PKCS5_PBKDF2_HMAC() function (but still got PKCS5_PBKDF2_HMAC_SHA1()...). I could not determine a simple reason for that...

Please try to uninstall your openssl library, download one from http://www.openssl.org/source/ and build it yourself. Then run the line above, again.

Btw: As long as we haven't found a fix for that problem, you can use the --iterations option with a lower value than the default of 75_000.

Answer 21 · 2012-06-12T11:03:11.000Z

Thanks, I've tried compiling various version of openssl from source, but still the same problem on availability of PKCS5_PBKDF2_HMAC function.

I might have a mix of ssl version, here are some results after compiling rvm with --with-openssl-dir option:

openssl$ rvm reinstall 1.9.3-p194 --with-openssl-dir=/usr/local
...
openssl$ which openssl
/usr/bin/openssl
openssl$ openssl version
OpenSSL 0.9.8r 8 Feb 2011
openssl$ /usr/local/ssl/bin/openssl version
OpenSSL 1.0.1c 10 May 2012

Do you think extconf.rb could mix the various install openssl libs?

I'll use the --iterations options waiting for another alternative.

Answer 22 · 2012-06-13T09:37:23.000Z

So you definitely have multiple versions of openssl on your machine the wrong one gets picked. You need to play with the compile option. Maybe this one (with just another openssl package...) works for you:

$ rvm pkg install openssl
$ rvm reinstall 1.9.3-p194 --with-openssl-dir=$rvm_path/usr

Answer 23 · 2012-06-13T10:11:03.000Z

Problem with rvm pkg install openssl is that it install openssl-0.9.8t whn 0.9.9 is required from @thiagobrandam comment.

I've tried to look for a way to change the version copying compiled openssl-1.0.1 form /usr/local/ssl in $rvm_path/usr but without success.

Answer 24 · 2012-06-13T11:37:38.000Z

@alx
Installing openssl via homebrew and reinstalling ruby with specific compile flags seems to solve the issue

LDFLAGS='-L/usr/local/Cellar/openssl/1.0.1c/lib' CPPFLAGS='-I/usr/local/Cellar/openssl/1.0.1c/include' rvm reinstall 1.9.3-p194 --with-openssl-dir=/usr/local

Answer 25 · 2012-06-13T18:12:04.000Z

Thanks @andiug

I tried it without success.

Is it normal that the --with-openssl-dir=/usr/local is not linking to the installed openssl brew?

~$ brew install openssl
==> Downloading http://openssl.org/source/openssl-1.0.1c.tar.gz
[...]
    LDFLAGS  -L/usr/local/Cellar/openssl/1.0.1c/lib
    CPPFLAGS -I/usr/local/Cellar/openssl/1.0.1c/include
~$  LDFLAGS='-L/usr/local/Cellar/openssl/1.0.1c/lib' CPPFLAGS='-I/usr/local/Cellar/openssl/1.0.1c/include' rvm reinstall 1.9.3-p194 --with-openssl-dir=/usr/local
[...]
Installing rubygems-1.8.24 for ruby-1.9.3-p194 ...
[...]
Install of ruby-1.9.3-p194 - #complete
~$ rvm 1.9.3-p194 && cd ~/.rvm/src/ruby-1.9.3-p194/ext/openssl && ruby extconf.rb
[...]
checking for PKCS5_PBKDF2_HMAC()... no
[...]
openssl$

Full logs: https://gist.github.com/2925586

Answer 26 · 2012-06-13T19:22:01.000Z

Thanks for the suggestions and thanks for trying them out. Some more ideas:

Can you please try to pass to extconf.rb directly
Try to pass /usr/local/Cellar as path instead of just /usr/local
In some thread, someone applied these options: --with-openssl-include=/where/you/installed/openssl/include --with-openssl-lib=/where/you/installed/openssl LD_RUN_PATH=/where/you/installed/openssl/lib make

Answer 27 · 2012-06-14T09:13:20.000Z

Yes! got it! Only 2 seconds now to load pws list :)

The third option was the good one, thanks all for your support!

I'll let someone else close this issue, in case something could be done to solve this, or if the readme needs to be updated.

direct extconf.rb

I'm not sure I understood this idea well, do you mean calling extconf with the default system ruby?

~$ ruby --version
ruby 1.8.7 (2011-12-28 patchlevel 357) [universal-darwin11.0]
~$ openssl version
OpenSSL 0.9.8r 8 Feb 2011
~$ cd ~/.rvm/src/ruby-1.9.3-p194/ext/openssl && ruby extconf.rb
...
checking for PKCS5_PBKDF2_HMAC()... no

with /usr/local/Cellar path

~$ LDFLAGS='-L/usr/local/Cellar/openssl/1.0.1c/lib' CPPFLAGS='-I/usr/local/Cellar/openssl/1.0.1c/include' rvm reinstall 1.9.3-p194 --with-openssl-dir=/usr/local/Cellar
...
~$ rvm use ruby-1.9.3-p194 && cd ~/.rvm/src/ruby-1.9.3-p194/ext/openssl && ruby extconf.rb
...
checking for PKCS5_PBKDF2_HMAC()... no

with applied options

~$ LD_RUN_PATH='/usr/local/Cellar/openssl/1.0.1c/lib' LDFLAGS='-L/usr/local/Cellar/openssl/1.0.1c/lib' CPPFLAGS='-I/usr/local/Cellar/openssl/1.0.1c/include' rvm reinstall 1.9.3-p194 --with-openssl-include=/usr/local/Cellar/openssl/1.0.1c/include --with-openssl-lib=/usr/local/Cellar/openssl/1.0.1c/lib
...
~$ rvm use ruby-1.9.3-p194 && cd ~/.rvm/src/ruby-1.9.3-p194/ext/openssl && ruby extconf.rb
...
checking for PKCS5_PBKDF2_HMAC()... yes

Answer 28 · 2012-06-14T21:48:25.000Z

Finaly! Updated the README and released 1.0.2 which outputs a warning when used with an old openssl version. Thanks for your efforts, especially @alx, @thiagobrandam and @andiug :)