No way to install `org`

[jiri]

Since org is included with emacs, use-package won't fetch it from a remote repo, even when using :pin – most likely due to the fact that package-installed-p also check built in packages. This can be problematic, because not only does it render use-package powerless to install a newer version of a package than is shipped with emacs (which can not only old but also buggy, as is the case with org in Emacs 25 devel), but also pretty much undermines :pin, which I would expect to ensure that the pinned version is installed.

Possible solutions are:

1. Do custom package checking bypassing package-installed-p
2. Always ensure the pinned version is installed, doing package-delete if necessary

Steps to reproduce using emacs -Q:

1. Install use-package
2. (use-package org :pin gnu)

[jiri]

Relevant elisp:

1. package-archive-contents – An alist mapping package name to their descriptors
2. package-desc-archive – An undocumented function in package.el that retrieves the archive name from the descriptor
3. package-install – When supplied with a package descriptor instead of a package name, install the package if that specific version hasn't been installed yet.

That last one is the key to solving this IMO.

[jiri]

This is the workaround I'm using for now, maybe it could be adapted into something inside use-package itself. It modifies the behavior of package-installed-p to only check if a packages was downloaded and installed using package.el.

(defun package-from-archive (f &rest args)
  (and (apply f args)
       (assq (car args) package-alist)))

(advice-add 'package-installed-p :around 'package-from-archive)

[thomasf]

I use the the org-plus-contrib package in the org elpa repository. It's not a general solution to the problem but it installs the latest org mode version.

...
(eval-and-compile
  (setq
   package-archives
   '(("melpa-stable" . "http://stable.melpa.org/packages/")
     ("melpa" . "http://melpa.org/packages/")
     ("marmalade"   . "http://marmalade-repo.org/packages/")
     ("org"         . "http://orgmode.org/elpa/")
     ("gnu"         . "http://elpa.gnu.org/packages/")
     ("sc"   . "http://joseito.republika.pl/sunrise-commander/")))
...
(use-package org
  :ensure org-plus-contrib
  :defer 7
...

[jiri]

Thanks for the info! Sadly, this isn't a workaround that would work for any other built-in package.

This seems to be a deeper issue with package.el itself, because it refuses to install org because it's built-in, even though it would get installed to a different place (~/.emacs.d/elpa). I've kicked off an email on emacs-devel about this, but until / unless this gets fixed, looking into a more fine-grained control over package installation might be desirable.

I suggest we either:

1. Ignore built-in packages by default and use :pin built-in to load them
2. Ensure that when a package gets pinned, that version is the one that's loaded
3. Introduce a customize variable to control this

[marcinant]

Hi!

org-mode is a pain in a** indeed.

Here is my workaround:

(eval-and-compile
  (let ((clp))
    (while (setq clp (locate-library "org"))
      (setq load-path
        (delete
         (directory-file-name (file-name-directory clp)) load-path))))

  (dolist (S (append (apropos-internal (concat "^" (symbol-name 'org) "-"))
             (apropos-internal (concat "^global-" (symbol-name 'org) "-"))))
    (when (and (fboundp S)
           (let ((sf (symbol-function S)))
         (and (listp sf) (eq (car sf) 'autoload))))
      (fmakunbound S))))

I know. It's a brute force method. However, it does its job. I couldn't find any better solution.

[Balooga]

(use-package org :ensure t) installs the latest 8.2.10 for me with no issues. However I break my load sequence into an 'init.el' which then loads my 'emacs.el'

I have the following in my init.el

(add-to-list 'package-archives
             '("melpa" . "http://melpa.org/packages/") t)
(add-to-list 'package-archives
             '("org" . "http://orgmode.org/elpa/") t)
(package-initialize)

(unless (package-installed-p 'use-package)
  (package-refresh-contents)
  (package-install 'use-package))
(eval-when-compile
  (require 'use-package))
(require 'bind-key)
(setq use-package-verbose nil)

And then the following in my 'emacs.el'

(use-package org
  :ensure t
  :defer t
  ;;:init (setq initial-major-mode 'org-mode) ;; Set mode of *scratch* buffer
  :bind (("C-c l" . org-store-link)
         ("C-c c" . org-capture)
         ("C-c a" . org-agenda)
         :map org-mode-map
         ;; ("C-h" . org-delete-backward-char)
         ("C-c !" . org-time-stamp-inactive))
  :mode ("\\.org$" . org-mode)
  :config
  (require 'org-id)
  ...
)

[shackra]

Well, I actually have a workaround that a kind user on Emacs Stack Exchange helped me with, here is the code:

(defun shackra/update-one-package (package)
  "Actualiza un paquete PACKAGE"
  (when (package-installed-p package)
    (let* ((newest-pkg (car-safe (cdr (assq package package-archive-contents))))
           (new-ver (and newest-pkg (package-desc-version newest-pkg)))
           (builtin-pkg (cdr (assq package package--builtins)))
           (installed-pkg (car-safe (cdr (assq package package-alist))))
           (old-dir (and installed-pkg (package-desc-dir installed-pkg)))
           (old-ver (or (and installed-pkg (package-desc-version installed-pkg))
                       (and builtin-pkg (package--bi-desc-version builtin-pkg)))))
      (when (and new-ver (version-list-< old-ver new-ver)) 
        ;; Instalamos la nueva versión de org-mode
        (condition-case nil
            ;; en caso de algún error tratando de bajar algún paquete, captura
            ;; el error para que no interfiera con la inicialización de Emacs
            (progn (package-install newest-pkg)
                   (message (format "Paquete «%s» actualizado de la versión %s a la versión %s"
                                    (package-desc-name newest-pkg) old-ver new-ver))))
        (unless old-dir
      (delete-directory old-dir t))))))

I'm only using it to update org when my Emacs start, before loading the rest of my configuration which is written on an org file. Hope it helps :)

[sri]

I don't use use-package, but I was running into the same problem described here
(came across this thread while doing a google search):

(package-installed-p 'org) returns t and (package-install 'org) doesn't do anything
because org is a builtin package and it thinks it is already installed.

This is what I do to get around that:

(let* ((package--builtins '()) ; <- this does the trick
       (missing (remove-if 'package-installed-p my-packages)))
  (when missing
    (package-refresh-contents)
    (mapc 'package-install missing)))

(Sorry about the reference -- didn't think adding the URL to this issue on that commit
would make that commit show up here.)

[theophilusx]

Just a cople of comments on this issue -

1. Using org-plus-contrib in addition to not fixing the underlying issue has another problem. If you then install packages which also depend on org, you end up with BOTH org and org-plus-contrib being installed. It is a shame that package.el doesn't have something like a generic package specification like some package managers where you could say something like "depends on org-mode" and then have packages state what services they provide i.e. "provides org-mode" rather than tying things to package names.

2. I wish org didn't actually do org and org-plus-contrib. would be far better IMO if they just did org and org-contrib, allowing users to add the contrib if they wanted. Would at least avoid the package dependency issue. Currently, if I want the contrib stuff, I have to accept that other packages, due to their dependencies, will also install org.

3. It would make some sense to call the bundled org package something else (though see problem in 1.) or not bundle org with emacs. At the moment, you have verison 8.x bundled with emacs 25.1, you have version 9.x in the 'gnu' repository, which is currently the same as the org repository, but at some point, org will update their version and we will have a 3rd version!

4. Things seem to get 'messy' if yo install an org related package from melpa.

Would be good if I could just have org pinned to a specific repo and be done with it, but that doesn't seem to work correctly when the package is also built-in/bundled with emacs. You also have to be far too careful about how you load things to avoid getting a mixed and unreliable org loaded.

Most of this is not a use-package problem directly, but things like use-package seem to be exposing some of the weaknesses in either package.el or the relationship between core emacs, package repositories and different user requirements (i.e. bleeding edge/latest features vs most stable etc).

[raxod502]

Another solution is to drop package.el and use an alternative package manager like straight.el which does not have this problem. Disclaimer: I'm the author. Note that straight.el has built-in integration with use-package and can be used as a drop-in replacement for the default package.el support. In that case, installing the latest version of Org from the source repository is as simple as

(use-package org)

[zzamboni]

@raxod502 I learned about straight from your comment, so I've decided to give it a try. It looks very promising. Unfortunately, when I install org, it still gives me an old version (8.2.10 instead of 9.1.2). If I run straight-get-recipe on org I get (org :type git :host github :repo "emacsmirror/org").

Do I need to specify my own recipe for org, or am I doing something wrong?

[dieggsy]

@zzamboni FWIW I use the following with straight.el:

(use-package org
  :recipe (:host github
           :repo "emacsmirror/org"
           :files ("lisp/*.el" "contrib/lisp/*.el"))
...
)

[raxod502]

@zzamboni to avoid cluttering this issue tracker can you open a ticket against straight.el?

[jwiegley]

I'm closing this as not something to be fixed in use-package, unless others disagree.

[Atman50]

My solution (Emacs 25.3):
The following also fixes the issue by looking for the latest version of org from (hopefully) the org repository and making sure it's installed. This will install the externally defined org-mode package if one does not already exist.

(unless (file-expand-wildcards (concat package-user-dir "/org-[0-9]*"))
  (package-install (elt (cdr (assoc 'org package-archive-contents)) 0)))
(require 'org)

[mzuther]

Sorry for practising necromancy – I have just found a one-liner that fixes the issue (Emacs 25.2):

(assq-delete-all 'org package--builtins)

[hubisan]

Sorry for practising necromancy – I have just found a one-liner that fixes the issue (Emacs 25.2):

(assq-delete-all 'org package--builtins)

Thanks for the solution. Had to make a two liner out of it (Emacs 27.1):

(assq-delete-all 'org package--builtins)
(assq-delete-all 'org package--builtin-versions)

[markokocic]

Thanks @hubisan, this seems to work for me, also in Emacs 28 branch.

However, I would still like that use-package actually honours the :pin property and really install org from gnu if I set :pin gnu instead of ignoring it completely and loading the builtin org version.

The same applies to other packages too, not only to org.