gvegayon/parallel

Can not set Stata directory on OSX with Stata MP 14.1

Rubensavvy opened this issue · 7 comments

Preliminaries

Before submitting an issue, please check (with x in brackets) that you:

  • Are using the newest release (see here for latest release version number).
  • Have checked that the examples in the help work.
  • Have checked that there is not already an existing issues for what you are reporting.
  • Have read the help (HTML version) and the gallery of examples.

Expected behavior and actual behavior

I’m trying to use "parallel do”, but when I run it I get an error that all processes encountered errors and the log is empty. Quick overview of the problem:

  • The problem seems to be that the Stata directory cannot be found.
  • If I manually try to alter the statapath, I get an error that it cannot set the Stata directory.
  • If I force the statapath, the command just keeps on running until I break
  • I tried the SSC and development version (only difference is that development version also stops with the error when forcing the statapath)

Some screenshots
error
statapath1
statapath2

Steps to reproduce the problem

Nothing works so base examples reproduce the problem:

parallel setclusters 2, f
sysuse auto
parallel, by(foreign): egen maxp = max(price)
tab maxp

Some relevant information
Everything works fine on another computer I have access to, running Windows.

System information

Stata 14MP
Mac OS X (10.12.5 - Sierra)

Output from creturn list:

System values

--------------------------------------------------------------------------------------------------------------------------
    c(current_date) = " 7 Aug 2017"
    c(current_time) = "19:14:43"
       c(rmsg_time) = 0                          (seconds, from set rmsg)
--------------------------------------------------------------------------------------------------------------------------
   c(stata_version) = 14.1
         c(version) = 14.1                       (version)
     c(userversion) = 14.1                       (version)
--------------------------------------------------------------------------------------------------------------------------
       c(born_date) = "01 Feb 2016"
          c(flavor) = "IC"
             c(bit) = 64
              c(SE) = 1
              c(MP) = 1
      c(processors) = 8                          (Stata/MP, set processors)
  c(processors_lic) = 8
 c(processors_mach) = 8
  c(processors_max) = 8
            c(mode) = ""
         c(console) = ""
--------------------------------------------------------------------------------------------------------------------------
              c(os) = "MacOSX"
           c(osdtl) = "10.12.5"
        c(hostname) = "rubens-imac.econ..."
    c(machine_type) = "Macintosh (Intel .."
       c(byteorder) = "lohi"
        c(username) = "rubensavelkoul"
--------------------------------------------------------------------------------------------------------------------------

Directories and paths

--------------------------------------------------------------------------------------------------------------------------
    c(sysdir_stata) = "/Applications/Sta.."      (sysdir)
     c(sysdir_base) = "/Applications/Sta.."      (sysdir)
     c(sysdir_site) = "/Applications/Sta.."      (sysdir)
     c(sysdir_plus) = "~/Library/Applica.."      (sysdir)
 c(sysdir_personal) = "~/Library/Applica.."      (sysdir)
 c(sysdir_oldplace) = "~/ado/"                   (sysdir)
          c(tmpdir) = "/var/folders/f0/k.."
--------------------------------------------------------------------------------------------------------------------------
         c(adopath) = "BASE;SITE;.;PERSO.."      (adopath)
             c(pwd) = "/Users/rubensavel.."      (cd)
          c(dirsep) = "/"
--------------------------------------------------------------------------------------------------------------------------

System limits

--------------------------------------------------------------------------------------------------------------------------
    c(max_N_theory) = 281474976710655
    c(max_k_theory) = 5000                       (set maxvar)
c(max_width_theory) = 1048576                    (set maxvar)
--------------------------------------------------------------------------------------------------------------------------
     c(max_matsize) = 11000
     c(min_matsize) = 10
--------------------------------------------------------------------------------------------------------------------------
    c(max_macrolen) = 4227143
        c(macrolen) = 645200                     (set maxvar)
         c(charlen) = 67783
      c(max_cmdlen) = 4227159
          c(cmdlen) = 645216                     (set maxvar)
     c(namelenbyte) = 128
     c(namelenchar) = 32
           c(eqlen) = 1337
--------------------------------------------------------------------------------------------------------------------------

Numerical and string limits

--------------------------------------------------------------------------------------------------------------------------
       c(mindouble) = -8.9884656743e+307
       c(maxdouble) = 8.9884656743e+307
       c(epsdouble) = 2.22044604925e-16
  c(smallestdouble) = 2.2250738585e-308
--------------------------------------------------------------------------------------------------------------------------
        c(minfloat) = -1.70141173319e+38
        c(maxfloat) = 1.70141173319e+38
        c(epsfloat) = 1.19209289551e-07
--------------------------------------------------------------------------------------------------------------------------
         c(minlong) = -2147483647
         c(maxlong) = 2147483620
--------------------------------------------------------------------------------------------------------------------------
          c(minint) = -32767
          c(maxint) = 32740
--------------------------------------------------------------------------------------------------------------------------
         c(minbyte) = -127
         c(maxbyte) = 100
--------------------------------------------------------------------------------------------------------------------------
    c(maxstrvarlen) = 2045
   c(maxstrlvarlen) = 2000000000
    c(maxvlabellen) = 32000
--------------------------------------------------------------------------------------------------------------------------

Current dataset

--------------------------------------------------------------------------------------------------------------------------
               c(N) = 74
               c(k) = 12
           c(width) = 43
         c(changed) = 0
        c(filename) = "/Applications/Sta.."
        c(filedate) = "13 Apr 2014 17:45"
--------------------------------------------------------------------------------------------------------------------------

Memory settings

--------------------------------------------------------------------------------------------------------------------------
          c(memory) = 33554432
          c(maxvar) = 5000                       (set maxvar)
         c(matsize) = 400                        (set matsize)
        c(niceness) = 5                          (set min_memory)
      c(min_memory) = 0                          (set min_memory)
      c(max_memory) = .                          (set max_memory)
     c(segmentsize) = 33554432                   (set segmentsize)
--------------------------------------------------------------------------------------------------------------------------

Output settings

--------------------------------------------------------------------------------------------------------------------------
            c(more) = "on"                       (set more)
            c(rmsg) = "off"                      (set rmsg)
              c(dp) = "period"                   (set dp)
        c(linesize) = 126                        (set linesize)
        c(pagesize) = 19                         (set pagesize)
         c(logtype) = "smcl"                     (set logtype)
         c(noisily) = 1
         c(eolchar) = "unix"                     (set eolchar)
      c(notifyuser) = "on"                       (set notifyuser)
         c(playsnd) = "off"                      (set playsnd)
  c(include_bitmap) = "on"                       (set include_bitmap)
--------------------------------------------------------------------------------------------------------------------------
           c(level) = 95                         (set level)
          c(clevel) = 95                         (set clevel)
--------------------------------------------------------------------------------------------------------------------------
  c(showbaselevels) = ""                         (set showbaselevels)
  c(showemptycells) = ""                         (set showemptycells)
     c(showomitted) = ""                         (set showomitted)
         c(fvlabel) = "on"                       (set fvlabel)
          c(fvwrap) = 1                          (set fvwrap)
        c(fvwrapon) = "word"                     (set fvwrapon)
        c(lstretch) = ""                         (set lstretch)
--------------------------------------------------------------------------------------------------------------------------
         c(cformat) = ""                         (set cformat)
         c(sformat) = ""                         (set sformat)
         c(pformat) = ""                         (set pformat)
--------------------------------------------------------------------------------------------------------------------------
  c(coeftabresults) = "on"                       (set coeftabresults)

Interface settings

--------------------------------------------------------------------------------------------------------------------------
      c(reventries) = 5000                       (set reventries)
     c(revkeyboard) = "on"                       (set revkeyboard)
     c(varkeyboard) = "on"                       (set varkeyboard)
     c(smoothfonts) = "on"                       (set smoothfonts)
         c(linegap) = 1                          (set linegap)
   c(scrollbufsize) = 204800                     (set scrollbufsize)
           c(maxdb) = 50                         (set maxdb)
--------------------------------------------------------------------------------------------------------------------------

Graphics settings

--------------------------------------------------------------------------------------------------------------------------
        c(graphics) = "on"                       (set graphics)
          c(scheme) = "s2color"                  (set scheme)
      c(printcolor) = "automatic"                (set printcolor)
       c(copycolor) = "automatic"                (set copycolor)
--------------------------------------------------------------------------------------------------------------------------

Efficiency settings

--------------------------------------------------------------------------------------------------------------------------
         c(adosize) = 1000                       (set adosize)
--------------------------------------------------------------------------------------------------------------------------

Network settings

--------------------------------------------------------------------------------------------------------------------------
        c(checksum) = "off"                      (set checksum)
        c(timeout1) = 30                         (set timeout1)
        c(timeout2) = 180                        (set timeout2)
--------------------------------------------------------------------------------------------------------------------------
       c(httpproxy) = "off"                      (set httpproxy)
   c(httpproxyhost) = ""                         (set httpproxyhost)
   c(httpproxyport) = 80                         (set httpproxyport)
--------------------------------------------------------------------------------------------------------------------------
   c(httpproxyauth) = "off"                      (set httpproxyauth)
   c(httpproxyuser) = ""                         (set httpproxyuser)
     c(httpproxypw) = ""                         (set httpproxypw)
--------------------------------------------------------------------------------------------------------------------------

Update settings

--------------------------------------------------------------------------------------------------------------------------
    c(update_query) = "off"                      (set update_query)
 c(update_interval) = 7                          (set update_interval)
   c(update_prompt) = "on"                       (set update_prompt)
--------------------------------------------------------------------------------------------------------------------------

Trace (program debugging) settings

--------------------------------------------------------------------------------------------------------------------------
           c(trace) = "off"                      (set trace)
      c(tracedepth) = 32000                      (set tracedepth)
        c(tracesep) = "on"                       (set tracesep)
     c(traceindent) = "on"                       (set traceindent)
     c(traceexpand) = "on"                       (set traceexpand)
     c(tracenumber) = "off"                      (set tracenumber)
     c(tracehilite) = ""                         (set tracehilite)
--------------------------------------------------------------------------------------------------------------------------

Mata settings

--------------------------------------------------------------------------------------------------------------------------
      c(matastrict) = "off"                      (set matastrict)
        c(matalnum) = "off"                      (set matalnum)
    c(mataoptimize) = "on"                       (set mataoptimize)
       c(matafavor) = "space"                    (set matafavor)
       c(matacache) = 2000                       (set matacache)
        c(matalibs) = "lmatabase;lmataad.."      (set matalibs)
     c(matamofirst) = "off"                      (set matamofirst)
--------------------------------------------------------------------------------------------------------------------------

Unicode settings

--------------------------------------------------------------------------------------------------------------------------
       c(locale_ui) = "nl_BE"                    (set locale_ui)
c(locale_functions) = "nl_BE"                    (set locale_functions)
  c(locale_icudflt) = "nl_BE"                    (unicode locale)
--------------------------------------------------------------------------------------------------------------------------

Other settings

--------------------------------------------------------------------------------------------------------------------------
            c(type) = "float"                    (set type)
         c(maxiter) = 16000                      (set maxiter)
   c(searchdefault) = "all"                      (set searchdefault)
             c(rng) = "default"                  (set rng)
     c(rng_current) = "mt64"
        c(rngstate) = "XAA00000000000000.."      (set rngstate)
       c(varabbrev) = "on"                       (set varabbrev)
      c(emptycells) = "keep"                     (set emptycells)
         c(odbcmgr) = "iodbc"                    (set odbcmgr)
      c(odbcdriver) = "unicode"                  (set odbcdriver)
--------------------------------------------------------------------------------------------------------------------------

Other

--------------------------------------------------------------------------------------------------------------------------
              c(pi) = 3.141592653589793
           c(alpha) = "a b c d e f g h i.."
           c(ALPHA) = "A B C D E F G H I.."
            c(Mons) = "Jan Feb Mar Apr M.."
          c(Months) = "January February .."
           c(Wdays) = "Sun Mon Tue Wed T.."
        c(Weekdays) = "Sunday Monday Tue.."
              c(rc) = 0                          (capture)
--------------------------------------------------------------------------------------------------------------------------

Can you manually verify that real path of Stata on your machine? Is the application "package" actually /Application/Stata 14/StataMP? Can you Ctrl-click on it and say "Show Package Contents" and then verify that there is the executable /Application/Stata 14/StataMP.app/Contents/MacOS/stata-mp?

Forgot to mention, but I already verified this.
In Mata, both fileexists(c("sysdir_stata")+"StataMP.app/Contents/MacOS/stata-mp") and fileexists(c("sysdir_stata")+"StataMP.app/Contents/MacOS/StataMP") return 1 as well.

I'm kind of lost now, what does this mean? Is there anything I can do? I should be able to enable the root user if that's necessary.

Btw, if it is of any use: I just tried on another Mac (El Capitan 10.11.6 and Stata 14MP) and am encountering the exact same problem.
Output from creturn list:
System values

-----------------------------------------------------------------------------
    c(current_date) = " 8 Aug 2017"
    c(current_time) = "00:08:18"
       c(rmsg_time) = 0                          (seconds, from set rmsg)
-----------------------------------------------------------------------------
   c(stata_version) = 14.1
         c(version) = 14.1                       (version)
     c(userversion) = 14.1                       (version)
-----------------------------------------------------------------------------
       c(born_date) = "01 Feb 2016"
          c(flavor) = "IC"
             c(bit) = 64
              c(SE) = 1
              c(MP) = 1
      c(processors) = 4                          (Stata/MP, set processors)
  c(processors_lic) = 8
 c(processors_mach) = 4
  c(processors_max) = 4
            c(mode) = ""
         c(console) = ""
-----------------------------------------------------------------------------
              c(os) = "MacOSX"
           c(osdtl) = "10.11.6"
        c(hostname) = "MacBook-Pro-van-R.."
    c(machine_type) = "Macintosh (Intel .."
       c(byteorder) = "lohi"
        c(username) = "rubensavelkoul"
-----------------------------------------------------------------------------

Directories and paths

-----------------------------------------------------------------------------
    c(sysdir_stata) = "/Applications/Sta.."      (sysdir)
     c(sysdir_base) = "/Applications/Sta.."      (sysdir)
     c(sysdir_site) = "/Applications/Sta.."      (sysdir)
     c(sysdir_plus) = "~/Library/Applica.."      (sysdir)
 c(sysdir_personal) = "~/Library/Applica.."      (sysdir)
 c(sysdir_oldplace) = "~/ado/"                   (sysdir)
          c(tmpdir) = "/var/folders/9w/b.."
-----------------------------------------------------------------------------
         c(adopath) = "BASE;SITE;.;PERSO.."      (adopath)
             c(pwd) = "/Users/rubensavel.."      (cd)
          c(dirsep) = "/"
-----------------------------------------------------------------------------

System limits

-----------------------------------------------------------------------------
    c(max_N_theory) = 281474976710655
    c(max_k_theory) = 5000                       (set maxvar)
c(max_width_theory) = 1048576                    (set maxvar)
-----------------------------------------------------------------------------
     c(max_matsize) = 11000
     c(min_matsize) = 10
-----------------------------------------------------------------------------
    c(max_macrolen) = 4227143
        c(macrolen) = 645200                     (set maxvar)
         c(charlen) = 67783
      c(max_cmdlen) = 4227159
          c(cmdlen) = 645216                     (set maxvar)
     c(namelenbyte) = 128
     c(namelenchar) = 32
           c(eqlen) = 1337
-----------------------------------------------------------------------------

Numerical and string limits

-----------------------------------------------------------------------------
       c(mindouble) = -8.9884656743e+307
       c(maxdouble) = 8.9884656743e+307
       c(epsdouble) = 2.22044604925e-16
  c(smallestdouble) = 2.2250738585e-308
-----------------------------------------------------------------------------
        c(minfloat) = -1.70141173319e+38
        c(maxfloat) = 1.70141173319e+38
        c(epsfloat) = 1.19209289551e-07
-----------------------------------------------------------------------------
         c(minlong) = -2147483647
         c(maxlong) = 2147483620
-----------------------------------------------------------------------------
          c(minint) = -32767
          c(maxint) = 32740
-----------------------------------------------------------------------------
         c(minbyte) = -127
         c(maxbyte) = 100
-----------------------------------------------------------------------------
    c(maxstrvarlen) = 2045
   c(maxstrlvarlen) = 2000000000
    c(maxvlabellen) = 32000
-----------------------------------------------------------------------------

Current dataset

-----------------------------------------------------------------------------
               c(N) = 74
               c(k) = 12
           c(width) = 43
         c(changed) = 0
        c(filename) = "/Applications/Sta.."
        c(filedate) = "13 Apr 2014 17:45"
-----------------------------------------------------------------------------

Memory settings

-----------------------------------------------------------------------------
          c(memory) = 33554432
          c(maxvar) = 5000                       (set maxvar)
         c(matsize) = 400                        (set matsize)
        c(niceness) = 5                          (set min_memory)
      c(min_memory) = 0                          (set min_memory)
      c(max_memory) = .                          (set max_memory)
     c(segmentsize) = 33554432                   (set segmentsize)
-----------------------------------------------------------------------------

Output settings

-----------------------------------------------------------------------------
            c(more) = "on"                       (set more)
            c(rmsg) = "off"                      (set rmsg)
              c(dp) = "period"                   (set dp)
        c(linesize) = 81                         (set linesize)
        c(pagesize) = 23                         (set pagesize)
         c(logtype) = "smcl"                     (set logtype)
         c(noisily) = 1
         c(eolchar) = "unix"                     (set eolchar)
      c(notifyuser) = "on"                       (set notifyuser)
         c(playsnd) = "off"                      (set playsnd)
  c(include_bitmap) = "on"                       (set include_bitmap)
-----------------------------------------------------------------------------
           c(level) = 95                         (set level)
          c(clevel) = 95                         (set clevel)
-----------------------------------------------------------------------------
  c(showbaselevels) = ""                         (set showbaselevels)
  c(showemptycells) = ""                         (set showemptycells)
     c(showomitted) = ""                         (set showomitted)
         c(fvlabel) = "on"                       (set fvlabel)
          c(fvwrap) = 1                          (set fvwrap)
        c(fvwrapon) = "word"                     (set fvwrapon)
        c(lstretch) = ""                         (set lstretch)
-----------------------------------------------------------------------------
         c(cformat) = ""                         (set cformat)
         c(sformat) = ""                         (set sformat)
         c(pformat) = ""                         (set pformat)
-----------------------------------------------------------------------------
  c(coeftabresults) = "on"                       (set coeftabresults)

Interface settings

-----------------------------------------------------------------------------
      c(reventries) = 5000                       (set reventries)
     c(revkeyboard) = "on"                       (set revkeyboard)
     c(varkeyboard) = "on"                       (set varkeyboard)
     c(smoothfonts) = "on"                       (set smoothfonts)
         c(linegap) = 1                          (set linegap)
   c(scrollbufsize) = 204800                     (set scrollbufsize)
           c(maxdb) = 50                         (set maxdb)
-----------------------------------------------------------------------------

Graphics settings

-----------------------------------------------------------------------------
        c(graphics) = "on"                       (set graphics)
          c(scheme) = "s2color"                  (set scheme)
      c(printcolor) = "automatic"                (set printcolor)
       c(copycolor) = "automatic"                (set copycolor)
-----------------------------------------------------------------------------

Efficiency settings

-----------------------------------------------------------------------------
         c(adosize) = 1000                       (set adosize)
-----------------------------------------------------------------------------

Network settings

-----------------------------------------------------------------------------
        c(checksum) = "off"                      (set checksum)
        c(timeout1) = 30                         (set timeout1)
        c(timeout2) = 180                        (set timeout2)
-----------------------------------------------------------------------------
       c(httpproxy) = "off"                      (set httpproxy)
   c(httpproxyhost) = ""                         (set httpproxyhost)
   c(httpproxyport) = 80                         (set httpproxyport)
-----------------------------------------------------------------------------
   c(httpproxyauth) = "off"                      (set httpproxyauth)
   c(httpproxyuser) = ""                         (set httpproxyuser)
     c(httpproxypw) = ""                         (set httpproxypw)
-----------------------------------------------------------------------------

Update settings

-----------------------------------------------------------------------------
    c(update_query) = "off"                      (set update_query)
 c(update_interval) = 7                          (set update_interval)
   c(update_prompt) = "on"                       (set update_prompt)
-----------------------------------------------------------------------------

Trace (program debugging) settings

-----------------------------------------------------------------------------
           c(trace) = "off"                      (set trace)
      c(tracedepth) = 32000                      (set tracedepth)
        c(tracesep) = "on"                       (set tracesep)
     c(traceindent) = "on"                       (set traceindent)
     c(traceexpand) = "on"                       (set traceexpand)
     c(tracenumber) = "off"                      (set tracenumber)
     c(tracehilite) = ""                         (set tracehilite)
-----------------------------------------------------------------------------

Mata settings

-----------------------------------------------------------------------------
      c(matastrict) = "off"                      (set matastrict)
        c(matalnum) = "off"                      (set matalnum)
    c(mataoptimize) = "on"                       (set mataoptimize)
       c(matafavor) = "space"                    (set matafavor)
       c(matacache) = 2000                       (set matacache)
        c(matalibs) = "lmatabase;lmataad.."      (set matalibs)
     c(matamofirst) = "off"                      (set matamofirst)
-----------------------------------------------------------------------------

Unicode settings

-----------------------------------------------------------------------------
       c(locale_ui) = "nl_BE"                    (set locale_ui)
c(locale_functions) = "nl_BE"                    (set locale_functions)
  c(locale_icudflt) = "nl_BE"                    (unicode locale)
-----------------------------------------------------------------------------

Other settings

-----------------------------------------------------------------------------
            c(type) = "float"                    (set type)
         c(maxiter) = 16000                      (set maxiter)
   c(searchdefault) = "all"                      (set searchdefault)
             c(rng) = "default"                  (set rng)
     c(rng_current) = "mt64"
        c(rngstate) = "XAA00000000000000.."      (set rngstate)
       c(varabbrev) = "on"                       (set varabbrev)
      c(emptycells) = "keep"                     (set emptycells)
         c(odbcmgr) = "iodbc"                    (set odbcmgr)
      c(odbcdriver) = "unicode"                  (set odbcdriver)
-----------------------------------------------------------------------------

Other

-----------------------------------------------------------------------------
              c(pi) = 3.141592653589793
           c(alpha) = "a b c d e f g h i.."
           c(ALPHA) = "A B C D E F G H I.."
            c(Mons) = "Jan Feb Mar Apr M.."
          c(Months) = "January February .."
           c(Wdays) = "Sun Mon Tue Wed T.."
        c(Weekdays) = "Sunday Monday Tue.."
              c(rc) = 0                          (capture)
-----------------------------------------------------------------------------

It's odd that you manual mata commands find the file, but that the parallel_setstatapath() doesn't. The function that checks this is mata: parallel_setstatapath(). Can you confirm parallel version? Sometimes people have two mixed-up installs. Also, could you try calling the function directly mata: parallel_setstatapath("") to ensure it doesn't find the file. It looks like it should work given the code and what is in your creturn. If it's a permission thing, could you open a terminal look at the output of ls -l for both the package and longer stata-mp? Could you also see if you can call the full stata-mp from the terminal?

I'm kind of lost now, what does this mean? Is there anything I can do? I should be able to enable the root user if that's necessary.

Sorry, what I said before was wrong. As Brian points out, a 1 means that the file does exist. My reply had to do with, as I mentioned before, installing Stata with sudo... in that case fileexists was returning a 0. Can you try doing what Brian suggests?

I only know basic Terminal, but your suggestions allowed me to figure out the problem. I was still figuring out what I had to do before issuing the ls -l in Terminal, and after adding Stata to my path I still couldn't use the command in Terminal (I was following these instructions: http://www.stata.com/support/faqs/mac/advanced-topics/). Definitely something fishy there.

While playing around with setting different directories, I noticed I couldn't cd "/Application/Stata 14/StataMP.app/Contents/MacOS/". Didn't know if I should have been able to, but I remembered there is two Applications folders on a Mac: "MacIntosh HD/Applications" and "Users/usr/Applications". Everything application is in the first folder on my computer, so I moved Stata to the latter (and renamed the folder temporarily "SStata 14" to not get confused) and then everything seemed to work. I thought moving Stata, worked, so I renamed that folder "Stata 14" and then it didn't work anymore. Renaming it again to SStata 14 did the trick again. Then I checked if renaming the folder to SStata 14 in "MacIntosh HD/Applications" also did the trick and it did...

I am not sure why this works, but I'll happily take it as a solution to my problem. If it helps, here are some screenshots of the output when issuing "mata: parallel_setstatapath()". I am not sure what that means, but thought it might help you understand the problem. Thanks for your suggestions and helping me out. If there is anything I can do to further understand the problem, let me know.

mata_setpath2
mata_setpath