ERROR: error loading package 'paxml'

I'm trying to run the PAX code in this repo:

I installed the prerequisites as mentioned in the repo:

python3 -m pip install -U pip
python3 -m pip install paxml praxis
cd paxml
bazel run -c opt --define=pax_task=lm \
    main -- \
    --exp=lm.decoder.ptb.PTBCharTransformerSmallSgd \
    --job_log_dir=/tmp/jax_log_dir/exp01 --alsologtostderr

To start training, I got the command line from here: https://github.com/google/paxml/blob/main/paxml/main.py#L19-L22

bazel run -c opt \
  third_party/py/paxml/tasks/lm/params:main -- \
  --exp=bert.BertAdamL4H128 \
  --job_log_dir=/tmp/jax_log_dir/exp01 --alsologtostderr

I'm encountering the following issue:

ERROR: Skipping 'paxml': error loading package 'paxml': Every .bzl file must have a corresponding package, but '//praxis:build-visibility.bzl' does not have one. Please create a BUILD file in the same or any parent directory. Note that this
BUILD file does not need to do anything except exist.
WARNING: Target pattern parsing failed.
ERROR: error loading package 'paxml': Every .bzl file must have a corresponding package, but '//praxis:build-visibility.bzl' does not have one. Please create a BUILD file in the same or any parent directory. Note that this BUILD file does not need to do anything except exist.
INFO: Elapsed time: 0.750s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (0 packages loaded)
FAILED: Build did NOT complete successfully (0 packages loaded)
    currently loading: paxml
    Fetching @com_google_protobuf; Restarting.

Though I see the BUILD is present here: https://github.com/google/paxml/blob/main/paxml/BUILD

Is there anything I'm doing wrong? Any suggestions to get it running?

Hi,

Thanks for the report.
We haven't pushed any library content on PyPi yet, so I doubt you've installed the library that way.

I'm not sure why you're getting this error message.
But as a temporary workaround and until we deploy proper packages and instructions, could you:

remove the problematic file build-visibility.bzl and any reference to it in the BUILD file.
update the visibility to a default value such as "//visibility:public"

We haven't pushed any library content on PyPi yet, so I doubt you've installed the library that way.

Interesting, the readme says to do that ...

paxml/README.md

Lines 29 to 32 in 93715a1

    
           ```bash 
        
           $ python3 -m pip install -U pip 
        
           $ python3 -m pip install paxml praxis 
        
           ```

@laurentes is there instructions written somewhere on how we can build and run PAX?

@laurentes

re: "remove the problematic file build-visibility.bzl and any reference to it in the BUILD file."

I commented out those specific lines in all the BUILD files I found: Is this right?

paxml/BUILD:load("//praxis:build-visibility.bzl", "JAX_VISIBILITY")
paxml/tools/BUILD:load("//praxis:build-visibility.bzl", "JAX_VISIBILITY")
paxml/tasks/test/params/BUILD:load("//praxis:build-visibility.bzl", "JAX_VISIBILITY")
paxml/tasks/BUILD:load("//praxis:build-visibility.bzl", "JAX_VISIBILITY")
paxml/tasks/lm/BUILD:load("//praxis:build-visibility.bzl", "JAX_VISIBILITY")
paxml/tasks/lm/params/BUILD:load("//praxis:build-visibility.bzl", "JAX_VISIBILITY")

re: "update the visibility to a default value such as "//visibility:public""

Where do I need to add this? Could be provide me with the snippet/example? Thanks!

@sharathts I was replying to the specific error message that @mjsML was referencing above.

I assume the BUILD file(s) has something like the following:

load("//praxis:build-visibility.bzl", "JAX_VISIBILITY")

package(default_visibility = JAX_VISIBILITY)

If build-visibility.bzl is the problem, I would remove it and update such snippet into just this line:

package(default_visibility = ["//visibility:public"])

like here:
https://github.com/bazelbuild/bazel/blob/master/examples/cpp/BUILD#L1

Thanks for the suggestion. I did this with all BUILD files:

-load("//praxis:build-visibility.bzl", "JAX_VISIBILITY")
+#load("//praxis:build-visibility.bzl", "JAX_VISIBILITY")
 
-package(default_visibility = JAX_VISIBILITY)
+#package(default_visibility = JAX_VISIBILITY)
+package(default_visibility = ["//visibility:public"])

Now I get the following error:

ERROR: /workspace/pax/paxml/BUILD:232:22: no such package 'praxis': BUILD file not found in any of the following directories. Add a BUILD file to a directory to mark it as a package.
 - /workspace/pax/praxis and referenced by '//paxml:main_lib'
ERROR: Analysis of target '//paxml:main' failed; build aborted:
INFO: Elapsed time: 1.387s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (39 packages loaded, 127 targets configured)
FAILED: Build did NOT complete successfully (39 packages loaded, 127 targets configured)

Hi @sharathts, is the targeted accelerator TPU or GPU? Knowing that would be helpful to reproduce the issue you are facing. Thanks.

The targeted accelerator is a GPU.

	```bash
	$ python3 -m pip install -U pip
	$ python3 -m pip install paxml praxis
	```