west cannot process a git branch name containing a single quote '
GarrettCarter-eaton opened this issue · 8 comments
If a repo that west clones and processes contains a branch name using a single quote, the following error is received:
Traceback (most recent call last):
File "/usr/local/bin/west", line 8, in <module>
sys.exit(main())
File "/usr/local/lib/python3.8/dist-packages/west/app/main.py", line 793, in main
app.run(argv or sys.argv[1:])
File "/usr/local/lib/python3.8/dist-packages/west/app/main.py", line 109, in run
self.run_command(argv)
File "/usr/local/lib/python3.8/dist-packages/west/app/main.py", line 342, in run_command
cmd.run(args, unknown, self.topdir, manifest=self.manifest,
File "/usr/local/lib/python3.8/dist-packages/west/commands.py", line 135, in run
self.do_run(args, unknown)
File "/usr/local/lib/python3.8/dist-packages/west/app/project.py", line 806, in do_run
self.update_all()
File "/usr/local/lib/python3.8/dist-packages/west/app/project.py", line 871, in update_all
self.update(project)
File "/usr/local/lib/python3.8/dist-packages/west/app/project.py", line 1039, in update
self.clean_refs_west(project, stats, take_stats)
File "/usr/local/lib/python3.8/dist-packages/west/app/project.py", line 1296, in clean_refs_west
_clean_west_refspace(project)
File "/usr/local/lib/python3.8/dist-packages/west/app/project.py", line 1471, in _clean_west_refspace
project.git(delete_ref_cmd)
File "/usr/local/lib/python3.8/dist-packages/west/manifest.py", line 807, in git
cmd_list = shlex.split(cmd)
File "/usr/lib/python3.8/shlex.py", line 311, in split
return list(lex)
File "/usr/lib/python3.8/shlex.py", line 300, in __next__
token = self.get_token()
File "/usr/lib/python3.8/shlex.py", line 109, in get_token
raw = self.read_token()
File "/usr/lib/python3.8/shlex.py", line 191, in read_token
raise ValueError("No closing quotation")
ValueError: No closing quotation
Example branch name:
LTK-18338-No-link-for-privacy-policy-on-WebUI's-Login-page
Thanks for the report!
I think I know what's happening and how to fix it but this looks tedious to reproduce because the quote must be in a remote branch, not just in a local branch.
Can you share a public manifest/repo that reproduces? A small one if possible.
Can you share a public manifest/repo that reproduces? A small one if possible.
https://github.com/mbolivar-ampere/west/tree/test-'-for-marc
Thanks @mbolivar-ampere . I still cannot reproduce, probably because the management of refs/west/* has changed.
@GarrettCarter-eaton which west version are you using? Please look at this bug template for "inspiration": https://github.com/zephyrproject-rtos/zephyr/issues/new?assignees=&labels=bug&projects=&template=001_bug_report.md&title=
To Reproduce
Manifest repo: https://github.com/GarrettCarter-eaton/west-quote-test-manifest
Dependency of manifest repo (containing quote in a branch name): https://github.com/GarrettCarter-eaton/west-quote-test-dep
west init -m https://github.com/GarrettCarter-eaton/west-quote-test-manifestwest update- See error
I'm noticing the behavior differs depending on what revision of the dependency repo you specify. When I specify a git SHA, West seems to process all the refs in the repo which leads to the quote processing failure.
Expected behavior
Expect west update to complete without failure. A single quote is a valid character in a git branch name in most systems.
Impact
Cannot complete the west update, leaving my dependency tree incomplete.
Logs and console output
PS C:\west-quote-test> west init -m https://github.com/GarrettCarter-eaton/west-quote-test-manifest
=== Initializing in C:\west-quote-test
--- Cloning manifest repository from https://github.com/GarrettCarter-eaton/west-quote-test-manifest
Cloning into 'C:\west-quote-test\.west\manifest-tmp'...
remote: Enumerating objects: 15, done.
remote: Counting objects: 100% (15/15), done.
remote: Compressing objects: 100% (10/10), done.
remote: Total 15 (delta 4), reused 0 (delta 0), pack-reused 0
Receiving objects: 100% (15/15), done.
Resolving deltas: 100% (4/4), done.
--- setting manifest.path to west-quote-test-manifest
=== Initialized. Now run "west update" inside C:\west-quote-test.
PS C:\west-quote-test> west update
=== updating west-quote-test-dep (west-quote-test-dep):
--- west-quote-test-dep: initializing
Initialized empty Git repository in C:/west-quote-test/west-quote-test-dep/.git/
--- west-quote-test-dep: fetching, need revision bb4143e
remote: Enumerating objects: 9, done.
remote: Counting objects: 100% (9/9), done.
remote: Compressing objects: 100% (5/5), done.
remote: Total 9 (delta 0), reused 0 (delta 0), pack-reused 0
Unpacking objects: 100% (9/9), 1.93 KiB | 86.00 KiB/s, done.
From https://github.com/GarrettCarter-eaton/west-quote-test-dep
* [new branch] branch-with-'-quote -> refs/west/branch-with-'-quote
* [new branch] main -> refs/west/main
* [new branch] other-rev -> refs/west/other-rev
Traceback (most recent call last):
File "C:\Users\eXXXXXXX\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "C:\Users\eXXXXXXX\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 86, in _run_code
exec(code, run_globals)
File "C:\Users\eXXXXXXX\AppData\Local\Programs\Python\Python310\Scripts\west.exe\__main__.py", line 7, in <module>
File "C:\Users\eXXXXXXX\AppData\Local\Programs\Python\Python310\lib\site-packages\west\app\main.py", line 866, in main
app.run(argv or sys.argv[1:])
File "C:\Users\eXXXXXXX\AppData\Local\Programs\Python\Python310\lib\site-packages\west\app\main.py", line 111, in run
self.run_command(argv)
File "C:\Users\eXXXXXXX\AppData\Local\Programs\Python\Python310\lib\site-packages\west\app\main.py", line 383, in run_command
self.run_builtin(args, unknown)
File "C:\Users\eXXXXXXX\AppData\Local\Programs\Python\Python310\lib\site-packages\west\app\main.py", line 424, in run_builtin
File "C:\Users\eXXXXXXX\AppData\Local\Programs\Python\Python310\lib\site-packages\west\commands.py", line 194, in run
self.do_run(args, unknown)
File "C:\Users\eXXXXXXX\AppData\Local\Programs\Python\Python310\lib\site-packages\west\app\project.py", line 823, in do_run
self.update_all()
File "C:\Users\eXXXXXXX\AppData\Local\Programs\Python\Python310\lib\site-packages\west\app\project.py", line 888, in update_all
self.update(project)
File "C:\Users\eXXXXXXX\AppData\Local\Programs\Python\Python310\lib\site-packages\west\app\project.py", line 1062, in update
self.clean_refs_west(project, stats, take_stats)
File "C:\Users\eXXXXXXX\AppData\Local\Programs\Python\Python310\lib\site-packages\west\app\project.py", line 1350, in clean_refs_west
_clean_west_refspace(project)
File "C:\Users\eXXXXXXX\AppData\Local\Programs\Python\Python310\lib\site-packages\west\app\project.py", line 1524, in _clean_west_refspace
project.git(delete_ref_cmd)
File "C:\Users\eXXXXXXX\AppData\Local\Programs\Python\Python310\lib\site-packages\west\manifest.py", line 807, in git
cmd_list = shlex.split(cmd)
File "C:\Users\eXXXXXXX\AppData\Local\Programs\Python\Python310\lib\shlex.py", line 315, in split
return list(lex)
File "C:\Users\eXXXXXXX\AppData\Local\Programs\Python\Python310\lib\shlex.py", line 300, in __next__
token = self.get_token()
File "C:\Users\eXXXXXXX\AppData\Local\Programs\Python\Python310\lib\shlex.py", line 109, in get_token
raw = self.read_token()
File "C:\Users\eXXXXXXX\AppData\Local\Programs\Python\Python310\lib\shlex.py", line 191, in read_token
raise ValueError("No closing quotation")
ValueError: No closing quotation
PS C:\west-quote-test> west -V
West version: v1.0.0
PS C:\west-quote-test>
Environment
- OS: Windows
- Toolchain: Zephyr SDK
- West 1.0.0
Thanks @GarrettCarter-eaton for the test case, it really saved my time. Fix submitted in #683, would you be in a position to test it? Check the west/README.txt for installing directly from source. If you go and do that, do pip uninstall west first.
Just FYI, tested with c936a4a and issue is fixed.
I can confirm that this bug is resolved in the v1.2.0 release of West.