Error listing Source Files Starting with Ñ

Question

Error listing Source Files Starting with Ñ

Opened this issue 2 months ago · 30 comments

Type: Bug

In the new version of Code for i, 2.13.0 when navigating in the Object Browser, it is not possible to view the members of source files that start with the letter Ñ. This issue did not exist in the previous version.

Steps to Replicate the Error:

Update to the latest version of Code for i. 2.13.0
Create a source file that starts with the letter Ñ (from IBM i).
Create a member for the source file (from IBM i).
Create a filter for this source in the VS Code
Attempt to view the members of the source file that starts with Ñ from the Object Browser.

Best regards,

Extension version: 2.13.0
VS Code version: Code 1.92.2 (fee1edb8d6d72a0ddff41e5f71a671c23ed924b9, 2024-08-14T17:29:30.058Z)
OS version: Windows_NT x64 10.0.22631
Modes:

System Info

Item	Value
CPUs	AMD Ryzen 5 3500U with Radeon Vega Mobile Gfx (8 x 2096)
GPU Status	2d_canvas: enabled canvas_oop_rasterization: enabled_on direct_rendering_display_compositor: disabled_off_ok gpu_compositing: enabled multiple_raster_threads: enabled_on opengl: enabled_on rasterization: enabled raw_draw: disabled_off_ok skia_graphite: disabled_off video_decode: enabled video_encode: enabled vulkan: disabled_off webgl: enabled webgl2: enabled webgpu: enabled webnn: disabled_off
Load (avg)	undefined
Memory (System)	9.94GB (3.36GB free)
Process Argv	--crash-reporter-id 587481dc-7481-4237-b6c2-fc757d6b10f8
Screen Reader	no
VM	0%

Answer 1 · 2024-08-29T16:05:24.000Z

This issue did not exist in the previous version.

What version are you referring to here?

Answer 2 · 2024-08-29T16:11:47.000Z

This issue did not exist in the previous version.

What version are you referring to here?

2.12.1.

Answer 3 · 2024-08-29T16:14:56.000Z

@rgarciale What is your QCCSID and typical job CCSID or user profile CCSID?

Answer 4 · 2024-08-29T16:22:40.000Z

cc @sebjulliand possibly related to #2215. Going to try and write a test case when I get more info.

Answer 5 · 2024-08-29T16:24:38.000Z

@rgarciale What is your QCCSID and typical job CCSID or user profile CCSID?

User CCSID is 65535 (*sysval)
SourceFile CCSID is 284

Answer 6 · 2024-08-29T16:25:25.000Z

@rgarciale What is the QCCSID?

Answer 7 · 2024-08-29T16:25:50.000Z

@rgarciale What is the QCCSID?

65535

Answer 8 · 2024-08-29T16:33:59.000Z

Are you able to go back to 2.12.1, and open the member successfully? Please share the contents of your Code for IBM i output channel after the member opens. Then, please do the same with 2.13.0. Thanks!

Answer 9 · 2024-08-29T16:47:30.000Z

Are you able to go back to 2.12.1, and open the member successfully? Please share the contents of your Code for IBM i output channel after the member opens. Then, please do the same with 2.13.0. Thanks!

v2.12.1
v2.12.1.txt

v2.13.0
v2.13.0.txt

Answer 10 · 2024-08-29T16:59:15.000Z

In Code for IBM i 2.12.1, I cannot create a source file with Ñ.

/home/LIAMA: /QOpenSys/usr/bin/qsh
system  "CRTSRCPF FILE(LIAMA/ÑHELLO) RCDLEN(112)"
{
    "code": 1,
    "signal": null,
    "stdout": "",
    "stderr": "CPD0012:  Characters in qualifier beginning 'ÑHELLO) RC' not valid.\nCPF0001:  Error found on *N command."
}

Are you able to do this @rgarciale?

Answer 11 · 2024-08-29T17:06:21.000Z

In Code for IBM i 2.12.1, I cannot create a source file with Ñ.

/home/LIAMA: /QOpenSys/usr/bin/qsh
system  "CRTSRCPF FILE(LIAMA/ÑHELLO) RCDLEN(112)"
{
    "code": 1,
    "signal": null,
    "stdout": "",
    "stderr": "CPD0012:  Characters in qualifier beginning 'ÑHELLO) RC' not valid.\nCPF0001:  Error found on *N command."
}

Are you able to do this @rgarciale?

yes I do.

/home/BCOINFRGL: /QOpenSys/usr/bin/qsh
system "CRTSRCPF FILE(LIBDREAMT/ÑPRUEBA) RCDLEN(112)"
{
"code": 0,
"signal": null,
"stdout": "",
"stderr": "CPC7301: Archivo ÑPRUEBA creado en la biblioteca LIBDREAMT.\nCPC2206: Se ha modificado la propiedad del objeto QZSHSYSTEM en QTEMP del tipo *USRSPC."
}

Answer 12 · 2024-08-29T17:10:12.000Z

@rgarciale After connecting to the system, are you able to click on the Report an Issue item and then share the contents that appear in this issue? That might help me track down some more information I need.

Answer 13 · 2024-08-29T17:13:56.000Z

@rgarciale After connecting to the system, are you able to click on the Report an Issue item and then share the contents that appear in this issue? That might help me track down some more information I need.

I cannot perform that action because company policies prevent me from generating the report.

Sorry for the inconvenience. :-(

Answer 14 · 2024-08-29T18:44:50.000Z

@rgarciale Have you tried changing your userprofile to have CCSID 284 instead of 65535? Does that make a difference?

65535 means no conversion at all - and you need conversion of the character Ñ, which is a variant of the character # in CCSID 37 (US).

Btw, what is the reason for creating a member with a variant character in the name? These variant characters are causing a lot of problems for IBM i programmers - and it is recommended to avoid using these characters in object names, member name, variable names etc. They were typically used to indicate a special type of variable, member, object, but they are not that useful anymore in the modern world with long RPG variable names, long SQL identifiers etc.

Answer 15 · 2024-08-29T19:36:51.000Z

@rgarciale Have you tried changing your userprofile to have CCSID 284 instead of 65535? Does that make a difference?

65535 means no conversion at all - and you need conversion of the character Ñ, which is a variant of the character # in CCSID 37 (US).

Btw, what is the reason for creating a member with a variant character in the name? These variant characters are causing a lot of problems for IBM i programmers - and it is recommended to avoid using these characters in object names, member name, variable names etc. They were typically used to indicate a special type of variable, member, object, but they are not that useful anymore in the modern world with long RPG variable names, long SQL identifiers etc.

I will request a change of the CCSID for my user profile to run the test, as I don't have permission to modify the profile myself.

Regarding the creation of source files with the character Ñ, it happens because the PDE tool we use for source control creates them this way. :-/

Best regards, and thank you for your help! :-)

Answer 16 · 2024-08-30T17:26:01.000Z

Hi @chrjorgensen,

I tried this advice that you provide

@rgarciale Have you tried changing your userprofile to have CCSID 284 instead of 65535? Does that make a difference?

DSPUSRPRF USRPRF(MY USER)

The detail:

ID de juego de caracteres . . . . .: 284

But this not make any difference for me, the problem it's still happening.

Any other idea? :(

Answer 17 · 2024-08-30T18:56:32.000Z

@e1mais Did you Connect and Reload Server Settings? The changed user setting is only picked up when doing this...

Answer 18 · 2024-08-30T19:20:28.000Z

@chrjorgensen Yes I did.

My steps:

F1 > dev:Reload window > code4i: Connect and Reload Server settings.

I also notice other thing on the tooltip when I hover the Source File name, those are named without Ñ character expand with all the information (Text, Members, Length, and the CCSID with the corresponding 284) but for those Ñ-named Source files the information is: Members: 0, Length works well, and for CCSID is ?

I attached you the tooltip info for better understanding

Answer 19 · 2024-09-02T14:43:39.000Z

Right, same problem hete.

Answer 20 · 2024-09-02T21:32:53.000Z

Hello @worksofliam, I was able to replicate the error on PUB400. These are the steps I followed to do so:

IBM i:
CHGPRF LANGID(ESP) CNTRYID(ES) CCSID(284)

CRTPF FILE(BCOINFRGL1/ÑTEST) RCDLEN(112) FILETYPE(*SRC) MAXMBRS(*NOMAX)

ADDPFM FILE(BCOINFRGL1/ÑTEST) MBR(PGM1) SRCTYPE(RPGLE)

WRKMBRPDM FILE(BCOINFRGL1/ÑTEST)

VS Code:

log.txt

Answer 21 · 2024-09-03T16:00:55.000Z

Hello, @worksofliam

It seems that the issue is with the cast char for bit data in the IBMContent.ts/getMemberList.
I removed the for bit data, and now the members are showing up.

However, the problem now is that the authority of the objects isn't changing, and they cannot be edited. I will continue investigating.

Answer 22 · 2024-09-10T21:12:02.000Z

@rgarciale I'd like to check something. Could you connect to your LPAR and click on Report an issue?

It will open a new issue template in your browser. Copy and paste the template here, I'd like to see some details about your LPAR. Thanks!

Answer 23 · 2024-09-10T21:51:31.000Z

@sebjulliand. Sure,

👉🏻 Issue text goes here.

⚠️ REMOVE THIS LINE AND ANY SENSITIVE INFORMATION BELOW! ⚠️

Context	Version
Code for IBM i version	2.13.0
Visual Studio Code version	1.93.0
Operating System	win32_x64

Active extensions

COBOL (cobol): 10.8.31
Code for IBM i Walkthroughs (vscode-ibmi-walkthroughs): 0.5.0
Configuration Editing (configuration-editing): 1.0.0
Db2 for IBM i (vscode-db2i): 1.3.1
ESLint (vscode-eslint): 3.0.10
Emmet (emmet): 1.0.0
Error Lens (errorlens): 3.20.0
Extension Authoring (extension-editing): 1.0.0
Git (git): 1.0.0
Git Base (git-base): 1.0.0
GitHub (github): 0.0.1
GitHub Authentication (github-authentication): 0.0.2
IBM i Debug (ibmidebug): 2.0.1
JSON Language Features (json-language-features): 1.0.0
Merge Conflict (merge-conflict): 1.0.0
NPM support for VS Code (npm): 1.0.1
Node Debug Auto-attach (debug-auto-launch): 1.0.0
TODO Highlight (vscode-todo-highlight): 1.0.5
TypeScript and JavaScript Language Features (typescript-language-features): 1.0.0

Remote system

Setting	Value
IBM i OS	V7R5M0
Tech Refresh	n/a
CCSID Origin	273
Runtime CCSID	273
Default CCSID	273
SQL	Enabled
Source dates	Disabled

Enabled features

/QOpenSys/pkgs/bin	/usr/bin	/QSYS.LIB	/QIBM/ProdData/IBMiDebugService/bin
bash	attr	QZDFMDB2.PGM	startDebugService.sh
chsh	iconv
find	setccsid
git
grep
ls
md5sum
sort
stat
tar
tn5250

Shell env

BUILDLIB=BCOINFRGL1
CURLIB=BCOINFRGL1
HOME=/home/BCOINFRGL
HOST=PUB400.COM
LIBLS=GAMES400 QTEMP QGPL
LOGIN=bcoinfrgl
LOGNAME=bcoinfrgl
MAIL=/var/spool/mail/bcoinfrgl
OLDPWD=/home/BCOINFRGL
PASE_USRGRP_LIMITED=N
PATH=/QOpenSys/pkgs/bin:/QOpenSys/usr/bin:/usr/ccs/bin:/QOpenSys/usr/bin/X11:/usr/sbin:.:/usr/bin:/QOpenSys/usr/bin:/usr/bin
PWD=/home/BCOINFRGL
SHELL=/QOpenSys/pkgs/bin/bash
SHLVL=1
SSH_CLIENT=186.26.118.186 35999 2222
SSH_CONNECTION=186.26.118.186 35999 185.113.5.134 2222
TZ=<UTC>0
USER=bcoinfrgl
USERNAME=BCOINFRGL
WORKDIR=/home/BCOINFRGL
_=/QOpenSys/pkgs/bin/env

Variants

{
  "american": "#@$",
  "local": "#§$"
}

Errors

[
  {
    "command": "/QOpenSys/usr/bin/qsh",
    "code": 1,
    "stderr": "CPD0012:  Characters in qualifier beginning 'ÑRPGLESRC)' not valid.\nCPF0001:  Error found on *N command.",
    "cwd": "/home/BCOINFRGL"
  },
  {
    "command": "cd \"/QSYS.LIB/BCOINFRGL1.LIB/ÑRPGLESRC.FILE\" && (ls | wc -l)",
    "code": 1,
    "stderr": "bash: line 1: cd: $'/QSYS.LIB/BCOINFRGL1.LIB/\\303\\221RPGLESRC.FILE': No such file or directory",
    "cwd": "/home/BCOINFRGL"
  },
  {
    "command": "/QOpenSys/usr/bin/qsh",
    "code": 1,
    "stderr": "CPD0012:  Characters in qualifier beginning 'ÑTEST) OBJ' not valid.\nCPF0001:  Error found on *N command.",
    "cwd": "/home/BCOINFRGL"
  }
]

Answer 24 · 2024-09-10T22:13:49.000Z

@sebjulliand

I have been doing some testing and have found the following:

When commands are executed directly from qsh, the objects are displayed with the # character.
For example:

However, when the command is executed from DB2, the objects are listed with the Ñ character. It seems this is causing some issues.

Answer 25 · 2024-09-11T07:17:00.000Z

There is an issue with the variant characters here. I assume that the issue template you sent was taken from PUB400, and not the LPAR your issue originated from.

As you can see, the LPAR CCSID is 273, and Ñ is not handled there because it's not a valid object name character with this CCSID.
Ñ is CCSID 284 variant of # and that's why it can be used when the job CCSID is 284.

I tried your use case above on PUB400, and it failed, as expected. But more importantly, I noticed the variant characters were still CCSID 273's, and not 284's, even after changing my profile's CCSID and reloading the settings.

All I can assume for now is:

variant characters are messing with some of our queries
if system variable QCCSID is set to something else than 65535, then variant characters are not loaded for the user's CCSID but for QCCSID's value

My guess here is that if:

the LPAR QCCSID is set to 284
OR if it is set to 65535 and your user profile's CCSID is 284

Then having Ñ in object names should be OK.
Otherwise, welcome to CCSID's hell 😁

Answer 26 · 2024-09-20T03:14:05.000Z

Hello @sebjulliand

I managed to get it working. I made the following change, and now I can see the sources and modify them. Obviously, it's not a real fix, but maybe this’ll help fix the issue.

And yeah, CCSID is a total pain ;-D

Answer 27 · 2024-09-26T23:26:51.000Z

Hello, @worksofliam @sebjulliand @chrjorgensen

I finally found the error.

The problem is not exclusive to the "Ñ" character. It happens when there is a character conversion due to the LPAR configuration between variantChars.american and variantChars.locale

For example, in PUB400, the "@" character has a conversion:

Variants

{
  "american": "#@$",
  "local": "#§$"
}

The issue arises when listing the objects using the getObjectList function, as they are listed in local format.

However, when retrieving members using getMemberList, the object filter is passed through the sysNameInAmerican function, which causes the members not to be listed correctly.

"I created a source that starts with @ in my library."

When I view it in VS Code, it appears in the LPAR format.

But when trying to retrieve the members, they are processed using the variantChars.american format

And it doesn't display anything.

To fix the issue, I modified the functions getObjectList and getMemberList so that when listing the members, it does not convert them in american format.

getObjectList

getMemberList

And now it works as it should.

Can this solution be implemented to fix the problem with character conversion?

Answer 28 · 2024-09-27T18:20:03.000Z

@rgarciale if the CCSID variant characters are not swapped with their CCSID 37 equivalent, it's going to crash for anyone having the variants rights (see #2239 (comment) above).

These variants are NOT those for CCSID 284; they are 273's:

284 variants are: Ñ@$.

It seems that since PUB400 system CCSID is set 273, changing your profile's CCSID to 284 will not let you work with the corresponding variants. The routine used to load the variants in Code for IBM i get those from the system CCSID if it's not 65635. The query used returns it like that and it doesn't seem we can do something about it.

Back on your original system (not PUB400), if the system CCSID is 65535 and your user profile's CCSID is 284, there is a good chance that browsing libraries/files/members with Ñ it should work.

Answer 29 · 2024-10-01T16:55:47.000Z

Hi @sebjulliand

I tried changing my user profile's CCSID to 284, as you suggested, but it still doesn't work.

I got the information from LPAR 65535.

Context	Version
Code for IBM i version	2.13.0
Visual Studio Code version	1.92.2
Operating System	win32_x64

Active extensions

COBOL (cobol): 10.8.31
Code for IBM i Walkthroughs (vscode-ibmi-walkthroughs): 0.5.0
Configuration Editing (configuration-editing): 1.0.0
Emmet (emmet): 1.0.0
Error Lens (errorlens): 3.20.0
Git (git): 1.0.0
Git Base (git-base): 1.0.0
GitHub (github): 0.0.1
IBM i Debug (ibmidebug): 2.0.1
JSON Language Features (json-language-features): 1.0.0
Merge Conflict (merge-conflict): 1.0.0
Node Debug Auto-attach (debug-auto-launch): 1.0.0
Print (vscode-print): 0.13.1
TODO Highlight (vscode-todo-highlight): 1.0.5
Text Marker (Highlighter) (text-marker): 1.11.0
TypeScript and JavaScript Language Features (typescript-language-features): 1.0.0

Remote system

Setting	Value
IBM i OS	V7R5M0
Tech Refresh	4
CCSID Origin	65535
Runtime CCSID	65535
Default CCSID	284
SQL	Enabled
Source dates	Enabled

Enabled features

/QOpenSys/pkgs/bin	/usr/bin	/QSYS.lib/ILEDITOR.lib	/QSYS.LIB	/QIBM/ProdData/IBMiDebugService/bin
bash	attr	GETNEWLIBL.PGM	QZDFMDB2.PGM	startDebugService.sh
find	iconv
git	setccsid
grep
ls
md5sum
sort
stat
tar
tn5250

Shell env

BUILDLIB=LIBRGL
CURLIB=LIBRGL
HOME=/home/XXX
HOST=XXX
LOGIN=xxx
LOGNAME=xxx
MAIL=/var/spool/mail/xxx
OLDPWD=/home/xxx
PASE_USRGRP_LIMITED=N
PATH=/QOpenSys/pkgs/bin:/QOpenSys/usr/bin:/usr/ccs/bin:/QOpenSys/usr/bin/X11:/usr/sbin:.:/usr/bin:/QOpenSys/usr/bin:/usr/bin
PWD=/home/xxx
SHELL=/QOpenSys/pkgs/bin/bash
SHLVL=1
SSH_CLIENT=00.00.00.00 62560 22
SSH_CONNECTION=00.00.00.00 62560 00.00.00.00 22
TZ=<UTC-06x00S>6
USER=xxx
USERNAME=xxx
WORKDIR=/home/xxx
_=/QOpenSys/pkgs/bin/env

Variants

{
  "american": "#@$",
  "local": "Ñ@$"
}

Errors

[
  {
    "command": "/QOpenSys/usr/bin/qsh",
    "code": 1,
    "stderr": "CPD0032:  No tiene autorización para el mandato CRTLIB en la biblioteca *LIBL.\nCPF0001:  Se ha encontrado un error en el mandato CRTLIB.\nCPC2206:  Se ha modificado la propiedad del objeto QZSHSYSTEM en QTEMP del tipo *USRSPC.",
    "cwd": "/home/BCOINFR"
  },
  {
    "command": "/QOpenSys/usr/bin/qsh",
    "code": 1,
    "stderr": "CPF9801:  No se ha encontrado el objeto QCPTOIMPF en la biblioteca QSYS.\nCPC2206:  Se ha modificado la propiedad del objeto QZSHSYSTEM en QTEMP del tipo *USRSPC.",
    "cwd": "/home/BCOINFR"
  },
  {
    "command": "/QOpenSys/usr/bin/qsh",
    "code": 1,
    "stderr": "CPF9801:  No se ha encontrado el objeto QCPFRMIMPF en la biblioteca QSYS.\nCPC2206:  Se ha modificado la propiedad del objeto QZSHSYSTEM en QTEMP del tipo *USRSPC.",
    "cwd": "/home/BCOINFR"
  }
]

Thanks for your time and suggestion, and sorry for being so persistent. It's just that some people in my workplace are using VS Code for IBM i, and I really love it, but we need it to work properly with the "Ñ". Otherwise, we'll have to stop using it. For now, we're stuck on version 2.12.1.

Answer 30 · 2024-10-15T17:58:17.000Z

Indeed, we have the same problem. Hope you can fix this to keep using code for i.