wader/fq

Fail to run test

Avimitin opened this issue · 25 comments

What version are you using (fq -v)?

$ fq -v
0.0.10

How was fq installed?

Can you reproduce the problem using the latest release or master branch?

Yes, same behavior as the 0.0.10 version.

What did you do?

go build -trimpath -buildmode=pie -mod=readonly -modcacherw \
     -ldflags "-linkmode external -extldflags \"-Wl,-O1,--sort-common,--asneeded,-z,relro,-z,now\"" \
     -o dist/fq \
     .
make test

Additional environment context

Tests are running in Arch Linux RISC-V qemu-user.

What result did you expect?

Test pass successfully

What did you see instead?

Test mess up the terminal, and exit with non-zero status but no error reported.

Full log
go test -timeout 20m    ./format ./internal/aheadreadseeker ./internal/ansi ./internal/asciiwriter ./internal/ctxreadseeker ./internal/hexdump ./internal/hexpairwriter ./internal/mathex ./internal/recoverfn ./internal/shquote ./pkg/bitio ./pkg/interp ./pkg/ranges
go: downloading github.com/pmezard/go-difflib v1.0.0
ok  	github.com/wader/fq/format	44.486s
ok  	github.com/wader/fq/internal/aheadreadseeker	0.068s
ok  	github.com/wader/fq/internal/ansi	0.085s
ok  	github.com/wader/fq/internal/asciiwriter	0.076s
ok  	github.com/wader/fq/internal/ctxreadseeker	0.097s
?   	github.com/wader/fq/internal/hexdump	[no test files]
ok  	github.com/wader/fq/internal/hexpairwriter	0.079s
ok  	github.com/wader/fq/internal/mathex	0.054s
ok  	github.com/wader/fq/internal/recoverfn	0.076s
ok  	github.com/wader/fq/internal/shquote	0.072s
ok  	github.com/wader/fq/pkg/bitio	0.212s
ok  	github.com/wader/fq/pkg/interp	20.591s
ok  	github.com/wader/fq/pkg/ranges	0.070s
CGO_ENABLED=0 go build -o fq -ldflags "-s -w" -trimpath .
testjq pkg/interp/args_test.jq
testjq pkg/interp/funcs_test.jq
testjq pkg/interp/query_test.jq
spawn sh -c NO_DECODE_PROGRESS=1 fq -o color=false -i . pkg/interp/testdata/test.mp3 2>&1

 ��[6nmp3> 
.
 ��[6nmp3> .
     |00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 12 13|0123456789abcdef0123|.{}: pkg/interp/testdata/test.mp3 (mp3)
0x000|49 44 33 04 00 00 00 00 00 23 54 53 53 45 00 00 00 0f 00 00|ID3......#TSSE......|  headers[0:1]:
*    |until 0x2c.7 (45)                                          |                    |
0x028|               ff fb 40 c0 00 00 00 00 00 00 00 00 00 00 00|     ..@............|  frames[0:3]:
0x03c|00 00 00 00 00 00 49 6e 66 6f 00 00 00 0f 00 00 00 02 00 00|......Info..........|
*    |until 0x283.7 (end) (599)                                  |                    |
     |                                                           |                    |  footers[0:0]:
 ��[6nmp3> 
 ��[6nmp3> range(100000) | d
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
^C ��[6nmp3> 
 ��[6nmp3> range(100000)
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
^C ��[6nmp3> 
[range(100000)] | d
 ��[6nmp3> [range(100000)] | d
[
  0,
  1,
  2,
  3,
  4,
  5,
  6,
  7,
  8,
  9,
  10,
  11,
  12,
  13,
  14,
  15,
  16,
  17,
  18,
  19,
  20,
  21,
  22,
  23,
  24,
  25,
  26,
  27,
  28,
  29,
  30,
  31,
  32,
  33,
  34,
  35,
  36,
  37,
  38,
  39,
  40,
  41,
  42,
  43,
  44,
  45,
  46,
  47,
  48,
  49,
  50,
  51,
  52,
  53,
  54,
  55,
  56,
  57,
  58,
  59,
  60,
  61,
  62,
  63,
  64,
  65,
  66,
  67,
  68,
  69,
  70,
  71,
  72,
  73,
  74,
  75,
  76,
  77,
  78,
  79,
  80,
  81,
  82,
  83,
  84,
  85,
  86,
  87,
  88,
  89,
  90,
  91,
  92,
  93,
  94,
  95,
  96,
  97,
  98,
  99,
  100,
  101,
  102,
  103,
  104,
  105,
  106,
  107,
  108,
  109,
  110,
  111,
  112,
  113,
  114,
  115,
  116,
  117,
  118,
  119,
  120,
  121,
  122,
  123,
  124,
  125,
  126,
  127,
  128,
  129,
  130,
  131,
  132,
  133,
  134,
  135,
  136,
  137,
  138,
  139,
  140,
  141,
  142,
  143,
  144,
  145,
  146,
  147,
  148,
  149,
  150,
  151,
  152,
  153,
  154,
  155,
  156,
  157,
  158,
  159,
  160,
  161,
  162,
  163,
  164,
  165,
  166,
  167,
  168,
  169,
  170,
  171,
  172,
  173,
  174,
  175,
  176,
  177,
  178,
  179,
  180,
  181,
  182,
  183,
  184,
  185,
  186,
  187,
  188,
  189,
  190,
  191,
  192,
  193,
  194,
  195,
  196,
  197,
  198,
  199,
  200,
  201,
  202,
  203,
  204,
  205,
  206,
  207,
  208,
  209,
  210,
  211,
  212,
  213,
  214,
  215,
  216,
  217,
  218,
  219,
  220,
  221,
  222,
  223,
  224,
  225,
  226,
  227,
  228,
  229,
  230,
  231,
  232,
  233,
  234,
  235,
  236,
  237,
  238,
  239,
  240,
  241,
  242,
  243,
  244,
  245,
  246,
  247,
  248,
  249,
  250,
  251,
  252,
  253,
  254,
  255,
  256,
  257,
  258,
  259,
  260,
  261,
  262,
  263,
  264,
  265,
  266,
  267,
  268,
  269,
  270,
  271,
  272,
  273,
  274,
  275,
  276,
  277,
  278,
  279,
  280,
  281,
  282,
  283,
  284,
  285,
  286,
  287,
  288,
  289,
  290,
  291,
  292,
  293,
  294,
  295,
  296,
  297,
  298,
  299,
  300,
  301,
  302,
  303,
  304,
  305,
  306,
  307,
  308,
  309,
  310,
  311,
  312,
  313,
  314,
  315,
  316,
  317,
  318,
  319,
  320,
  321,
  322,
  323,
  324,
  325,
  326,
  327,
  328,
  329,
  330,
  331,
  332,
  333,
  334,
  335,
  336,
  337,
  338,
  339,
  340,
  341,
  342,
  343,
  344,
  345,
  346,
  347,
  348,
  349,
  350,
  351,
  352,
  353,
  354,
  355,
  356,
  357,
  358,
  359,
  360,
  361,
  362,
  363,
  364,
  365,
  366,
  367,
  368,
  369,
  370,
  371,
  372,
  373,
  374,
  375,
  376,
  377,
  378,
  379,
  380,
  381,
  382,
  383,
  384,
  385,
  386,
  387,
  388,
  389,
  390,
  391,
  392,
  393,
  394,
  395,
  396,
  397,
  398,
  399,
  400,
  401,
  402,
  403,
  404,
  405,
  406,
  407,
  408,
  409,
  410,
  411,
  412,
  413,
  414,
  415,
  416,
  417,
  418,
  419,
  420,
  421,
  422,
  423,
  424,
  425,
  426,
  427,
  428,
  429,
  430,
  431,
  432,
  433,
  434,
  435,
  436,
  437,
  438,
  439,
  440,
  441,
  442,
  443,
  444,
  445,
  446,
  447,
  448,
  449,
  450,
  451,
  452,
  453,
  454,
  455,
  456,
  457,
  458,
  459,
  460,
  461,
  462,
  463,
  464,
  465,
  466,
  467,
  468,
  469,
  470,
  471,
  472,
  473,
  474,
  475,
  476,
  477,
  478,
  479,
  480,
  481,
  482,
  483,
  484,
  485,
  486,
  487,
  488,
  489,
  490,
  491,
  492,
  493,
  494,
  495,
  496,
  497,
  498,
  499,
  500,
  501,
  502,
  503,
  504,
  505,
  506,
  507,
  508,
  509,
  510,
  511,
  512,
  513,
  514,
  515,
  516,
  517,
  518,
  519,
  520,
  521,
  522,
  523,
  524,
  525,
  526,
  527,
  528,
  529,
  530,
  531,
  532,
  533,
  534,
  535,
  536,
  537,
  538,
  539,
  540,
  541,
  542,
  543,
  544,
  545,
  546,
  547,
  548,
  549,
  550,
  551,
  552,
  553,
  554,
  555,
  556,
  557,
  558,
  559,
  560,
  561,
  562,
  563,
  564,
  565,
  566,
  567,
  568,
  569,
  570,
  571,
  572,
  573,
  574,
  575,
  576,
  577,
  578,
  579,
  580,
  581,
  582,
  583,
  584,
  585,
  586,
  587,
  588,
  589,
  590,
  591,
  592,
  593,
  594,
  595,
  596,
  597,
  598,
  599,
  600,
  601,
  602,
  603,
  604,
  605,
  606,
  607,
  608,
  609,
  610,
  611,
  612,
  613,
  614,
  615,
  616,
  617,
  618,
  619,
  620,
  621,
  622,
  623,
  624,
  625,
  626,
  627,
  628,
  629,
  630,
  631,
  632,
  633,
  634,
  635,
  636,
  637,
  638,
  639,
  640,
  641,
  642,
  643,
  644,
  645,
  646,
  647,
  648,
  649,
  650,
  651,
  652,
  653,
  654,
  655,
  656,
  657,
  658,
  659,
  660,
  661,
  662,
  663,
  664,
  665,
  666,
  667,
  668,
  669,
  670,
  671,
  672,
  673,
  674,
  675,
  676,
  677,
  678,
  679,
  680,
  681,
  682,
  683,
  684,
  685,
  686,
  687,
  688,
  689,
  690,
  691,
  692,
  693,
  694,
  695,
  696,
  697,
  698,
  699,
  700,
  701,
  702,
  703,
  704,
  705,
  706,
  707,
  708,
  709,
  710,
  711,
  712,
  713,
  714,
  715,
  716,
  717,
  718,
  719,
  720,
  721,
  722,
  723,
  724,
  725,
  726,
  727,
  728,
  729,
  730,
  731,
  732,
  733,
  734,
  735,
  736,
  737,
  738,
  739,
  740,
  741,
  742,
  743,
  744,
  745,
  746,
  747,
  748,
  749,
  750,
  751,
  752,
  753,
  754,
  755,
  756,
  757,
  758,
  759,
  760,
  761,
  762,
  763,
  764,
  765,
  766,
  767,
  768,
  769,
  770,
  771,
  772,
  773,
  774,
  775,
  776,
  777,
  778,
  779,
  780,
  781,
  782,
  783,
  784,
  785,
  786,
  787,
  788,
^C ��[6nmp3> 
[range(100000)]
 ��[6nmp3> [range(100000)]
[
  0,
  1,
  2,
  3,
  4,
  5,
  6,
  7,
  8,
  9,
  10,
  11,
  12,
  13,
  14,
  15,
  16,
  17,
  18,
  19,
  20,
  21,
  22,
  23,
  24,
  25,
  26,
  27,
  28,
  29,
  30,
  31,
  32,
  33,
  34,
  35,
  36,
  37,
  38,
  39,
  40,
  41,
  42,
  43,
  44,
  45,
  46,
  47,
  48,
  49,
  50,
  51,
  52,
  53,
  54,
  55,
  56,
  57,
  58,
  59,
  60,
  61,
  62,
  63,
  64,
  65,
  66,
  67,
  68,
  69,
  70,
  71,
  72,
  73,
  74,
  75,
  76,
  77,
  78,
  79,
  80,
  81,
  82,
  83,
  84,
  85,
  86,
  87,
  88,
  89,
  90,
  91,
  92,
  93,
  94,
  95,
  96,
  97,
  98,
  99,
  100,
  101,
  102,
  103,
  104,
  105,
  106,
  107,
  108,
  109,
  110,
  111,
  112,
  113,
  114,
  115,
  116,
  117,
  118,
  119,
  120,
  121,
  122,
  123,
  124,
  125,
  126,
  127,
  128,
  129,
  130,
  131,
  132,
  133,
  134,
  135,
  136,
  137,
  138,
  139,
  140,
  141,
  142,
  143,
  144,
  145,
  146,
  147,
  148,
  149,
  150,
  151,
  152,
  153,
  154,
  155,
  156,
  157,
  158,
  159,
  160,
  161,
  162,
  163,
  164,
  165,
  166,
  167,
  168,
  169,
  170,
  171,
  172,
  173,
  174,
  175,
  176,
  177,
  178,
  179,
  180,
  181,
  182,
  183,
  184,
  185,
  186,
  187,
  188,
  189,
  190,
  191,
  192,
  193,
  194,
  195,
  196,
  197,
  198,
  199,
  200,
  201,
  202,
  203,
  204,
  205,
  206,
  207,
  208,
  209,
  210,
  211,
  212,
  213,
  214,
  215,
  216,
  217,
  218,
  219,
  220,
  221,
  222,
  223,
  224,
  225,
  226,
  227,
  228,
  229,
  230,
  231,
  232,
  233,
  234,
  235,
  236,
  237,
  238,
  239,
  240,
  241,
  242,
  243,
  244,
  245,
  246,
  247,
  248,
  249,
  250,
  251,
  252,
  253,
  254,
  255,
  256,
  257,
  258,
  259,
  260,
  261,
  262,
  263,
  264,
  265,
  266,
  267,
  268,
  269,
  270,
  271,
  272,
  273,
  274,
  275,
  276,
  277,
  278,
  279,
  280,
  281,
  282,
  283,
  284,
  285,
  286,
  287,
  288,
  289,
  290,
  291,
  292,
  293,
  294,
  295,
  296,
  297,
  298,
  299,
  300,
  301,
  302,
  303,
  304,
  305,
  306,
  307,
  308,
  309,
  310,
  311,
  312,
  313,
  314,
  315,
  316,
  317,
  318,
  319,
  320,
  321,
  322,
  323,
  324,
  325,
  326,
  327,
  328,
  329,
  330,
  331,
  332,
  333,
  334,
  335,
  336,
  337,
  338,
  339,
  340,
  341,
  342,
  343,
  344,
  345,
  346,
  347,
  348,
  349,
  350,
  351,
  352,
  353,
  354,
  355,
  356,
  357,
  358,
  359,
  360,
  361,
  362,
  363,
  364,
  365,
  366,
  367,
  368,
  369,
  370,
  371,
  372,
  373,
  374,
  375,
  376,
  377,
  378,
  379,
  380,
  381,
  382,
  383,
  384,
  385,
  386,
  387,
  388,
  389,
  390,
  391,
  392,
  393,
  394,
  395,
  396,
  397,
  398,
  399,
  400,
  401,
  402,
  403,
  404,
  405,
  406,
  407,
  408,
  409,
  410,
  411,
  412,
  413,
  414,
  415,
  416,
  417,
  418,
  419,
  420,
  421,
  422,
  423,
  424,
  425,
  426,
  427,
  428,
  429,
  430,
  431,
  432,
  433,
  434,
  435,
  436,
  437,
  438,
  439,
  440,
  441,
  442,
  443,
  444,
  445,
  446,
  447,
  448,
  449,
  450,
  451,
  452,
  453,
  454,
  455,
  456,
  457,
  458,
  459,
  460,
  461,
  462,
  463,
  464,
  465,
  466,
  467,
  468,
  469,
  470,
  471,
  472,
  473,
  474,
  475,
  476,
  477,
  478,
  479,
  480,
  481,
  482,
  483,
  484,
  485,
  486,
  487,
  488,
  489,
  490,
  491,
  492,
  493,
  494,
  495,
  496,
  497,
  498,
  499,
  500,
  501,
  502,
  503,
  504,
  505,
  506,
  507,
  508,
  509,
  510,
  511,
  512,
  513,
  514,
  515,
  516,
  517,
  518,
  519,
  520,
  521,
  522,
  523,
  524,
  525,
  526,
  527,
  528,
  529,
  530,
  531,
  532,
  533,
  534,
  535,
  536,
  537,
  538,
  539,
  540,
  541,
  542,
  543,
  544,
  545,
  546,
  547,
  548,
  549,
  550,
  551,
  552,
  553,
  554,
  555,
  556,
  557,
  558,
  559,
  560,
  561,
  562,
  563,
  564,
  565,
  566,
  567,
  568,
  569,
  570,
  571,
  572,
  573,
  574,
  575,
  576,
  577,
  578,
  579,
  580,
  581,
  582,
  583,
  584,
  585,
  586,
  587,
  588,
  589,
  590,
  591,
  592,
  593,
  594,
  595,
  596,
  597,
  598,
  599,
  600,
  601,
  602,
  603,
  604,
  605,
  606,
  607,
  608,
  609,
  610,
  611,
  612,
  613,
  614,
  615,
  616,
  617,
  618,
  619,
  620,
  621,
  622,
  623,
  624,
  625,
  626,
  627,
  628,
  629,
  630,
  631,
  632,
  633,
  634,
  635,
  636,
  637,
  638,
  639,
  640,
  641,
  642,
  643,
  644,
  645,
  646,
  647,
  648,
  649,
  650,
  651,
  652,
  653,
  654,
  655,
  656,
  657,
  658,
  659,
  660,
  661,
  662,
  663,
  664,
  665,
  666,
  667,
  668,
  669,
  670,
  671,
  672,
  673,
  674,
  675,
  676,
  677,
  678,
  679,
  680,
  681,
  682,
  683,
  684,
  685,
  686,
  687,
  688,
  689,
  690,
  691,
  692,
  693,
  694,
  695,
  696,
  697,
  698,
  699,
  700,
  701,
  702,
  703,
  704,
  705,
  706,
  707,
  708,
  709,
  710,
  711,
  712,
  713,
  714,
  715,
  716,
  717,
  718,
  719,
  720,
  721,
  722,
  723,
  724,
  725,
  726,
  727,
  728,
  729,
  730,
  731,
  732,
  733,
  734,
  735,
  736,
  737,
  738,
  739,
  740,
  741,
  742,
  743,
  744,
  745,
  746,
  747,
  748,
  749,
  750,
  751,
  752,
  753,
  754,
  755,
  756,
  757,
  758,
  759,
  760,
  761,
  762,
  763,
  764,
  765,
  766,
  767,
  768,
  769,
  770,
  771,
  772,
  773,
  774,
  775,
  776,
  777,
  778,
  779,
  780,
  781,
  782,
  783,
  784,
  785,
  786,
  787,
  788,
  789,
  790,
  791,
  792,
  793,
  794,
  795,
  796,
  797,
  798,
  799,
  800,
  801,
  802,
  803,
  804,
  805,
  806,
  807,
  808,
  809,
  810,
  811,
  812,
  813,
  814,
  815,
  816,
  817,
  818,
  819,
  820,
  821,
  822,
  823,
  824,
  825,
  826,
  827,
  828,
  829,
  830,
  831,
  832,
  833,
  834,
  835,
  836,
  837,
  838,
  839,
  840,
  841,
  842,
  843,
  844,
  845,
  846,
  847,
  848,
  849,
  850,
  851,
  852,
  853,
  854,
  855,
  856,
  857,
  858,
  859,
  860,
  861,
  862,
  863,
  864,
  865,
  866,
  867,
  868,
  869,
  870,
  871,
  872,
  873,
  874,
  875,
  876,
  877,
  878,
  879,
  880,
  881,
  882,
  883,
  884,
  885,
  886,
  887,
  888,
  889,
  890,
  891,
  892,
  893,
  894,
  895,
  896,
  897,
  898,
  899^C ��[6nmp3> 
 ��[6nmp3> �[1G�[Jmake: *** [Makefile:32: testcli] Error 1
wader commented

Hi, thanks for the issue. Seems like the expect based cli tests are unhappy, they are a bit sensitive when running on slow hardware as they use timeouts, but based on the log i don't really see what went wrong.

Could you try run PATH="$PATH:$PWD" expect pkg/cli/test_repl.exp && echo SUCCESS || echo FAIL and see what happens? the PATH stuff is to make sure there is a fq binary in PATH.

Are you able to build a binary and does the REPL seems to work fine? completion, ctlr-c, ctrl-d etc?

Is this happening in some packaging or CI system?

What version of expect is installed?

If you want to run only the go and jq tests you can do make testgo testjq, the testcli target is the one that uses expect.

Could you try run PATH="$PATH:$PWD" expect pkg/cli/test_repl.exp && echo SUCCESS || echo FAIL and see what happens? the PATH stuff is to make sure there is a fq binary in PATH.

It print out the log like the above, and exit 1 and print out "FAIL".

Is this happening in some packaging or CI system?

Yes, this happen when I try to create a package for Arch Linux.

What version of expect is installed?

5.45.4

Are you able to build a binary and does the REPL seems to work fine? completion, ctlr-c, ctrl-d etc?

After running the ./fq -ni command, no prompt was appeared. But input help <enter> can get help output, and ctrl-d is also working.

And manually run the test command is also working.

$ NO_DECODE_PROGRESS=1 ./fq -o color=false -i . pkg/interp/testdata/test.mp3
     |00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f|0123456789abcdef|.{}: pkg/interp/testdata/test.mp3 (mp3)
0x000|49 44 33 04 00 00 00 00 00 23 54 53 53 45 00 00|ID3......#TSSE..|  headers[0:1]:
*    |until 0x2c.7 (45)                              |                |
0x020|                                       ff fb 40|             ..@|  frames[0:3]:
0x030|c0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00|................|
*    |until 0x283.7 (end) (599)                      |                |
     |                                               |                |  footers[0:0]:
wader commented

Could you try run PATH="$PATH:$PWD" expect pkg/cli/test_repl.exp && echo SUCCESS || echo FAIL and see what happens? the PATH stuff is to make sure there is a fq binary in PATH.

It print out the log like the above, and exit 1 and print out "FAIL".

Is this happening in some packaging or CI system?

Yes, this happen when I try to create a package for Arch Linux.

What version of expect is installed?

5.45.4

Strange, and there are other arch architectures where packaging work fine?

What is the host your running this on? thinking how i can get a dev env to reproduce

wader commented

Are you able to build a binary and does the REPL seems to work fine? completion, ctlr-c, ctrl-d etc?

After running the ./fq -ni command, no prompt was appeared. But input help <enter> can get help output, and ctrl-d is also working.

So no null> prompt but other things seems to work? you can move the cursor around and edit text without issues? does adding -M change anything?

wader commented

@piggynl you didn't run into this when running on risc-v? related to #420

wader commented

Could it be qemu-user that does things with the tty etc?

Could you try run PATH="$PATH:$PWD" expect pkg/cli/test_repl.exp && echo SUCCESS || echo FAIL and see what happens? the PATH stuff is to make sure there is a fq binary in PATH.

It print out the log like the above, and exit 1 and print out "FAIL".

Is this happening in some packaging or CI system?

Yes, this happen when I try to create a package for Arch Linux.

What version of expect is installed?

5.45.4

Strange, and there are other arch architectures where packaging work fine?

What is the host your running this on? thinking how i can get a dev env to reproduce

You can try this guidance to create a qemu user environment. https://github.com/felixonmars/archriscv-packages/wiki/Setup-Arch-Linux-RISC-V-Development-Environment

Could it be qemu-user that does things with the tty etc?

I've tried to build the package on SiFive Unmatched board, which is a native RISC-V platform, but it also fail on the same test.

wader commented

Ok! thanks for testing. Hope i will get time to look into this soon, is a bit extra threshold when you have to get a dev env first :)

wader commented

Sorry for no progress here. Could you try to do just:

$ fq -i | hexdump -C

and press ctrl-d?

Would look something like:

$ fq -i | hexdump -C
00000000  20 08 1b 5b 36 6e 6e 75  6c 6c 3e 20 5e 44 0a 1b  | ..[6nnull> ^D..|
00000010  5b 31 47 1b 5b 4a                                 |[1G.[J|
00000016

do you see a "null>" prompt if you do:

echo -e '\x20\x08\x1b\x5b\x36\x6e\x6e\x75\x6c\x6c\x3e'

A bit fishy that this is RISC-V specific for some reason

I try to test this from fq 0.1.0, however the test is just pass.

wader commented

You mean the pkg/cli/test_repl.exp test works now? see a prompt when testing fq -i manually?

wader commented

Hey @thesamesam first thanks for packaging for gentoo! I noticed https://gitweb.gentoo.org/repo/gentoo.git/plain/dev-util/fq/fq-0.1.0.ebuild has issues with this test also. How did it fail? also on RISC-V?

Hi @wader! No problem & thanks for writing fq!

I was considering posting an issue last night about the test failure but I saw the PR for 0.2.0 pending so I thought I'd wait and retest when that was out rather than bother you if it turned out to be imminently fixed.

I still seem to hit it with 0.2.0, but I can file a new bug if you want because it's on amd64:

[...]
GOROOT_FINAL='$GOROOT' /usr/lib/go/pkg/tool/linux_amd64/link -o $WORK/b001/exe/a.out -importcfg $WORK/b001/importcfg.link -buildmode=exe -buildid=XH5Uc2U2kcMf6jZYUZ1z/r6-JQXKMJmoUqSd9RV2p/1hLD9a0r0viaD5hE1s7i/XH5Uc2U2kcMf6jZYUZ1z -w -extld=x86_64-pc-linux-gnu-gcc $WORK/b001/_pkg_.a
/usr/lib/go/pkg/tool/linux_amd64/buildid -w $WORK/b001/exe/a.out # internal
cp $WORK/b001/exe/a.out fq
rm -r $WORK/b001/
pkg/interp/query.jq.test
pkg/interp/args.jq.test
pkg/interp/funcs.jq.test
1 of 1 tests passed
1 of 1 tests passed
9 of 9 tests passed
send: spawn id exp4 not open
    while executing
"send "range(100000)\n""
    (file "pkg/cli/test_repl.exp" line 36)
spawn sh -c NO_DECODE_PROGRESS=1 fq -o color=false -i . pkg/interp/testdata/test.mp3 2>&1
make: *** [Makefile:29: testcli] Error 1
make: *** Waiting for unfinished jobs....
rm -r $WORK/b262/
[...]

Here's the full log when it fails: build.log.gz. I'm not very familiar with expect so let me know if there's some way to get more output out of it, or if I should try anything.

But if I disable some of our sandboxes (FEATURES="-sandbox -usersandbox"), it works (works ok if I run it manually in a normal shell too):

[...]
cp $WORK/b001/exe/a.out fq
rm -r $WORK/b001/
pkg/interp/query.jq.test
pkg/interp/args.jq.test
pkg/interp/funcs.jq.test
1 of 1 tests passed
1 of 1 tests passed
9 of 9 tests passed
rm -r $WORK/b262/
pkg/cli/test_exp.sh
pkg/cli/test_exp.sh
pkg/cli/test_exp.sh
rm -r $WORK/b001/
[...]
wader commented

@thesamesam Thanks for testing. Ok that is a good clue...so maybe this is related to SELinux etc (i guess sandbox in gentoo is about that?). I suspect it might be that expect wants to create/write to pty:s etc

@Avimitin could SELinux etc be involved in your case also?

I wonder it we could modif test_repl.exp to not even run fq and see if it still fails, would also rule it has to do with fq itself.

Note that test*.exp tests are mostly to sanity that we haven't messed up the main/cli glue code, CI runs them on all PR:s so if that passes we're probably fine

@thesamesam I think we can continue in this issue for now, i suspect both issue are related.

You mean the pkg/cli/test_repl.exp test works now? see a prompt when testing fq -i manually?

I couldn't reproduce now, it is a bit flaky. I will try to verify if it is some performance issue.

wader commented

@Avimitin Ok thanks, we can of course tweak the timeouts if it helps. But if so t seem unrelated to issue @thesamesam has

@Avimitin Ok thanks, we can of course tweak the timeouts if it helps. But if so t seem unrelated to issue @thesamesam has

I don't think increasing timeouts could help. It seems like the test just can't pass when any other process which require a large amount of resources (CPU...) is running. So I suspect that this issue is more relatable to the performance of RISC-V QEMU image. But I don't know why the expect test will require that much resources to run.

wader commented

I see, the test_repl.exp works by doing range(100000) and then send a ctrl-c to abort it so it might use quite a bit of CPU for a short while. Do the other *.exp test work? they work by aborting and close a blocked read on stdin

Sorry, I lost track of this - I was going to file another bug, but it seems like it passes now for me on Gentoo with 0.3.0!

wader commented

No worries. Strange, don't think i've touch anything related to this. So all expect tests worked on RISC-V now? wonder if qemu has changed something? :)

wader commented

Close this or still a problem?

It's OK to close this.