Patching of array of strings only updating first three entries
marshallward opened this issue · 9 comments
Following the bug reported by @lisha992610 in #80
If I use this namelist:
&test_nml
start_date = '2016-04-30_00:00:00','2016-04-30_00:00:00','2016-04-30_00:00:00','2016-04-30_00:00:00','2016-04-30_00:00:00'
/
and apply the following patch code:
import f90nml
p = {
'test_nml': {
'start_date': [
'2017-04-30_00:00:00',
'2017-04-30_00:00:00',
'2017-04-30_00:00:00',
'2017-04-30_00:00:00',
'2017-04-30_00:00:00'
]
}
}
f90nml.patch('x.nml', p, 'p.nml')
then only the first three entries are updated:
&test_nml
start_date = '2017-04-30_00:00:00','2017-04-30_00:00:00','2017-04-30_00:00:00','2016-04-30_00:00:00','2016-04-30_00:00:00'
/
I'm still working through it, but there's clearly a bug in the (p_idx - 1) <= n_vals_remain
test, since it is effectively being double-counted.
As we increase the index of our patch array, the number of remaining values is also going to decrease. In this case, we are terminating the update in the middle (p_idx=2
) rather than the end (p_idx=4
).
This was a relatively recent change to accommodate a more complicated patch update, so this must be a new bug. I will try to resolve it today.
Actually, looks like just replacing n_vals_remain
with len(patch_values)
may be all we need here... just going through testing now.
Confirming that 18f5d2d, which was added to support patches to arrays shorter than the patch, broke the much simpler case of an array update.
Neither appears to have been in the test suite, which I am currently amending.
There was also bug in the count_values
token, which was not considering self.token
in its count while also incorrectly mislabeling multiple spaces as a valid patch token. This was "cancelling out" the errors in many cases, but became severe when the p_idx
/ n_vals_remain
bug was identified and fixed.
Testing is now working, and the test suite is being expanded.
I've pushed a change (ad84533) which hopefully addresses the issue.
@lisha992610 can you please test this?
@marshallward Thank you for your contribution.
I am not sure if I could update the python package with "pip install f90nml --upgrade" .
I have tested again, the problem still remains.
My code is following:
Since the parsing of the 'start_date' is recognized as list in python. I tried to create a same list and create new file with modification.
patch_nml = {'share': {'start_date': start_date, 'end_date': end_date}}
f90nml.patch('namelist.wps', patch_nml, 'namelist_modified.wps')
Initial namelist has 5 element in start_date, even I tried to create a list with 10 elements, only 5 appear in the namelist. regarding the update problem, still only 3 elements were updated.
Thank you.
The pip version has not yet been updated. You'll need to pull the code via git and install it manually.
There's some basic instructions in the README if you need help with this.
Any update on this? Otherwise I'll just assume that it's working and will close this issue.
I'll treat this as resolved, please reopen if it's still happening though.
(And note that the PyPI version installed by pip has not been updated, you will have to to a git checkout of the latest version)