capn-freako/PyBERT

Use ASCII format for configuration file

dghan0219 opened this issue · 5 comments

dghan0219 commented

PyBERT used to use an ASCII format for the configuration file. This allowed me to create files with a program and then start PyBERT and I'd pick the configuration file I wanted to use for that simulation. Now that PyBERT 3.4.2 uses a binary format, I can't do that. Consider loading and saving the old or new formats or switching back to the ASCII format.

capn-freako commented

Yes, I'm getting the following error when I attempt to use David Patterson's change to YAML encoded configuration files:

RuntimeError: The following error occured:
	found unconstructable recursive node
  in "/Users/dbanas/Documents/Projects/PyBERT/misc/Support/Testing/v3.4.5_relTest.yaml", line 26, column 11
The configuration was NOT loaded.

Here's the relevant portion of the (previously saved) *.yaml file:

mod_type: &id001 !!python/object/new:traits.trait_list_object.TraitListObject
  listitems:
  - 0
  state:
    item_validator: !!python/object/apply:builtins.getattr
    - *id001
    - _item_validator
    name: mod_type
    name_items: mod_type_items

It appears that &id001 is causing problems, but I don't know what to do about this;
@jdpatt , any thoughts?

capn-freako commented

This GitHub issue may pertain.

The very last comment is:

It should be noted that one should never attempt to directly use yaml.dump

And we are using yaml.dump;
anyone know what the preferred alternative is?

jdpatt commented

Looks like the traitsui are not directly serializable by the yaml library. So instead of a TraitListObject, it needs a python list or it throws its hands up. I think the best bet will be some clean up and let each class serialize itself into something yaml can handle and then a new configuration class handle loading/exporting.

  • #77 should get fixed while we fix this.
  • This commit is where it reverted back to pickle.

FWIW, I don't think the issue above is saying you can't use it. They are saying you should use the serialization class of their library that implements yaml and not yaml.dump directly. Basically, what I think we should upgrade to!

Later this week, I will pull master and update what I see here.

jdpatt commented

That seems to be the issue!

Traitsui serializes out in a format that the yaml.Loader doesn't understand how to reconstruct. Also seeing that it doesn't even store the current value just the length!!!

mod_type: &id001 !!python/object/new:traits.trait_list_object.TraitListObject
  listitems:
  - 2
  state:
    item_validator: !!python/object/apply:builtins.getattr
    - *id001
    - _item_validator
    name: mod_type
    name_items: mod_type_items

We could make our own loader like some examples I found but it seems all I had to do to get past your issue was change line 38 in pybert_cfg.py to self.mod_type = list(the_PyBERT.mod_type) so that it is an actual thing that yaml loader understands. I've got a hotfix to enable yaml again and fix this saving/loading issue. Before @capn-freako merges that PR, I'll take a look at fixing #77 as well.

Future looking, I still think a revamp would be great which could support up-rev'ing older ones or at least telling the user what is up. Add things such as a file format version. Right now a full config file looks like this:

!!python/object:pybert.pybert_cfg.PyBertCfg
R0: 1.452
Rdc: 0.1876
Theta0: 0.02
Z0: 100.0
alpha: 0.01
bit_rate: 10
cac: 1.0
ch_file: ''
cin: 0.5
cout: 0.5
ctle_file: ''
ctle_mode: 'Off'
ctle_mode_tune: 'Off'
ctle_offset: 0.0
ctle_offset_tune: 0.0
decision_scaler: 0.5
delta_t: 0.1
do_sweep: false
eye_bits: 1600
f_step: 10
gain: 0.5
impulse_length: 0.0
l_ch: 1.0
lock_sustain: 500
mod_type:
- 2
n_ave: 100
n_lock_ave: 500
n_taps: 5
nbits: 8000
nspb: 32
num_sweeps: 1
pattern_len: 127
peak_freq: 5.0
peak_mag: 10.0
pn_freq: 0.437
pn_mag: 0.001
rel_lock_tol: 0.1
rin: 100
rn: 0.001
rs: 100
rx_ami_file: ''
rx_bw: 12.0
rx_dll_file: ''
rx_use_ami: false
rx_use_getwave: false
rx_use_ts4: false
sum_bw: 12.0
sum_ideal: true
sweep_aves: 1
sweep_num: 1
thresh: 6
tx_ami_file: ''
tx_dll_file: ''
tx_tap_tuners:
- !!python/tuple
  - true
  - 0.0
- !!python/tuple
  - false
  - 0.0
- !!python/tuple
  - false
  - 0.0
- !!python/tuple
  - false
  - 0.0
tx_taps:
- !!python/tuple
  - true
  - 0.0
- !!python/tuple
  - false
  - 0.0
- !!python/tuple
  - false
  - 0.0
- !!python/tuple
  - false
  - 0.0
tx_use_ami: false
tx_use_getwave: false
tx_use_ts4: false
use_ch_file: false
use_ctle_file: false
use_dfe: true
use_dfe_tune: true
v0: 0.67
vod: 1.0
w0: 10000000.0

And a reduced (just to keep it short) example what I'm thinking the about:

version: '0.0.1'
simulation:
  bit_rate: 10
  debug: true
  do_sweep: false
  eye_bits: 1600
channel:
  ch_file: ''
receiver:
  ctle:
    file: ''
    mode: 'Off'
    mode_tune: 'Off'
    offset: 0.0
    offset_tune: 0.0

@dghan0219 how has the ASCII file format been (since originally it was binary) or any feedback on what you'd prefer in a future configuration file format?

dghan0219 commented

I like the ASCII format. It's easy to edit with a text editor, and it's easy to create and change with a program I wrote that then kicks off PyBERT. Adding issue 89 as a new feature would make launching PyBERT with the configuration file even easier.

  • 👍1