louisabraham/har2requests

You need exactly one of "params" or "text" in field postData

Lightdm opened this issue · 16 comments

Lightdm commented

I'm new to .har files and only a bit experienced with automated python webrequests.
I exported a .har file from firefox, and after a lot of hussle managed to find this, and then get it to run, but I'm getting some error I don't understand...

Traceback (most recent call last):
  File "c:\users\MYNAME\appdata\local\programs\python\python37\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "c:\users\MYNAME\appdata\local\programs\python\python37\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Users\MYNAME\AppData\Local\Programs\Python\Python37\Scripts\har2requests.exe\__main__.py", line 7, in <module>  File "c:\users\MYNAME\appdata\local\programs\python\python37\lib\site-packages\click\core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "c:\users\MYNAME\appdata\local\programs\python\python37\lib\site-packages\click\core.py", line 782, in main
    rv = self.invoke(ctx)
  File "c:\users\MYNAME\appdata\local\programs\python\python37\lib\site-packages\click\core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "c:\users\MYNAME\appdata\local\programs\python\python37\lib\site-packages\click\core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "c:\users\MYNAME\appdata\local\programs\python\python37\lib\site-packages\har2requests\__init__.py", line 197, in main
    entry["request"], entry["response"], entry["startedDateTime"]
  File "c:\users\MYNAME\appdata\local\programs\python\python37\lib\site-packages\har2requests\__init__.py", line 65, in from_json
    ), 'You need exactly one of "params" or "text" in field postData'
AssertionError: You need exactly one of "params" or "text" in field postData
louisabraham commented

Note: you can use triple backticks (```) to indicate blocks of code

Well, it looks like Firefox doesn't follow the specs (http://www.softwareishard.com/blog/har-12-spec/#postData)

Note that text and params fields are mutually exclusive.

I'm creating an unsafe mode.

Try the following, if it works I might push to master.

pip install -U git+https://github.com/louisabraham/har2requests.git@unsafe
har2requests --unsafe file.har
Lightdm commented

Thanks for the tip :)
I had some problems getting that unsafe installed but got it now (using pip install -U https://github.com/louisabraham/har2requests/archive/unsafe.zip)

Now I'm getting

E:\DIRECTORY>har2requests --unsafe FILE.har
c:\users\MYNAME\appdata\local\programs\python\python37\lib\site-packages\har2requests\__init__.py:74: UserWarning: You need exactly one of "params" or "text" in field postData
  warnings.warn(POSTDATA_WARNING)
Traceback (most recent call last):
  File "C:\Users\MYNAME\AppData\Local\Programs\Python\Python37\Scripts\har2requests-script.py", line 11, in <module>
    load_entry_point('har2requests==0.1.2', 'console_scripts', 'har2requests')()
  File "c:\users\MYNAME\appdata\local\programs\python\python37\lib\site-packages\click\core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "c:\users\MYNAME\appdata\local\programs\python\python37\lib\site-packages\click\core.py", line 782, in main
    rv = self.invoke(ctx)
  File "c:\users\MYNAME\appdata\local\programs\python\python37\lib\site-packages\click\core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "c:\users\MYNAME\appdata\local\programs\python\python37\lib\site-packages\click\core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "c:\users\MYNAME\appdata\local\programs\python\python37\lib\site-packages\har2requests\__init__.py", line 206, in main
    unsafe=unsafe
  File "c:\users\MYNAME\appdata\local\programs\python\python37\lib\site-packages\har2requests\__init__.py", line 78, in from_json
    postData = Request.dict_from_har(pd["text"])
  File "c:\users\MYNAME\appdata\local\programs\python\python37\lib\site-packages\har2requests\__init__.py", line 100, in dict_from_har
    return {x["name"]: x["value"] for x in j}
  File "c:\users\MYNAME\appdata\local\programs\python\python37\lib\site-packages\har2requests\__init__.py", line 100, in <dictcomp>
    return {x["name"]: x["value"] for x in j}
TypeError: string indices must be integers
louisabraham commented

Sorry, I made a typo. Could you try again?

Lightdm commented

As you can see in my edit, I tried it with the unsafe mode, and still got an error, so I downloaded a .har from chrome, and tried the original version of har2requests, and got the original Error again
And I just checked, the unsafe release (now installed properly, had to get git, apparently I didn't have it anymore) gives the same new error for the chrome .har

louisabraham commented

Hi, I just made a few modifications and added logging, could you try?

Lightdm commented 
Traceback (most recent call last):
  File "C:\Users\MYNAME\AppData\Local\Programs\Python\Python37\Scripts\har2requests-script.py", line 11, in <module>
    load_entry_point('har2requests==0.1.2', 'console_scripts', 'har2requests')()
  File "c:\users\MYNAME\appdata\local\programs\python\python37\lib\site-packages\pkg_resources\__init__.py", line 489, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "c:\users\MYNAME\appdata\local\programs\python\python37\lib\site-packages\pkg_resources\__init__.py", line 2793, in load_entry_point
    return ep.load()
  File "c:\users\MYNAME\appdata\local\programs\python\python37\lib\site-packages\pkg_resources\__init__.py", line 2411, in load
    return self.resolve()
  File "c:\users\MYNAME\appdata\local\programs\python\python37\lib\site-packages\pkg_resources\__init__.py", line 2417, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "c:\users\MYNAME\appdata\local\programs\python\python37\lib\site-packages\har2requests\__init__.py", line 74
    warnings.warn(POSTDATA_WARNING + f"\n{request}\n{'-'*10})
                                                            ^
SyntaxError: EOL while scanning string literal
louisabraham commented

lol sorry I make some typos, could you try again?

Lightdm commented 
Traceback (most recent call last):
  File "C:\Users\MYNAME\AppData\Local\Programs\Python\Python37\Scripts\har2requests-script.py", line 11, in <module>
    load_entry_point('har2requests==0.1.2', 'console_scripts', 'har2requests')()
  File "c:\users\MYNAME\appdata\local\programs\python\python37\lib\site-packages\click\core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "c:\users\MYNAME\appdata\local\programs\python\python37\lib\site-packages\click\core.py", line 782, in main
    rv = self.invoke(ctx)
  File "c:\users\MYNAME\appdata\local\programs\python\python37\lib\site-packages\click\core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "c:\users\MYNAME\appdata\local\programs\python\python37\lib\site-packages\click\core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "c:\users\MYNAME\appdata\local\programs\python\python37\lib\site-packages\har2requests\__init__.py", line 470, in main
    unsafe=unsafe,
  File "c:\users\MYNAME\appdata\local\programs\python\python37\lib\site-packages\har2requests\__init__.py", line 333, in from_json
    assert params + text == 1, POSTDATA_WARNING
AssertionError: You need exactly one of "params" or "text" in field postData
louisabraham commented

I don't think you used the latest version, can you install it with: pip install -U git+https://github.com/louisabraham/har2requests.git@unsafe (I tested that line)

Lightdm commented

I think last time I updated it used a cached version, but this time I think it worked, still gives me the same error, but have you seen that it's a different error than the first one i got? Now it says "POSTDATA_WARNING", so something has changed.

louisabraham commented

I'm sure it didn't work because the line numbers don't match the source code.

Maybe try to uninstall first and resinstall with these options?

pip uninstall har2requests
python -c "import har2requests" # make sure it raises an error
pip install -U --no-cache-dir git+https://github.com/louisabraham/har2requests.git@unsafe
Lightdm commented

I got the error, that there is no such module, even made sure that i couldn't use the command, but after installing with the no-cache command still got the same error as in #8 (comment)

louisabraham commented

Oh maybe it worked. Could you give me what is above? There should be a line with Exception while parsing.

Please provide both outputs without and with --unsafe

Lightdm commented

I--- I just realized I didn't use the --unsafe option...
I am so sorry.

Lightdm commented

Now I only need to figure out, how to work with what your programm gave me...
Anyways, thank you very much :) I am sorry for the inconvenience I caused...

louisabraham commented

No problem, you made me improve my program with the unsafe option. Could you tell me the warnings it gives? It will help other users.