OSError: [Errno 22] Invalid argument

Question

OSError: [Errno 22] Invalid argument

Closed this issue 7 years ago · 15 comments

when I run slack-backup for the first time, it creates 2 folders (files & images) and 1 file slackrw.sqlite. It then downloads 12 images to the images folder and returns the following error:

Traceback (most recent call last):
File "C:\ProgramData\Anaconda3\Scripts\slack-backup", line 10, in
command.main()
File "C:\ProgramData\Anaconda3\lib\site-packages\slack_backup\command.py", line 141, in main
args.func(args)
File "C:\ProgramData\Anaconda3\lib\site-packages\slack_backup\command.py", line 62, in fetch_data
slack.update()
File "C:\ProgramData\Anaconda3\lib\site-packages\slack_backup\client.py", line 53, in update
self.update_channels()
File "C:\ProgramData\Anaconda3\lib\site-packages\slack_backup\client.py", line 69, in update_channels
channel = o.Channel()
File "", line 4, in init
File "C:\ProgramData\Anaconda3\lib\site-packages\sqlalchemy\orm\state.py", line 414, in _initialize_instance
manager.dispatch.init_failure(self, args, kwargs)
File "C:\ProgramData\Anaconda3\lib\site-packages\sqlalchemy\util\langhelpers.py", line 66, in exit
compat.reraise(exc_type, exc_value, exc_tb)
File "C:\ProgramData\Anaconda3\lib\site-packages\sqlalchemy\util\compat.py", line 187, in reraise
raise value
File "C:\ProgramData\Anaconda3\lib\site-packages\sqlalchemy\orm\state.py", line 411, in _initialize_instance
return manager.original_init(*mixed[1:], **kwargs)
File "C:\ProgramData\Anaconda3\lib\site-packages\slack_backup\objects.py", line 87, in init
self.update(data_dict)
File "C:\ProgramData\Anaconda3\lib\site-packages\slack_backup\objects.py", line 94, in update
self.created = datetime.fromtimestamp(data_dict.get('created', 0))
OSError: [Errno 22] Invalid argument

I don't see any log files. Should they be in mylogs folder? What number should I set quiet to in common to generate logs. I've tried 2 and 0.

I'm not sure if you are able to resolve this without having access to the data, but I thought I would ask if you have any ideas what is happening and how to resolve it?

Thanks,
Richard

Answer 1 · 2018-03-12T15:24:46.000Z

I've tried running the script on a different Slack channel and I receive the exact same error. As with above, it creates 2 folders (files & images) under an assets folder and 1 sqlite file. It then downloads 40 some images to the images folder, stops and returns the error.

Answer 2 · 2018-04-13T22:54:11.000Z

Anyone have any ideas on how to resolve the above issue?

Answer 3 · 2018-04-25T20:42:36.000Z

Initially, while you start slack-backup, it will fetch channels and create corresponding records in the database. One of the channels field - created - is a datetime object, but during creation of it (in line 69) it will create an empty Channel object. For some reason, default value for an empty datetime object, which is created using datetime.datetime.fromtimestamp(0) on Windows raises OSError, instead of expected timestamp placed in 1970-01-01 01:00:00 as it does under Linux. I'll investigate that further.
In the meantime you could use a VM or linux subsystem on windows as a workaround for this issue.

Answer 4 · 2018-05-01T10:46:00.000Z

I've fixed this issue with c0c1e7c commit. @aberja could you please check if the latest commits are working for you?

Answer 5 · 2018-05-02T05:48:34.000Z

Thank you for working on this.

I updated slack-backup and ran the following on 3 different Slack groups:
python "C:\ProgramData\Anaconda3\Scripts\slack-backup" fetch

The FIRST Slack group generated the following error messages:

Traceback (most recent call last):
File "C:\ProgramData\Anaconda3\lib\site-packages\slack_backup\client.py", line 159, in _create_message
filter(o.User.slackid == data['user']).one()
KeyError: 'user'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "C:\ProgramData\Anaconda3\Scripts\slack-backup", line 10, in
command.main()
File "C:\ProgramData\Anaconda3\lib\site-packages\slack_backup\command.py", line 141, in main
args.func(args)
File "C:\ProgramData\Anaconda3\lib\site-packages\slack_backup\command.py", line 62, in fetch_data
slack.update()
File "C:\ProgramData\Anaconda3\lib\site-packages\slack_backup\client.py", line 54, in update
self.update_history()
File "C:\ProgramData\Anaconda3\lib\site-packages\slack_backup\client.py", line 133, in update_history
self._create_message(msg, channel)
File "C:\ProgramData\Anaconda3\lib\site-packages\slack_backup\client.py", line 162, in _create_message
filter(o.User.slackid == data['comment']['user']).one()
KeyError: 'comment'

The SECOND and THIRD Slack groups both generated the following error messages:

Traceback (most recent call last):
File "C:\ProgramData\Anaconda3\Scripts\slack-backup", line 10, in
command.main()
File "C:\ProgramData\Anaconda3\lib\site-packages\slack_backup\command.py", line 141, in main
args.func(args)
File "C:\ProgramData\Anaconda3\lib\site-packages\slack_backup\command.py", line 62, in fetch_data
slack.update()
File "C:\ProgramData\Anaconda3\lib\site-packages\slack_backup\client.py", line 53, in update
self.update_channels()
File "C:\ProgramData\Anaconda3\lib\site-packages\slack_backup\client.py", line 72, in update_channels
self._update_channel(channel, data)
File "C:\ProgramData\Anaconda3\lib\site-packages\slack_backup\client.py", line 258, in _update_channel
channel)
File "C:\ProgramData\Anaconda3\lib\site-packages\slack_backup\client.py", line 233, in _get_create_obj
datetime.fromtimestamp(data['last_set'])).
OSError: [Errno 22] Invalid argument

If there is anything else you need from me, please do not hesitate to ask.

Answer 6 · 2018-05-02T11:15:36.000Z

@aberja, could you try to do all the steps again from scratch? starting from cloning repository, preparing an ini file, making sure, that there is no db file around? What I did yesterday, was (on fresh installed windows):

install fresh python 3,
clone the repository,
create an ini file in the repository directory,
add repository path to PYTHONPATH variable,
install dependences (sqlalchemy and slackclient),
and run python scripts/slack-backup fetch inside toplevel slack-backup repo.

Works like a charm. I did that again today, just to make sure.

Answer 7 · 2018-05-02T16:19:24.000Z

I followed the steps you outlined on a different computer than the one I was using.

Unfortunately, I obtained the same Keyerror 'user' error for the first Slack group. This Slack group is actually open to the public. So I'm not sure if you would like to join and see if there is an issue in the group that is causing the program to fail? If so, the invite link is: https://www.quantconnect.com/slack Alternatively, I've copied the full log and can email the log to you if you would like.

For kicks and giggles and tried downloading the messages using a different program found here: https://github.com/mcoster/slack-auto-export It worked without any errors.

Answer 8 · 2018-05-02T16:48:29.000Z

On the second computer, I just ran the script on the second Slack group and I received the same Errno 22 error. It appears to be that we are dealing with 2 different errors. The first one is a 'user' error while this one is Errno 22.

Also, this one crashes prior to downloading any files, whereas the first Slack group referred to above was able to download some files.

In the Debug log, it appears to crash while Fetching and updating channel info in DB. I can see that it starts to update channels, but stops before getting thru all of them:

2018-05-02 11:41:33,123 INFO: Fetching and update channels information in DB
2018-05-02 11:41:33,125 DEBUG: Starting new HTTPS connection (1): slack.com
2018-05-02 11:41:33,373 DEBUG: https://slack.com:443 "POST /api/channels.list HTTP/1.1" 200 2092
2018-05-02 11:41:33,379 INFO: Update channel alliancechallenges' information in DB 2018-05-02 11:41:33,389 INFO: Update channel attackvideo' information in DB
2018-05-02 11:41:33,391 INFO: Update channel basedesigns' information in DB 2018-05-02 11:41:33,395 INFO: Update channel current-match-intel' information in DB
2018-05-02 11:41:33,397 INFO: Update channel defensevideo' information in DB 2018-05-02 11:41:33,400 INFO: Update channel general' information in DB
Traceback (most recent call last):
File "slack-backup", line 10, in
command.main()
File "C:\ProgramData\Anaconda3\lib\site-packages\slack_backup\command.py", line 141, in main
args.func(args)
File "C:\ProgramData\Anaconda3\lib\site-packages\slack_backup\command.py", line 62, in fetch_data
slack.update()
File "C:\ProgramData\Anaconda3\lib\site-packages\slack_backup\client.py", line 53, in update
self.update_channels()
File "C:\ProgramData\Anaconda3\lib\site-packages\slack_backup\client.py", line 72, in update_channels
self._update_channel(channel, data)
File "C:\ProgramData\Anaconda3\lib\site-packages\slack_backup\client.py", line 258, in _update_channel
channel)
File "C:\ProgramData\Anaconda3\lib\site-packages\slack_backup\client.py", line 233, in _get_create_obj
datetime.fromtimestamp(data['last_set'])).
OSError: [Errno 22] Invalid argument

Answer 9 · 2018-05-02T18:31:34.000Z

Ok, I've created an account and joined quantconnect team, and turns out there are some cases where user_id is not propagated because of bot generated messages/attachements/comments. I'll look at this closer tomorrow.

Answer 10 · 2018-05-03T09:22:43.000Z

I've fixed issue with crashing on non-existing user in commit 641d670. As for failure on _get_create_obj datetime.fromtimestamp(data['last_set'])) I've added logging of data, which could help in debugging. Could you check it again?

Answer 11 · 2018-05-03T15:11:45.000Z

Great job, the non-exiting user issue appears to be fixed :)

With respect to the Errno 22 error, here is message I get:

2018-05-03 10:07:18,422 ERROR: Failed on data: {'creator': '',
'last_set': 0,
'value': 'This channel is for team-wide communication and announcements. All '
'team members are in this channel.'}
Traceback (most recent call last):
File "C:\ProgramData\Anaconda3\lib\site-packages\slack_backup\client.py", line 266, in _get_create_obj
datetime.fromtimestamp(data['last_set'])).
OSError: [Errno 22] Invalid argument
Traceback (most recent call last):
File "C:\ProgramData\Anaconda3\Scripts\slack-backup", line 10, in
command.main()
File "C:\ProgramData\Anaconda3\lib\site-packages\slack_backup\command.py", line 141, in main
args.func(args)
File "C:\ProgramData\Anaconda3\lib\site-packages\slack_backup\command.py", line 62, in fetch_data
slack.update()
File "C:\ProgramData\Anaconda3\lib\site-packages\slack_backup\client.py", line 55, in update
self.update_channels()
File "C:\ProgramData\Anaconda3\lib\site-packages\slack_backup\client.py", line 74, in update_channels
self._update_channel(channel, data)
File "C:\ProgramData\Anaconda3\lib\site-packages\slack_backup\client.py", line 294, in _update_channel
channel)
File "C:\ProgramData\Anaconda3\lib\site-packages\slack_backup\client.py", line 266, in _get_create_obj
datetime.fromtimestamp(data['last_set'])).
OSError: [Errno 22] Invalid argument

Answer 12 · 2018-05-03T16:36:17.000Z

Ok, so the culprit is 'last_set': 0 in data dict, and the issue we already bump on with datetime.datetime.fromtimestamp() function with argument 0. Give me a sec.

Answer 13 · 2018-05-03T16:52:55.000Z

Commit 5e7f474 should take care of that issue now :)

Answer 14 · 2018-05-03T20:26:07.000Z

Yes! It's working now, great job. tyvm

Answer 15 · 2018-05-04T17:20:40.000Z

You're welcome. I'm closing the issue in that case :)