downloading templates folder and re-uploading breaks problem editor 2.18
Closed this issue · 17 comments
Problem and how to manifest
My university's server is at 2.18, and this evening I followed a practice I used all summer with 2.17, to edit many problems at once on my computer -- much faster than one-by-one on webwork:
- make archive of templates on webwork
- download templates.tgz to my computer
- unpack and edit a bunch of files
- re-tar the folder
- upload, and unpack-and-delete on webwork
I've successfully used this for a while now, to do things like merge content from courses, change filenames, move problems with filters, etc. Dozens of times this summer I've walked this loop to accomplish goals practically impossible through the Webwork online interface, using Python and other tools. So I had reason to believe it would still work this evening. (Please don't tell me "you shouldn't have been doing that" -- that won't be helpful in solving this problem, and it would belittle my experience and goals)
This was my first time doing this on 2.18, and 🔥. Now when I try to use the problem editor, I get this:
I think this is a bug having to do with the hardcopy template system in 2.18
Evidence
In the Course Configuration page, I can use the Hardcopy theme and Hardcopy theme for problem editor dropdowns, and I can see some ._ files available:
🧐 my guess at an explanation
downloading/uploading cannot create files that start with ._ I tried to make a file named that just now, and it refused giving me a message "Your file name may not begin with a dot". so somehow a config is pointing to this ._ file, and the download-upload loop loses it?
Something about the hardcopy selecting is picking up files it shouldn't. before i replaced the templates folder, the files were there? (were they? I can't see dotfiles) now they're not, and for some reason that breaks the problem editor.
That's my hunch.
My immediate ask
I'm about to launch my course for the Fall 2023 semester, and I have more work to do before it can happen. Is there a quick fix my sysadmin could use? Copy in those files, or just make them symlinks to the actual files?
Did you download to a Mac? A "dot underscore" file is a file that Mac OS makes, which has info in it about the file that the file itself does not have. These are typically created when you open a folder or file with Finder. So I am guessing that your Mac OS created these files (which are not actually .xml files) and then you packaged up the templates folder including these dot underscore files, and now they are in there. The error is because they end in .xml, so WW thinks they are valid hardcopy theme files. But they are not, and moreover their hidden status might be causing other problems for WW trying to use them.
Because they start with a dot, they are hidden and you can't see them in your File Manager. Otherwise the immediate thing to do is to delete them, which your server admin can do.
We will work on making such files be ignored, so this error would not occur. But you may want to be aware of how Mac OS will always create these if you navigate around in Finder. Your templates folder tree may have many more of them, but it's just that they don't usually cause trouble and are invisible.
One more thing you could do is delete your entire hardcopyThemes folder, and then upload a clean replacement. I'm just not sure what the right thing to do is in Mac OS to delete such files, unless you use the Terminal.
Did you download to a Mac?
Yes.
Those ._ files are not on my Mac. I did my due diligence. Before posting, I looked using ls -a in the folder on my mac, and they're not there.
To get your access back to the editor, try renaming the hardcopyThemes folder to something else for backup. Then use File Manager in WW to make a new hardcopyThemes folder. It can be empyt. One at a time, you could upload the one or two custom themes three that it looks like you created. But upload with the File Manager one at a time, not as a tgz.
Indeed, simply renaming the folder from hardcopyThemes to literally anything else, and i'm back able to use the editor on Webwork. I don't understand, how did the ._ files get there? They're not on my mac, I never made them.
Indeed, there is only one file in my entire templates folder I uploaded in this process that starts with ._:
I am very skeptical this was my mac's fault,
They're not on my mac, I never made them.
If I'm not wrong, then as I said, your Finder made them.
Then why can't I find them?
ah, it's not FInder's fault per se, it's an issue with the tar command as provided on MacOS. https://superuser.com/questions/259703/get-mac-tar-to-stop-putting-filenames-in-tar-archives
Then why can't I find them?
I don't know. All I can say is that dot underscore files are created by Mac OS. It is very unlikely they were created by WeBWorK or its OS and related tools.
I feel satisfied I know how to not cause this again. But I still think this is a bug in Webwork, let's not get distracted. I think the bug is:
if a ._*.xml file is in hardcopyThemes, then the editor breaks. common cause: naive use of tar on a Mac. (how would the user know otherwise)
But I still think this is a bug in Webwork,
I believe I already agreed:
We will work on making such files be ignored, so this error would not occur.
Just trying to leave a succinct conclusion at the bottom, Alex. Have a good night, and thanks a lot for helping me find a quick fix.
I opened PR #2186 to address this. It generally takes up to a week for these to be reviewed and merged. I don't recommend merging this to your installation before others here have reviewed it, as they may see better ways to accomplish the goal. Or see bad side effects that I am missing.
I finally got around to adding a page on the wiki about hardcopy themes: https://webwork.maa.org/wiki/Hardcopy_Themes. Aside from the issue with the bad xml files here, I'm curious to hear if you had success writing and using new hardcopy themes.
@ofloveandhate PR #2186 has been merged into main now. If you or your system admin can update WeBWorK, by going to /opt/webwork/webwork2 and doing git pull, then restarting webwork2 (with sudo systemctl restart webwork2 on ubuntu), you should have the new code and be able to use your workflow.
Thank you so very much!
I believe this issue is fixed and can be closed.