Feature Development

[onizenso]

I've been developing a feature to compress and encrypt the pappy project files, and have run into a bit of a hanging point.

How should I hook into the core pappy structure to check if the encrypted archive exists, and keep pappy from creating new project files?

Also, I am writing the feature as two separate plugins (one for encryption, one for compression), how do I call/import the compression feature from within the encryption feature?

Thanks in advance for any help/direction you can give me,
onizenso

[onizenso]

Please disregard the part about importing plugins into each other, found the answer in the "Writing Plugins" documentation:

Under the "Using defer.inlineCallbacks With a Command" section:
from pappyproxy.console import load_reqlist

Still trying how best to incorporate my plugin into pappy.py, but think I am slowly getting there.

Very open to any help/direction, sorry for the bit of noobishness,
onizenso
Embarassed I didn't RTFM more closely before asking this question.

[roglew]

So I'm not sure if writing a plugin is the best way to go about this. Maybe pass a project file in as a command line argument, extract/decrypt the contents of the file to a temporary directory, do work, then reencrypt/compress the directory back into the file when the program quits. To do this I would probably:

1. Add some variables to the class in config.py to store information passed into the program
2. Parse the command line arguments to get the name of the project file (same place as where it checks for lite mode) and put the settings into the configuration object. You'll probably have to do something tricky with configuration since the session object right now takes in a configuration object before it starts.
3. Add some methods to the session object to deal with setting up/cleaning up a temporary working directory (be sure to watch permissions. data file/cmdhistory shouldn't be world-readable, maintain original permissions if you can)
4. Call the setup methods in the session object's start() method and the cleanup methods in the session's cleanup() method (this way cleanup will happen even if the program is killed with Ctl-C)

For the most part, I don't think you'll need to write a plugin since by the time plugins can have their commands run, files are already opened and being used. Plus in terms of usability it's probably more intuitive to pass in a project file from the command line.

Loading configuration might be tricky since we need to decrypt/decompress the directory since the config file is in the directory that's encrypted/compressed. I would probably make sure that decompressing/decrypting is the first thing to happen in the session object's start() method, then make sure that loading config.json is included in the setup process so that the rest of the code afterwards can assume that the config is set up correctly.

I sort of rambled a bit so if I didn't describe something well let me know. I'm really glad to have someone working a feature like this!

[onizenso]

Totally agree, plugin doesn't seem to be the way to go anymore. Everything I was trying looked really hacky and gross. I can probably rewrite the plugin as a core/config feature, and still re-use alot of the code I've already written. Just need to change the approach. Better to do it right than have it be really fragile+ugly.

Most of what you are recommending is pretty clear, except I'm a little unclear on part 2. When you say project file, do you mean the name of the encrypted zip/bz2 archive?

Are you talking about something like:
On first run, create temp directory
Copy all existing project files (macros, config.json, etc.) into temp directory
Change config to point to temp directory as project root
Do work in temp directory
At close, compress all files in the temp directory into project.[zip/tar.bz2]
Encrypt archive, naming it with user supplied name (or default if none supplied)
Delete temp directory

Thanks for the detailed help on this one, if (probably when) anything else comes up I won't hesitate to post it here. Happy to be helping out.

[roglew]

yeah, exactly that. When I say project file I'm talking about the compressed/encrypted file. I'm imagining something like you start pappy with like pappy -e project.zip or whatever. So you would check if the flag was given and store the filename as a variable in the config object. Then in the actual code, use the filename in the config object.

[onizenso]

ok, totally makes sense. I was essentially doing a similar thing with the plugin, but way more clunky. Think this approach will definitely clean things up a bit.

Will be careful to preserve permissions. Should have something working soon fingers-crossed

[onizenso]

Almost finished with the crypto feature!

Still need to write some more tests to verify all the edge cases work, but the main functionality is complete. However, it is still a pretty fragile feature at this point.

If the user forgets to pass in the name of the encrypted project archive, pappy just starts in clear-text mode.

For example:
pappy -c proj.crypt starts in crypto mode
project -c silently falls back to clear text mode

Was trying to allow for default crypt file naming, should I just force passing in the archive name? Or should I skip allowing for command line passing, and just load the name from the config file/default settings in config.py?

Also, when starting in crypto mode, if the user has previously encrypted the project, it will ask them for their password to decrypt the file. If they enter the wrong password, an exception is raised, but pappy just falls back to clear-text mode.

How should I halt pappy to force the user to restart and try decryption again?

Thanks for any help/advice in advance.

P.s. At what point do you think it is appropriate for me to submit a pull request?

[roglew]

Awesome! If the user tries to run pappy with the flag but no name, I'd say print an error than quit. What do you mean by how to halt pappy to restart and try decryption again? Would a while loop with a try/except block work? Something like:

while True:
    try:
        do_decryption(arg)
        break
    except DecryptionException:
        print 'Incorrect password, try again?'
        if not some_confirmation_function():
            quit_pappy()

Just make sure that it's run before the listeners are opened and the console is started.

As long as you would feel comfortable using the feature for real work it should be fine to submit as a pull request. I guess as long as it encrypts/decrypts, is easy to use, doesn't open any new security issues, and doesn't crash all the time it should be fine.

[roglew]

Integrated