PZS-NG (Project-ZS - Next Generation) -------------------------------------- Introduction: ------------- PZS-NG is based on Project-ZS by Dark0n3. It is what is commonly known as a zipscript, or a post check script for file uploads. Its task is, among others, to check the integrity of the files uploaded, and make sure a "release" is complete and not corrupted. Of course, this is not the only thing done by this zipscript ;). Read on to find out why this zipscript is considered the best of the bunch by most. Find us on the web: http://www.pzs-ng.com http://bugs.pzs-ng.com History: -------- PZS-NG is a continuance of the original Project-ZS by Dark0n3, which stopped development in June(?) 2002, and remained stagnant for a long time. Up until now various coders/scripters have made patches to fix bugs or make some enhancements, but the patches have at times been overlapping, or complete replacements of source-files, making it hard for the siteops to implement them all. In April, 2004, daxxar and psxc collected the various patches into packages, but soon found out it was better to pool resources and make a unified version. After rounding up/threatening some of the scene's brightest boys, they got the result - PZS-NG. The Team: --------- daxxar (all-round programmer) psxc (all-round programmer) iwdisb (C specialist) js (C specialist) freezy3k (C specialist) iono (Tcl guru) themolester (Tcl guru) avizion (Tcl guru, webmaster) juanker (Tcl guru) dakrer (Tcl guru) neoxed (Tcl guru) You can find us in #pzs-ng on EFnet for support and/or bug reporting. Supported Platforms: -------------------- PZS-NG should compile fine on the following platforms: - Linux - FreeBSD - OpenBSD - OSX/Darwin (the zipscript should work, but sitewho etc may need some makefile options. Contact psxc or one of the devs for help) - (AIX - untested, but the original docs say it works there) - (Solaris - soon *g*) 64-bit processors - should work. Since glftpd is pre-compiled on a 32bit platform (usually) certain bins in pzs-ng will be compiled in 32bit mode. Make sure you have the 32bit libs installed. For more info, see README.AMD64. The *BSD platforms currently have no support for the -m32 flag. Ask psxc or one of the devs if you are stuck on this. Only the i386 platform is tested, so if you try it on anything else, don't hesitate to contact us and inform us of the result. Supported FTP Daemons: ---------------------- - glftpd 1.xx - glftpd 2.xx - cuftpd 1.x Basic description of How Things Work (tm): ------------------------------------------ As mentioned in the Introduction earlier, a zipscript checks the integrity of the files, and keep tabs on when a release is complete. How it does this depends on the filetype. 'ZIP' files have an integrity code build within the file itself, which makes it easy to verify the file. To keep track of whether or not a release based on zip-files is complete, a file named file-id.diz is scanned. The most common method of checking files, however, is by 'SFV'. Unlike zip, the SFV file is a text file which stores the filenames and a CRC (cyclic redundancy check) code. The files belonging to/listed in the sfv file can be of any type, of which the most common are rar and mp3. The CRC code listed in the file is compared to a CRC code calculated on the fly by either the FTP daemon, or the program itself (more on this later). It is also quite easy to find out if a release is complete by counting files in the SFV file. Features of PZS-NG: ------------------- PZS-NG would not be considered the best zipscript by most unless it had features beyond simple file checking. Here's a list of a few things it can do: - Log information about sfv files (how many files expected etc) - Log information about mp3 files (genre, year, quality etc) - Log information about the first uploaded file (who did it, speed etc) - Log information about halfway (when a release is halfway, who is leading, speed etc) - Log information about complete (who won, speed, percent, who raced etc) - Log information about a race (who takes the lead, who is racing, what speed, percent etc) - include information about the release in the release dir (who raced, won, any information on the media files, speed etc) - creation of -missing files (to easily spot what files are missing in a release) - create (in)complete dirs/files (to show what releases are incomplete, and info on the release itself when it's complete) - execute external script based on filetypes, when a file is uploaded, and/ or when a release complete. - MORE! Along with the included sitebot (more on this later) we can pass this information on to an IRC channel, which in turn will make the site seem alive, and help couriers in their work. Using a bot also has a high fun-factor :) Installation: ------------- See INSTALL for general (and glftpd-specific) instructions, and in addition INSTALL.cuftpd for cuftpd-specific instructions. sitebot/README (and INSTALL.cuftpd) has instructions regarding the sitebot (dZSbot). Compiled Binaries: ------------------ There are a few other binaries compiled, but they are mostly used by 3rd party scripts. Here's a short description of each compiled binary: - cleanup - This little bin will clean out dead symlinks. Where it search you can specify in zsconfig.h in the define of "cleanupdirs". Please note that this script does *NOT* scan recursively, meaning you really have to insert the names of dirs to be scanned in this variable. Setting this to '/site/incoming/' will only search /site/incoming, not /site/incoming/apps/ or any other dir just below. Please note that dead symlinks in the mp3 genre/group/year/etc will be scanned automatically - there is no need to add these to 'cleanupdirs'. This script may be used as a cscript to 'site wipe' or 'site nuke' for instance, in which case it will only scan the current dir. You may also run it in 'view' mode, by giving it rootpath to glftpd. This will only list incomplete dirs, not remove any links. How to test: chroot /glftpd /bin/cleanup (in chroot, in a site dir) /bin/glftpd something /glftpd/bin/cleanup /glftpd - datacleaner - This one will remove the racedata of dirs no longer found on your site. Please note that this script may be run in crontab (example 1), as a command from shell (example 2) or as a cscript (example 3). Please note that if you try to use it as a cscript to anything but RMD, it will scan recursively, relative to where you are. This means it's possible to use it as a cscript to 'site wipe' or 'site nuke' for instance, but it may take some time for it to go thru all the data. How to test: chroot /glftpd /bin/datacleaner chroot /glftpd /bin/datacleaner /site/path/to/file chroot /glftpd /bin/datacleaner "RMD /path/to/file" (no /site in front!) chroot /glftpd /bin/datacleaner "NUKE /path/to/file" (no /site in front!) - postdel - Should be run after you delete something (as a cscript). Will re-check the release and update the racedata accordingly. Please note that this script will *only* work as a cscript to the DELE command. How to test: chroot /glftpd /bin/postdel "DELE /site/path/to/filename" - postunnuke - Should be run after you unnuke something (as a cscript). Will re-check the full release and update the racedata accordingly. Please note that this script will *only* work as a cscript to the site UNNUKE command. How to test: chroot /glftpd cd /site/archive/something /bin/postunnuke "site UNNUKE some.release-here sorry.erroneus.nuke" - dl_speedtest - used to measure download speeds. It will write the output to glftpd.log only. How to test: chroot /glftpd cd /site/speedtest export USER=something; export GROUP=something; export SPEED=2011 /bin/dl_speedtest "RETR /site/speedtest/file" - ng-undupe - Removes entry in dupelog after a file fails sfv check. How to test: chroot /glftpd /bin/ng-undupe filename - ng-deldir - Marks a directory as deleted in dirlog when it is removed cause it's banned. How to test: chroot /glftpd /bin/ng-deldir /site/path/to/dir - racedebug - Debugging bin that reads the racedata directly and prints out a report on racers, files, speed, crc etc. How to test: chroot /glftpd /bin/racedebug /ftp-data/pzs-ng/path/to/racedata - racestats - Mostly used by 3rd party scripts. Will give raceinfo of a release in cookie format. How to test: chroot /glftpd /bin/racestats /site/path/to/dir - zipscript-c - The zipscript. Should be run from within glftpd after a file is uploaded. Will do various tests, create stats, run external commands etc, according to your config. How to test: chroot /glftpd /bin/zipscript-c <filename> <path> <crc-code> You can recover (or check) your compiled config by the following syntax: chroot /glftpd /bin/zipscript-c --config chroot /glftpd /bin/zipscript-c --fullconfig # Also shows defaulted values - ng-chown - currently not used. It's a bin designed to chown files/dirs in your site to a specified used/group. When used it needs the +s bit set. Take care, though - using this bin with +s may be a security risk. - rescan - Used to re-check a release. Mostly used as a site command. It can take the following arguments (only one allowed): --quick - skips files that are already marked as checked, and crc-checks the ones that are not. --normal - check all files regardless if they previously have been checked and found ok. --chroot=<DIRNAME> - chroot() to DIRNAME before starting the rescan. --dir=<DIRNAME> - chdir() to DIRNAME before starting the rescan. <NAME><*> - only recheck the file named NAME or all files starting with NAME*. Wildcard can only be at the end, not beginning or in the middle. How to test: /glftpd/bin/rescan --chroot=/glftpd --dir=/site/linux/suse15 --normal site rescan --dir=/site/linux/suse15 site rescan --quick