C implementation of createrepo
Run createrepo -h
for usage syntax.
Package build requires - Pkg name in Fedora/Ubuntu:
- bzip2 (http://bzip.org/) - bzip2-devel/libbz2-dev
- cmake (http://www.cmake.org/) - cmake/cmake
- expat (http://expat.sourceforge.net/) - expat-devel/libexpat1-dev
- file (http://www.darwinsys.com/file/) - file-devel/libmagic-dev
- glib2 (http://developer.gnome.org/glib/) - glib2-devel/libglib2.0-dev
- libcurl (http://curl.haxx.se/libcurl/) - libcurl-devel/libcurl4-openssl-dev
- libxml2 (http://xmlsoft.org/) - libxml2-devel/libxml2-dev
- python (http://python.org/) - python2-devel/libpython2.7-dev
- rpm (http://www.rpm.org/) - rpm-devel/librpm-dev
- openssl (http://www.openssl.org/) - openssl-devel/libssl-dev
- sqlite3 (https://sqlite.org/) - sqlite-devel/libsqlite3-dev
- xz (http://tukaani.org/xz/) - xz-devel/liblzma-dev
- zlib (http://www.zlib.net/) - zlib-devel/zlib1g-dev
- Documentation: doxygen (http://doxygen.org/) - doxygen/doxygen
- Documentation: sphinx (http://sphinx-doc.org/) - python-sphinx/
- Test requires: check (http://check.sourceforge.net/) - check-devel/check
- Test requires: python-nose (https://nose.readthedocs.org/) - python-nose/python-nose
- Experimental support: drpm (https://github.com/rpm-software-management/drpm)
From your checkout dir:
mkdir build
cd build/
cmake ..
make
To build the documentation, from the build/ directory: make doc
Note: For build with debugging symbols you could use (from the build/ directory):
cmake -DCMAKE_BUILD_TYPE:STRING=DEBUG .. && make
E.g. when you want to try weak and rich dependencies.
cmake -DRPM_PATH="/home/tmlcoch/git/rpm" .. && make
Note: The RPM must be buit in that directory
Commands I am using for building of RPM:
cd /home/tmlcoch/git/rpm
CPPFLAGS='-I/usr/include/nss3/ -I/usr/include/nspr4/' ./autogen.sh --rpmconfigure --with-vendor=redhat --with-external-db --with-lua --with-selinux --with-cap --with-acl --enable-python
make clean && make
At first, you have to checkout drpm library from the https://git.fedorahosted.org/git/drpm.git and build it.
git clone ssh://git.fedorahosted.org/git/drpm.git
cd drpm/
make
Then run cmake
for createrepo_c with param -DDRPM_PATH="/home/tmlcoch/git/drpm"
where the path is path to your build of drpm library.
cmake -DDRPM_PATH="/home/tmlcoch/git/drpm" .. && make
By default, cmake should set up things to build for Python 2, but you can do a build for Python 3 like this::
cmake -DPYTHON_DESIRED=3 .
Enable legacy SUSE/Mageia/Mandriva weakdeps support (Default: ON)
Threaded XZ encoding (Default: OFF)
Note: This option is disabled by default, because Createrepo_c parallelize a lot of tasks (including compression) by default, this only adds extra threads on XZ library level which causes thread bloat and for most usecases doesn't bring any performance boost. On regular hardware (e.g. less-or-equal 4 cores) this option may even cause degradation of performance.
utils/make_tarball.sh [git revision]
Without git revision specified HEAD is used.
Modify createrepo_c.spec and run:
utils/make_rpm.sh
Note: Current .spec for Fedora rawhide
All unit tests run from librepo checkout dir
make tests && make test
Note: For a verbose output of testing use: make ARGS="-V" test
build/tests/run_gtester.sh
Note: The C tests have to be builded by make tests
)!
PYTHONPATH=`readlink -f ./build/src/python/` nosetests -s tests/python/tests/
Or, for an alternative python version, specify the appropriate nosetests executable:
PYTHONPATH=`readlink -f ./build/src/python/` nosetests-3.4 -s tests/python/tests/
In original createrepo sha
is a nickname for the sha1
checksum.
Createrepo_c mimics this behaviour.
- Repodata in repo/ are has checksum xxx
- Params: --update --checksum=yyy repo/
- All package checksums are recalculated into yyy
- Only new and changed packages has yyy checksums other packages has still xxx checksums
- Some packages in repo/ are symlinks
- Params: --skip-symlinks repo/
- Symlinked packages are ignored
- Symlinked packages are processed (https://bugzilla.redhat.com/show_bug.cgi?id=828848)
- A somebody else's repo is somewhere
- The repo items have set a base path to http://foo.com/
- We want to create metadata for our repo
- Some packages in our repo are same as packages in somebody else's repo
- We want to speed up creation of our repodata with combo --update and --update-md-path=somebody_else's_repo
- Params: --update --update-md-path=ftp://somebody.else/repo our_repo/
- All our packages have no base path set (if we don't set --baseurl explicitly)
- Some packages in metadata (which was same in our repo and in somebody else's repo) have base path set to http://foo.com/
- (https://bugzilla.redhat.com/show_bug.cgi?id=875029)
- A repo with old metadata exists
- We want to update metadata
- Params: --update repo/
- All is fine
- Some paths in filelists.xml are crippled (https://bugzilla.redhat.com/show_bug.cgi?id=835565)
- A repo with repodata created with --simple-md-filenames exists
- We want to update repodata to have checksums in filenames
- Params: --update repo/
- All repodata contains checksum in the name
- All repodata contains checksum in the name
- There are old metadata without checksum in the name too
- (https://bugzilla.redhat.com/show_bug.cgi?id=836917)
Modifyrepo_c is compatible with classical Modifyrepo except some misbehaviour:
- TODO: Report bugs and add reference here
When there is need to do several modification to repository (repomd.xml
)
a batch file could be used.
Batch file is Modifyrepo_c specific. It is not supported by the classical Modifyrepo - at least not yet.
# Add:
# [<path/to/file>]
# <options>
# Metadata that use a bunch of config options
[some/path/comps.xml]
type=group
compress=true
compress-type=gz
unique-md-filenames=true
checksum=sha256
new-name=group.xml
# Metadata that use default settings
[some/path/bar.xml]
# Remove:
# [<metadata name>]
# remove=true
[updateinfo]
remove=true
Option name | Description | Supported value(s) | Default |
---|---|---|---|
path | Path to the file. When specified it override the path specified in group name (name between [] parenthesis) | Any string | group name (string between '[' ']') |
type | Type of the metadata | Any string | Based on filename |
remove | Remove specified file/type from repodata | true or false |
false |
compress | Compress the new metadata before adding it to repo | true or false |
true |
compress-type | Compression format to use | gz , bz2 , xz |
gz |
checksum | Checksum type to use | md5 , sha , sha1 , sha224 , sha256 , sha384 , sha512 |
sha256 |
unique-md-filenames | Include the file's checksum in the filename | true or false |
true |
new-name | New name for the file. If compress is true , then compression suffix will be appended. If unique-md-filenames is true , then checksum will be prepended. |
Any string | Original source filename |
- Lines beginning with a '#' and blank lines are considered comments.
- If
remove=true
is used, no other config options should be used