/createrepo_c

C implementation of the createrepo.

Primary LanguageCGNU General Public License v2.0GPL-2.0

createrepo_c

C implementation of createrepo

Run createrepo -h for usage syntax.

Devel tips

Building

Package build requires - Pkg name in Fedora/Ubuntu:

From your checkout dir:

mkdir build
cd build/
cmake ..
make

To build the documentation, from the build/ directory:

make doc

Note: For build with debugging symbols you could use (from the build/ directory):

cmake -DCMAKE_BUILD_TYPE:STRING=DEBUG .. && make

Building from an rpm checkout

E.g. when you want to try weak and rich dependencies.

cmake -DRPM_PATH="/home/tmlcoch/git/rpm" .. && make

Note: The RPM must be built in that directory

Commands I am using for building the RPM:

cd /home/tmlcoch/git/rpm
CPPFLAGS='-I/usr/include/nss3/ -I/usr/include/nspr4/' ./autogen.sh --rpmconfigure --with-vendor=redhat --with-external-db --with-lua --with-selinux --with-cap --with-acl --enable-python
make clean && make

Building with delta rpm support (drpm)

At first, you have to checkout the drpm library from https://github.com/rpm-software-management/drpm.git and build it.

git clone https://github.com/rpm-software-management/drpm.git
cd drpm/
make

Then run cmake for createrepo_c with param -DDRPM_PATH="/home/tmlcoch/git/drpm" where the path is path to your build of drpm library.

cmake -DDRPM_PATH="/home/tmlcoch/git/drpm" .. && make

Building for a different Python version

By default, cmake should set up things to build for Python 3, but you can do a build for Python 2 like this::

cmake -DPYTHON_DESIRED=2 .

Other build options

-DENABLE_LEGACY_WEAKDEPS=ON

Enable legacy SUSE/Mageia/Mandriva weakdeps support (Default: ON)

-DENABLE_THREADED_XZ_ENCODER=ON

Threaded XZ encoding (Default: OFF)

Note: This option is disabled by default, because Createrepo_c parallelizes a lot of tasks (including compression) by default; this only adds extra threads on XZ library level which causes thread bloat and for most usecases doesn't bring any performance boost. On regular hardware (e.g. less-or-equal 4 cores) this option may even cause degradation of performance.

Build tarball

utils/make_tarball.sh [git revision]

Without git revision specified HEAD is used.

Build RPM package

Modify createrepo_c.spec and run:

utils/make_rpm.sh

Note: Current .spec for Fedora rawhide

Testing

All unit tests run from librepo checkout dir

Build C tests && run c and python tests

make tests && make test

Note: For a verbose output of testing use: make ARGS="-V" test

Run only C unittests (from your checkout dir):

build/tests/run_gtester.sh

Note: The C tests have to be builded by make tests)!

Run only Python unittests (from your checkout dir):

PYTHONPATH=`readlink -f ./build/src/python/` nosetests -s tests/python/tests/

Or, for an alternative python version, specify the appropriate nosetests executable:

PYTHONPATH=`readlink -f ./build/src/python/` nosetests-3.4 -s tests/python/tests/

Links

Bugzilla

Important notes

In original createrepo sha is a nickname for the sha1 checksum. Createrepo_c mimics this behaviour.

Differences in behavior between createrepo_c and createrepo

Checksums after update

Use case:

  • Repodata in repo/ are has checksum xxx
  • Params: --update --checksum=yyy repo/

createrepo_c result:

  • All package checksums are recalculated into yyy

original createrepo result:

  • Only new and changed packages has yyy checksums other packages has still xxx checksums

Skip symlinks param

Use case:

  • Some packages in repo/ are symlinks
  • Params: --skip-symlinks repo/

createrepo_c result:

  • Symlinked packages are ignored

original createrepo result:

Base path from update-md-path repo

Use case:

  • A somebody else's repo is somewhere
  • The repo items have set a base path to http://foo.com/
  • We want to create metadata for our repo
  • Some packages in our repo are same as packages in somebody else's repo
  • We want to speed up creation of our repodata with combo --update and --update-md-path=somebody_else's_repo
  • Params: --update --update-md-path=ftp://somebody.else/repo our_repo/

createrepo_c results:

  • All our packages have no base path set (if we don't set --baseurl explicitly)

original createrepo result:

Crippled paths in filelists.xml after update

Use case:

  • A repo with old metadata exists
  • We want to update metadata
  • Params: --update repo/

createrepo_c results:

  • All is fine

original createrepo result:

--update leaves behind some old repodata files

Use case:

  • A repo with repodata created with --simple-md-filenames exists
  • We want to update repodata to have checksums in filenames
  • Params: --update repo/

createrepo_c results:

  • All repodata contains checksum in the name

original createrepo result:

Modifyrepo_c

Modifyrepo_c is compatible with classical Modifyrepo except some misbehaviour:

  • TODO: Report bugs and add reference here

Batch file

When there is need to do several modification to repository (repomd.xml) a batch file could be used.

Batch file is Modifyrepo_c specific. It is not supported by the classical Modifyrepo - at least not yet.

Example

# Add:
#   [<path/to/file>]
#   <options>

# Metadata that use a bunch of config options
[some/path/comps.xml]
type=group
compress=true
compress-type=gz
unique-md-filenames=true
checksum=sha256
new-name=group.xml

# Metadata that use default settings
[some/path/bar.xml]

# Remove:
#   [<metadata name>]
#   remove=true

[updateinfo]
remove=true

Supported options

Option name Description Supported value(s) Default
path Path to the file. When specified it override the path specified in group name (name between [] parenthesis) Any string group name (string between '[' ']')
type Type of the metadata Any string Based on filename
remove Remove specified file/type from repodata true or false false
compress Compress the new metadata before adding it to repo true or false true
compress-type Compression format to use gz, bz2, xz gz
checksum Checksum type to use md5, sha, sha1, sha224, sha256, sha384, sha512 sha256
unique-md-filenames Include the file's checksum in the filename true or false true
new-name New name for the file. If compress is true, then compression suffix will be appended. If unique-md-filenames is true, then checksum will be prepended. Any string Original source filename

Notes

  • Lines beginning with a '#' and blank lines are considered comments.
  • If remove=true is used, no other config options should be used