Excessive memory use / time taken to clone large git repository

[gallafent]

I find that the time taken and memory consumption when cloning a repository are excessive to the point that hg-git is unusable for me! Here's the status of an hg-git clone operation after I'd left it running overnight (well over 12 hours clockface time in fact), just before I stopped it! I don't know how far through the clone operation it had got. Note over six hours CPU, … and counting.

$ ps aux --sort -vsz | head -2
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
bill 8368 28.5 79.1 312228 198952 ? R Oct25 267:27 /usr/bin/python /usr/bin/hg clone git://gitorious.org/+qt-iphone/qt/qt-iphone-clone.git

(Sorry, don't have the console output, I disowned it to allow it to run after I logged out!)

For comparison, this operation takes less than six minutes clockface time (and less than a minute of CPU!) using git natively on the same system.

$ command time git clone git://gitorious.org/+qt-iphone/qt/qt-iphone-clone.git qt-iphone-clone
Initialized empty Git repository in /home/bill/clonetest/qt-iphone-clone/.git/
remote: Counting objects: 460484, done.
remote: Compressing objects: 100% (124223/124223), done.
remote: Total 460484 (delta 370117), reused 422259 (delta 332296)
Receiving objects: 100% (460484/460484), 335.19 MiB | 2.63 MiB/s, done.
Resolving deltas: 100% (370117/370117), done.
43.84user 13.18system 5:47.55elapsed 16%CPU (0avgtext+0avgdata 619072maxresident)k
739480inputs+1827408outputs (620major+90704minor)pagefaults 0swaps

I'm using hg 1.6.3-1, mercurial-git 0.2.3-1 and dulwich 0.6.1-1 packages of ubuntu 10.10.

I hope this provides a useful test-case to help solve this performance problem! Same order of magnitude in clockface time would be fine for me!