Abysmal performance on basic commands involving nixpkgs
eclairevoyant opened this issue ยท 14 comments
Commands like
nix build -f https://github.com/NixOS/nixpkgs/archive/refs/heads/master.tar.gz hello
or
nix profile install nixpkgs#hello
gets stuck for tens of minutes (!) after unzipping, sometimes never completing. SIGINT does nothing, have to force stop the app.
The evaluation speed (in a proot at least) is very slow. Once a specific version of nixpkgs has been evaluation cached then things get a lot faster. Running just nixpkgs#<whatever>
is generally a bad idea as nixpkgs updates too quickly. Using a specific revision is faster as is just adding/removing packages from a flake.
taking hours to eval makes this basically unusable ๐คท
and why would proot be related? eval happens in-memory.
and proot claims to "not cause overhead".
Running nix-on-droid switch
takes about a minute for me once evaluation caching is done, which for me is very usable. Perhaps you could try and profile things to see where time is being spent?
once evaluation caching is done
I feel like this is missing the point - if eval takes over an hour with no guarantee of completion whenever the revision needs updating, I'd have to sit there waiting and hoping it ever completes. There's obviously a design flaw and I'd hope it can be addressed. Or at least, part of the social contract of using open-source software is reporting issues as they are encountered, if no one has done so prior.
OK, my stance on this.
- IDK who claims "proot doesn't cause overhead", proot tanks IO real hard, and nixpkgs eval is very IO-heavy.
- proot is the project's chosen way to run unprivileged without recompiling nixpkgs. I have neither the bandwidth to implement our ways of relocating /nix/store, nor the resources to recompile nixpkgs to a new location. Whoever has either of the two, reach out to me and we'll figure something out.
- Hour-long eval is horrible UX and entirely unreasonable.
- on a 5 year old Samsung S10e 6GB with LineageOS 19 and wakelock held,
nix build -f https://github.com/NixOS/nixpkgs/archive/refs/heads/master.tar.gz hello
(i.e., download + unpack + eval) takes less than 8 minutes for me. rm -rf ~/.cache/nix/eval-cache-v5; nix build -f https://github.com/NixOS/nixpkgs/archive/refs/heads/master.tar.gz gnumake
immediately after this evaluates in 3.2 secondstime nix profile install nixpkgs#ncdu
downloads/unpacks/evals in under 8 minutes.- That being said, if you claim that eval takes an hour for you, I'm sorry, but I can neither reproduce that nor advise what should you do about it, besides maybe suggesting you acquire a wakelock and/or follow guidance from https://dontkillmyapp.com
- IDK who claims "proot doesn't cause overhead",
Apologies, I misread the comment here: https://wiki.termux.com/wiki/PRoot
What it actually said was chroot has no overhead, which makes more sense WRT the performance seen here.
If proot fundamentally tanks IO, then fair enough, strace
indicates most of the time spent on newfsastat
calls.
6.
time nix profile install nixpkgs#ncdu
downloads/unpacks/evals in under 8 minutes.
8 min is quite reasonable and closer to what I'd expect on a phone. If I can get it there I'd be happy with that.
7. if you claim that eval takes an hour for you, I'm sorry, but I can neither reproduce that nor advise what should you do about it, besides maybe suggesting you acquire a wakelock and/or follow guidance from https://dontkillmyapp.com
Battery saver is off, and "allow background usage" is on. Though, I'd be surprised if doze mode was involved for a foreground app while the screen is on?
Also, I think the link provided is outdated, as I don't see any option for "battery optimization" as mentioned in https://dontkillmyapp.com/google
FWIW I use a Pixel 7 with the current (June) release of android 14.
Other helpful info might be that, the initial bootstrap of nix-on-droid takes about 3 min. I'll run the time
command on the fresh install for comparison.
Initial bootstrap definitely involves more than eval'ing a hello, so something is killing the performance afterwards. IDK whether an app can run out of Doze credits while being in foreground, but try (quoting from same device):
- App info - Battery usage - Unrestricted
- Notifications - Nix - Expand - Acquire wakelock
I'm not super knowledgeable about Nix yet, but I'll try to provide useful and relevant information, let me know if I miss anything and I'll follow up with that info.
Device Info + App info
I'm using a Pixel 5 on LineageOS 21, rooted with KernelSU (probably irrelevant but thought I should mention just in case).
The app's battery restriction setting thing is set to Unrestricted and I always acquire a wakelock via the notification when using it. I'm running it with the screen on, app in the foreground, and charger plugged in. Of course that's not how I always use it, but for right now, since I've been configuring things while switching over from Termux, this is how I've been using it
I didn't time it perfectly, but seem to have encountered the same behavior as the person who created this issue. SIGINT did not work for me either, I assume since it was waiting on IO or something along those lines? However, I wanted to add that I don't experience the problem anymore now that I've copied over my dotfiles from my PC. Here is the (I think) relevant part of my config:
nixOnDroidConfigurations.default = nix-on-droid.lib.nixOnDroidConfiguration {
modules = [
{nix.registry.nixpkgs.flake = nixpkgs;}
...
];
};
I put ...
to denote that I omitted the rest of the stuff in that block, because I don't think it matters, but if it does, let me know and I'll add it back. I use this on my PC because I use nixpkgs unstable and don't wanna have to redownload ~40MB every now and then if I run a quick nix shell nixpkgs#foo
. As I said before, I'm not super knowledgeable about Nix yet, but I think this is what was mentioned in this comment:
Once a specific version of nixpkgs has been evaluation cached then things get a lot faster. Running just
nixpkgs#<whatever>
is generally a bad idea as nixpkgs updates too quickly. Using a specific revision is faster as is just adding/removing packages from a flake.
Basically this comment is a "I also experience this issue, here's the config that seemingly makes the issue disappear", and hopefully the info helps. Again, let me know if I should add any more details.
@Arian04 Could you time the commands from #374 (comment) and post more objective results?
TL;DR: I timed those commands with my current config, then commented out the lines that I felt had "fixed" the issue, then switch'd into the new config, then re-timed those commands, and got perfectly reasonable results both times, so sorry about the noise. Either I was super impatient before and interpreted the times as like 5x longer than they were, or I'm just no longer able to reproduce the issue :(
Testing details
Testing with current config
nix build -f https://github.com/NixOS/nixpkgs/archive/refs/heads/master.tar.gz hello
- took 3:17, was just showing
[41.7 MiB DL]
for a little over 2 minutes of that time I believe
rm -rf ~/.cache/nix/eval-cache-v5; nix build -f https://github.com/NixOS/nixpkgs/archive/refs/heads/master.tar.gz gnumake
- running this immediately afterwards evaluates in a little under a second
time nix profile install nixpkgs#ncdu
- takes 4 seconds
Testing with modified config
- After running the previous test I ran
nix-collect-garbage
, thennix-on-droid switch --flake
'd my config with{nix.registry.nixpkgs.flake = nixpkgs;}
commented out. Then re-did those tests with the following results. - also I removed a line setting NIX_PATH in my home-manager config that I think was doing a related thing maybe?
- I actually had to
nix profile rollback
twice because of weird errors while trying to switch, but yea.
nix build -f https://github.com/NixOS/nixpkgs/archive/refs/heads/master.tar.gz hello
- same as before
rm -rf ~/.cache/nix/eval-cache-v5; nix build -f https://github.com/NixOS/nixpkgs/archive/refs/heads/master.tar.gz gnumake
- same as before
time nix profile install nixpkgs#ncdu
- takes 3:29
Sorry, I missed responding to this earlier.
- App info - Battery usage - Unrestricted
EDIT: this is indeed set.
Original comment:
As mentioned, I still don't see this option in battery settings:
- Notifications - Nix - Expand - Acquire wakelock
I did try this and it says "wakelock held" in the notification, but the screen does still shut off after some time (1 min in my case, which is my display timeout) - is that expected?
Beyond this, I've run time nix build nixpkgs#hello
; the vast majority of the nearly 15 min it appeared to be paused on [41.9 MiB DL] evaluating derivation 'flake:nixpkgs#hello'
due to the background I/O. The following was the output:
time nix build nixpkgs#hello
real 13m45.525s
user 0m32.591s
sys 4m43.273s
Leaving the phone plugged in vs unplugged did not appear to significantly impact the time spent here, over multiple runs this was the best time I saw.
I'm not sure if this time can be improved, if this is fundamentally a proot limitation then I'll close this out.
As mentioned, I still don't see this option in battery settings:
I noticed the option got moved around a bit in an Android update. you can find it if you click on the preference item itself, but not the toggle switch on the right. so basically click around where the actual "Allow background usage" text is, and it'll show you the submenu "underneath" it, if that makes sense
Ah, yes it's set to unrestricted already.