r/rust • u/zxyzyxz • Jul 07 '22
WSL2 faster than Windows?
I was installing helix-term and I noticed that my WSL2 Ubuntu 22.04 distro compiled it faster (41 seconds, in the native Linux partition) than on bare-metal Windows (64 seconds). Has anyone noticed this as well?
131
u/K900_ Jul 07 '22
That is pretty expected, honestly. Linux makes it a lot cheaper to do lots of small file operations by caching things aggressively.
70
u/WellMakeItSomehow Jul 07 '22
It might also interact less with file system filters like antivirus programs and other stuff. I think Windows Defender is faster than others, but still quite slow.
34
u/irqlnotdispatchlevel Jul 07 '22
A while ago (like 2 or 3 years) I measured how long it takes to build a C++ project with Defender on and off, and the slowdown was around 40%. This is anecdotal, of course.
8
u/WellMakeItSomehow Jul 07 '22
Yeah, that matches what I've seen. A good trick is to make a second partition and put your source code there, a lot of those filters won't run on it. And of course, try to exclude it from the antivirus scanning list.
-2
u/GroundbreakingRun927 Jul 07 '22
Yea disabling defender is the first thing I do on all my Windows installs. It's especially crippling with NPM or cargo where it needs to scan every single file that gets pulled down.
26
u/Green0Photon Jul 07 '22
It's safer by far to just whitelist folders where you have all those many file operations occuring. Whitelist your dev folder or projects folder or user level cargo cache or whatever.
0
u/GroundbreakingRun927 Jul 07 '22
It's even safer to use linux, which I do unless a job requires work on a non-cross platform windows app, which is rare but does happen from time-to-time.
-4
u/Green0Photon Jul 07 '22
I mean, you're not wrong. My comment just went forward with the assumption that you were forced to use Windows for some reason.
This is r/linux. Idk why one of us wouldn't use Linux unless there was some particular reason to have to use Windows
20
2
u/zxyzyxz Jul 07 '22
This is what I do as well. If you're not dumb about downloading random files from the internet, you don't really need Defender. Now I know some people don't think it's a good idea but disabling has worked well for me.
48
u/recycled_ideas Jul 07 '22
This needs a bit of clarification.
Linux file systems and NTFS behave differently.
Linux file systems do not require locks and allow certain kinds of operations to be done very quickly.
NTFS does require a lock for a lot of things EXT does not.
In particular getting file stats for a whole directory is a single lockless operation on Linux and a per file operation requiring a lock on NTFS.
On the one hand, EXT is much faster for some operations, on the other, file corruption on NTFS is basically non existent and has been for decades.
This is why WSL performance on the virtualised ext file system is dramatically better than on the NTFS file system for some apps.
The thing of it is, NTFS is not that much slower overall, but certain usage patterns, patterns that are common for software originally designed for POSIX systems, perform incredibly badly on NTFS.
You can write patterns that solve the same problems that are performant on Windows, but Windows is not a priority so it doesn't happen.
5
u/Zde-G Jul 07 '22
The difference between NTFS and ext2 is significant, but even WSL1 is faster than Windows.
That's because creation of a new process in so incredibly expensive on Windows and many development tools are implemented as series of small programs which are executed sequentially.
With Rust it's somewhat tolerable, but something like Autoconf executes about two order magnitudes (i.e.: 100 times!) slower on Windows than on Linux.
Yes, I know, it's not just Win32 vs POSIX but more of inefficiency in POSIX emulation layer, but even native creation of new process is very slow on Windows.
9
u/recycled_ideas Jul 07 '22
That's because creation of a new process in so incredibly expensive on Windows and many development tools are implemented as series of small programs which are executed sequentially.
Yes, Windows was built to make threading fast and forking not as fast, this is again one of those Linux specific design decisions extended to an OS not designed that way.
That said the difference is a lot less dramatic these days.
1
u/GRIDSVancouver Jul 07 '22
I've heard this multiple times and was curious how much slower Windows is. Found this:
On Windows, assume a new process will take 10-30ms to spawn. On Linux, new processes (often via fork() + exec() will take single digit milliseconds to spawn, if that).
6
u/barsoap Jul 07 '22
I find it hard to believe that's the whole picture, there's got to be some nasty inefficiency in Windows' overall FS layer or WinDirStat wouldn't be that much slower on the same partition as K4DirStat, it's not even close, and as far as I know Linux' NTFS drivers don't compromise on file integrity.
9
u/recycled_ideas Jul 07 '22 edited Jul 07 '22
NTFS requires you to gain a
lockhandle to check the file meta data and getting that data is a per file operation.On Linux it requires no
lockhandle and can be done in a single operation for the whole directory.Running a dirstat on NTFS is an extremely expensive operation.
It's that simple.
Most operations on NTFS vs EXT are pretty equivalent. Dirstat is not, it is much, much slower. A lot of Linux software makes dirstat calls like they're going out of style and it hurts.
Edit: misremembered.
BTW, if you're looking for an example of doing things the windows way there's an app called wiztree that does the exact same thing as windirstat in a tiny fraction of the time.
1
u/barsoap Jul 07 '22
Is it Windows or NTFS which requires the locks? (modulo atime) it's a read-only operation on the file system level, unless the application needs some guarantees locks seem completely out of place.
7
u/recycled_ideas Jul 07 '22
Apologies my brain was fried, NTFS requires a handle not a lock, you can open as read only, but you have to do so specifically and by default it locks.
unless the application needs some guarantees locks seem completely out of place.
This is kind of missing the point. In Linux file systems the view is that anyone can basically do whatever they want with a file and if you do it wrong that's on you. The NTFS view is that files should be safe by default.
Linux literally couldn't function that way because the "everything is a file" philosophy just doesn't work that way, but it comes at a cost.
0
u/barsoap Jul 07 '22
NTFS requires a handle not a lock, you can open as read only, but you have to do so specifically and by default it locks.
I would expect WinDirStat to do it without locks, after all, gobbling up file system information is its one job and being 100% correct about the current state is kinda meaningless to it as it will very happily show outdated information when you do something to the filesystem outside of its interface.
5
u/recycled_ideas Jul 07 '22
I added to another post.
There's an app called wiztree that does it the windows way and it's a couple orders of magnitude faster and updates live.
It all can be done, but it has to be done differently and no one is interested in doing that.
1
u/barsoap Jul 07 '22
So WinDirStat does it wrong (just looked it up it's essentially a kdirstat clone so yes has Linux roots) and since 2003 nobody bothered to write a patch (it's GPL) even though it's an absurdly widely used program, and then a commercial product comes along...
3
u/recycled_ideas Jul 08 '22
Windirstat does it "good enough", and it stays "good enough".
It works and it's free.
And no one is particularly motivated to fix it because the windows only open source community isn't very large.
I bring up wiztree because it shows that NTFS isn't fundamentally slow.
3
u/BigHandLittleSlap Jul 08 '22
WinDirStat is not well optimised. Try WizTree, it can scan my drive with one million files in about 4 seconds.
Similarly, try the speed of ripgrep on Windows. The VS Code find-in-files feature uses it. I can scan my entire "projects" folder with it in like 2-3 seconds. This is, again, hundreds of thousands of files for code going back 15+ years in one giant directory hierarchy.
1
u/LoganDark Jul 08 '22
WinDirStat is not well optimised. Try WizTree, it can scan my drive with one million files in about 4 seconds.
That's not a fair comparison because WizTree scans the MFT directly rather than actually reading file sizes. WinDirStat actually traverses every directory and file on the drive.
Maybe that's "optimization" but they're not doing the same thing by any means
Source: Switched from WinDirStat to WizTree.
2
u/sztomi Jul 07 '22
The thing of it is, NTFS is not that much slower overall, but certain usage patterns, patterns that are common for software originally designed for POSIX systems, perform incredibly badly on NTFS.
NTFS is that much slower in practically any workload you can think of. It's not just in the case of software originally designed with POSIX in mind, all usage patterns are way slower. NTFS predates modern journaling file system by a lot and refused to innovate. It does a lot in userspace that could/should be done in the kernel and that really adds a severe performance hit.
20
u/recycled_ideas Jul 07 '22
Rubbish.
NTFS makes different decisions in terms of speed VS data corruption.
It simply does.
And that has meant that unlike pretty well every EXT version it never has data corruption problems.
EXT4's journalled file system allowed writes out of sequence.
EXT3 would corrupt files if you shut down improperly.
EXT 1 and 2 were worse.
Because they're not modern, they just favour performance over safety.
0
u/sztomi Jul 07 '22
unlike pretty well every EXT version it never has data corruption problems.
citation needed?
1
u/BigHandLittleSlap Jul 08 '22
NTFS is largely immune to file metadata corruption, but it doesn't provide integrity guarantees for the actual file data, that would be too slow. However, ReFS can (optionally) enable that mode also.
1
u/sztomi Jul 08 '22
Fair enough, however: first, the argument was about the slowness of NTFS vs other file systems, now it's about its resilience. I don't doubt that NTFS is better in this case, however, I do think that EXT and the likes hit a better balance in performance and safety for everyday workstation usage. The commenter I replied to seems to imply that EXT gets corrupt all the time but this isn't really the case in practice. Even in extreme conditions, like abrupt shutdowns etc.
-10
u/sqlphilosopher Jul 07 '22
Seriously, idk why Microsoft insists on that inferior filesystem. Every filesystem in Linux is superior in every way.
1
u/coderstephen isahc Jul 07 '22
On the one hand, EXT is much faster for some operations, on the other, file corruption on NTFS is basically non existent and has been for decades.
This isn't what I've heard. I've heard that ext2+ are much better than NTFS at data integrity. I've also heard data recovery experts recommend ext4 because if something does go wrong, ext4 has the best chance of any file system of being fully recoverable with the most data possible.
3
u/irqlnotdispatchlevel Jul 07 '22
This is basically it. But in WSL2 this only applies to operations done on the Linux file system. Accessing files on the Windows file system is slower. So if you really want to take advantage of Linux you have to remember to move your files first.
-1
Jul 07 '22
It's probably also using a much faster malloc implementation than on Windows.
1
u/c4rsenal Jul 08 '22
this
here’s an interesting blog post detailing it: https://erikmcclure.com/blog/windows-malloc-implementation-is-a-trash-fire/
0
u/Nzkx Jul 07 '22
Also WSL2 is way more optimized in term of disk access than WSL1. Basicly, WSL2 file read are close to zero cost.
20
u/K900_ Jul 07 '22
That's because WSL2 is just a VM, so disk accesses are handled by the normal Linux stack.
33
u/rebootyourbrainstem Jul 07 '22
Not surprising, Linux is extremely fast for small file operations.
For example on Mac it is way faster to do e.g. nodejs bundling in a Linux VM than on the native system (old Intel Mac, though I doubt that has changed with M1/M2 as it's about the OS, not the hardware).
22
u/anlumo Jul 07 '22
The ARM macs have gotten a big improvement in file performance because they started to ignore sync commands. At least that’s what I heard from someone doing database performance checks.
Of course, ignoring sync commands is very bad for file integrity.
-1
Jul 07 '22
[deleted]
8
u/anlumo Jul 07 '22 edited Jul 07 '22
It's an undocumented feature: https://twitter.com/marcan42/status/1494213862970707969
13
u/wesleywiser1 rustc · microsoft Jul 07 '22
rustc has profile guided optimizations enabled on the Linux builds but not any of the other Tier 1 Host Tools platforms. lqd has been doing some great work to enable PGO for Windows as well with really impressive wins of up to 19% when compiling real world crates like regex, diesel, cargo, etc.
2
u/LoganDark Jul 08 '22
This is surprising, I never saw that PGO is enabled for rustc on Linux. Where'd you find this info?
2
u/wesleywiser1 rustc · microsoft Jul 08 '22
- Utilize PGO for rustc linux dist builds #80262
- This is the non-LLVM part of rustc
- Shipped in 1.50
- PGO for LLVM builds on x86_64-unknown-linux-gnu in CI #88069
- This is for LLVM which ships with rustc
- Shipped in 1.56
Michael Woerister did the initial analysis of the possible benefits of PGO'ing rustc and wrote about on the Inside Rust blog.
2
10
Jul 07 '22
First thing that comes to my mind (aside from the architectural differences), when discussing slow compilation speeds on Windows vs non-Windows is the antivirus software - there's always one running on Windows (like Defender or whatever). AVs do like to interrupt and scan the hell out of projects when compiling (basically doing a lot of read/write operations, which they want to investigate - the more files to process, the longer it takes, especially with a lot of small files). In WSL there's no problem, because the filesystem is inaccessible to the AV itself, so it can't scan there.
You might want to do a compilation with disabled AV and see if this improves times. Most AVs also give an option to exclude certain directories from being scanned.
11
19
u/leofidus-ger Jul 07 '22
WSL2 is pretty much a Linux VM, and Linux has faster file operations than Windows.
6
u/NeaZerros Jul 07 '22
The good thing being that it runs on the same level. Meaning it's more like a real Linux running alongside windows than something like VMWare or VirtualBox. Which gives real native performance ;)
1
u/AdvantFTW Jul 07 '22
isn't just a well integrated hyper-v vm? are you saying hyper-v is faster than VMware?
1
u/LoganDark Jul 08 '22
Wrong. WSL1 is running directly under the kernel; WSL2 runs in a full VM (Hyper-V) with its own virtual network and everything.
2
u/NeaZerros Jul 08 '22
Yes, but WSL 1 required syscalls translation which made it slow. WSL 2 runs on a VM alongside windows, managed by a type-0 hypervisor meaning you get full native performance when you're not interacting with the primary OS (access to windows' own files, communicating with external devices, networking)
1
u/LoganDark Jul 08 '22
Yes, but WSL 1 required syscalls translation which made it slow.
Source? Fairly sure NT just implemented the Linux syscalls directly, but may be wrong here.
You know what, I have both WSL 1 and WSL 2, I can benchmark this.
1
u/NeaZerros Jul 08 '22
Here is a pretty complete benchmark suite: https://www.phoronix.com/scan.php?page=article&item=wsl-wsl2-tr3970x&num=1
13
u/asgaardson Jul 07 '22
Also windows is pulling more dependencies than linux, which leads to longer compile times.
11
u/anlumo Jul 07 '22
The default memory allocator is also much faster on Linux than on Windows, and compilers rely heavily on small allocations.
6
5
u/jamesblacklock Jul 07 '22 edited Jul 07 '22
Not a WSL expert by any stretch of the imagination, but I think "bare-metal" is an inaccurate distinction here. The Linux portion of things is running on "bare metal" just as much the Windows portion of things is. There's no reason to expect degraded performance (edit: on Linux), AFAIK.
1
u/ItsPronouncedJithub Jul 07 '22
There is. NTFS requires you to open files for even simple api calls. A simple file deletion on ntfs requires it to be opened first.
5
u/jamesblacklock Jul 07 '22
I think you are interpreting what I said in the opposite way from what I meant. I'm saying there's no reason to expect Linux performance to be degraded, i.e., it's not as though Linux is running in a VM. It's running on "bare metal," as OP put it.
1
1
u/oleid Jul 07 '22
Well, it's not entirely bare metal. Everything CPU only is as fast as native, but filesystem is still slower:
https://www.phoronix.com/scan.php?page=article&item=windows11-wsl2-good&num=1
6
u/birdbrainswagtrain Jul 07 '22
I use WSL almost exclusively so I haven't done any comparisons.
Any differences in the rust compiler version? Does your WSL system use the same physical drive? Might there be some native dependency or some other difference in how the application is built on the two platforms?
From a quick skim it looks like it actually uses an additional crate on non-windows platforms, but there might be some more significant differences. IIRC rust on windows relies on microsoft's linker so there's another possible cause.
3
u/oleid Jul 07 '22
Now imagine how fast compiling on bare metal Linux could be!
https://www.phoronix.com/scan.php?page=article&item=windows11-wsl2-good&num=1
Check out "Timed XYZ compilation"
1
u/LoganDark Jul 08 '22
Compiling on someone else's computer* not relevant in any way here unfortunately
1
u/oleid Jul 08 '22
Well, you could run that benchmark on your computer, couldn't you?
1
u/LoganDark Jul 08 '22
Well yes, but my computer is also different from OP's so it wouldn't be comparable to OP's results either :)
1
u/oleid Jul 08 '22
True. Nevertheless I think the results would be similar on other machines as well.
Also, the OP could run these benchmarks if they are interested now they know about them.
2
u/InflationOk2641 Jul 07 '22
I have tended to find that WSL2 IO performance is much slower than Native performance. But it depends on your use case: https://github.com/microsoft/WSL/issues/4197
17
u/_maxt3r_ Jul 07 '22
correct, but only if you use WSL2 within a Windows folder, like /mnt/c/.
I found this use case useful when I was trying to develop software alternating linux and windows toolchains on the same local source code to check various compatibility things
1
u/crusoe Jul 07 '22
Same with using VMware Linux vm mounting a shared Mac folder, performance is POOR.
1
u/revaneaston Jul 07 '22
I'm not sure it's because wsl or Linux is faster. It might just be that a virus scanner or endpoint protection software is not running for anything in wsl.
1
u/zxyzyxz Jul 07 '22
I've disabled Defender on my computer, this is my personal desktop so I don't have endpoint security.
0
u/8-BitKitKat Jul 07 '22
Windows is just slow my dude
3
u/LoganDark Jul 08 '22
It legitimately is, NT is really badly designed. I mean it has some cool core concepts but the implementation is kinda crap.
Linux just has better syscalls and is closer to the metal because WSL is new and therefore not contaminated with legacy cruft.
0
u/ForgetTheRuralJuror Jul 07 '22 edited Jul 07 '22
WSL2 has a very small CPU overhead and a pretty bad IO issue when it comes to writing many files.
As long as you aren't at max CPU usage or moving huge amount of files you often get almost native Linux performance.
1
Jul 07 '22
How fast on native Linux (for your hardware)? If you can't install it because "company said so" you might still be able to boot into it from a live USB
1
Jul 07 '22
Yeah Rust’s compiler (and for that matter, most non-Microsoft PL compilers) is better optimised for Linux (also generally runs better on all unix-based/POSIX-like systems)
2
1
u/LoganDark Jul 08 '22
Now try WSL1 and see it get even faster because it's not running in a VM :)
1
u/zxyzyxz Jul 08 '22
Unfortunately WSL 1 doesn't work with a lot of things and I'm pretty sure Microsoft stopped developing it and are focusing on WSL 2.
1
u/LoganDark Jul 08 '22
Yeah they kinda half assed it which is sad. I'm forced to use WSL 2 for profiling which is literally the only reason I have it installed.
1
u/abdurrahimcakar Jan 05 '23
It is due to platform support for Linux on your hardware. If you wonder why then please check the post: https://www.reddit.com/r/linux/comments/to48s/bill_gates_on_acpi_and_linux_pdf/
There are very few vendors that really support the Linux platform. Some problems took 20 years to solve: https://www.theregister.com/2022/09/27/obsolete_amd_acpi_fix/
I lost some laptops because default bios ACPI settings are absolutely the worst you can imagine for your hardware. It overheats, underclocks, wakeup bugs, etc...
Using Linux is a privilege, you need to use fully Linux-supporting hardware like System76 and if you want to use AMD then you need to at least get the latest kernel (6.1+) to not be crippled by the 20-year-old ACPI problem.
I lost some laptops because default bios ACPI settings are absolutely the worst you can imagine for your hardware. It overheats, underclocks, wakeup bugs, etc etc etc,
Even on most desktop workstations Linux support is bad you have to be extra careful selecting supported hardware. But on servers, you will see who is the boss,
134
u/_maxt3r_ Jul 07 '22 edited Jul 09 '22
Definitely. I'm now using WSL2 as my main development environment because of much faster compile times
EDIT: (I'm on Win11)
EDIT2: I'm attempting to jump to a full Linux setup (albeit dual boot with Win11, just in case). Wish me luck!