Windows Subsystem for Linux as a theoretical chemist

Windows Subsystem for Linux. Why ?

In my lab a lot of people use Linux whereas others use macOS. It’s because a lot of programs used in quantum chemistry were designed for Unix type OS. Consequently in order to work decently we need a Unix type OS. We use for example SSH, python, fortran compilers and a lot of quantum chemistry software designed for Unix. I think that’s because they are console programs and Unix type OS so they are suitable for console usage. There are a lot of great shells like bash compared to Windows and its cmd (even though I can admit that Microsoft has made a big effort with Windows PowerShell) I am still not convinced by the graphical user interfaces present on Linux. There is also a big lack of compatibility with many common GUI programs. I have also experimented drivers issues with printers and GPU.

This is how I schematize things if I must make OS ranking.

So I understand why many of my colleges choose macOS. It seems to be the best of two worlds (I also use macOS in my office). And it’s almost the only way for that (Without dual-boot or virtualization)… Until 2016. Since this year there are also a good way for that included in the core of Windows 10.

Bash on Ubuntu On Windows

At the build 2016 conference, Kevin Galo in the head of Windows developer platform, presented a way to run native Linux binaries and Linux distributions directly on Windows 10.

At this point this technology was called Bash on Ubuntu on Windows (You can watch the video of the keynote bellow). At this time only Ubuntu distribution was available.

A year later this technology went out of beta with major improvements and supports of any Linux distributions and the name “Bash on Ubuntu On Windows” was replaced by “Windows Subsystem For Linux” (a.k.a WSL).

The environment

With WSL you can have a Linux command line interface with your favorite distribution and your favorite shell directly inside the Windows console host (or any alternative console available on Windows). Your Windows hard drives are automatically mounted (typically on /mnt/<driveleter>) and your Windows programs are also added to the path. And so you can run native Linux programs and Windows programs. inside a true Linux environment and you can interact with your Windows files in it.

It’s magic ! No it’s Linux => Windows syscalls conversion.

A syscall is the way that a program interact with the kernel to ask it for something (Like open a file, write on a file, request memory allocation…). Because the binaries are Linux binaries. They attend to interact with a Linux Kernel. So how does it work ? This is thanks to the magic of a technology called pico processes. Pico processes are a category of processes handled by Windows where we have a minimal process. i.e. no Windows dll has loaded, also a pico provider which is a kernel-mode driver is placed between the Windows kernel and the pico process. It provide the interface between the kernel and the pico process. So the WSL pico provider (lxss.sys) provide all the kernel interface to convert Linux syscalls to Windows kernel equivalent syscalls. For a Linux process inside a Windows pico process the kernel is a true Linux kernel and the provider ensures compatibility.

Pico process overview

Syscalls conversion

WARNING

Not all syscalls are implemented so There is no guarantee that a specific program works !

A well known example is the docker daemon.

How to install it on your machine

NOTE

For the installation process I added direct links to Windows 10 settings App pages and Microsoft store pages when its relevant to simplify your installation process. So this links work only on Windows 10.

prerequisites to install WSL

• A 64Bits operating system (x86-64 or arm64)

• Windows 10 fall creator update or greater

You can check your system configuration in the About page of the settings App. We must have the above configuration.

If you have a 64Bits OS but an older version of Windows 10 you must update your OS. Even if you have a compatible version, in order to have a more stable WSL and all the new features, it’s recommended to use the latest version of Windows 10. You can use Windows Update from the Windows Settings app to update your Windows 10 version for free.

Check for update now

Install the Windows Subsystem for Linux optional feature

WSL is distributed as an optional feature of windows 10. You can install it in different ways. Each following section presents a way to install WSL. You can choose the most convenient way for you. It doesn’t matter.

Warning

To install a Windows feature an administrator account is required. If you are not the administrator of your PC you must ask him to install it.

By command line

Generally people who want to install WSL are comfortable with command line interfaces. So it’s may be the simpliest way for you.

With PowerShell

Open a PowerShell console as administrator and type the following command

Enable-WindowsOptionalFeature -Online -FeatureName Microsoft-Windows-Subsystem-Linux

Reboot the PC

With DISM

Open a PowerShell or cmd console as administrator and type the following command

Dism /online /Enable-Feature /FeatureName:Microsoft-Windows-Subsystem-Linux

Reboot the PC

With the chocolatey package manager

Open a PowerShell or cmd console as administrator and type the following command

choco install Microsoft-Windows-Subsystem-Linux --source Windowsfeatures

Reboot the PC

By graphical user interface

You can find the Windows Subsystem for Linux as an optional feature on the control panel.

Open Apps and features from the Windows Settings app and click on Programs and Features => Turn Windows features on or off

TIP

To go directly to this window.

Press Windows Key + R

Write optionalfeatures on the run box and press enter

check Windows Subsystem for Linux and click on the ok button.

Follow instructions.

Download distribution from the store

Warning

At this point the Windows Subsystem for Linux optional feature may be installed.

An administrator account is no longer required.

Now go to the WSL page on the Microsoft store to chose your distribution and download the associated app.

TIP

You can install and use multiple distros at the same time

After the app installation, launch it. The distribution was present as taball file inside the app and at the first launch it was automatically extracted and the app ask you to create a user account. Set your Linux username and password (It can be different to your Windows user name and passwords.)

After you can play with the Linux distribution you have chosen and you can install and run programs from the Linux world in it exactly like a true Linux OS.

if you are not convinced that this is a real Linux environment you can type the following line of code on the distribution console

file /bin/bash

you can see

/bin/bash: ELF 64-bit […] for GNU/Linux […]

so bash is in the Executable and Linkable Format, the format used by Linux OS and many BSD OS for executable files and it’s compiled for the Linux kernel. So bash (and the other binaries of the distribution) are the same Linux binaries as true Linux installation.

you can also try

uname -a

you can see something like this

Linux <pc-name> 4.4.0-<Windowsbuldversion>-Microsoft #1-Microsoft <Windowsbuilddate> x86_64 x86_64 x86_64 GNU/Linux

You can see that is not a Linux generic kernel but it’s considered as a Linux Microsoft kernel. This is the way that a Linux program can see that is currently running on WSL and not a real Linux OS. A Linux program can also get the Windows 10 build number by this mean.

The Linux distributions distributed on the Microsoft Store are published by it’s maintainers and are almost the same as the ISO distributed versions (But without graphical programs by default and for Kali Linux without pentest programs) we can download all programs supported by a distribution with its packages manager thanks to programs being downloaded on the same repositories. Of course you can also compile programs from sources, or download Linux binaries directly from the web and run it on WSL. It’s also possible to compile from source on WSL and distribute the binary on the Web (But by precaution and may be excess of caution I recommend you to test it on a real Linux OS).

Useful resources

Official installation guide

Launch WSL

From graphical user interface

• Launch the distro app you installed previously to open a console window with this distro.
• Since October 2018 update it’s also possible to launch the default distro from Windows explorer. For that press shift key and right click. Then click on Open a Linux Shell here

From command line

To launch WSL from windows command line there are multiple possibilities

• Launch a specific distro in the Linux home directory

  <distroname>
  

  or

  wsl ~ -d <distroname>
  

• Launch the default distro in the current directory

  wsl
  

• Launch the default distro in the Linux home directory

  wsl ~
  

• Launch the the default distro with the bash shell (and override the default shell) in the current directory

  bash
  

• Launch the the default distro with the bash shell (and override the default shell) in the Linux home directory

  bash ~
  

• Launch a specific distro in the current directory

  wsl -d <distroname>
  

For more info about how to run WSL and learn how to configure the default distro (If we have multiple distros installed) check the documentation here

Windows Linux interoperability with WSL

Windows drives

• You have access to your Windows fixed drives by default (typically on /mnt)

• You can mount other Windows drive like USB key with the Linux classical mount command like this

  sudo mount -t drvfs F: /mnt/f
  

  TIP

  To mount a Windows drive and set owner and group to your user and your main group run :

  sudo mount -t drvfs F: /mnt/f -o uid=$(id -u),gid=$(id -g)
  

  A lot of options are available

  More info here

Run executable

• You can run a Windows binary from WSL if it is on the Windows Path like this (The name of the Windows executable ending with .exe)

  cmd.exe
  

• You can run Linux command line from Windows console as follows

  wsl git clone https://github.com/username/repo
  

  or like this

  <disroname> run git clone https://github.com/username/repo
  

  See the official guide to invoking WSL

• You can also combine Windows and Linux command like this For example to print Windows version with Linux lolcat (Rainbow colors) in cmd you can type

  ver | wsl lolcat
  

More information here

Share environment variables

You can share environment variable between Linux and Windows. Further info here.

So with a good configuration you can do this:

  cd $USERPROFILE

And go to your Windows user directory.

More interop and other WSL features come twice a year with each new version of Windows 10.

Change distribution launch behavior

You can configure the wsl functionalities of a distro in the in the /etc/wsl.conf file

More info here

WSL vs other technologies

An overview of WSL and its alternative which I have tested on windows. I have not really used Cygwin in real situations, therefore I have no idea of it’s performances.

Conclusion

Even if a bare-metal Linux install (on the hard disk) is the best choice for those who want to run Linux binaries only, those who dislike Linux GUI or have drivers problem on Linux; but want a Linux command line interface or want to run Linux binaries, WSL is probably an excellent choice. So in this case, for Windows users, WSL is a good alternative to switch to macOS. But even if details, its structure differs from Linux, because macOS is a Unix-like OS the majority of Linux software are macOS compatible and It remains an excellent choice.

Useful resources

Official documentation

Learn About Windows Console & Windows Subsystem For Linux (WSL)

WSL in Windows command line blog

Please read the documentation above before asking me questions !! Thanks.

If you think that this blog-post can be improved please post a comment below.–>

WSL on Twitter

@craigaloewen Program Manager of WSL

@richturn_ms Senior Program Manager of the Windows console

#WSLinux

Update

WSL2

Since Windows 10 May 2020 update (v2004) WSL2 was released and its architecture is very different (and include a true Linux kernel) and its performances and features are increased see the documentation. or the following deep dive video from Microsoft Build 2019.

GPU compute

Since Windows 11 or Windows 10 21H2 WSL2 now support GPU compute

WSL GUI apps

Since Windows 11 WSL2 now support Linux GUI apps