In May 2020, Microsoft released another update number 2004. Among the innovations, I was interested in WSL2-Windows Subsystem for Linux 2. Because the name has the same word – Linux, which many people have a chill on the skin. In fairness, it should be noted that Linux causes such feelings quite deservedly. But, WSL2 is not the case! Therefore, if you are an enthusiast in AI development and use Windows, I recommend that you pay attention to this tool, it deserves it.
When Windows is a matter of habit
Most of the professional financial software is developed for the Windows operating system. This is why few financiers are familiar with the Linux operating system. At the same time, if you have taken machine learning courses or read books about Keras or TensorFlow, you have probably come across a recommendation to use the Linux operating system in order to avoid all sorts of problems or misunderstandings with the libraries used and their compatibility.
This recommendation is related to the fact that almost all libraries used in machine learning and for processing large data sets are originally written for Linux. And in the process of adapting them to Windows, errors may occur.
From my personal experience, I can say that if the required library is not installed or gives an error in the Windows operating system, this is not the worst case. It is much worse if the library will work, but not as it should, and it will be difficult or simply impossible for you to check it. As a result, not getting the expected result, you can turn away from the working idea.
But what if you have a high-performance computer with a powerful GPU and Windows operating system, and you want to use AI libraries in your work without the risk of getting unpredictable results? The solution to this problem is to install the Linux operating system next to Windows. Ubuntu is the most common operating system in the AI environment. it is also listed in the TensorFlow and Keras installation manuals (previously, these libraries were separate). However, without the help of a familiar system administrator, I could not correctly configure the Ubuntu operating system to get maximum performance from the first time.
Having installed and configured Ubuntu to solve AI problems, we face another problem – in the process of training a neural network, which can take several days or even weeks (for example, in the case of optimization), we do not have the opportunity to use the usual software products, because they are in a different operating system.
WSL2 can be a solution that allows you to combine the use of familiar software products in the Windows environment and the reliability of AI development in the Linux environment.
Features, limitations, and performance
I consider WSL2 to be a promising solution for the simple reason that at the moment the Linux kernel version is 4.19.104-microsoft-standard and it does not support the GPU, but it fully works in multithreaded mode. And as you know, AI training on the CPU is much slower than on the GPU, which is not always acceptable for my tasks. If it is acceptable for you to train models on the CPU, then WSL2 can become a full-fledged alternative to a separate Linux installation.
Installed in wsl2 Linux (the Microsoft store has Ubuntu, Kali, Fedora, and others) is like an application that can be easily removed and re-installed. This allows you to learn Linux terminal commands and system settings, experiment with the virtual environment for TensorFlow, and much more without any worries about completely reinstalling the operating system. At the same time, you get a stable development environment for AI and simultaneous access to familiar Windows applications.
To compare the performance of Tensorflow in Windows and WSL2, I chose the convolutional neural network for recognizing a set of numbers MNIST. My test configuration: i7-8700k, DDR4 4000Mhz, virtual environment created in MiniConda, TensorFlow 2.3 and JupyterLab installed using pip. Full training results: Windows – 2 min 15 sec, WSL2-1 min 39 sec. As you can see, the learning rate of the AI model on WSL2 was 27% faster.
Installing Miniconda and Tensorflow in WSL2
The process of installing WSL2 in Windows 10 is described in sufficient detail on the Microsoft website. I will only add that before installing WSL2, it is recommended to activate virtualization in the tinctures of your BIOS/UEFI if it is still activated.
After installing WSL2 in the Microsoft Store, I recommend that you choose to download the version of Ubuntu without a number. In fact, this will be the latest version of Ubuntu 20. After downloading, installing, and running for the first time, you will be prompted to set a username and password. Next, you will see a classic terminal window.
Then go to the Miniconda website, find the “Linux installers” section, copy the link to the Python 3. x installation file (currently this is https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh), type the
wget command in the Ubuntu terminal and right-click, the link to download Miniconda will be added to the end of the line, press > Enter.
After downloading the file, you can start installing it.
bash Mini command in the terminal and press > Tab to fill at the end of the file name, then >Enter. After carefully reading the agreement, you will be asked to answer
yes. Next, by pressing > Enter, we agree to specify the default installation directory and after installation, we agree
yes to initialize Miniconda. The installation process is complete.
Close the Ubuntu window and reopen it. You will see the same terminal only with the prefix (base), which means that the installation was successful and now you can use the wsl2 terminal as a classic Conda Prompt terminal!
First, create your first Conda virtual environment in WSL2 and install the main libraries. To do this, run the following commands sequentially:
conda create -n tf-cpu python=3.6
conda activate tf-cpu
pip install tensorflow
pip install jupyterlab
conda install nodejs jupyter lab
Copy the generated address of the Jupyter page and paste it into your browser. Everything is ready, you can test the learning speed of the AI model from the example above.
And in conclusion
I think that WSL2, after adding GPU support, can become a full-fledged alternative to a separate Linux installation. Until this happens, I use WSL2 for small AI models and learn the tricks of working with the Linux command line.
Therefore, if you, like me, need Linux only as a stable platform for developing and working with AI models, then I am sure that you will highly appreciate the potential of WSL2 for solving these problems.