Merge pull request #1337 from fredizzimo/wsl_setup

Add script for setting up Windows Subsystem for Linux

.gitignore

@@ -22,6 +22,7 @@ quantum/version.h
 .idea/
 CMakeLists.txt
 .DS_STORE
+/util/wsl_downloaded
 
 # Eclipse/PyCharm/Other IDE Settings
 .cproject

docs/Build-Environment-Setup.md

@@ -1,25 +1,32 @@
 ### Windows 10
 
-Due to some issues with the "Windows (Vista and later)" instructions below, we now recommend following these instructions if you use Windows, which will allow you to use the Windows Subsystem for Linux to compile the firmware. If you are not using Windows 10 with the Anniversary Update installed (which came out in July 2016), you will need to use one of the other methods, such as Docker, Vagrant, or the instructions for Vista and later. 
+#### Creators Update
+If you have Windows 10 with Creators Update or later, you can build and flash the firmware directly. Before the Creators Update, only building was possible. If you don't have it yet or if are unsure, follow [these instructions](https://support.microsoft.com/en-us/instantanswers/d4efb316-79f0-1aa1-9ef3-dcada78f3fa0/get-the-windows-10-creators-update).
 
-If you use this method, you will need to use a standalone tool to flash the firmware to the keyboard after you compile it. We recommend the official [QMK Firmware Flasher](https://github.com/qmk/qmk_firmware_flasher/releases). This is because the Windows 10 Subsystem for Linux lacks [libUSB support](https://wpdev.uservoice.com/forums/266908-command-prompt-console-bash-on-ubuntu-on-windo/suggestions/13355724-unable-to-access-usb-devices-from-bash), so it can't access the keyboard's microcontroller. Please add your vote for Microsoft to fix this issue using the link!
+#### Windows Subsystem for Linux
+In addition to the Creators Update, you need Windows 10 Subystem for Linux, so install it following [these instructions](http://www.howtogeek.com/249966/how-to-install-and-use-the-linux-bash-shell-on-windows-10/). If you already have the Windows 10 Subsystem for Linux from the Anniversary update it's recommended that you [upgrade](https://betanews.com/2017/04/14/upgrade-windows-subsystem-for-linux/) it to 16.04LTS, because some keyboards don't compile with the toolchains included in 14.04LTS. Note that you need to know what your are doing if you chose the `sudo do-release-upgrade` method.
 
-Here are the steps
+#### Git
+If you already have cloned the repository on your Windows file system you can ignore this section.
 
-1. Install the Windows 10 subsystem for Linux, following [these instructions](http://www.howtogeek.com/249966/how-to-install-and-use-the-linux-bash-shell-on-windows-10/).
+You will need to clone the repository to your Windows file system using the normal Git for Windows and **not** the WSL Git. So if you haven't installed Git before, [download](https://git-scm.com/download/win) and install it. Then [set it up](https://git-scm.com/book/en/v2/Getting-Started-First-Time-Git-Setup), it's important that you setup the e-mail and user name, especially if you are planning to contribute. 
-2. If you have  cloned the repository using git before August 20, 2016, clean up the line endings from wherever you currently access git:
-   1. Make sure that you have no changes you haven't committed by running `git status`. ANY UNCOMMITTED CHANGES WILL BE PERMANENTLY LOST.
-   2. Run `git rm --cached -r .`
-   3. Run `git reset --hard`
-3. Open "Bash On Ubuntu On Windows" from the start menu
-4. With the bash window open, navigate to your copy of the [qmk_firmware repository](https://github.com/qmk/qmk_firmware) using the `cd` command. The harddisks can be accessed from `/mnt/<driveletter>`. For example, your main hard drive (C:) can be accessed by executing the command `cd /mnt/c`. If your username is John and the qmk_firmware folder is in your Downloads folder, you can move to it with the command `cd /mnt/c/Users/John/Downloads/qmk_firmware`. You can use the Tab key as you go to help you autocomplete the folder names.
-5. Run `sudo util/install_dependencies.sh`.  **This will run `apt-get upgrade`.**
-6. After a while the installation will finish, and you are good to go
 
-**Note** From time to time, the dependencies might change, so just run `install_dependencies.sh` again if things are not working.
+Once Git is installed, open the Git bash command and change the directory to where you want to clone QMK, note that you have to use forward slashes, and that your c drive is accessed like this `/c/path/to/where/you/want/to/go`. Then run `git clone --recurse-submodules https://github.com/qmk/qmk_firmware`, this will create a new folder `qmk_firmware` as a subfolder of the current one.
 
-**Warning:** If you edit Makefiles or shell scripts, make sure you are using an editor that saves the files with Unix line endings. Otherwise the compilation might not work.
+#### Toolchain setup
+The Toolchain setup is done through the Windows Subsystem for Linux, and the process is fully automated. If you want to do everything manually, there are no other instructions than the scripts themselves, but you can always open issues and ask for more information.
 
+1. Open "Bash On Ubuntu On Windows" from the start menu. 
+2. Go to the directory where you cloned `qmk_firmware`. Note that the paths start with `/mnt/` in the WSL, so you have to write for example `cd /mnt/c/path/to/qmk_firmware`. 
+3. Run `util/wsl_install.sh` and follow the on-screen instructions.
+4. Close the Bash command window, and re-open it.
+5. You are ready to compile and flash the firmware!
+
+#### Some important things to keep in mind
+* You can run `util/wsl_install.sh` again to get all the newest updates.
+* Your QMK repository need to be on a Windows file system path, since WSL can't run executables outside it.
+* The WSL Git is **not** compatible with the Windows Git, so use the Windows Git Bash or a windows Git GUI for all Git operations
+* You can edit files either inside WSL or normally using Windows, but note that if you edit makefiles or shell scripts, make sure you are using an editor that saves the files with Unix line endings. Otherwise the compilation might not work.
 
 ### Windows (Vista and later)
 1. If you have ever installed WinAVR, uninstall it.
@@ -109,4 +116,4 @@ If you have any problems building the firmware, you can try using a tool called
 1. If you haven't already, obtain this repository ([https://github.com/qmk/qmk_firmware](https://github.com/qmk/qmk_firmware)). You can either download it as a zip file and extract it, or clone it using the command line tool git or the Github Desktop application.
 2. Open up a terminal or command prompt and navigate to the `qmk_firmware` folder using the `cd` command. The command prompt will typically open to your home directory. If, for example, you cloned the repository to your Documents folder, then you would type `cd Documents/qmk_firmware`. If you extracted the file from a zip, then it may be named `qmk_firmware-master` instead.
 3. To confirm that you're in the correct location, you can display the contents of your current folder using the `dir` command on Windows, or the `ls` command on Linux or Mac. You should see several files, including `readme.md` and a `quantum` folder. From here, you need to navigate to the appropriate folder under `keyboards/`. For example, if you're building for a Planck, run `cd keyboards/planck`.
-4. Once you're in the correct keyboard-specific folder, run the `make` command. This should output a lot of information about the build process. More information about the `make` command can be found below.
+4. Once you're in the correct keyboard-specific folder, run the `make` command. This should output a lot of information about the build process. More information about the `make` command can be found below.

util/activate_wsl.sh

@@ -0,0 +1,17 @@
+#!/bin/bash
+
+function export_variables {
+    local util_dir=~/qmk_utils
+    local download_dir=$util_dir/wsl_downloaded
+
+    export DFU_PROGRAMMER=$download_dir/dfu-programmer/dfu-programmer.exe
+    export DFU_UTIL=$download_dir/dfu-util-0.9-win64/dfu-util.exe
+    export TEENSY_LOADER_CLI=$download_dir/teensy_loader_cli.exe
+    export BATCHISP=batchisp.exe
+}
+
+export_variables
+
+
+
+

util/drivers.txt

@@ -0,0 +1,46 @@
+# The format is
+# driver,desc,vid,pid,guid
+# Use a comma as a separator without spaces
+# Driver can be one of winusb,libusb,libusbk
+# Use Windows Powershell and type [guid]::NewGuid() to generate guids
+winusb,Kiibohd DFU Bootloader,1C11,B007,aa5a3f86-b81e-4416-89ad-0c1ea1ed63af
+libusb,ATxmega16C4,03EB,2FD8,23266ee7-5423-4cc4-993b-034571c43a90
+libusb,ATxmega32C4,03EB,2FD9,d4b62886-2ac8-4534-aa24-eae0a2c3ce43
+libusb,ATxmega64C3,03EB,2FD6,08467ca7-9b5a-41d2-8d8a-4a26d0b5285b
+libusb,ATxmega128C3,03EB,2FD7,1ca69799-6d95-46cf-be69-5b3d0eb915e6
+libusb,ATxmega256C3,03EB,2FDA,216ddc8b-6c67-4cc0-b934-482829a483a0
+libusb,ATxmega384C3,03EB,2FDB,0e4e3347-6025-4d49-ba80-2375ea690c28
+libusb,ATxmega64A1U,03EB,2FE8,2553d8fa-7de1-44a6-bdbf-57be8bb37e28
+libusb,ATxmega128A1U,03EB,2FED,6d9fd0ff-755d-4e29-bd29-df0a9a7544b9
+libusb,ATxmega64A4U,03EB,2FDD,bcf5e7c3-44a1-4fd1-971f-9ef9843f6291
+libusb,ATxmega128A4U,03EB,2FDE,3f976bb6-36ca-44cc-a728-844bc1d0d168
+libusb,ATxmega64B3,03EB,2FDF,de280c81-c12a-4ca7-bf34-566151786418
+libusb,ATxmega128B3,03EB,2FE0,2ad1ffeb-eb83-4e78-b34a-d5633771991f
+libusb,ATxmega64B1,03EB,2FE1,002874a6-7fc7-413b-9ac4-2b52c5a230bd
+libusb,ATxmega128B1,03EB,2FEA,60ea9d08-2ae6-4434-b743-ce6f73537136
+libusb,ATxmega256A3BU,03EB,2FE2,5949bd0a-8bd4-417b-b1c5-7d249836bf0d
+libusb,ATxmega16A4U,03EB,2FE3,cc3172b0-e86a-4758-914e-951bca6ca7f5
+libusb,ATxmega32A4U,03EB,2FE4,f44c515f-7d17-4612-a532-ee620afb22b2
+libusb,ATxmega64A4U,03EB,2FE5,c1af4f1c-045f-40c9-893a-3ad4adb2e67d
+libusb,ATxmega128A3U,03EB,2FE6,26f275f0-d6b2-46d8-8334-e4de66996c74
+libusb,ATxmega192A3U,03EB,2FE7,b7b50d98-0429-4235-8f08-5466e4f83ed4
+libusb,UC3,03EB,2FDC,972d9af7-d71b-44c7-a895-9340b362f545
+libusb,ATUC3,03EB,2FE9,d5855d0a-f82e-4df5-9c14-2b0b1dcb65bd
+libusb,AT32UC3C,03EB,2FEB,1eeb52aa-fd24-47fd-8a76-056446d1a54f
+libusb,ATxmega256A3U,03EB,2FEC,198fa8ea-3157-4863-b9a8-a3f6fe027367
+libusb,ATmega8U2,03EB,2FEE,14018055-46f4-4c62-aa03-e8fafeedaf72
+libusb,ATmega16U2,03EB,2FEF,007274da-b75f-492e-a288-8fc0aff8339f
+libusb,ATmega32U2,03EB,2FF0,ddc2c572-cb6e-4f61-a6cc-1a5de941f063
+libusb,AT32UC3A3,03EB,2FF1,8b614283-36c0-46a2-890d-65f61b5b6201
+libusb,ATmega32U6,03EB,2FF2,a207dd90-2814-4418-b5b7-4b708fdf1bfd
+libusb,ATmega16U4,03EB,2FF3,3180d426-bf93-4578-a693-2efbc337da8e
+libusb,ATmega32U4,03EB,2FF4,5f9726fd-f9de-487a-9fbd-8b3524a7a56a
+libusb,AT32UC3B,03EB,2FF6,ef90068a-277a-44db-805a-9b83a6beb29a
+libusb,AT90USB82,03EB,2FF7,062fa2ab-f9d8-4a0d-83c1-df0521cfd0f6
+libusb,AT32UC3A,03EB,2FF8,24080a67-3874-4fb8-8808-fb4cc297c466
+libusb,AT90USB64,03EB,2FF9,c6a708ad-e97d-43cd-b04a-3180d737a71b
+libusb,AT90USB162,03EB,2FFA,de67bff5-6e39-4e9c-8dfe-de7fce113716
+libusb,AT90USB128,03EB,2FFB,fd217df3-59d0-440a-a8f3-4c0c8c84daa3
+libusb,AT89C5130,03EB,2FFD,31b69a56-9ac0-4fab-a3ae-cd7bb7021ec5
+libusb,AT8XC5122,03EB,2FFE,395a6118-8568-41b2-913a-d16912722342
+libusb,AT89C5132,03EB,2FFF,266ca4bc-5e59-4a7b-82dc-6e8732373d40

util/wsl_install.sh

@@ -0,0 +1,159 @@
+#!/bin/bash
+
+download_dir=wsl_downloaded
+
+function install_utils {
+    rm -f -r $download_dir
+    mkdir $download_dir
+
+    pushd $download_dir
+
+    echo "Installing dfu-programmer"
+    wget 'http://downloads.sourceforge.net/project/dfu-programmer/dfu-programmer/0.7.2/dfu-programmer-win-0.7.2.zip'
+    unzip -d dfu-programmer dfu-programmer-win-0.7.2.zip
+
+    echo "Installing dfu-util"
+    wget 'http://dfu-util.sourceforge.net/releases/dfu-util-0.9-win64.zip'
+    unzip dfu-util-0.9-win64.zip
+
+    echo "Installing teensy_loader_cli"
+    wget 'https://www.pjrc.com/teensy/teensy_loader_cli_windows.zip'
+    unzip teensy_loader_cli_windows.zip
+
+    echo "Installing Atmel Flip"
+    wget 'http://www.atmel.com/images/Flip%20Installer%20-%203.4.7.112.exe'
+    mv Flip\ Installer\ \-\ 3.4.7.112.exe FlipInstaller.exe
+
+    echo "Downloading the QMK driver installer"
+    wget -qO- https://api.github.com/repos/qmk/qmk_driver_installer/releases | grep browser_download_url | head -n 1 | cut -d '"' -f 4 | wget -i -
+
+    rm -f *.zip
+
+    popd > /dev/null
+}
+
+function install_drivers {
+    pushd $download_dir
+    cmd.exe /C qmk_driver_installer.exe $1 $2 ../drivers.txt
+    popd > /dev/null
+}
+
+dir=$(cd -P -- "$(dirname -- "$0")" && pwd -P)
+
+if [[ $dir != /mnt/* ]];
+then
+    echo
+    echo "You need to clone the qmk_firmware repository outside the linux filesystem."
+    echo "Otherwise the windows executables can't be run."
+    exit 1
+fi
+
+pushd "$dir"
+
+while true; do
+    echo
+    echo "Do you want to install all toolchain dependencies needed for compiling QMK?"
+    echo "This will run install_dependencies.sh, which calls apt-get upgrade."
+    echo "If you don't want that, you can install the dependencies manually."
+    read -p "(Y/N) " res
+    case $res in
+        [Yy]* ) sudo ./install_dependencies.sh; break;;
+        [Nn]* ) break;;
+        * ) echo "Invalid answer";;
+    esac
+done
+
+echo "Installing dependencies needed for the installation (unzip, wget)"
+echo "This will ask for the sudo password"
+sudo apt-get install unzip wget
+
+
+if [ ! -d "$download_dir" ]; then
+    install_utils
+else
+    while true; do
+        echo
+        read -p "The utils seem to already be downloaded, do you want to re-download them and update to the newest version (Y/N) " res
+        case $res in
+            [Yy]* ) install_utils; break;;
+            [Nn]* ) break;;
+            * ) echo "Invalid answer";;
+        esac
+    done
+fi
+
+while true; do
+    echo
+    read -p "Flip need to be installed if you want to use that for programming, do you want to install it now? (Y/N) " res
+    case $res in
+        [Yy]* ) cmd.exe /c $download_dir\\FlipInstaller.exe; break;;
+        [Nn]* ) break;;
+        * ) echo "Invalid answer";;
+    esac
+done
+
+
+while true; do
+    echo
+    echo "Which USB drivers do you want to install?"
+    echo "(A)all - All supported drivers will be installed"
+    echo "(C)onnected - Only drivers for connected keyboards (in bootloader/flashing mode) will be installed"
+    echo "(F)force - Like all, but will also override existing drivers for connected keyboards"
+    echo "(N)one - No drivers will be installed, flashing your keyboard will most likely not work"
+    read -p "(A/C/F/N)? " res
+    case $res in
+        [Aa]* ) install_drivers --all; break;;
+        [Cc]* ) install_drivers; break;;
+        [Ff]* ) install_drivers --all --force; break;;
+        [Nn]* ) break;;
+        * ) echo "Invalid answer";;
+    esac
+done
+
+echo 
+echo "Creating a softlink to the utils directory as ~/qmk_utils."
+echo "This is needed so that the the make system can find all utils it need."
+read -p "Press any key to continue (ctrl-c to abort)"
+ln -sfn "$dir" ~/qmk_utils
+
+if grep "^source ~/qmk_utils/activate_wsl.sh$" ~/.bashrc
+then
+    echo
+    echo "The line source ~/qmk_utils/activate_wsl.sh is already added to your /.bashrc"
+    echo "Not adding it twice"
+else
+    while true; do
+        echo
+        echo "Do you want to add 'source ~/qmk_utils/activate_wsl.sh' to the end of you .bashrc file?"
+        echo "Without this make won't find the needed utils, so if you don't want to do it automatically,"
+        echo "then you have to do it manually."
+        read -p "(Y/N)? " res
+        case $res in
+            [Yy]* ) echo "source ~/qmk_utils/activate_wsl.sh" >> ~/.bashrc; break;;
+            [Nn]* ) break;;
+            * ) echo "Invalid answer";;
+        esac
+    done
+fi
+
+while true; do
+    echo
+    echo "Do you want to add a symlink to the QMK repository in your home directory for convenience?"
+    echo "This will create a folder 'qmk_firmware' in your home directory."
+    echo "In the future you can use this folder instead of the full path on your windows file system"
+    read -p "(Y/N)? " res
+    case $res in
+        [Yy]* ) ln -sfn "$dir/.." ~/qmk_firmware; break;;
+        [Nn]* ) break;;
+        * ) echo "Invalid answer";;
+    esac
+done
+
+echo
+echo "******************************************************************************"
+echo "Installation completed!"
+echo "You need to open a new batch command prompt for all the utils to work properly"
+echo "******************************************************************************"
+
+popd > /dev/null
+