crosstool-ng build fail on Debian 10

[mrx23dot]

sudo apt-get install make unrar-free autoconf automake libtool gcc g++ gperf
flex bison texinfo gawk ncurses-dev libexpat-dev python-dev python python-serial
sed git unzip bash help2man wget bzip2 libtool-bin
git clone --recursive https://github.com/pfalcon/esp-open-sdk.git
make

Now configured for "xtensa-lx106-elf"
make[2]: Leaving directory '/home/test/esp-open-sdk/crosstool-NG'
sed -r -i.org s%CT_PREFIX_DIR=.*%CT_PREFIX_DIR="/home/test/esp-open-sdk/xtensa-lx106-elf"% .config
sed -r -i s%CT_INSTALL_DIR_RO=y%"#"CT_INSTALL_DIR_RO=y% .config
cat ../crosstool-config-overrides >> .config
./ct-ng build
make[2]: Entering directory '/home/test/esp-open-sdk/crosstool-NG'
[INFO ] Performing some trivial sanity checks
[INFO ] Build started 20210824.105431
[INFO ] Building environment variables
[INFO ] =================================================================
[INFO ] Retrieving needed toolchain components' tarballs
[ERROR]
[ERROR] >>
[ERROR] >> Build failed in step 'Retrieving needed toolchain components' tarballs'
[ERROR] >> called in step '(top-level)'
[ERROR] >>
[ERROR] >> Error happened in: do_expat_get[scripts/build/companion_libs/210-expat.sh@12]
[ERROR] >> called from: do_companion_libs_get[scripts/build/companion_libs.sh@15]
[ERROR] >> called from: main[scripts/crosstool-NG.sh@591]
[ERROR] >>
[ERROR] >> For more info on this error, look at the file: 'build.log'
[ERROR] >> There is a list of known issues, some with workarounds, in:
[ERROR] >> 'share/doc/crosstool-ng/crosstool-ng-1.22.0-60-g37b07f6f-dirty/B - Known issues.txt'
[ERROR]
[ERROR] (elapsed: 0:03.81)

[mrx23dot]

build.log:

[DEBUG] Not at this location: "http://downloads.sourceforge.net/project/expat/expat/2.1.0/expat-2.1.0.zip"
[DEBUG] Trying 'http://downloads.sourceforge.net/project/expat/expat/2.1.0/expat-2.1.0'
[DEBUG] ==> Executing: 'wget' '--passive-ftp' '--tries=3' '-nc' '--progress=dot:binary' '-T' '10' '-O' '/home/test/esp-open-sdk/crosstool-NG/.build/tarballs/expat-2.1.0.tmp-dl' 'http://downloads.sourceforge.net/project/expat/expat/2.1.0/expat-2.1.0'

[ALL ] --2021-08-24 10:54:35-- http://downloads.sourceforge.net/project/expat/expat/2.1.0/expat-2.1.0
[ALL ] Resolving downloads.sourceforge.net (downloads.sourceforge.net)... 204.68.111.105
[ALL ] Connecting to downloads.sourceforge.net (downloads.sourceforge.net)|204.68.111.105|:80... connected.
[ALL ] HTTP request sent, awaiting response... 404 Not Found
[ALL ] 2021-08-24 10:54:35 ERROR 404: Not Found.
[ALL ]
[DEBUG] Not at this location: "http://downloads.sourceforge.net/project/expat/expat/2.1.0/expat-2.1.0"
[ERROR]

[mrx23dot]

looks like they renamed the file:
https://sourceforge.net/projects/expat/files/expat/2.1.0/expat-2.1.0-RENAMED-VULNERABLE-PLEASE-USE-2.3.0-INSTEAD.tar.gz/download

[glenn20]

The following patch changes the expat version to 2.4.1 as suggested by bitninja at https://forum.micropython.org/viewtopic.php?f=16&t=11188.

--- a/crosstool-NG/config/companion_libs/expat.in
+++ b/crosstool-NG/config/companion_libs/expat.in
@@ -6,9 +6,9 @@ choice
 # Don't remove next line
 # CT_INSERT_VERSION_BELOW
 
-config EXPAT_V_2_1_0
+config EXPAT_V_2_4_1
     bool
-    prompt "2.1.0"
+    prompt "2.4.1"
 
 endchoice
 
@@ -16,4 +16,4 @@ config EXPAT_VERSION
     string
 # Don't remove next line
 # CT_INSERT_VERSION_STRING_BELOW
-    default "2.1.0" if EXPAT_V_2_1_0
+    default "2.4.1" if EXPAT_V_2_4_1

Save the snippet as ~/patchfile, cd esp-open-sdk and patch -p1 < patchfile.
This patch should be applied beforemake runs the bootstrap command in crosstool-NG to ensure the generated scripts are all built with the new settings from expat.in. It works for me if I apply the patch on a freshly cloned folder before I run make.

[mrx23dot]

Could we integrate this into the next release?

[eku]

@mrx23dot you may want to use this fork.

[glenn20]

Thanks - I've been experimenting with different forks - but that one looks nice and complete.

[ChrisMacGregor]

Alternately, a fork with the minimal set of changes to get a clean build on Ubuntu 21.10 of an apparently working toolchain can be found here:

#391
https://github.com/ChrisMacGregor/esp-open-sdk/tree/builds-on-Ubuntu-21.10
Tarball of built toolchain: https://github.com/ChrisMacGregor/esp-open-sdk/releases/tag/builds-on-Ubuntu-21.10

A maintained fork with more-current versions of things, such as https://github.com/esp-open-sdk/esp-open-sdk or https://github.com/someburner/esp-open-sdk, may be a better option. I'm offering this minimally-tweaked option for those who prefer that, or for comparison purposes if for some reason the newer tools don't work for you, or whatever.

[nopnop2002]

@ChrisMacGregor

I used your folk on Ubuntu22.04.
But i got this error:

[INFO ]  Installing cross-gdb
[ERROR]    configure: error: failure running python-config --includes
[ERROR]    gmake[4]: *** [Makefile:8725: configure-gdb] Error 1
[ERROR]    gmake[3]: *** [Makefile:845: all] Error 2
[ERROR]
[ERROR]  >>
[ERROR]  >>  Build failed in step 'Installing cross-gdb'
[ERROR]  >>        called in step '(top-level)'
[ERROR]  >>
[ERROR]  >>  Error happened in: CT_DoExecLog[scripts/functions@257]
[ERROR]  >>        called from: do_debug_gdb_build[scripts/build/debug/300-gdb.sh@120]
[ERROR]  >>        called from: do_debug[scripts/build/debug.sh@35]
[ERROR]  >>        called from: main[scripts/crosstool-NG.sh@646]
[ERROR]  >>
[ERROR]  >>  For more info on this error, look at the file: 'build.log'
[ERROR]  >>  There is a list of known issues, some with workarounds, in:
[ERROR]  >>      'share/doc/crosstool-ng/crosstool-ng-1.22.0-60-g37b07f6f-dirty/B - Known issues.txt'
[ERROR]
[ERROR]  (elapsed: 28:25.61)

Your fork is the best I've tried.
gcc is ready.

$ ./xtensa-lx106-elf/bin/xtensa-lx106-elf-gcc --version
xtensa-lx106-elf-gcc (crosstool-NG crosstool-ng-1.22.0-60-g37b07f6f-dirty) 4.8.5
Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.