PR : Spyder 6 compatibility

[Zybulon]

Hello,

I have updated the code to be compatible with Spyder 6.

[Zybulon]

The tests do not pass. They failed not because of the corrections I made but because of the dependencies are too old.
Honestly I don't think I would have time to make the tests pass. So I think I'll stop here, unless someone update the tests.

[Zybulon]

@steff456 could you review the PR so that it can be merged with the main branch ?

[ccordoba12]

@Zybulon, thanks a lot for your work on this! We'll take a look at it to release a new version as soon as we can (probably in two or three weeks).

[bnavigator]

Any news here?

[juliangilbey]

Amazing, thanks @Zybulon!

Two extra patches I would make:

1. pipes is deprecated, so in setupbase.py, replace import pipes with import shlex and change pipes.quote to shlex.quote. (Later versions of pipes just said from shlex import quote anyway!)
2. distutils is deprecated. It is used in setupbase.py and spyder_terminal/confpage.py, so these should be replaced.

[juliangilbey]

In package.json, we could also get rid of promise-polyfill (removed in e0ff7f8) and fetch (included in NodeJS and never imported). I´m not sure why we need babel-loader or @babel/core listed explicitly, either.

[juliangilbey]

OK, I've managed to build the package with your PR, but the tests didn't all run. Here are the further things I've found:

1. It seems fine to remove promise-polyfill, fetch, babel-loader and @babel/core; at least doing so does not break any tests.
2. The file .yarnrc is ignored and should therefore be removed; the --modules-folder option is no longer valid. So the build should do (the equivalent of) mv node_modules spyder_terminal/server/static/components at the end of build_static. Otherwise things break quite badly.
3. In spyder_terminal/server/static/index.html, there are two references to the components directory. With the previous change, the jquery script will be found. But the xterm.css file has moved; that link should be changed to <link rel="stylesheet" href="static/components/@xterm/xterm/css/xterm.css" />

In spite of all this, there is still one test that fails consistently for me: spyder_terminal/tests/test_terminal.py::test_terminal_paste_1 - @Zybulon, was this the same one that failed for you? Here is the rather unhelpful but somewhat disturbing failure log:

____________________________ test_terminal_paste_1 _____________________________

setup_terminal = <spyder_terminal.terminalplugin.TerminalPlugin object at 0x7f56f43ed880>
qtbot_module = <pytestqt.qtbot.QtBot object at 0x7f57015c0830>

    @flaky(max_runs=3)
    @pytest.mark.skipif((os.environ.get('CI') and
                         (sys.platform.startswith('linux') or WINDOWS)),
                         reason="Doesn't work on Linux and Windows CIs")
    def test_terminal_paste_1(setup_terminal, qtbot_module):
    [... code snipped ...]
        term.view.paste()
        qtbot_module.wait(1000)
>       qtbot_module.waitUntil(lambda: check_paste(term, expected),
                               timeout=TERM_UP)
E       pytestqt.exceptions.TimeoutError: waitUntil timed out in 40000 milliseconds

spyder_terminal/tests/test_terminal.py:200: TimeoutError
----------------------------- Captured Qt messages -----------------------------
webpack://spyder_terminal/./spyder_terminal/server/static/js/main.js?:None:1:

    QtCriticalMsg: Uncaught SyntaxError: Invalid or unexpected token
---------------------------- Captured stdout setup -----------------------------
Warning: None of the following fonts is installed: ['']
Warning: None of the following fonts is installed: ['']

----------------------------- Captured stdout call -----------------------------

----------------------------- Captured stderr call -----------------------------
[15376:15474:0107/211243.542459:ERROR:database.cc(1696)] Cookie sqlite error 5, errno 0: database is locked, sql: PRAGMA auto_vacuum
[15376:15474:0107/211243.542497:ERROR:database.cc(1696)] Cookie sqlite error 5, errno 0: database is locked, sql: PRAGMA journal_mode=TRUNCATE
[15376:15474:0107/211243.542514:ERROR:database.cc(1696)] Cookie sqlite error 5, errno 0: database is locked, sql: SELECT 1 FROM sqlite_master WHERE type=? AND name=?
[15376:15474:0107/211243.542562:ERROR:database.cc(1696)] Cookie sqlite error 5, errno 0: database is locked, sql: SELECT 1 FROM sqlite_master WHERE type=? AND name=?
[15376:15474:0107/211243.542571:ERROR:database.cc(1696)] Cookie sqlite error 5, errno 0: database is locked, sql: CREATE TABLE meta(key LONGVARCHAR NOT NULL UNIQUE PRIMARY KEY, value LONGVARCHAR)
[15376:15474:0107/211243.542749:ERROR:database.cc(1696)] Cookie sqlite error 5, errno 0: database is locked, sql: PRAGMA auto_vacuum
[15376:15474:0107/211243.542763:ERROR:database.cc(1696)] Cookie sqlite error 5, errno 0: database is locked, sql: PRAGMA journal_mode=TRUNCATE
[15376:15474:0107/211243.542777:ERROR:database.cc(1696)] Cookie sqlite error 5, errno 0: database is locked, sql: SELECT 1 FROM sqlite_master WHERE type=? AND name=?
[15376:15474:0107/211243.542809:ERROR:database.cc(1696)] Cookie sqlite error 5, errno 0: database is locked, sql: SELECT 1 FROM sqlite_master WHERE type=? AND name=?
[15376:15474:0107/211243.542819:ERROR:database.cc(1696)] Cookie sqlite error 5, errno 0: database is locked, sql: CREATE TABLE meta(key LONGVARCHAR NOT NULL UNIQUE PRIMARY KEY, value LONGVARCHAR)

What is that syntax error in main.js? Or is that just a red herring?

[juliangilbey]

Actually, I wrote:

2. The file .yarnrc is ignored and should therefore be removed; the --modules-folder option is no longer valid. So the build should do (the equivalent of) mv node_modules spyder_terminal/server/static/components at the end of build_static. Otherwise things break quite badly.

But only xterm.css and jquery.min.{js,map} are needed, so only these files need to be copied into the final distribution. So it would be sufficient to do (the equivalent of):

cp node_modules/jquery/dist/jquery.min.* spyder_terminal/server/static/js
cp node_modules/@xterm/xterm/css/xterm.css spyder_terminal/server/static/css

and then modify spyder_terminal/server/static/index.html to point to the new file locations.

[juliangilbey]

1. It seems fine to remove promise-polyfill, fetch, babel-loader and @babel/core; at least doing so does not break any tests.

Nope; @babel/core is required.

[Zybulon]

In spite of all this, there is still one test that fails consistently for me: spyder_terminal/tests/test_terminal.py::test_terminal_paste_1 - @Zybulon, was this the same one that failed for you? Here is the rather unhelpful but somewhat disturbing failure log:

Yes that's it. But I did not notice the Captured stderr call. For me, the problem is related to the test rather than the actual code and I have no idea how to solve it.

[Zybulon]

Actually the success of this test is not reproductible. For example on my repository it worked.

[dalthviz]

Hi, sorry for such a late reply and thank you so much for all your work @Zybulon ! I gave this a check and indeed I think some of the points mentioned by @juliangilbey (thank you for giving this a check too!) are currently needed, particularly:

2. The file .yarnrc is ignored and should therefore be removed; the --modules-folder option is no longer valid. So the build should do (the equivalent of) mv node_modules spyder_terminal/server/static/components at the end of build_static. Otherwise things break quite badly.
3. In spyder_terminal/server/static/index.html, there are two references to the components directory. With the previous change, the jquery script will be found. But the xterm.css file has moved; that link should be changed to

I will close and reopen this to retrigger the CI to have a fresh CI log but, besides that, I will most probably start working on the missing changes and pushing this effort forward by opening a new PR next week (preserving your commits of course), so just to be sure, is that okay with you @Zybulon or maybe you would prefer to address those things and continue working on this here? Let us know!

[Zybulon]

Hi, that's okay for me, thanks !

[ccordoba12]

Closing in favor of PR #355.

Thanks @Zybulon and @juliangilbey for your help!