"OSError: [WinError 6] The handle is invalid" when running invoke

[farrrb]

I sometimes get the following error, when I execute invoke on a windows machine. This error causes to abruptly terminate the following tasks (without further notice). If I run invoke with debug and echo flags I get the following output (see below).

My environment:
python 3.8.0 (and I also tried python 3.6.1)
Invoke 1.3.0

The error message (with -d and -e flag):
invoke.program.run: Received a possibly-skippable exception: <UnexpectedExit: cmd='cd C:\\work\\git\\template\\build && C:\\work\\git\\template\\set_env.bat && cmake -G "Ninja" ..' exited=None> Exception ignored in: <function Popen.__del__ at 0x03D1A9B8> Traceback (most recent call last): File "C:\tools\python\lib\subprocess.py", line 945, in __del__ self._internal_poll(_deadstate=_maxsize) File "C:\tools\python\lib\subprocess.py", line 1344, in _internal_poll if _WaitForSingleObject(self._handle, 0) == _WAIT_OBJECT_0: OSError: [WinError 6] The handle is invalid

[cod3monk3y]

I'm also seeing this in python 3.8.0/invoke 1.3.0.

Repro is as simple as running "invoke build" with the following tasks.py file. Exception occurs randomly, and frequently. This happens for nearly all the commands in my tasks.py file, and never generates any warning messages. I've had to wrap all my commands with try/except blocks in order to see the output when it occurs.

import traceback
from invoke import task

@task
def build(ctx):
    try:
        print("BUILD started...")
        ctx.run("echo hello")
    except Exception as ex:
        print(f"Exception: [{ex}]")
        traceback.print_exc()
    finally:
        print("BUILD Finished")

Output:

(.venv-brokevoke) C:\experimental\broken_invoke>invoke build
BUILD started...
hello
BUILD Finished

(.venv-brokevoke) C:\experimental\broken_invoke>invoke build
BUILD started...
hello
Exception: [Encountered a bad command exit code!

Command: 'echo hello'

Exit code: None

Stdout: already printed

Stderr: already printed

]
Traceback (most recent call last):
  File "C:\experimental\broken_invoke\tasks.py", line 8, in build
    ctx.run("echo hello")
  File "c:\experimental\broken_invoke\.venv-brokevoke\lib\site-packages\invoke\context.py", line 94, in run
    return self._run(runner, command, **kwargs)
  File "c:\experimental\broken_invoke\.venv-brokevoke\lib\site-packages\invoke\context.py", line 101, in _run
    return runner.run(command, **kwargs)
  File "c:\experimental\broken_invoke\.venv-brokevoke\lib\site-packages\invoke\runners.py", line 291, in run
    return self._run_body(command, **kwargs)
  File "c:\experimental\broken_invoke\.venv-brokevoke\lib\site-packages\invoke\runners.py", line 442, in _run_body
    raise UnexpectedExit(result)
invoke.exceptions.UnexpectedExit: Encountered a bad command exit code!

Command: 'echo hello'

Exit code: None

Stdout: already printed

Stderr: already printed


BUILD Finished

[cod3monk3y]

I've hacked up a workaround. Essentially, there's a race condition in 1.3.0 where the stdio and stderr threads "die" before the process does, and this exits the polling loop, causing the return code to be None. My change (clearly not for primetime) simply polls the process up to an additional 3 seconds to wait for it to complete. Details are in the commit message.

I've tried 1.3.0 on multiple machines (Windows 10, Python 3.7.x and 3.8.x) and it's failing regularly even with a simple task and nearly always when there are multiple tasks chained together. With this workaround, everything is much more stable.

[mrcljx]

Also encountering "Encountered a bad command exit code!" (None) on macOS

[krupan]

I think I'm seeing the same or a similar thing on archlinux, also invoke 1.3, python 3.8.1.

      invoke.program.run: Received a possibly-skippable exception: <UnexpectedExit: cmd='cd /home/bryan/work/open_source_eda_grand_dream/eda-tool-src/iverilog && git archive v10.3-maintenance --prefix iverilog-v10.3/ -o iverilog-v10.3.src.tar' exited=None>

I can get it to repeat pretty consistently with this small task:

      @task
      def xz(c):
          c.run('echo "xz-ing"')
          c.run(f'xz -9 /a-pretty-large-file')
          c.run('echo "done xz-ing"')

The xz compression takes about a minute 40 on my machine and I thought maybe I was hitting a timeout. Turns out I can fix it by doing some else after the xz command:

      @task
      def xz_yo(c):
          c.run('echo "xz-ing"')
          c.run(f'xz -9 /a-pretty-large-file; echo $?')
          c.run('echo "done xz-ing"')

However, that's not fixing the problem consistently in my actual tasks. This is what I see in debug mode:

      invoke.executor.execute: Finished loading collection & shell env configs
      xz-ing
      invoke.program.run: Received a possibly-skippable exception: <UnexpectedExit: cmd='xz -9 /a-pretty-large-file' exited=None>

It doesn't run the command after the xz. When not in debug mode it exits and does not give me any clue as to why.

[bitprophet]

This def feels like #660 - thanks for the extra reports, please follow that issue.