Selector.select() hangs when there is nothing to select

[AlekseyKladov]

BPO	25680
Nosy	@gvanrossum, @vadmium, @1st1, @russelldavis
PRs	bpo-29255: Wait in KqueueSelector when no fds are registered #19508

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2015-11-20.14:18:22.941>
labels = ['expert-asyncio']
title = 'Selector.select() hangs when there is nothing to select'
updated_at = <Date 2020-04-15.20:06:54.831>
user = 'https://bugs.python.org/AlekseyKladov'

bugs.python.org fields:

activity = <Date 2020-04-15.20:06:54.831>
actor = 'gvanrossum'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['asyncio']
creation = <Date 2015-11-20.14:18:22.941>
creator = 'Aleksey Kladov'
dependencies = []
files = []
hgrepos = []
issue_num = 25680
keywords = ['patch']
message_count = 13.0
messages = ['254975', '254977', '255060', '255081', '255100', '255101', '255116', '366163', '366188', '366533', '366541', '366545', '366548']
nosy_count = 6.0
nosy_names = ['gvanrossum', 'neologix', 'martin.panter', 'yselivanov', 'Aleksey Kladov', 'russelldavis']
pr_nums = ['19508']
priority = 'normal'
resolution = None
stage = 'resolved'
status = 'open'
superseder = None
type = None
url = 'https://bugs.python.org/issue25680'
versions = ['Python 3.5']

[AlekseyKladov]

The following hangs on Linux

>>> import selectors
>>> s = selectors.DefaultSelector()
>>> s.select()

On Mac it returns an empty list.

[vstinner]

Your code doesn't make sense. You listen to 0 file descriptor and wait forever until you get events, which will never occur. I'm not even sure that it should be allowed by the Selector API.

What's the point of your bug report? Does your application really rely on the behaviour of the selector when no file descriptor is registered?

The behaviour changes between Linux and OS X because the "default" selector is a different selector on Linux (epoll) and OS X (kqueue? select?)

[AlekseyKladov]

What's the point of your bug report?

To show surprising platform-dependent API behavior. I don't know what is the correct behavior here, but it should be cross platform. Seems like the most sane option is to throw an exception.

Does your application really rely on the behaviour of the selector when no file descriptor is registered?

It was developed on Mac and relied (quite probably incorrectly) on the empty list result. When I run it on linux, it unexpectedly hanged.

So, I would like to either

• observe an exception on all platforms
• observe an empty list on all platforms
• observe infinite blocking on all platforms

I think this special case should also be mentioned in the docs.

[vadmium]

I would expect it to hang (until the timeout expired) of no files are added. If it returns an empty list, that should mean the timeout expired.

I guess Mac uses KqueueSelector. I would certainly expect SelectSelector to hang. Maybe the underlying kqueue implementation behaves differently though. Perhaps Python should handle this as a special case in the selectors module?

[vstinner]

Aleksey Kladov: "To show surprising platform-dependent API behavior."

Hum ok, it makes sense :-) I agree that it would be better to have a portable behaviour in the selectors module which is supposed to be an abstraction providing a portable behaviour. We already wrote code to have the same behaviour when a file descriptor is closed (handle EBADF).

Aleksey Kladov: "I think this special case should also be mentioned in the docs."

I disagree, we can do better than documenting "bugs" :-)

select.select([], [], [], None) raise an OSError "An invalid argument was supplied" on Windows. I propose to use this behaviour on all platforms for all selectors.

I should be easy to write a patch in selectors, is anyone interested to write a patch?

[AlekseyKladov]

I disagree, we can do better than documenting "bugs" :-)

The result of select.select([], [], [], None) should defintelly be documented.

Three platoforms made three different decisions in similar situation, so any behavior is reasonable and it should be made clear which one is used by Python.

[gvanrossum]

I don't think we should "fix" the select module; it's advertised as a wrapper around the various syscalls, and if the platforms disagree on what that syscall (e.g. select()) does we shouldn't try to fix it. (Documenting the differences is fine.)

However the *selectors* module is a different thing. It defines a better abstraction that's supposedly independent from the underlying syscall. I agree with Aleksey that it should just hang when no FDs are registered and no timeout is given -- this is the only logical extension of its behavior when a FD is registered or a timeout is given. This could be used to wait until e.g. a signal arrives.

It's illogical that it would behave differently if a FD was registered that will never fire, or if a timeout of a billion seconds was given (although that may be an easy hack to wait forever if the underlying syscall doesn't like this).

The asyncio package would probably be broken because of this except that it always has a FD registered (the self-pipe).

[russelldavis]

Looks like this is caused by bpo-29255

[gvanrossum]

That may well be correct. Do you want to submit a PR to fix it?

[russelldavis]

@gvanrossum PR is ready for review: #19508

[gvanrossum]

New changeset ba1bcff by Russell Davis in branch 'master':
bpo-29255: Wait in KqueueSelector.select when no fds are registered (GH-19508)
ba1bcff

[russelldavis]

I think this got auto-closed due to a link in the PR. Note that, per #19508 (comment), the behavior is still inconsistent on windows.

I think the solution there will have to be a call to sleep() when the list of fds is empty.

[gvanrossum]

How ironic, the other issue had to be closed manually. :-)

Reopening this one.

[gvanrossum]

I there still a difference between platforms here? Because then we should follow the recommendation of a previous comment.

[vadmium]

It sounds like Windows is still different, but GH-73442 is specifically open about that

[gvanrossum]

Okay, let's close, since the original issue is definitely solved.