py.test with xdist is not executing tests parametrized with random values

[pytestbot]

Originally reported by: BitBucket: pytry, GitHub: pytry

---

I have noticed the following strange behaviour for pytest and xdist.

When trying to run the test that is parametrized with some randomly selected values the test are not actually run. The same test is executed without any problems if xdist is not used.

Following code can be used to reproduce this.

#!python
import pytest
import random

PARAMS_NUMBER = 3
PARAMS = []

for i in range(PARAMS_NUMBER):
    PARAMS.append(random.randrange(0, 1000))

@pytest.mark.parametrize('rand_par', PARAMS)
def test_random_param(rand_par):

    assert 500 > rand_par

Without xdists it works fine.

With xdist no test is executed at all with the following output

#!console
============================= test session starts =============================
platform win32 -- Python 2.7.3 -- py-1.4.24 -- pytest-2.6.2
plugins: xdist
gw0 [3] / gw1 [3] / gw2 [3] / gw3 [3]
scheduling tests via LoadScheduling

==============================  in 1.93 seconds ===============================

Versions I'm using:

python 2.7.3
pytest 2.6.2
pytest-xdist 1.11

Additional note:

With some older versions (xdist 1.8 and pytest 2.4.X or 2.5.X do not remember exactly) the xdist was stopping on assertion in dsession.py

#!python
assert collection == col

Thanks in advance for any help :)

---

• Bitbucket: https://bitbucket.org/pytest-dev/pytest/issue/594

[pytestbot]

Original comment by holger krekel (BitBucket: hpk42, GitHub: hpk42):

---

Each subprocess does its own collection of tests. With -n 2 two subprocesses will collect different tests and thus pytest cannot correctly schedule the tests (or not at all as in your case). There should be an error, not sure why nothing shows up, maybe @flub knows.

[pytestbot]

Original comment by holger krekel (BitBucket: hpk42, GitHub: hpk42):

---

To work with random values you can do this:

#!python

import random
import pytest

@pytest.fixture(params=[0,1,2])
def rand_par():
    return random.randrange(0, 1000)

def test_random_param(rand_par):
    assert 500 > rand_par

With this, the collection itself is non-random, only the execution involves random numbers.

The only reason i leave this issue open is that we need to check about the error reporting.

[pytestbot]

Original comment by Bruno Oliveira (BitBucket: nicoddemus, GitHub: nicoddemus):

---

@hpk42 I think PR9 might be responsible for not reporting anything at all; the purpose of that patch was to not consume the error that happened when xdist collected different number of tests because of some other internal error (for instance that error during .pyc writing, which has been fixed since).

Because no internal error is happening here as each slave just has a different collection, I think that problem is just being logged but no further message is appearing in the console. If I'm correct, --debug should provide more information.

I would be happy to work on this if someone can give me a pointer or two how can I produce an error in xdist when we detect that different nodes were collected, but without overwriting any previous error that might have happened in a previous step.

[pytestbot]

Original comment by BitBucket: pytry, GitHub: pytry:

---

Thanks @hpk42

I'll use proposed solution for random values.

Are You going to add some proper message in such case? So the user does not have to think why tests are not running at all?
Honestly now it can be quite confusing (as I have written before it was simply crashing on assertion so at least user had info where to start investigation :) )

[pytestbot]

Original comment by Bruno Oliveira (BitBucket: nicoddemus, GitHub: nicoddemus):

---

Hi @Pytry,

This has been fixed by this PR. :)

Cheers,

[pytestbot]

Original comment by BitBucket: pytry, GitHub: pytry:

---

Cool :)
Thanks for the update @nicoddemus

[pytestbot]

Original comment by Floris Bruynooghe (BitBucket: flub, GitHub: flub):

---

@nicoddemus In which case this issue can be closed?

[pytestbot]

Original comment by Bruno Oliveira (BitBucket: nicoddemus, GitHub: nicoddemus):

---

Sure, sorry about that. Thanks @flub.