Professional Documents
Culture Documents
Holger Krekel
http://merlinux.eu
Pycon US 2009, Chicago
H. Krekel (Pycon US 2009, Chicago) pytest advanced testing tutorial March 26, 2009 1 / 78
If you have time, please install
H. Krekel (Pycon US 2009, Chicago) pytest advanced testing tutorial March 26, 2009 1 / 78
my technical background
H. Krekel (Pycon US 2009, Chicago) pytest advanced testing tutorial March 26, 2009 2 / 78
my current background
H. Krekel (Pycon US 2009, Chicago) pytest advanced testing tutorial March 26, 2009 4 / 78
What’s your background?
H. Krekel (Pycon US 2009, Chicago) pytest advanced testing tutorial March 26, 2009 5 / 78
Python
H. Krekel (Pycon US 2009, Chicago) pytest advanced testing tutorial March 26, 2009 6 / 78
Testing to the rescue!
H. Krekel (Pycon US 2009, Chicago) pytest advanced testing tutorial March 26, 2009 7 / 78
The test tool question
H. Krekel (Pycon US 2009, Chicago) pytest advanced testing tutorial March 26, 2009 8 / 78
my current answer
H. Krekel (Pycon US 2009, Chicago) pytest advanced testing tutorial March 26, 2009 9 / 78
If failures are not helpful ...
H. Krekel (Pycon US 2009, Chicago) pytest advanced testing tutorial March 26, 2009 10 / 78
The Automated Test question
H. Krekel (Pycon US 2009, Chicago) pytest advanced testing tutorial March 26, 2009 11 / 78
my current answer
H. Krekel (Pycon US 2009, Chicago) pytest advanced testing tutorial March 26, 2009 12 / 78
What does “code changes work out” mean?
H. Krekel (Pycon US 2009, Chicago) pytest advanced testing tutorial March 26, 2009 14 / 78
Common Test terminology
H. Krekel (Pycon US 2009, Chicago) pytest advanced testing tutorial March 26, 2009 15 / 78
py.test strives to test it all
current focus:
unittesting
functional
integration tests
H. Krekel (Pycon US 2009, Chicago) pytest advanced testing tutorial March 26, 2009 16 / 78
A pragmatic view on test types
H. Krekel (Pycon US 2009, Chicago) pytest advanced testing tutorial March 26, 2009 17 / 78
Small Tests: one aspect
H. Krekel (Pycon US 2009, Chicago) pytest advanced testing tutorial March 26, 2009 18 / 78
Medium Tests: two aspects
H. Krekel (Pycon US 2009, Chicago) pytest advanced testing tutorial March 26, 2009 19 / 78
Large Tests: end-to-end
H. Krekel (Pycon US 2009, Chicago) pytest advanced testing tutorial March 26, 2009 20 / 78
generally speaking
H. Krekel (Pycon US 2009, Chicago) pytest advanced testing tutorial March 26, 2009 21 / 78
my current answer
H. Krekel (Pycon US 2009, Chicago) pytest advanced testing tutorial March 26, 2009 22 / 78
pytest advanced features (30 minutes)
H. Krekel (Pycon US 2009, Chicago) pytest advanced testing tutorial March 26, 2009 24 / 78
observations
H. Krekel (Pycon US 2009, Chicago) pytest advanced testing tutorial March 26, 2009 25 / 78
setting up state close to code
def test_something(self):
... use self.inst1 ...
assert "things are ok"
H. Krekel (Pycon US 2009, Chicago) pytest advanced testing tutorial March 26, 2009 26 / 78
observations
H. Krekel (Pycon US 2009, Chicago) pytest advanced testing tutorial March 26, 2009 27 / 78
meet py.test “funcargs”
goals:
fully separate test setup from test code
setup app bootstraps in one place
allow to group tests in classes or files logically
H. Krekel (Pycon US 2009, Chicago) pytest advanced testing tutorial March 26, 2009 28 / 78
basic funcarg mechanism
H. Krekel (Pycon US 2009, Chicago) pytest advanced testing tutorial March 26, 2009 29 / 78
Example test function
def test_somefunction(myarg):
assert myarg == ’test_somefunction’
H. Krekel (Pycon US 2009, Chicago) pytest advanced testing tutorial March 26, 2009 30 / 78
Example conftest.py
class ConftestPlugin:
def pytest_funcarg__myarg(self, py-
funcitem):
return pyfuncitem.name
H. Krekel (Pycon US 2009, Chicago) pytest advanced testing tutorial March 26, 2009 31 / 78
observations
funcargs:
test functions receive value by specifying a name
makers are registered after command line parsing
makers have meta-access to“collected function item”
H. Krekel (Pycon US 2009, Chicago) pytest advanced testing tutorial March 26, 2009 32 / 78
notes on “named based lookup”
H. Krekel (Pycon US 2009, Chicago) pytest advanced testing tutorial March 26, 2009 33 / 78
Exercise
H. Krekel (Pycon US 2009, Chicago) pytest advanced testing tutorial March 26, 2009 34 / 78
Using Plugins and Extensions
H. Krekel (Pycon US 2009, Chicago) pytest advanced testing tutorial March 26, 2009 36 / 78
Customizing py.test
H. Krekel (Pycon US 2009, Chicago) pytest advanced testing tutorial March 26, 2009 37 / 78
Conftest.py
H. Krekel (Pycon US 2009, Chicago) pytest advanced testing tutorial March 26, 2009 38 / 78
Specifying plugins
H. Krekel (Pycon US 2009, Chicago) pytest advanced testing tutorial March 26, 2009 39 / 78
Writing a local conftest plugin
H. Krekel (Pycon US 2009, Chicago) pytest advanced testing tutorial March 26, 2009 40 / 78
Exercise
H. Krekel (Pycon US 2009, Chicago) pytest advanced testing tutorial March 26, 2009 41 / 78
other Plugin Examples
H. Krekel (Pycon US 2009, Chicago) pytest advanced testing tutorial March 26, 2009 42 / 78
if time permits ...
H. Krekel (Pycon US 2009, Chicago) pytest advanced testing tutorial March 26, 2009 43 / 78
Break
H. Krekel (Pycon US 2009, Chicago) pytest advanced testing tutorial March 26, 2009 44 / 78
Writing Plugins (30 minutes)
H. Krekel (Pycon US 2009, Chicago) pytest advanced testing tutorial March 26, 2009 45 / 78
py.test collection objects
H. Krekel (Pycon US 2009, Chicago) pytest advanced testing tutorial March 26, 2009 46 / 78
test collection tree
H. Krekel (Pycon US 2009, Chicago) pytest advanced testing tutorial March 26, 2009 47 / 78
py.test.collect.* filesystem objects
Directory
File
H. Krekel (Pycon US 2009, Chicago) pytest advanced testing tutorial March 26, 2009 48 / 78
always available Attributes
H. Krekel (Pycon US 2009, Chicago) pytest advanced testing tutorial March 26, 2009 49 / 78
Python object collectors
H. Krekel (Pycon US 2009, Chicago) pytest advanced testing tutorial March 26, 2009 50 / 78
Exercise “collection tree”
H. Krekel (Pycon US 2009, Chicago) pytest advanced testing tutorial March 26, 2009 51 / 78
Hooks and Events
H. Krekel (Pycon US 2009, Chicago) pytest advanced testing tutorial March 26, 2009 52 / 78
Configuration Hooks
def pytest_addoption(self, parser):
""" called before commandline pars-
ing. """
H. Krekel (Pycon US 2009, Chicago) pytest advanced testing tutorial March 26, 2009 53 / 78
Collection Hooks
def pytest_collect_file(self, path, parent):
""" return Collection node or None. """
H. Krekel (Pycon US 2009, Chicago) pytest advanced testing tutorial March 26, 2009 54 / 78
Python collection hooks
::
def pytest pymodule makeitem(self, modcol, nam
“”“ return custom item/collector for
a python object in a module, or
None. ”“”
H. Krekel (Pycon US 2009, Chicago) pytest advanced testing tutorial March 26, 2009 55 / 78
function test run hooks
::
def pytest pyfunc call(self, pyfuncitem, args, kwa
“”“ return True if we consumed/did
the call to the python function
item. ”“”
def pytest item makereport(self, item, excinfo, w
“”“ return ItemTestReport event for
the given test outcome. ”“”
H. Krekel (Pycon US 2009, Chicago) pytest advanced testing tutorial March 26, 2009 56 / 78
Reporting hooks
::
def pytest report teststatus(self, event): “”“
return shortletter and verbose
word. ”“”
def pytest terminal summary(self, terminalreporte
“”“ add additional section in
terminal summary reporting. ”“”
H. Krekel (Pycon US 2009, Chicago) pytest advanced testing tutorial March 26, 2009 57 / 78
Event methods
H. Krekel (Pycon US 2009, Chicago) pytest advanced testing tutorial March 26, 2009 58 / 78
Warning
H. Krekel (Pycon US 2009, Chicago) pytest advanced testing tutorial March 26, 2009 59 / 78
Writing cross-project plugins
H. Krekel (Pycon US 2009, Chicago) pytest advanced testing tutorial March 26, 2009 60 / 78
Exercise
H. Krekel (Pycon US 2009, Chicago) pytest advanced testing tutorial March 26, 2009 61 / 78
Distributed Testing (45 minutes)
H. Krekel (Pycon US 2009, Chicago) pytest advanced testing tutorial March 26, 2009 63 / 78
test distribution modes
H. Krekel (Pycon US 2009, Chicago) pytest advanced testing tutorial March 26, 2009 64 / 78
send test to one other python interpreter
py.test --dist=each --
tx popen//python=python2.4
H. Krekel (Pycon US 2009, Chicago) pytest advanced testing tutorial March 26, 2009 65 / 78
send test to three different interpreters
py.test --dist=each \
--tx=popen//python=python2.4 \
--tx=popen//python=python2.5 \
--tx=popen//python=python2.6
H. Krekel (Pycon US 2009, Chicago) pytest advanced testing tutorial March 26, 2009 66 / 78
Example: Speed up test runs
or in short:
py.test -n 3
Especially for longer running tests or tests requiring a lot
of IO this can lead to considerable speed ups!
H. Krekel (Pycon US 2009, Chicago) pytest advanced testing tutorial March 26, 2009 67 / 78
send tests to remote SSH accout
H. Krekel (Pycon US 2009, Chicago) pytest advanced testing tutorial March 26, 2009 68 / 78
Sending tests to remote socket servers
H. Krekel (Pycon US 2009, Chicago) pytest advanced testing tutorial March 26, 2009 69 / 78
Exercise: Move settings into conftest
H. Krekel (Pycon US 2009, Chicago) pytest advanced testing tutorial March 26, 2009 70 / 78
Distribution modes
H. Krekel (Pycon US 2009, Chicago) pytest advanced testing tutorial March 26, 2009 71 / 78
how does py.test do all this?
it uses py.execnet!
H. Krekel (Pycon US 2009, Chicago) pytest advanced testing tutorial March 26, 2009 72 / 78
py.execnet
H. Krekel (Pycon US 2009, Chicago) pytest advanced testing tutorial March 26, 2009 73 / 78
xspecs: Exec environment specs
general form:
key1=value1//key2=value2//key3=value3
H. Krekel (Pycon US 2009, Chicago) pytest advanced testing tutorial March 26, 2009 74 / 78
xspec key meanings
H. Krekel (Pycon US 2009, Chicago) pytest advanced testing tutorial March 26, 2009 75 / 78
xspec examples
::
ssh=wyvern//python=python2.4//chdir=mycach
popen//python=2.5//nice=20
socket=192.168.1.4:8888
H. Krekel (Pycon US 2009, Chicago) pytest advanced testing tutorial March 26, 2009 76 / 78
Exercise
H. Krekel (Pycon US 2009, Chicago) pytest advanced testing tutorial March 26, 2009 77 / 78
Feedback round
H. Krekel (Pycon US 2009, Chicago) pytest advanced testing tutorial March 26, 2009 78 / 78