A tool for finding the causes of unstable Python tests

pomponchik1 pts0 comments

GitHub - mgaitan/pytest-leak-finder: Find the test that's leaking before the one that fails · GitHub

/" data-turbo-transient="true" />

Skip to content

Search or jump to...

Search code, repositories, users, issues, pull requests...

-->

Search

Clear

Search syntax tips

Provide feedback

--><br>We read every piece of feedback, and take your input very seriously.

Include my email address so I can be contacted

Cancel

Submit feedback

Saved searches

Use saved searches to filter your results more quickly

-->

Name

Query

To see all available qualifiers, see our documentation.

Cancel

Create saved search

Sign in

/;ref_cta:Sign up;ref_loc:header logged out"}"<br>Sign up

Appearance settings

Resetting focus

You signed in with another tab or window. Reload to refresh your session.<br>You signed out in another tab or window. Reload to refresh your session.<br>You switched accounts on another tab or window. Reload to refresh your session.

Dismiss alert

{{ message }}

mgaitan

pytest-leak-finder

Public

Notifications<br>You must be signed in to change notification settings

Fork

Star

main

BranchesTags

Go to file

CodeOpen more actions menu

Folders and files<br>NameNameLast commit message<br>Last commit date<br>Latest commit

History<br>30 Commits<br>30 Commits

.github/workflows

.github/workflows

demo

demo

docs

docs

tests

tests

.gitignore

.gitignore

CHANGELOG.md

CHANGELOG.md

LICENSE

LICENSE

Makefile

Makefile

README.md

README.md

pyproject.toml

pyproject.toml

pytest_leak_finder.py

pytest_leak_finder.py

uv.lock

uv.lock

View all files

Repository files navigation

pytest-leak-finder

You have a test that passes when executed alone but fails when running its suite. What's happening? My two cents that some previous test keeps the things dirty. But wich one/s, maybe the previous are a lot, right?

This plugin helps to find a culprit by doing a binary search (alla git bisect) on the collected tests before the target.

The first time it will collect the first half of those tests plus the failing one (the target). If the target fails, we are in a good path, so, a new bisect is applied. When the target doesn't fail, it changes the "half" to bisect the next time.

Consider the following example:

$ pytest -v demo/test_demo.py<br>collected 6 items<br>tests/test_demo.py::test1 PASSED<br>tests/test_demo.py::test2 PASSED<br>tests/test_demo.py::test3 PASSED<br>tests/test_demo.py::test4 PASSED<br>tests/test_demo.py::test5 FAILED<br>tests/test_demo.py::test6 PASSED

$ pytest -v --lf demo/test_demo.py<br>collected 6 items / 5 deselected / 1 selected<br>tests/test_demo.py::test5 PASSED

You can use pytest-leak-finder to find the problematic test.

On the first run will set the failed test as the "target" and will stop the session.

$ pytest -v --leak-finder demo/test_demo.py<br>collected 6 items

tests/test_demo.py::test1 PASSED<br>tests/test_demo.py::test2 PASSED<br>tests/test_demo.py::test3 PASSED<br>tests/test_demo.py::test4 PASSED<br>tests/test_demo.py::test5 FAILED

===================================================== Leak finder =====================================================<br>Target set to: pytest-leak-finder/tests/test_demo.py::test5

Next step: a<br>Current target is: pytest-leak-finder/tests/test_demo.py::test5<br>============================================= 1 failed, 4 passed in 0.13s =============================================

The second execution will run the first half of the tests passed before the target (step "a", composed by test1 and test2).

If the target still fail, that path would followed deeper by dividing again. But in this example<br>it passes, so we'll discard it, and asumme the other half was the one that include the leak.<br>That's the reason why the "next step" will be "ba".

$ pytest -v --leak-finder demo/test_demo.py<br>collected 6 items / 3 deselected / 3 selected<br>demo/test_demo.py::test1 PASSED [ 33%]<br>demo/test_demo.py::test2 PASSED [ 66%]<br>demo/test_demo.py::test5 PASSED [100%]

===================================================== Leak finder =====================================================<br>We reach the target and nothing failed. Let's change the last half.

Next step: ba<br>Current target is: pytest-leak-finder/demo/test_demo.py::test5<br>=========================================== 3 passed, 3 deselected in 0.03s ===========================================

So, the new step will be "B-A", i.e. test3

$ pytest -v --leak-finder demo/test_demo.py<br>collected 6 items / 3 deselected / 3 selected<br>tests/test_demo.py::test3 PASSED<br>tests/test_demo.py::test5 FAILED

===================================================== Leak finder =====================================================<br>We found a leak!

Leak found in: pytest-leak-finder/demo/test_demo.py::test3<br>Last step was: ba

And there it is, test3 was the problematic test we were looking for!

About<br>Find the test that's leaking before the one that fails<br>mgaitan.github.io/pytest-leak-finder/<br>Resources<br>Readme<br>MIT license<br>Activity<br>Stars<br>6 stars<br>Watchers<br>1 watching<br>Forks<br>1 fork<br>Report...

test_demo tests leak passed finder pytest

Related Articles