Toms Testing

 For ant introduction to Dependency Injection and Service Locator, see the post in the bliki . The problem with dependency injection frameworks are: With the frameworks: I don't want to annotate or anything. I want to write vanilla code There should not be one single place that has to know about every thing else. It is the ultimate violation about the open-closed-principle. Even if it is in main. Configuration explodes with dependencies of nested stuff. Main is on the top level and has no idea what some low level stuff should need Allowing only two presets (like dev and prod) sounds just stupid.  Tests should no have to configure anything they don't need, but should be able to configure anything they want All in all dependency injections feels soooo heavy. Service Locators on the other hand - especially dynamic ones - are very easy to implement, understand and lightweight to use. But they have two major downsides: They hide dependencies You have to implement it in a way that t...

The naive idea is to change the source code and then compile to create a mutant. But this costs a lot of compile time. A better idea: For every file Pass it to an extendable set of mutation-adders Ever mutation-adder adds its mutation wrapped in an if, which checks an environment variable, like EXPERIMENT_ID Every mutation increments the required EXPERIMENT_ID by one We compile the project It runs normally, as EXPERIMENT_ID is not set and thus every if is false, and thus disabling the mutation We now shuffle the set of experiment-ids and set it one after each other, which "causes" the mutation. This is very fast and trivial to parallelize - even on a cluster

For `./test.sh` it is often useful to pass all arguments.  $ cat test.sh python3 manage.py test ${@:1} --parallel --fail-fast This is useful to configure a tool but keep the flexibility of its cli. Open problem: auto-complete

 Implementing the algorithm with TLA+ can output interfaces in any languages together with tests agains this interface. Implementing these interfaces connects an implementation with the algorithm. The wiring of the interfaces via DI is generated by the model/algorithm. The database of implementation is provider by the implementer.

Git Test Selection Use code-coverage to determine, which line of code is executed by which tests. Take the git diff of changed lines, take the set-union of the sets of tests per line and run them.  Flaky-Detection and Handling If a test fails, rerun it n times. If it passes k times, mark it as green but increase its flakiness-counter by some sensible amount. Report the statistics. Fail Fast Remember which tests failed if we touched a file. This creates a database. If we run the tests, we order the tests in a way, that the most "dangerous" run first. This should move the moment of failure nearer to typing of the line. React quickly Rerun the tests as soon as the file-system changed. If production code changed, calculate the line-diffs and execute the tests first, that use that line (like in Git Test Selection). Report now, that these passed and test some random n tests in the background. The goal is to finish faster than the eyes can focus the test-output window. Prepare the n...

Docs should: Describe the algorithm Give usage-examples Give instructions on how to use function Use-Cases? Alternatives? Motivation? If you describe as much as possible via formal languages, tools can read them too and help you. It also removes ambiguities.  You could use: TLA+ to describe the algorithm Usage-examples via doctests in form of unit-tests Instructions on how to use the function via DbC: Pre-Conditions, Post-Conditions and Invariants (the blend into the algorithm-description of TLA+ The docstrings are parsed and a documentation is created while using the new power gained by the tools (maybe create a playground, etc.)