What would also be nice is an “abort on static_test failure” flag. E.g. when I’m sure that the code is working, I don’t want to waste a full murdock cycle on fixing an incorrectly spelled word or a whitespace error.
What would also be nice is an “abort on static_test failure” flag
Or just drop the static tests from Murdock and only run Murdock when the static tests are green and the Ready for CI build label is set.
The problem with doing so is with the pr_check check that will fail with fixup commits. So in this case, Murdock could only be triggered once the PR is squashed and we don’t want that.
Maybe we could have the pr_check (with commit comments and fixup commit checks) in a different action.
Or we combine it with what currently is the check-label action. One job to check the labels, one to check the commit messages.
I’m very much for having one authorative CI, compared to checking labels with github-actions, checking docs with circle ci and checking compilation/tests with Murdock…
I disagree. We should follow the Unix philosophy here (“Make each program do one thing well”). The different CI solutions that we use come with different strength:
- Murdock is very good for building,
- Circle CI can build the documentation and provide a preview link,
- Github actions are well integrated with Github (labels) and very flexible.
A single CI that proposes just the 3 points above doesn’t exist and having one authoritative CI, like Murdock, that fulfills them would need some work each time a new feature needs to be developed/configured and would come with (usability) limitations. For example, Murdock already fulfills points 1 and 3 (without the Docker container flexibility) but not point 2 (documentation preview is not available). Adding this last feature can be quite some work, adding it to Circle CI was a breeze and requires no infrastructure maintenance.
The diversity that we have now (static tests/basic builds/python tests on GH actions, doc build/preview on Circle CI, massive builds on Murdock) goes into the right direction: use the best solutions that exist for a given problem and avoid re-inventing the wheel.
The next step is to remove the static tests from Murdock and to improve the workflow that allows to trigger Murdock, as discussed above.
I think it is better to reduce the different CI tools if feasible. It may not always be though.
The things to consider are the level of expertise required to accomplish the task vs the level of expertise we have. Having fewer CI tools means we may require more expertise but that effort is not distributed across many different tools. The bus factor should also be considered. It is a benefit to have a few people understand one tool rather than everyone owning their own.
That being said something like Circle CI doing the documentation is simple enough that anyone can come in without much effort to maintain or correct it. Maybe it would make sense to move the documentation task to GH actions or something but as of now I think it is not critical.
Something like what Murdock does is niche and non-trivial to implement on other CI tools.
Maybe we should first loosely define a list of tasks.
- building
- binaries
- documentation
- testing
- serving
- build results
- test results
- documentation
- static checks
- github checks etc…