Dear RIOTers,
we are happy to inform you that our alternative self-hosted CI server
(reachable via: https://ci.riot-os.org/riot-os/riot) is now online and ready to process your pull requests on Github. This is why, from now on, you will see two status messages
on every new pull requests (one from Travis and one from the alternative
system).
What does this mean for you as contributor or maintainer?
Merge decisions will still be based on the output of Travis.
However, you might want to check the output of the alternative
system to get faster feedback on your code changes.
Why are we rolling out our own test infrastructure?
The reason for this is two-fold:
- We want to support features Travis simply cannot supply
such as:
-
Hardware-based tests: executing test cases on target devices
-
Device volunteering: allowing trusted developers to connect
their device over the internet to our test infrastructure, so that hardware-based tests can be run on their devices.
- We experienced several major service degradations of
Travis (often lasting more than several days) over the course of the last few months. We want a system that is
more predictable, directly under our control and also customizable.
How we use or whether we use exactly this system (together with Travis or by itself) in the future, is a decision of the developer community. That is why we would like to ask you for your thoughts on the alternative CI system.
In addition to posts to this mailing list, you may also create an issue on our issue tracker: https://github.com/RIOT-OS/RIOT/issues?q=is%3Aopen+is%3Aissue+label%3ACI-Infrastructure
Here are some additional facts you might find useful:
- The current state of the server:
-
The CI server software we are using is Strider-CD (https://github.com/strider-cd/strider)
-
The current set of functionality is comparable to Travis (except for some UI
issues which are subject to change)
- The run time for static tests only (PRs with a missing ‘Ready for Ci build’ label)
is less than a minute
- The build queue only serves RIOT. Therefore, the backlog of build/test jobs
on the server is relatively small
- Test results are currently displayed as one log file with a test summary at the
end. This is subject to change in the near future.
- Full PR tests take about 45 min. to finish. If tests are skipped by the build script,
this number is considerably lower (around 20 min.)
- We are considering migrating the system to a more powerful machine. On a modern
Core-i7 tests/builds finish after about 25-30 min… We hope to further decrease this
time with some fine-tuning of the build script.
- Future plans:
-
UI changes: build matrix instead of just one log file
-
Hardware tests allowing automated tests to be run directly on devices
-
Device volunteering
-
Build slaves for scalability
Best,
Philipp
