[RFC]: stop testing latest dependencies
#194
Comments
Ocramius
added
BC Break
RFC
|
Yeah makes sense. With the lock file constantly updated, that now makes the latest tests redundant. +1 from me. |
|
I was going to say "hell no", but then read the bit about updating |
|
The point of latest is to notice those upstream breakages. And point of lock file is to quickly detect when those breakages are not something introduced by tested changes. With lock file maintenance in place the lock file covers both points so it make sense to remove latest. BUT lock file is only usable for the php version it was generated on. So latest would need to stay on any other php versions. |
|
The problem with
Kinda agree that we need to somehow probe newer PHP versions with newer dependencies, and that we cannot run Perhaps sticking with |
|
I see the point here and I am fine with the change. Still want to say that my current pipelines (in pet projects) do depend somehow on that (which I only trigger via weekly builds as I do not have renovate up and running - yet). So whatever change we make here should ofc reflect the needs of the laminas ecosystem but we should keep in mind where this all comes from and who is using this - since we published it as a publicly available action, we should kinda try to stick with decisions we made before - even tho they start being inconvenient at some point. If we start building a CI pipeline around renovate, we should properly note this in the documentation and create proper documentation on how to have a proper build pipeline along with renovate + the laminas CI stuff. |
|
I'll implement this (it's mostly code removal), and will patch the docs accordingly. /cc @lcobucci IMO this is the last |
…erated pipelines This change removes `latest` from default composer dependencies in the generated CI matrix. The rationale is that `latest` dependencies tend to break our builds, and we usually run @renovate-bot or @dependabot on our repositories, keeping both `composer.json` and `composer.lock` fairly updated. Because of this kind of disciplined approach, we can assume that `latest` dependencies are already regularly tested by refreshing `composer.lock` regularly, and further testing with explicit `composer update` is considered risky, because it moves into unexplored land, breaking builds that are otherwise quite stable. We also don't want to break builds by contributors, or builds that upgrade perfectly safe to upgrade dependencies, just because a specific `latest` upstream dependency broke, and we didn't really touch it. The final objective is that CI can break, but only in commits that change `composer.json` and `composer.lock`, and those are handled every day by automation. Therefore, the default testing approach will cover `lowest` and `locked` dependencies, which both move much less than `latest`, and don't cause instability.
…erated pipelines This change removes `latest` from default composer dependencies in the generated CI matrix. The rationale is that `latest` dependencies tend to break our builds, and we usually run @renovate-bot or @dependabot on our repositories, keeping both `composer.json` and `composer.lock` fairly updated. Because of this kind of disciplined approach, we can assume that `latest` dependencies are already regularly tested by refreshing `composer.lock` regularly, and further testing with explicit `composer update` is considered risky, because it moves into unexplored land, breaking builds that are otherwise quite stable. We also don't want to break builds by contributors, or builds that upgrade perfectly safe to upgrade dependencies, just because a specific `latest` upstream dependency broke, and we didn't really touch it. The final objective is that CI can break, but only in commits that change `composer.json` and `composer.lock`, and those are handled every day by automation. Therefore, the default testing approach will cover `lowest` and `locked` dependencies, which both move much less than `latest`, and don't cause instability.
…erated pipelines This change removes `latest` from default composer dependencies in the generated CI matrix. The rationale is that `latest` dependencies tend to break our builds, and we usually run @renovate-bot or @dependabot on our repositories, keeping both `composer.json` and `composer.lock` fairly updated. Because of this kind of disciplined approach, we can assume that `latest` dependencies are already regularly tested by refreshing `composer.lock` regularly, and further testing with explicit `composer update` is considered risky, because it moves into unexplored land, breaking builds that are otherwise quite stable. We also don't want to break builds by contributors, or builds that upgrade perfectly safe to upgrade dependencies, just because a specific `latest` upstream dependency broke, and we didn't really touch it. The final objective is that CI can break, but only in commits that change `composer.json` and `composer.lock`, and those are handled every day by automation. Therefore, the default testing approach will cover `lowest` and `locked` dependencies, which both move much less than `latest`, and don't cause instability.
…erated pipelines This change removes `latest` from default composer dependencies in the generated CI matrix. The rationale is that `latest` dependencies tend to break our builds, and we usually run @renovate-bot or @dependabot on our repositories, keeping both `composer.json` and `composer.lock` fairly updated. Because of this kind of disciplined approach, we can assume that `latest` dependencies are already regularly tested by refreshing `composer.lock` regularly, and further testing with explicit `composer update` is considered risky, because it moves into unexplored land, breaking builds that are otherwise quite stable. We also don't want to break builds by contributors, or builds that upgrade perfectly safe to upgrade dependencies, just because a specific `latest` upstream dependency broke, and we didn't really touch it. The final objective is that CI can break, but only in commits that change `composer.json` and `composer.lock`, and those are handled every day by automation. Therefore, the default testing approach will cover `lowest` and `locked` dependencies, which both move much less than `latest`, and don't cause instability.
…erated pipelines This change removes `latest` from default composer dependencies in the generated CI matrix. The rationale is that `latest` dependencies tend to break our builds, and we usually run @renovate-bot or @dependabot on our repositories, keeping both `composer.json` and `composer.lock` fairly updated. Because of this kind of disciplined approach, we can assume that `latest` dependencies are already regularly tested by refreshing `composer.lock` regularly, and further testing with explicit `composer update` is considered risky, because it moves into unexplored land, breaking builds that are otherwise quite stable. We also don't want to break builds by contributors, or builds that upgrade perfectly safe to upgrade dependencies, just because a specific `latest` upstream dependency broke, and we didn't really touch it. The final objective is that CI can break, but only in commits that change `composer.json` and `composer.lock`, and those are handled every day by automation. Therefore, the default testing approach will cover `lowest` and `locked` dependencies, which both move much less than `latest`, and don't cause instability.
…erated pipelines This change removes `latest` from default composer dependencies in the generated CI matrix. The rationale is that `latest` dependencies tend to break our builds, and we usually run @renovate-bot or @dependabot on our repositories, keeping both `composer.json` and `composer.lock` fairly updated. Because of this kind of disciplined approach, we can assume that `latest` dependencies are already regularly tested by refreshing `composer.lock` regularly, and further testing with explicit `composer update` is considered risky, because it moves into unexplored land, breaking builds that are otherwise quite stable. We also don't want to break builds by contributors, or builds that upgrade perfectly safe to upgrade dependencies, just because a specific `latest` upstream dependency broke, and we didn't really touch it. The final objective is that CI can break, but only in commits that change `composer.json` and `composer.lock`, and those are handled every day by automation. Therefore, the default testing approach will cover `lowest` and `locked` dependencies, which both move much less than `latest`, and don't cause instability.
|
When I read this correct, most issues were runtime deprecations? Pretty sure that we explicitly want these - I was like "hell disable phpunit deprecation exceptions" in the past 😁 Might still be a way to go. I was working on testing if we can have Jobs which may fail. Could still be something to be added to the matrix as a dedicated Job tho. Having deprecation warnings in phpunit confuse a lot but I see why we might want them - its just not a good place to trigger contributors attention because of that. |
RFC
While running tests over the last few days, I noticed that we have test failure due to runtime depreciations (which I still firmly believe to be an abomination) when running
latestdependencies against the lowest supported PHP version of a repository.We have to consider 2 problems in this test matrix entry:
composer.lockof all repositories on a nightly basis, therefore detecting upstream breakagesGiven that updating lock files is equivalent to testing latest dependencies, I therefore suggest dropping the
latestentries from our generated CI matrix.Note: I counted this as BC break, because it changes assumptions around testing depth
The text was updated successfully, but these errors were encountered: