Manual Tests Matrix

To prevent any kind of regressions from ending up in production, we follow a manual testing matrix for each pull request before it is merged into master. This matrix is based upon the various functions performed by the bot.

---

Heads up: All these tests need to be performed by setting up your own version of the oppiabot on your device. Refer to README page for further details.

The matrix is listed below:

1. Merge Conflict Check:

   • Test Case 1: The bot should put up PR: don't merge - HAS MERGE CONFLICTS label and post a comment on all open pull requests having merge conflicts.

     • Create at least 4 to 6 pull requests with deliberate merge conflicts on your test repository before installing the bot.
     • Install your version of the bot on the test repository.
     • This installation will trigger the merge conflict check and the bot should put up PR: don't merge - HAS MERGE CONFLICTS label and post a comment Hi @<PR-Author>. The latest commit in this PR has resulted in a merge conflict. Please follow this link if you need help to resolve the conflict. Thanks! on all pull requests having merge conflicts.**

   • Test Case 2: The bot should remove the label automatically on its next run for all open pull requests whose conflicts have been resolved.

     • Resolve the conflict for any of the pull requests and wait for the bot to shoot up. You can also restart the server (or Heroku dyno) to fire up the bot instantaneously.
     • The bot should remove the above-added label from all the pull requests which earlier had merge conflicts but which have been resolved now.
     • The important part of the manual test for this check is the load, that is, the number of open pull requests present for the bot to scan. The higher is the number, the better is the manual testing. 15 to 20 pull requests might be perfect for this test.

Once the changes made by you pass both the automated as well as manual tests, feel free to submit a pull request.