added Document Synchronization integration tests #1805
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
pkukielka
reviewed
Jun 25, 2024
| this.javaClass.getMethod(methodName) | ||
| ?: throw IllegalStateException( | ||
| "No method with name $methodName found in ${this.javaClass.name}") | ||
| if (method.isAnnotationPresent(TestFile::class.java)) { |
There was a problem hiding this comment.
Maybe we could use the default test file if no annotation is added?
There was a problem hiding this comment.
Good call; I'll clean that up.
There was a problem hiding this comment.
Looks like you missed this one @steveyegge
There was a problem hiding this comment.
Yeah, there really isn't a good choice for a default right now, since the only test file is specific to DocumentCodeTest.
But I went ahead and used that as the default for now, until we get more tests with disk files.
There was a problem hiding this comment.
I had to roll this change back because it broke all the new tests, which don't have testResources files on disk.
pkukielka
approved these changes
Jun 25, 2024
steveyegge
force-pushed
the
stevey/doc-synchronization-tests
branch
from
e050427
to
bda3bbe
Compare
steveyegge
changed the title
steveyegge
force-pushed
the
stevey/doc-synchronization-tests
branch
from
c56438e
to
f752745
Compare
dominiccooney
approved these changes
Jun 27, 2024
There was a problem hiding this comment.
Looks good. Note, one bit of feedback from @pkukielka you said you'll address.
| @@ -18,16 +19,25 @@ import org.junit.runners.Suite | |||
| * automatically after the platform version bump. | |||
| * | |||
| * Multiple recording files can be used, but each should have its own suite with tearDown() method | |||
| * nad define unique CODY_RECORDING_NAME. | |||
| * and define unique CODY_RECORDING_NAME. | |||
There was a problem hiding this comment.
...while we're here...
Suggested change
| * and define unique CODY_RECORDING_NAME. | |
| * and define a unique CODY_RECORDING_NAME. |
| """ | ||
| class Foo { | ||
| console.log("hello there!🎉🎂 | ||
| 🥳🎈") |
There was a problem hiding this comment.
Resulting indentation is interesting...
pkukielka
reviewed
Jun 27, 2024
| @@ -42,7 +42,8 @@ open class CodyIntegrationTextFixture : BasePlatformTestCase() { | |||
|
|
|||
| override fun tearDown() { | |||
| try { | |||
| FixupService.getInstance(myFixture.project).getActiveSession()?.apply { | |||
There was a problem hiding this comment.
Shouldn't that whole file be removed and that line modified in the CodyIntegrationTextFixture.kt?
Then git should correctly show it as rename, and not like now as whole new file added.
There was a problem hiding this comment.
Good catch; fixed that.
steveyegge
force-pushed
the
stevey/doc-synchronization-tests
branch
3 times, most recently
from
014fd7f
to
6340bfd
Compare
they still don't pass but they are getting closer
renamed testing/requestWorkspaceDocuments -> testing/workspaceDocuments and put in some more asssertions
This prevents it from spewing an error over a graphql call
DocumentCodeTest is still failing
steveyegge
force-pushed
the
stevey/doc-synchronization-tests
branch
from
e068545
to
dddffa7
Compare
was failing during tests
## Test plan Add account that exists 1. Using settings, add the account that already exists in the list
… having a pro llm selected before (#1866) ## Test plan 1. Having a pro account 2. switch to a pro llm in the chat 3. remove the token from the web 4. re-authenticate via github
Closes https://linear.app/sourcegraph/issue/CODY-988/p1-authorization-and-log-in-log-out ## Test plan Repeat for dotcom and enterprise user account: **Scenario 1** 1. Remove all existing accounts 2. Make sure you are redirected to the login panel 3. Login using one of the available providers 4. Make sure that both `cody.auth.login/clicked` and `cody.auth.signin.token/clicked` telemetry events were fired **Scenario 2** 1. Click on Cody status bar and choose `Manage Accounts` 2. Click to add new account 3. Make sure that `cody.auth.login/clicked` was fired 4. Fill the account details (server and token) and confirm 5. Make sure that `cody.auth.signin.token/clicked` was fired
No functional changes. Features should work as before. ## Test plan 1. Sign in panel features working
## Test plan Delete the token from the keychain externally You can run a separate instance of IDE and remove the account there. 1. Log in to the account 2. Open another IDE, log in to the same account, apply, then remove the account, apply 3. Switch back to the first ide 4. Close and reopen the project 5. Login panel visible
## Test plan 1. `Sign In with <>` and `Get started` buttons work
…where (#1872) Closes https://linear.app/sourcegraph/issue/CODY-2130/improve-how-actions-appear-in-shiftshift-menu ## Changes I moved action properties like `text` and `description` to `plugin.xml`. It is just easier to edit and inspect this way. To be honest, Geminit 1.5 did most of the work: > Given the following plugin.xml file and properties file, please modify plugin.xml so each action have additional propertes `text` and `description` with values from properties file. > Additionally for every action please add such tag: `<override-text place="GoToAction" text="Cody: $TEXT"/>` where `$TEXT` is taken from properties file as well. > For some actions like: `<action id="Cody.Accounts.LogInToSourcegraphAction" class="com.sourcegraph.cody.config.LogInToSourcegraphAction"/>` you may have to change them to form such as: > ``` > <action id="Cody.Accounts.LogInToSourcegraphAction" class="com.sourcegraph.cody.config.LogInToSourcegraphAction" text="Log In to Sourcegraph"> > <override-text place="GoToAction" text="Cody: Log In to Sourcegraph"/> > </action> >``` > > Please output whole content of the modified plugin.xml file. ## Test plan 1. Hit `Shift shift` 2. Type 'Cody:' 
1. Bump cody commit to include changes from sourcegraph/cody#4766 2. Add startup telemetry event. I realised that only v2 telemetry events sent directly from IJ contains info about IDE version and we want to have at least one such event for every user IDE. Install action telemetry event would not be sufficient because it would not tell us anything about existing installations. Same as in sourcegraph/cody#4766
Add the missing value for the ollama docs ## Test plan <!-- All pull requests REQUIRE a test plan: https://sourcegraph.com/docs/dev/background-information/testing_principles Why does it matter? These test plans are there to demonstrate that are following industry standards which are important or critical for our customers. They might be read by customers or an auditor. There are meant be simple and easy to read. Simply explain what you did to ensure your changes are correct! Here are a non exhaustive list of test plan examples to help you: - Making changes on a given feature or component: - "Covered by existing tests" or "CI" for the shortest possible plan if there is zero ambiguity - "Added new tests" - "Manually tested" (if non trivial, share some output, logs, or screenshot) - Updating docs: - "previewed locally" - share a screenshot if you want to be thorough - Updating deps, that would typically fail immediately in CI if incorrect - "CI" - "locally tested" --> 
Our current setup produces a lot of stable releases that often are not unhidden eventually. That makes the QA and awaiting JB approval process parallel. Based on the recent experience only 1/8 releases goes public. That is a waste of JB approval team's time and CI time. Let's change the setup. The nightly version handling does not change but the default release script does not longer publish the stable version. Instead the separate workflow can be triggered on the specific tag that publishes the stable version. ## Test plan 1. push a tag for a nightly release 2. trigger the stable release manually to be tested once merged
Closes https://linear.app/sourcegraph/issue/CODY-2095/organize-keymap-settings ## Changes 1. Proper grouping was added for all actions in plugin.xml 2. The same grouping is used for nice visual organisation of keymap management ## Test plan Visual verification in `Settings > Keymap` (see screenshots):  
## Test plan Full QA according to a test plan. As for launch script args change that is covered by automatic tests.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR introduces a new set of integration tests for document synchronization testing.
This PR is dependent on a Cody-side PR, and the tests will not pass until that PR is merged.
Test plan
This is not ready to merge yet; I'll update this section when it's ready to go.