-
Notifications
You must be signed in to change notification settings - Fork 61
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[CI] Sync tests with Device farm #1605
Merged
Merged
Changes from 62 commits
Commits
Show all changes
69 commits
Select commit
Hold shift + click to select a range
831b059
Testing device farm with Sync
nhachicha bca12b9
Adding missing conf
nhachicha b6f136b
Testing Baasaas
nhachicha 2388808
Fixing config
nhachicha e1c930d
Using correct auxiliary param name
nhachicha 675bc40
Update pr.yml
nhachicha 42e3ffa
debugging
nhachicha c4d9a6f
debug
nhachicha 1ad5a05
debug
nhachicha d68cb13
debug
nhachicha 05e3017
debug
nhachicha 2d5ad10
debug
nhachicha bd162dd
debug
nhachicha 58e53ca
debug
nhachicha 3e4ae6e
Using BAAS CLI
nhachicha 4790747
First step to cleanup BAAS instance
nhachicha 9bdd55f
Trigger manually GHA
nhachicha 1a3309f
Fixing job syntax
nhachicha 982ede3
Using github token for private repo
nhachicha c32881d
using GITHUB_TOKEN
nhachicha e49209e
using Github cli to clone
nhachicha a72e733
login with github token
nhachicha 09377ce
removing token from env
nhachicha 0dceaf2
Using REALM_CI_PAT
nhachicha 82d959a
using REALM_REPO_PAT
nhachicha ba6c6a6
using BAASAAS_REPO_PAT
nhachicha 85c5096
using BAAS_CLI
nhachicha ec1dcd8
Using APIKEY as env
nhachicha 836a469
using bash
nhachicha f032af4
Force running jobs
nhachicha 9353e04
disable non Linux JVM jobs
nhachicha c24801c
fixing never
nhachicha 7927e8b
disable copying jvm
nhachicha 27065ad
Revert changes
nhachicha 2cd6561
Adding JVM Sync tests with BAAS
nhachicha 46ce108
Add conditional run for windows test (BAAS CLI runs on bash)
nhachicha 7c7cf59
Fixing syntax
nhachicha 6ef9e6e
Wrong conditional runs
nhachicha e914723
More Syntax fix
nhachicha 7fc8bd3
Removing deprecated set-output
nhachicha f22d43e
Removing finally
nhachicha ddb0ac2
Using tag when starting baas with cli
nhachicha 2782aa4
Custom Sync report
nhachicha 509aae0
Adding macOS & iOS Sync tests
nhachicha 0fcb4aa
Device farm replay
nhachicha f4b4318
adding dummy step
nhachicha 9c03bf4
revert change
nhachicha ed740b5
Merge remote-tracking branch 'origin/feature/github-actions' into nh/…
nhachicha 472fb71
separating device farm between sync and non-sync
nhachicha 3a04d80
Using correct container id
nhachicha 0bb0d73
Using bash shell consistently and enabling JVM windows Sync Tests
nhachicha 81af914
Prepend script with back slash
nhachicha 9b7fe75
Testing windows powershell cli script
nhachicha ef06756
Using correct syntax for clone with branch
nhachicha b6897bc
- Removing Windows filter
nhachicha 3e9038a
GHA syntax
nhachicha 02c3e5c
Fixing syntax
nhachicha c872563
test
nhachicha 1d6e0d6
Rever changes
nhachicha 9f9cf12
fixing auxiliary apps syntax
nhachicha c59c32a
Using sequence
nhachicha da627e9
fixing auxiliary apps syntax
nhachicha e6e785c
Using common device farm action for Sync and non-Sync tests
nhachicha ba24644
Adding changelog
nhachicha c9c32f3
Parsing BAAS instance id properly
nhachicha 874c20b
Merge remote-tracking branch 'origin/feature/github-actions' into nh/…
nhachicha 85f6e1e
Fixing Android Build Sync APK test flag
nhachicha f823ff5
Remove unused device farm sync test for Android (using common device …
nhachicha a4288a1
Update .github/workflows/pr.yml
nhachicha File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
72 changes: 72 additions & 0 deletions
72
.github/actions/run-android-device-farm-sync-test/action.yml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,72 @@ | ||
name: 'Run Android tests on Device Farm' | ||
inputs: | ||
apk-path: | ||
required: true | ||
apk-auxiliary-path: | ||
required: true | ||
app-id: | ||
required: true | ||
project-arn: | ||
required: true | ||
device-pool-arn: | ||
required: true | ||
arguments: | ||
default: _ | ||
outputs: | ||
test-results-path: | ||
value: ${{ steps.get-test-results.outputs.results-path }} | ||
runs: | ||
using: "composite" | ||
steps: | ||
- name: Run the tests | ||
uses: nhachicha/aws-devicefarm/test-application@master | ||
|
||
id: run-tests | ||
with: | ||
project_arn: ${{ inputs.project-arn }} | ||
device_pool_arn: ${{ inputs.device-pool-arn }} | ||
app_file: ${{ inputs.apk-path }} | ||
app_auxiliary_files: ${{ inputs.apk-auxiliary-path }} | ||
app_type: ANDROID_APP | ||
test_type: APPIUM_PYTHON | ||
test_package_file: https://github.com/realm/aws-devicefarm-sample-data/releases/download/0.3/sample_env_python3.zip | ||
test_package_type: APPIUM_PYTHON_TEST_PACKAGE | ||
test_spec_file: test_spec-${{ inputs.app-id }}.yaml | ||
test_spec_type: APPIUM_PYTHON_TEST_SPEC | ||
remote_src: true | ||
test_spec: | | ||
version: 0.1 | ||
phases: | ||
install: | ||
commands: | ||
- export PYTHON_VERSION=3 | ||
test: | ||
commands: | ||
- adb shell pm list packages | grep realm | ||
- adb shell am instrument -w -r io.realm.sync.testapp.test/androidx.test.runner.AndroidJUnitRunner | egrep 'OK \([0-9]+ test[s]?\)' | ||
|
||
- run: | | ||
Install-Module -Name AWSPowerShell -Force | ||
echo "::group::Data" | ||
echo (ConvertFrom-Json '${{ steps.run-tests.outputs.data }}' | ConvertTo-Json) | ||
echo "::endgroup::" | ||
Import-Module AWSPowerShell | ||
$runs = Get-DFRunList -Arn ${{ inputs.project-arn }} | ||
$jobs = Get-DFJobList -Arn $runs[0].Arn | ||
$suites = Get-DFSuiteList -Arn $jobs[0].Arn | ||
$fileArtifacts = Get-DFArtifactList -Arn $suites[1].Arn -Type File | ||
echo "All File Artifacts:" | ||
echo $fileArtifacts | ||
$logCatArtifacts = $fileArtifacts | Where-Object { $_.Name -EQ "Logcat" } | ||
if ($logCatArtifacts) { | ||
echo "LogCat Artifacts:" | ||
echo $logCatArtifacts | ||
echo "::group::Logcat" | ||
Invoke-WebRequest -Uri $logCatArtifacts[0].Url | Select-Object -Expand RawContent | ||
echo "::endgroup::" | ||
} else { | ||
Write-Warning "No logCatArtifacts found." | ||
} | ||
shell: pwsh | ||
if: always() | ||
name: Device Farm Output |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -28,6 +28,8 @@ on: | |
value: ${{ jobs.check-cache.outputs.packages-android-cache-hit }} | ||
android-test-base-apk-cache-hit: | ||
value: ${{ jobs.check-cache.outputs.android-test-base-apk-cache-hit }} | ||
android-test-sync-apk-cache-hit: | ||
value: ${{ jobs.check-cache.outputs.android-test-sync-apk-cache-hit }} | ||
packages-macos-x64-cache-hit: | ||
value: ${{ jobs.check-cache.outputs.packages-macos-x64-cache-hit }} | ||
packages-macos-arm64-cache-hit: | ||
|
@@ -63,6 +65,7 @@ jobs: | |
packages-jvm-cache-hit: ${{ steps.jvm-cache.outputs.cache-hit }} | ||
packages-android-cache-hit: ${{ steps.android-cache.outputs.cache-hit }} | ||
android-test-base-apk-cache-hit: ${{ steps.android-test-base-apk.outputs.cache-hit }} | ||
android-test-sync-apk-cache-hit: ${{ steps.android-test-sync-apk.outputs.cache-hit }} | ||
packages-macos-x64-cache-hit: ${{ steps.macos-x64-cache.outputs.cache-hit }} | ||
packages-macos-arm64-cache-hit: ${{ steps.macos-arm64-cache.outputs.cache-hit }} | ||
packages-ios-x64-cache-hit: ${{ steps.ios-x64-cache.outputs.cache-hit }} | ||
|
@@ -276,6 +279,18 @@ jobs: | |
path: ./packages/test-base/build/outputs/apk/androidTest/debug/test-base-debug-androidTest.apk | ||
key: android-base-test-apk-key-${{ steps.packages-cache-key.outputs.sha }} | ||
|
||
# | ||
# Android Sync Test APK | ||
# | ||
- name: Check Android Sync Test APK | ||
id: android-test-sync-apk | ||
uses: cmelchior/cache@main | ||
with: | ||
key: android-sync-test-apk-key-${{ steps.packages-cache-key.outputs.sha }} | ||
path: | | ||
./packages/test-sync/build/outputs/apk/androidTest/debug/test-sync-debug-androidTest.apk | ||
./packages/test-sync/build/outputs/apk/debug/test-sync-debug.apk | ||
|
||
- name: Save Android Test APK | ||
uses: actions/upload-artifact@v3 | ||
if: always() && !cancelled() && steps.android-test-base-apk.outputs.cache-hit == 'true' | ||
|
@@ -284,13 +299,32 @@ jobs: | |
path: ./packages/test-base/build/outputs/apk/androidTest/debug/test-base-debug-androidTest.apk | ||
retention-days: 1 | ||
|
||
- name: Save Android Sync Test APK | ||
uses: actions/upload-artifact@v3 | ||
if: always() && !cancelled() && steps.android-test-sync-apk.outputs.cache-hit == 'true' | ||
with: | ||
name: android-sync-test-apk-${{ steps.find-library-version.outputs.label }} | ||
retention-days: 1 | ||
path: | | ||
./packages/test-sync/build/outputs/apk/androidTest/debug/test-sync-debug-androidTest.apk | ||
./packages/test-sync/build/outputs/apk/debug/test-sync-debug.apk | ||
|
||
- name: Delete Android Test APK cache files | ||
id: delete-cache-android-base-test-apk | ||
uses: JesseTG/[email protected] | ||
if: always() && !cancelled() && steps.android-test-base-apk.outputs.cache-hit == 'true' | ||
with: | ||
path: ./packages/test-base/build/outputs/apk/androidTest/debug/test-base-debug-androidTest.apk | ||
|
||
- name: Delete Android Sync Test APK cache files | ||
id: delete-cache-android-sync-test-apk | ||
uses: JesseTG/[email protected] | ||
if: always() && !cancelled() && steps.android-test-sync-apk.outputs.cache-hit == 'true' | ||
with: | ||
path: | | ||
./packages/test-sync/build/outputs/apk/androidTest/debug/test-sync-debug-androidTest.apk | ||
./packages/test-sync/build/outputs/apk/debug/test-sync-debug.apk | ||
|
||
# | ||
# MacOS arm64 | ||
# | ||
|
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Unless I am missing it, the only difference between this action and
run-android-device-farm-test
seems to be the package name:io.realm.sync.testapp.test
vs.io.realm.testapp.test
. Isn't there a way we can combine them?I guess
app_auxiliary_files
is also a difference, but couldn't we just make that optional?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Updated with common action used for sync and non-sync device farm tests