Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CI] Sync tests with Device farm #1605

Merged
merged 69 commits into from
Jan 10, 2024
Merged
Show file tree
Hide file tree
Changes from 62 commits
Commits
Show all changes
69 commits
Select commit Hold shift + click to select a range
831b059
Testing device farm with Sync
nhachicha Dec 13, 2023
bca12b9
Adding missing conf
nhachicha Dec 13, 2023
b6f136b
Testing Baasaas
nhachicha Dec 13, 2023
2388808
Fixing config
nhachicha Dec 13, 2023
e1c930d
Using correct auxiliary param name
nhachicha Dec 14, 2023
675bc40
Update pr.yml
nhachicha Dec 14, 2023
42e3ffa
debugging
nhachicha Dec 14, 2023
c4d9a6f
debug
nhachicha Dec 14, 2023
1ad5a05
debug
nhachicha Dec 14, 2023
d68cb13
debug
nhachicha Dec 14, 2023
05e3017
debug
nhachicha Dec 14, 2023
2d5ad10
debug
nhachicha Dec 14, 2023
bd162dd
debug
nhachicha Dec 14, 2023
58e53ca
debug
nhachicha Dec 14, 2023
3e4ae6e
Using BAAS CLI
nhachicha Dec 15, 2023
4790747
First step to cleanup BAAS instance
nhachicha Dec 15, 2023
9bdd55f
Trigger manually GHA
nhachicha Dec 15, 2023
1a3309f
Fixing job syntax
nhachicha Dec 15, 2023
982ede3
Using github token for private repo
nhachicha Dec 15, 2023
c32881d
using GITHUB_TOKEN
nhachicha Dec 15, 2023
e49209e
using Github cli to clone
nhachicha Dec 15, 2023
a72e733
login with github token
nhachicha Dec 15, 2023
09377ce
removing token from env
nhachicha Dec 15, 2023
0dceaf2
Using REALM_CI_PAT
nhachicha Dec 15, 2023
82d959a
using REALM_REPO_PAT
nhachicha Dec 15, 2023
ba6c6a6
using BAASAAS_REPO_PAT
nhachicha Dec 15, 2023
85c5096
using BAAS_CLI
nhachicha Dec 16, 2023
ec1dcd8
Using APIKEY as env
nhachicha Dec 16, 2023
836a469
using bash
nhachicha Dec 16, 2023
f032af4
Force running jobs
nhachicha Dec 16, 2023
9353e04
disable non Linux JVM jobs
nhachicha Dec 16, 2023
c24801c
fixing never
nhachicha Dec 16, 2023
7927e8b
disable copying jvm
nhachicha Dec 16, 2023
27065ad
Revert changes
nhachicha Dec 18, 2023
2cd6561
Adding JVM Sync tests with BAAS
nhachicha Dec 18, 2023
46ce108
Add conditional run for windows test (BAAS CLI runs on bash)
nhachicha Dec 18, 2023
7c7cf59
Fixing syntax
nhachicha Dec 18, 2023
6ef9e6e
Wrong conditional runs
nhachicha Dec 18, 2023
e914723
More Syntax fix
nhachicha Dec 18, 2023
7fc8bd3
Removing deprecated set-output
nhachicha Dec 18, 2023
f22d43e
Removing finally
nhachicha Dec 18, 2023
ddb0ac2
Using tag when starting baas with cli
nhachicha Dec 18, 2023
2782aa4
Custom Sync report
nhachicha Dec 19, 2023
509aae0
Adding macOS & iOS Sync tests
nhachicha Dec 19, 2023
0fcb4aa
Device farm replay
nhachicha Dec 19, 2023
f4b4318
adding dummy step
nhachicha Dec 19, 2023
9c03bf4
revert change
nhachicha Dec 20, 2023
ed740b5
Merge remote-tracking branch 'origin/feature/github-actions' into nh/…
nhachicha Dec 20, 2023
472fb71
separating device farm between sync and non-sync
nhachicha Dec 20, 2023
3a04d80
Using correct container id
nhachicha Dec 20, 2023
0bb0d73
Using bash shell consistently and enabling JVM windows Sync Tests
nhachicha Dec 21, 2023
81af914
Prepend script with back slash
nhachicha Dec 21, 2023
9b7fe75
Testing windows powershell cli script
nhachicha Dec 21, 2023
ef06756
Using correct syntax for clone with branch
nhachicha Dec 21, 2023
b6897bc
- Removing Windows filter
nhachicha Dec 21, 2023
3e9038a
GHA syntax
nhachicha Dec 21, 2023
02c3e5c
Fixing syntax
nhachicha Dec 21, 2023
c872563
test
nhachicha Dec 21, 2023
1d6e0d6
Rever changes
nhachicha Dec 22, 2023
9f9cf12
fixing auxiliary apps syntax
nhachicha Dec 22, 2023
c59c32a
Using sequence
nhachicha Dec 22, 2023
da627e9
fixing auxiliary apps syntax
nhachicha Dec 22, 2023
e6e785c
Using common device farm action for Sync and non-Sync tests
nhachicha Jan 9, 2024
ba24644
Adding changelog
nhachicha Jan 9, 2024
c9c32f3
Parsing BAAS instance id properly
nhachicha Jan 9, 2024
874c20b
Merge remote-tracking branch 'origin/feature/github-actions' into nh/…
nhachicha Jan 9, 2024
85f6e1e
Fixing Android Build Sync APK test flag
nhachicha Jan 10, 2024
f823ff5
Remove unused device farm sync test for Android (using common device …
nhachicha Jan 10, 2024
a4288a1
Update .github/workflows/pr.yml
nhachicha Jan 10, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
72 changes: 72 additions & 0 deletions .github/actions/run-android-device-farm-sync-test/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
name: 'Run Android tests on Device Farm'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unless I am missing it, the only difference between this action and run-android-device-farm-test seems to be the package name: io.realm.sync.testapp.test vs. io.realm.testapp.test. Isn't there a way we can combine them?

I guess app_auxiliary_files is also a difference, but couldn't we just make that optional?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated with common action used for sync and non-sync device farm tests

inputs:
apk-path:
required: true
apk-auxiliary-path:
required: true
app-id:
required: true
project-arn:
required: true
device-pool-arn:
required: true
arguments:
default: _
outputs:
test-results-path:
value: ${{ steps.get-test-results.outputs.results-path }}
runs:
using: "composite"
steps:
- name: Run the tests
uses: nhachicha/aws-devicefarm/test-application@master

id: run-tests
with:
project_arn: ${{ inputs.project-arn }}
device_pool_arn: ${{ inputs.device-pool-arn }}
app_file: ${{ inputs.apk-path }}
app_auxiliary_files: ${{ inputs.apk-auxiliary-path }}
app_type: ANDROID_APP
test_type: APPIUM_PYTHON
test_package_file: https://github.com/realm/aws-devicefarm-sample-data/releases/download/0.3/sample_env_python3.zip
test_package_type: APPIUM_PYTHON_TEST_PACKAGE
test_spec_file: test_spec-${{ inputs.app-id }}.yaml
test_spec_type: APPIUM_PYTHON_TEST_SPEC
remote_src: true
test_spec: |
version: 0.1
phases:
install:
commands:
- export PYTHON_VERSION=3
test:
commands:
- adb shell pm list packages | grep realm
- adb shell am instrument -w -r io.realm.sync.testapp.test/androidx.test.runner.AndroidJUnitRunner | egrep 'OK \([0-9]+ test[s]?\)'

- run: |
Install-Module -Name AWSPowerShell -Force
echo "::group::Data"
echo (ConvertFrom-Json '${{ steps.run-tests.outputs.data }}' | ConvertTo-Json)
echo "::endgroup::"
Import-Module AWSPowerShell
$runs = Get-DFRunList -Arn ${{ inputs.project-arn }}
$jobs = Get-DFJobList -Arn $runs[0].Arn
$suites = Get-DFSuiteList -Arn $jobs[0].Arn
$fileArtifacts = Get-DFArtifactList -Arn $suites[1].Arn -Type File
echo "All File Artifacts:"
echo $fileArtifacts
$logCatArtifacts = $fileArtifacts | Where-Object { $_.Name -EQ "Logcat" }
if ($logCatArtifacts) {
echo "LogCat Artifacts:"
echo $logCatArtifacts
echo "::group::Logcat"
Invoke-WebRequest -Uri $logCatArtifacts[0].Url | Select-Object -Expand RawContent
echo "::endgroup::"
} else {
Write-Warning "No logCatArtifacts found."
}
shell: pwsh
if: always()
name: Device Farm Output
15 changes: 10 additions & 5 deletions .github/actions/run-android-device-farm-test/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -53,11 +53,16 @@ runs:
echo "All File Artifacts:"
echo $fileArtifacts
$logCatArtifacts = $fileArtifacts | Where-Object { $_.Name -EQ "Logcat" }
echo "LogCat Artifacts:"
echo $logCatArtifacts
echo "::group::Logcat"
Invoke-WebRequest -Uri $logCatArtifacts[0].Url | Select-Object -Expand RawContent
echo "::endgroup::"
if ($logCatArtifacts) {
echo "LogCat Artifacts:"
echo $logCatArtifacts
echo "::group::Logcat"
Invoke-WebRequest -Uri $logCatArtifacts[0].Url | Select-Object -Expand RawContent
echo "::endgroup::"
} else {
Write-Warning "No logCatArtifacts found."
}

shell: pwsh
if: always()
name: Device Farm Output
34 changes: 34 additions & 0 deletions .github/workflows/include-check-cache.yml
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,8 @@ on:
value: ${{ jobs.check-cache.outputs.packages-android-cache-hit }}
android-test-base-apk-cache-hit:
value: ${{ jobs.check-cache.outputs.android-test-base-apk-cache-hit }}
android-test-sync-apk-cache-hit:
value: ${{ jobs.check-cache.outputs.android-test-sync-apk-cache-hit }}
packages-macos-x64-cache-hit:
value: ${{ jobs.check-cache.outputs.packages-macos-x64-cache-hit }}
packages-macos-arm64-cache-hit:
Expand Down Expand Up @@ -63,6 +65,7 @@ jobs:
packages-jvm-cache-hit: ${{ steps.jvm-cache.outputs.cache-hit }}
packages-android-cache-hit: ${{ steps.android-cache.outputs.cache-hit }}
android-test-base-apk-cache-hit: ${{ steps.android-test-base-apk.outputs.cache-hit }}
android-test-sync-apk-cache-hit: ${{ steps.android-test-sync-apk.outputs.cache-hit }}
packages-macos-x64-cache-hit: ${{ steps.macos-x64-cache.outputs.cache-hit }}
packages-macos-arm64-cache-hit: ${{ steps.macos-arm64-cache.outputs.cache-hit }}
packages-ios-x64-cache-hit: ${{ steps.ios-x64-cache.outputs.cache-hit }}
Expand Down Expand Up @@ -276,6 +279,18 @@ jobs:
path: ./packages/test-base/build/outputs/apk/androidTest/debug/test-base-debug-androidTest.apk
key: android-base-test-apk-key-${{ steps.packages-cache-key.outputs.sha }}

#
# Android Sync Test APK
#
- name: Check Android Sync Test APK
id: android-test-sync-apk
uses: cmelchior/cache@main
with:
key: android-sync-test-apk-key-${{ steps.packages-cache-key.outputs.sha }}
path: |
./packages/test-sync/build/outputs/apk/androidTest/debug/test-sync-debug-androidTest.apk
./packages/test-sync/build/outputs/apk/debug/test-sync-debug.apk

- name: Save Android Test APK
uses: actions/upload-artifact@v3
if: always() && !cancelled() && steps.android-test-base-apk.outputs.cache-hit == 'true'
Expand All @@ -284,13 +299,32 @@ jobs:
path: ./packages/test-base/build/outputs/apk/androidTest/debug/test-base-debug-androidTest.apk
retention-days: 1

- name: Save Android Sync Test APK
uses: actions/upload-artifact@v3
if: always() && !cancelled() && steps.android-test-sync-apk.outputs.cache-hit == 'true'
with:
name: android-sync-test-apk-${{ steps.find-library-version.outputs.label }}
retention-days: 1
path: |
./packages/test-sync/build/outputs/apk/androidTest/debug/test-sync-debug-androidTest.apk
./packages/test-sync/build/outputs/apk/debug/test-sync-debug.apk

- name: Delete Android Test APK cache files
id: delete-cache-android-base-test-apk
uses: JesseTG/[email protected]
if: always() && !cancelled() && steps.android-test-base-apk.outputs.cache-hit == 'true'
with:
path: ./packages/test-base/build/outputs/apk/androidTest/debug/test-base-debug-androidTest.apk

- name: Delete Android Sync Test APK cache files
id: delete-cache-android-sync-test-apk
uses: JesseTG/[email protected]
if: always() && !cancelled() && steps.android-test-sync-apk.outputs.cache-hit == 'true'
with:
path: |
./packages/test-sync/build/outputs/apk/androidTest/debug/test-sync-debug-androidTest.apk
./packages/test-sync/build/outputs/apk/debug/test-sync-debug.apk

#
# MacOS arm64
#
Expand Down
Loading
Loading