Refactoring, additional tests, README updates (#62)

* Additional integration tests
* Refactorings, clenaups, and README updates
* Added missing comments/kDoc to new interfaces
* Added known issues and limitations to README.md
* Updates and fixes
* Added github workflow and fixed/workaround some tests
* fixed missing imports
* build automation
* fixed bug causing integration tests to hang
* update README files

.github/workflows/gradle.yml

@@ -0,0 +1,38 @@
+name: CI - R3 Corda/EVM onnly
+
+on:
+  push:
+    paths:
+      - 'src/r3/atomic-swap/**'
+  pull_request:
+    paths:
+      - 'src/r3/atomic-swap/**'  
+
+permissions:
+  contents: read
+
+jobs:
+  build:
+
+    runs-on: ubuntu-latest
+
+    steps:
+    - uses: actions/checkout@v3
+      with:
+        submodules: recursive
+
+    - name: Run R3 harmonia testnet
+      run: |
+        docker run --platform linux/amd64 -d -p 8545:8545 edoardoierina/r3-harmonia-testnet:latest
+        
+    - name: Set up JDK 8
+      uses: actions/setup-java@v3
+      with:
+        java-version: '8'
+        distribution: 'temurin'
+
+    - name: Build with Gradle
+      uses: gradle/gradle-build-action@bd5760595778326ba7f1441bcf7e88b49de61a25 # v2.6.0
+      with:
+        arguments: clean test --continue
+        build-root-directory: 'src/r3/atomic-swap/corda'

src/r3/atomic-swap/corda/README.md

@@ -1,91 +1,122 @@
-# Corda-EVM interop
+# Corda-EVM Interoperability
+
+## Introduction
+This project is an experimental reference implementation of Corda-EVM interoperability. It is not intended for production use and may have limitations and bugs. Please use this code for reference and experimentation only.
 
 ## License
 
 This project is licensed under the Apache License 2.0. See the `LICENSE` file for details.
 
 ## Development Status
 
-The atomic swap reference code is currently under development but nearing completion. Some of its components are in the process of refinement and preparation for community access.
+This project is an experimental reference implementation and is considered complete for its intended purpose. While it may undergo minor updates to address any critical issues that may arise, no major changes are expected in the near future.
+
+### Component Overview
+
+The following is a list of the main components included of this project. Some of these, while dev complete, are possibly subject to limitations or known issues that are highlighted in a separate section of this document.
 
-### Component Overview and Status
+1. **EVM Interoperability Service**: This service enables flows to execute asynchronous EVM transactions and calls, waiting for the result.
 
-The following is a list of components included in this project. All of these are subject to future changes and are currently under development.
+2. **Identity Module**: This module allows for the configuration of the EVM identity a flow will operate with. It also supports the implementation of custom identity modules leveraging Hardware Security Modules (HSMs) or other protocols to ensure the safety of the private key used for signing EVM transactions. The reference `UnsecureRemoteEvmIdentityFlow` allows you to specify the private key and RPC endpoint and is only meant as a reference for implementing safer options.
 
-1. **EVM Interoperability Service**: This service enables flows to execute asynchronous EVM transactions and calls, waiting for the result. It is fully implemented and tested. 
+3. **Web3 Interfaces**: Current implementation supports direct interaction with ERC20 tokens, standard Web3 APIs like querying blocks and transactions, the EVM `SwapVault` contract which allows swapping ERC20, ERC721, ERC1155 assets against a Corda asset.
 
-2. **Identity Module**: This module allows for the configuration of the EVM identity a flow will operate with. It also supports the implementation of custom identity modules leveraging Hardware Security Modules (HSMs) or other protocols to ensure the safety of the private key used for signing EVM transactions. The basic module is fully implemented and tested, and other modules are being implemented.
+4. **Atomic Swap Flows**: Atomic swap flows for executing Corda-EVM DvP and PvP have been implemented. A sample project with tests demonstrates how an EVM asset is swapped with a Corda asset in a completely fair, risk-less, balanced way.
 
-3. **Web3 Interfaces**: Current implementation supports interaction with ERC20 tokens and standard Web3 APIs like querying blocks and transactions. These features are fully implemented and tested. Support for ERC721 and ERC1155 tokens is in the pipeline and will be added soon.
+5. **Event Subscription Service**: there is no real, full support in this project for event subscription but rather a simplistic, incomplete interface for doing so. Triggering flows in response to EVM events can be done using existing projects like [Eventeum](https://github.com/eventeum/eventeum) or Web3 javascript frameworks like [Web3.js](https://web3js.org) or [Ethers.js](https://ethers.org)
 
-4. **Atomic Swap Flows**: Basic atomic swap flows for executing Corda-EVM DvP and PvP scenarios are under active development.
+6. **Patricia Merkle Trie Component**: Allows EVM events and transaction validation, production of events and transaction proofs, and verification of proofs.
 
-5. **Event Subscription Service**: This component is responsible for subscribing to and handling events. It is currently under active development and nearing completion. Some features are experimental and will be moved to a separate external module in future iterations.
+7. **EVM / Solidity Project for Atomic Swap**: This project will implement atomic swap of EVM assets like ERC20, ERC721, ERC1155.
 
-6. **Patricia Merkle Trie Component**: This upcoming component will support EVM events and transaction validation, production of events and transaction proofs, and verification of proofs.
+## Known Issues and Limitations
 
-#### Coming Soon
+### Memory Storage / Persistence
+Some services required by the application use memory storage for simplicity rather than persistent storage. Persistent storage is necessary to retain references to transactions and associated data (e.g., signatures, identity) between distinct flows. For non-experimental use, persistent storage is required.
 
-7. **Full Atomic Swap Flows**: Full atomic swap flows for executing Corda-EVM DvP and PvP scenarios are in the pipeline and will be implemented in the near future.
+### Nonce
+The EVM transaction model uses the account nonce to maintain the order and uniqueness of transactions originating from a specific Ethereum address. The nonce represents the number of transactions sent from that address and ensures that transactions are executed in the correct sequence and not replayed. While the EVM interface available from the flows handles the asynchronous model of EVM transactions and calls, it lacks a recovery function should any issue arise with the nonce that is not already handled by the underlying Web3j framework.
 
-8. **EVM / Solidity Project for Atomic Swap**: This project will implement atomic swap of EVM assets like ERC20, ERC721, ERC1155, with live upgradeability to extend with other assets. It is in finalization phase and will be released soon.
+### Checkpointing
+While some further testing and investigation are required, Corda flows' checkpointing may, in some cases, cause an EVM transaction to be repeated. For non-experimental use, proper deduplication of the EVM interface calls should be ensured.
 
-## Experimental Code
+### EVM Events
+The EVM interface, implemented as a Corda service, has a very simplistic and incomplete event registration mechanism. It is not intended for use outside of highly experimental cases, and we recommend that events be registered, filtered, and handled outside of a Corda node functioning as a coordination mechanism used to trigger related flows on Corda nodes.
 
-Please note that this project currently contains some experimental code, particularly related to EVM events subscription and handling. This code is intended to be moved to an external module in the future.
+### Tests and Test Network
+Integration tests rely on a local Hardhat node instance running and initialized with a deployment script that deploys the required contracts. This implies that accounts, keys, and addresses used by the integration tests are hardcoded in a base class used by all the tests.
 
-## Integration Tests
+#### Manual Test Network Setup
+Please refer to the EVM project's README.md for instructions in this regard before executing any Corda project test.
+#### Docker Test Network Setup
+The Corda project has Gradle tasks meant to start a Docker container with the test network ready for executing the integration tests included in the projects.
 
-The integration tests in this project are currently hardcoded to a locally set up test EVM instance. Development is in progress to enable these tests to execute in a fully automated manner, independent of manually set up testing environment.
+You can either start the docker instance by explicitly executing
+```
+./gradlew startDockerContainer
+```
+or passing the useDockerTestnet as a property to your gradle build or test commands, as
+```
+./gradlew build -PuseDockerTestnet
+```
+or
+```
+./gradlew test -PuseDockerTestnet
+```
 
+You must have a recent Docker version installed on your machine.
 ## Build and Run
 
 ### Prerequisites
 
 This project requires and has been tested with the following tools:
 
 - gradle - version v5.6.4
-- node.js - version v16.19.0 (our recommended installer is NVM available [here](https://github.com/nvm-sh/nvm))
-- npm - version v9.7.1 (comes with node.js)
-- npx - version v9.7.1 (to install npx enter `npm install -g npx`)
+- if planning to setup the test network manually
+    - node.js - version v16.19.0 (our recommended installer is NVM available [here](https://github.com/nvm-sh/nvm))
+    - npm - version v9.7.1 (comes with node.js)
+    - npx - version v9.7.1 (to install npx enter `npm install -g npx`)
+- if planning to use the docker container to run the dockerized test network
+    - Docker Desktop (tested with version 4.25.2)
 
-
-### Building and Deploying the Project
+### Building the Project
 
 #### Build
 
 To build the Corda project, enter the following command from the root folder:
+
 ```
 ./gradlew build -x test
 ```
 or
-```
+```  
 ./gradlew build 
+```    
+if you have manually prepared the EVM test environment, or
+```  
+./gradlew build -PuseDockerTestnet
 ```
-if you have manually prepared the EVM test environment.
-
-#### Deploy
-
-Plain and Dockerized deployment is under development
+if you want to use Docker and have Docker installed on your machine.
 
 ### Testing
 
-To run the tests you need to set up the test environment first.
+#### Manual Test Network Setup
 
-To set up the test environment proceed as follows:
-- change directory to samples/testnet and open two terminals to that directory
-- on the first terminal run `npm install` and wait for the required packages to be installed - this step is required once.
-- again on the first terminal run `npx hardhat node` - it will print a number of accounts and will start printing block numbers in the form `Mined empty block range #m to #n`
-- on the second terminal, once the first the hardhat node is running, enter `npx hardhat run deploy.js --network localhost` and wait for the shell prompt to return (without errors)
+To run the tests you need to set up the test environment first, refer to the [Integration Tests / Test Network Setup Section in the EVM Project's README.md](../evm/README.md#integration-tests--test-network-setup). Once you have setup the network manually, from the Corda project root enter:
 
-If you followed the steps above correctly, on the second terminal you will see the following output:
-
-Gold Tethered (GLDT) Token deployed to: 0x5FbDB2315678afecb367f032d93F642f64180aa3</br>
-Silver Tethered (SLVT) Token deployed to: 0xc6e7DF5E7b4f2A278906862b61205850344D4e7d</br>
+```
+./gradlew test
+```
 
+#### Automated Test Network Setup
 
-The test environment is now ready and you can enter the following command:
+Start the tests using the Gradle property useDockerTestnet:
+```
+./gradlew test -PuseDockerTestnet
+```
 
+Optionally start the network using Gradle and then run the tests:
 ```
-./gradlew test 
+./gradlew startDockerContainer
+./gradlew test
 ```

src/r3/atomic-swap/corda/build.gradle

@@ -37,6 +37,64 @@ buildscript {
     }
 }
 
+task startDockerContainer {
+    doLast {
+        exec {
+            environment 'PATH', System.getenv('PATH') + ":/usr/local/bin"
+            commandLine 'sh', '-c', 'if [ -z "$(docker ps -q -f name=r3-harmonia-testnet-instance)" ]; then docker run --rm -d --name r3-harmonia-testnet-instance --platform linux/amd64 -p 8545:8545 edoardoierina/r3-harmonia-testnet; else echo "Container already running."; fi'
+        }
+    }
+}
+
+task waitForContainerReady {
+    doLast {
+        def retries = 0
+        def maxRetries = 36
+        def healthy = false
+        def outputStream = new ByteArrayOutputStream()
+
+        while (!healthy && retries < maxRetries) {
+            exec {
+                environment 'PATH', System.getenv('PATH') + ":/usr/local/bin"
+                commandLine 'sh', '-c', 'docker inspect --format="{{.State.Health.Status}}" r3-harmonia-testnet-instance'
+                standardOutput = outputStream
+                ignoreExitValue = true
+            }
+
+            def healthStatus = outputStream.toString().trim()
+            outputStream.reset() // Reset the output stream for the next iteration
+
+            if (healthStatus == 'healthy') {
+                healthy = true
+            } else {
+                println "Waiting for container to become ready... (${retries}/${maxRetries})"
+                Thread.sleep(5000) // 5 seconds
+                retries++
+            }
+        }
+
+        if (!healthy) {
+            throw new GradleException("Container did not become ready within the expected time")
+        }
+
+        println "Container is now ready."
+    }
+}
+
+task stopDockerContainer  {
+    doLast {
+        try {
+            exec {
+                environment 'PATH', System.getenv('PATH') + ":/usr/local/bin"
+                commandLine 'sh', '-c', 'docker stop r3-harmonia-testnet-instance'
+                ignoreExitValue = true
+            }
+        } catch (Exception e) {
+            println "An error occurred while stopping the Docker container: ${e.message}"
+        }
+    }
+}
+
 allprojects { // Properties that you need to compile your project (The application)
     apply from: "${rootProject.projectDir}/repositories.gradle"
     apply plugin: 'net.corda.plugins.cordapp'
@@ -47,6 +105,9 @@ allprojects { // Properties that you need to compile your project (The applicati
         mavenLocal()
         mavenCentral()
         maven { url 'https://download.corda.net/maven/corda' }
+        maven { url 'https://download.corda.net/maven/corda-dev' }
+        maven { url "https://download.corda.net/maven/corda-dependencies" }
+        maven { url "https://download.corda.net/maven/corda-dependencies-dev" }
         maven { url 'https://jitpack.io' }
     }
 
@@ -59,6 +120,25 @@ allprojects { // Properties that you need to compile your project (The applicati
         }
     }
 
+    tasks.withType(Test) {
+        if (project.hasProperty('useDockerTestnet')) {
+            dependsOn startDockerContainer, waitForContainerReady
+            finalizedBy stopDockerContainer
+        }
+    }
+
+    tasks.named('build') {
+        if (project.hasProperty('useDockerTestnet')) {
+            dependsOn startDockerContainer, waitForContainerReady
+            finalizedBy stopDockerContainer
+        }
+    }
+
+    test {
+        maxParallelForks = 1
+        forkEvery = 0
+    }
+
     jar {
         preserveFileTimestamps = false
         reproducibleFileOrder = true

src/r3/atomic-swap/corda/constants.properties

@@ -1,13 +1,12 @@
 cordaReleaseGroup=net.corda
 cordaCoreReleaseGroup=net.corda
-cordaVersion=4.9
-cordaCoreVersion=4.9
+cordaVersion=4.9.8
+cordaCoreVersion=4.9.8
 gradlePluginsVersion=5.0.12
 kotlinVersion=1.2.71
 junitVersion=4.12
-quasarVersion=0.7.13_r3
+quasarVersion=0.7.15_r3
 log4jVersion=2.17.1
 platformVersion=11
 slf4jVersion=1.7.30
-nettyVersion=4.1.68.Final
-cordaNodeDriverVersion=4.9
+nettyVersion=4.1.77.Final

src/r3/atomic-swap/corda/evm-interop-contracts/src/main/kotlin/com/r3/corda/evminterop/DefaultEventEncoder.kt

@@ -4,6 +4,7 @@ import com.r3.corda.evminterop.dto.TransactionReceipt
 import com.r3.corda.evminterop.dto.TransactionReceiptLog
 import net.corda.core.serialization.CordaSerializable
 import org.web3j.abi.DefaultFunctionEncoder
+import org.web3j.abi.TypeEncoder
 import org.web3j.abi.datatypes.*
 import org.web3j.abi.datatypes.generated.Bytes32
 import org.web3j.abi.datatypes.generated.Int256
@@ -35,36 +36,46 @@ object DefaultEventEncoder {
      *         expected address.
      */
     fun encodeEvent(contractAddress: String, eventSignature: String, vararg params: Any): EncodedEvent {
-        val paramTypesString = eventSignature.substringAfter('(').substringBefore(')')
-        val paramTypes = paramTypesString.split(',').map { it.trim() }
+        val paramTypes = eventSignature.substringAfter('(').substringBefore(')').split(",")
 
-        fun toWeb3jType(value: Any, type: String): Pair<Type<out Serializable>, Boolean> {
+        val typesWithValues = params.zip(paramTypes).map { (value, typeString) ->
             val isIndexed = value is Indexed<*>
-            val unwrappedValue = if (isIndexed) (value as Indexed<*>).indexedValue else value
-
-            return Pair(when (type) {
-                "string" -> Utf8String(unwrappedValue as String)
-                "uint256" -> Uint256(unwrappedValue as BigInteger)
-                "uint8" -> Uint8(unwrappedValue as BigInteger)
-                "int256" -> Int256(unwrappedValue as BigInteger)
-                "address" -> Address(unwrappedValue as String)
-                "bool" -> Bool(unwrappedValue as Boolean)
-                "bytes" -> DynamicBytes(unwrappedValue as ByteArray)
-                "bytes32" -> unwrappedValue as Bytes32
-                else -> throw IllegalArgumentException("Unsupported type: $type")
-            }, isIndexed)
+            val actualValue = if (isIndexed) (value as Indexed<*>).indexedValue else value
+
+            val type = when (typeString.trim()) {
+                "string" -> Utf8String(actualValue as String)
+                "uint256" -> Uint256(actualValue as BigInteger)
+                "uint8" -> Uint8(actualValue as BigInteger)
+                "int256" -> Int256(actualValue as BigInteger)
+                "address" -> Address(actualValue as String)
+                "bool" -> Bool(actualValue as Boolean)
+                "bytes" -> DynamicBytes(actualValue as ByteArray)
+                "bytes32" -> actualValue as Bytes32//StaticBytes32(actualValue as ByteArray)
+                else -> throw IllegalArgumentException("Unsupported type: $typeString")
         }
 
-        val web3jParamsWithIndexedInfo = params.zip(paramTypes).map { (value, type) -> toWeb3jType(value, type) }
+            Triple(type, isIndexed, typeString.trim())
+        }
 
-        val indexedParams = web3jParamsWithIndexedInfo.filter { it.second }.map { it.first }
-        val nonIndexedParams = web3jParamsWithIndexedInfo.filterNot { it.second }.map { it.first }
+        val topic0 = Hash.sha3String(eventSignature)
+        val topics = mutableListOf(topic0)
+
+        typesWithValues.filter { it.second }.forEach { (type, _, typeString) ->
+            val topic = when {
+                typeString == "string" || typeString == "bytes" -> if(typeString == "string") Hash.sha3String(type.toString()) else Hash.sha3(
+                    TypeEncoder.encode(type))
+                type is Address -> Numeric.toHexStringWithPrefixZeroPadded(Numeric.toBigInt(type.value), 64) // Ensures 32 bytes length with 0x prefix
+                type is BytesType -> Numeric.toHexStringWithPrefixZeroPadded(BigInteger(type.value), 64)
+                type is NumericType -> Numeric.toHexStringWithPrefixZeroPadded(type.value as BigInteger, 64)
+                else -> throw IllegalArgumentException("Unsupported indexed type: $typeString")
+            }
+            topics.add(topic)
+        }
 
-        val topic0 = Hash.sha3String(whitespaceRegex.replace(eventSignature, ""))
-        val topics = listOf(topic0) + indexedParams.map { Hash.sha3String(it.toString()) }
-        val data = Numeric.prependHexPrefix(DefaultFunctionEncoder().encodeParameters(nonIndexedParams))
+        val data = typesWithValues.filterNot { it.second }
+            .joinToString("") { TypeEncoder.encode(it.first) }
 
-        return EncodedEvent(contractAddress, topics, data)
+        return EncodedEvent(contractAddress, topics, Numeric.prependHexPrefix(data))
     }
 }