Enhance Test Coverage for Receipt Insertion, Batch Processing, and Sender Accounts #572

carlosvdr · 2025-01-06T16:01:12Z

This new tests cover are here to enhance the testing experience and help ensure tap V2 wont affect the processes in a "bigger picture"
As well as added more test types around receiving receipts. For this a new method was added regarding batch inserts for receipts into the DB, trying to be have a more accurate representation of how TAP works on a day to day basis.

Regarding the tests on the "bigger picture" , tests for several layers were added.
Such as a full process from sender account all the way from starting it, creating a sender allocation up to the point of marking allocation as last for redeeming.

Another test added from the sender account manager layer which tries to do the same process as the previous layer but starting from the manager.

And finally a test for the full Tap Agent and simulate as closely as possible the full process of it

coveralls · 2025-01-06T16:06:51Z

Pull Request Test Coverage Report for Build 12959664917

Details

310 of 320 (96.88%) changed or added relevant lines in 7 files are covered.
4 unchanged lines in 1 file lost coverage.
Overall coverage increased (+0.5%) to 78.022%

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
crates/tap-agent/src/agent/sender_account.rs	2	3	66.67%
crates/tap-agent/src/agent/sender_accounts_manager.rs	2	3	66.67%
crates/tap-agent/src/agent/sender_allocation.rs	98	100	98.0%
crates/tap-agent/src/test.rs	202	208	97.12%

Files with Coverage Reduction	New Missed Lines	%
crates/tap-agent/src/agent/sender_allocation.rs	4	89.3%

Totals
Change from base Build 12955204222:	0.5%
Covered Lines:	6983
Relevant Lines:	8950

💛 - Coveralls

suchapalaver

@carlosvdr can you squash/fix the commit history, please 🙏

suchapalaver · 2025-01-14T21:14:30Z

@carlosvdr , approving just to remove the block 👍

crates/tap-agent/src/agent/sender_account.rs

crates/tap-agent/src/agent/sender_accounts_manager.rs

crates/tap-agent/src/agent/sender_account.rs

suchapalaver

I really want to see what @gusinacio thinks, @carlosvdr!

gusinacio · 2025-01-20T19:58:44Z

crates/tap-agent/src/agent/sender_accounts_manager.rs

+    lazy_static! {
+        static ref MOCK_EMPTY_ESCROW_SUBGRAPH: OnceCell<Arc<(MockServer, MockGuard)>> =
+            OnceCell::new();
+    }
+    async fn mock_escrow_subgraph_empty_response() -> Arc<(MockServer, MockGuard)> {
+        MOCK_EMPTY_ESCROW_SUBGRAPH
+            .get_or_try_init(|| async {
+                let mock_ecrow_subgraph_server: MockServer = MockServer::start().await;
+                let _mock_ecrow_subgraph = mock_ecrow_subgraph_server
+                    .register_as_scoped(
+                        Mock::given(method("POST"))
+                            .and(body_string_contains("TapTransactions"))
+                            .respond_with(ResponseTemplate::new(200).set_body_json(
+                                json!({ "data": {
+                                        "transactions": [],
+                                    }
+                                }),
+                            )),
+                    )
+                    .await;
+                Ok::<_, anyhow::Error>(Arc::new((mock_ecrow_subgraph_server, _mock_ecrow_subgraph)))
+            })
+            .await
+            .unwrap()
+            .clone()
+    }


Hey, I see what you did here, but I think it's not needed to use lazy_static, since it's used by a single test.

Also, since you are using register_as_scoped, it doesn't make sense to have a scoped route that is never dropped.

Maybe rstest is better for this context.

I do agree with you on not using lazy_Static for this and also about the register_as_scoped
Regarding rstest I think for this specific one, might not be needed since as you said its only used once for one test, could just put it inside the same test actually.

rstest allow you to "compose" fixtures, which could be nice. But yeah, no worries. I'm just suggesting things in case you want to try it out.

crates/tap-agent/src/tap_agent_test.rs

crates/tap-agent/src/agent/sender_accounts_manager.rs

crates/tap-agent/src/agent/sender_account.rs

crates/tap-agent/src/agent/sender_allocation.rs

gusinacio · 2025-01-21T18:12:47Z

crates/tap-agent/src/agent/sender_allocation.rs

        pgpool: PgPool,
        sender_aggregator_endpoint: String,
        escrow_subgraph_endpoint: &str,
+        rav_request_receipt_limit: u64,
        sender_account: Option<ActorRef<SenderAccountMessage>>,
    ) -> SenderAllocationArgs {


This may be a little out of the scope, but one way to make this better is to add bon and annotate this function with #[bon::builder]. You are able to set default and only provide member that you actually need to change.

https://bon-rs.com/

Feel free to use it or not, it's your choice.

Oh yes, I did liked this bon thing

Hey, you didn't use bon here. Add it and provide a default value.

Suggested change

pgpool: PgPool,

sender_aggregator_endpoint: String,

escrow_subgraph_endpoint: &str,

rav_request_receipt_limit: u64,

sender_account: Option<ActorRef<SenderAccountMessage>>,

) -> SenderAllocationArgs {

pgpool: PgPool,

sender_aggregator_endpoint: String,

escrow_subgraph_endpoint: &str,

#[builder(default = 1000)] rav_request_receipt_limit: u64,

sender_account: Option<ActorRef<SenderAccountMessage>>,

) -> SenderAllocationArgs {

Hey, no updates here?

crates/tap-agent/src/agent/sender_account.rs

crates/tap-agent/src/agent/sender_allocation.rs

gusinacio · 2025-01-24T12:45:22Z

crates/tap-agent/src/agent/sender_allocation.rs

    async fn create_sender_allocation(
        pgpool: PgPool,
        sender_aggregator_endpoint: String,
        escrow_subgraph_endpoint: &str,
+        rav_request_receipt_limit: Option<u64>,
        sender_account: Option<ActorRef<SenderAccountMessage>>,
    ) -> (ActorRef<SenderAllocationMessage>, Arc<Notify>) {


You can remove all the calls to sender_aggregator_endpoint that contains DUMMY_URL.

Suggested change

async fn create_sender_allocation(

pgpool: PgPool,

sender_aggregator_endpoint: String,

escrow_subgraph_endpoint: &str,

rav_request_receipt_limit: Option<u64>,

sender_account: Option<ActorRef<SenderAccountMessage>>,

) -> (ActorRef<SenderAllocationMessage>, Arc<Notify>) {

async fn create_sender_allocation(

pgpool: PgPool,

#[builder(default = DUMMY_URL.to_string())]

sender_aggregator_endpoint: String,

escrow_subgraph_endpoint: &str,

rav_request_receipt_limit: Option<u64>,

sender_account: Option<ActorRef<SenderAccountMessage>>,

) -> (ActorRef<SenderAllocationMessage>, Arc<Notify>) {

any updates here?

gusinacio · 2025-01-24T12:49:42Z

crates/tap-agent/src/agent/sender_allocation.rs

        pgpool: PgPool,
        sender_aggregator_endpoint: String,
        escrow_subgraph_endpoint: &str,
+        rav_request_receipt_limit: u64,
        sender_account: Option<ActorRef<SenderAccountMessage>>,
    ) -> SenderAllocationArgs {


Hey, you didn't use bon here. Add it and provide a default value.

Suggested change

pgpool: PgPool,

sender_aggregator_endpoint: String,

escrow_subgraph_endpoint: &str,

rav_request_receipt_limit: u64,

sender_account: Option<ActorRef<SenderAccountMessage>>,

) -> SenderAllocationArgs {

pgpool: PgPool,

sender_aggregator_endpoint: String,

escrow_subgraph_endpoint: &str,

#[builder(default = 1000)] rav_request_receipt_limit: u64,

sender_account: Option<ActorRef<SenderAccountMessage>>,

) -> SenderAllocationArgs {

gusinacio

In general, I can see a great progress here. I think that bon is still new to you but don't forget about defaults, this allows you to provide only what is different from default values.

gusinacio · 2025-01-25T18:25:01Z

crates/tap-agent/tests/tap_agent_test.rs

+    pub static PREFIX_ID: AtomicU32 = AtomicU32::new(0);
+    let prefix = format!(
+        "test-{}",
+        PREFIX_ID.fetch_add(1, std::sync::atomic::Ordering::SeqCst)
+    );


Since you are not running in an environment with multiple tests, you can remove the prefix and use None

gusinacio · 2025-01-25T18:27:48Z

crates/tap-agent/tests/tap_agent_test.rs

+#[sqlx::test(migrations = "../../migrations")]
+async fn test_start_tap_agent(pgpool: PgPool) {
+    let (prefix, notify, (_actor_ref, _handle)) = start_agent(pgpool.clone()).await;
+    flush_messages(&notify).await;


Do you really need to flush the messages for this test to work?

spawn(..).await is supposed to wait for all senders and allocations to be created. If it works, maybe you can remove TestableActor.

gusinacio · 2025-01-25T18:29:27Z

crates/tap-agent/tests/sender_account_test.rs

+    Mock, MockServer, ResponseTemplate,
+};
+
+pub static PREFIX_ID: AtomicU32 = AtomicU32::new(0);


Do you need prefix id here?

gusinacio · 2025-01-25T18:33:00Z

crates/tap-agent/tests/sender_account_test.rs

+
+    let rav_marked_as_last = sqlx::query!(
+        r#"
+                SELECT * FROM scalar_tap_ravs WHERE last = true;


Could you specify WHERE allocation_id = ALLOCATION_0?

gusinacio · 2025-01-25T18:36:24Z

crates/tap-agent/tests/sender_account_manager_test.rs

+        .cast(SenderAccountMessage::UpdateAllocationIds(HashSet::new()))
+        .unwrap();
+
+    actor_ref.unwrap().stop_children_and_wait(None, None).await;


This line kills the children which makes the prior line useless.
I recommend removing this and it should work the same.

Maybe add an assert to check if SenderAllocation actor is stopped.

gusinacio · 2025-01-25T18:49:25Z

crates/tap-agent/src/agent/sender_allocation.rs

+            .pgpool(pgpool.clone())
+            .escrow_subgraph_endpoint(&mock_escrow_subgraph_server.uri())
+            .sender_account(sender_account)
+            .sender_aggregator_endpoint(get_grpc_url().await)


Remove this line

gusinacio · 2025-01-25T18:49:32Z

crates/tap-agent/src/agent/sender_allocation.rs

+            .pgpool(pgpool.clone())
+            .escrow_subgraph_endpoint(&mock_escrow_subgraph_server.uri())
+            .sender_account(sender_account)
+            .sender_aggregator_endpoint(get_grpc_url().await)


gusinacio · 2025-01-25T18:49:40Z

crates/tap-agent/src/agent/sender_allocation.rs

+            .pgpool(pgpool.clone())
+            .escrow_subgraph_endpoint(&mock_escrow_subgraph_server.uri())
+            .sender_account(sender_account)
+            .sender_aggregator_endpoint(get_grpc_url().await)


gusinacio · 2025-01-25T18:53:58Z

crates/tap-agent/src/agent/sender_account.rs

-        .await;
+        let (sender_account, notify, prefix, _) = create_sender_account()
+            .pgpool(pgpool)
+            .initial_allocation(HashSet::new())


If this is the default implementation, you could use default and not provide it here.

same for all arguments that can be "default"

gusinacio · 2025-01-25T18:58:53Z

crates/tap-agent/src/test.rs

+    initial_allocation: HashSet<Address>,
+    rav_request_trigger_value: u128,
+    max_amount_willing_to_lose_grt: u128,
+    escrow_subgraph_endpoint: Option<&str>,
+    network_subgraph_endpoint: Option<&str>,
+    rav_request_receipt_limit: u64,
+    aggregator_endpoint: Option<Url>,


Suggested change

initial_allocation: HashSet<Address>,

rav_request_trigger_value: u128,

max_amount_willing_to_lose_grt: u128,

escrow_subgraph_endpoint: Option<&str>,

network_subgraph_endpoint: Option<&str>,

rav_request_receipt_limit: u64,

aggregator_endpoint: Option<Url>,

#[Builder(default = HashSet::new())]

initial_allocation: HashSet<Address>,

...

carlosvdr self-assigned this Jan 6, 2025

carlosvdr requested a review from gusinacio January 6, 2025 16:51

carlosvdr marked this pull request as ready for review January 13, 2025 19:39

carlosvdr changed the title ~~Extra tap tests~~ Enhance Test Coverage for Receipt Insertion, Batch Processing, and Sender Accounts” Jan 14, 2025

carlosvdr changed the title ~~Enhance Test Coverage for Receipt Insertion, Batch Processing, and Sender Accounts”~~ Enhance Test Coverage for Receipt Insertion, Batch Processing, and Sender Accounts Jan 14, 2025

suchapalaver requested changes Jan 14, 2025

View reviewed changes

carlosvdr force-pushed the 868b324jv-tap-tests branch from e3ef764 to 025024c Compare January 14, 2025 21:01

carlosvdr requested a review from suchapalaver January 14, 2025 21:01

suchapalaver previously approved these changes Jan 14, 2025

View reviewed changes

suchapalaver reviewed Jan 14, 2025

View reviewed changes

crates/tap-agent/src/agent/sender_account.rs Outdated Show resolved Hide resolved

suchapalaver reviewed Jan 14, 2025

View reviewed changes

crates/tap-agent/src/agent/sender_accounts_manager.rs Outdated Show resolved Hide resolved

carlosvdr dismissed suchapalaver’s stale review via c59262a January 15, 2025 17:41

suchapalaver reviewed Jan 15, 2025

View reviewed changes

crates/tap-agent/src/agent/sender_account.rs Outdated Show resolved Hide resolved

carlosvdr force-pushed the 868b324jv-tap-tests branch from 8140179 to 7930988 Compare January 17, 2025 19:09

carlosvdr requested a review from suchapalaver January 17, 2025 19:09

suchapalaver reviewed Jan 20, 2025

View reviewed changes

gusinacio requested changes Jan 21, 2025

View reviewed changes

carlosvdr force-pushed the 868b324jv-tap-tests branch from f6dd24e to a0ca003 Compare January 23, 2025 22:15

carlosvdr requested a review from gusinacio January 23, 2025 22:16

gusinacio requested changes Jan 24, 2025

View reviewed changes

carlosvdr force-pushed the 868b324jv-tap-tests branch from 78d1b5e to 262efd2 Compare January 24, 2025 18:57

test: Enhance test coverage for tap-agent

34313f8

carlosvdr force-pushed the 868b324jv-tap-tests branch from 262efd2 to 34313f8 Compare January 24, 2025 20:49

carlosvdr requested a review from gusinacio January 24, 2025 20:51

carlosvdr added 2 commits January 24, 2025 14:51

style: add license

f1a2c93

style: cargo clippy

a4bb78a

carlosvdr requested a review from suchapalaver January 24, 2025 21:01

style: cargo fmt

d33ce55

carlosvdr added 2 commits January 24, 2025 18:03

test: fix tests and cleanup

3be8d30

test: remove dummy ulr

46c7745

gusinacio requested changes Jan 25, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Enhance Test Coverage for Receipt Insertion, Batch Processing, and Sender Accounts #572

Enhance Test Coverage for Receipt Insertion, Batch Processing, and Sender Accounts #572

carlosvdr commented Jan 6, 2025 •

edited

Loading

coveralls commented Jan 6, 2025 •

edited

Loading

suchapalaver left a comment

suchapalaver commented Jan 14, 2025

suchapalaver left a comment

gusinacio Jan 20, 2025

gusinacio Jan 21, 2025

carlosvdr Jan 23, 2025

gusinacio Jan 23, 2025

gusinacio Jan 21, 2025

gusinacio Jan 21, 2025

carlosvdr Jan 23, 2025

gusinacio Jan 24, 2025

gusinacio Jan 25, 2025

gusinacio Jan 24, 2025

gusinacio Jan 25, 2025

gusinacio Jan 24, 2025

gusinacio left a comment

gusinacio Jan 25, 2025

gusinacio Jan 25, 2025

gusinacio Jan 25, 2025

gusinacio Jan 25, 2025

gusinacio Jan 25, 2025

gusinacio Jan 25, 2025

gusinacio Jan 25, 2025

gusinacio Jan 25, 2025

gusinacio Jan 25, 2025

gusinacio Jan 25, 2025

gusinacio Jan 25, 2025

gusinacio Jan 25, 2025

Enhance Test Coverage for Receipt Insertion, Batch Processing, and Sender Accounts #572

Are you sure you want to change the base?

Enhance Test Coverage for Receipt Insertion, Batch Processing, and Sender Accounts #572

Conversation

carlosvdr commented Jan 6, 2025 • edited Loading

coveralls commented Jan 6, 2025 • edited Loading

Pull Request Test Coverage Report for Build 12959664917

Details

💛 - Coveralls

suchapalaver left a comment

Choose a reason for hiding this comment

suchapalaver commented Jan 14, 2025

suchapalaver left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gusinacio left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

carlosvdr commented Jan 6, 2025 •

edited

Loading

coveralls commented Jan 6, 2025 •

edited

Loading