Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor(NODE-6187): refactor to use TimeoutContext abstraction #4131

Merged
merged 23 commits into from
Jun 21, 2024

Conversation

W-A-James
Copy link
Contributor

@W-A-James W-A-James commented Jun 4, 2024

Description

What is changing?

New TimeoutContext abstraction
  • Created TimeoutContext abstraction that centralizes creation of Timeout instances and implementation of legacy or CSOT timeout behaviour
  • Pass in TimeoutContext to server.command invocations for all CommandOperation classes
  • Added unit tests for TimeoutContext
Is there new documentation needed for these changes?

What is the motivation for this change?

Release Highlight

Fill in title or leave empty for no highlight

Double check the following

  • Ran npm run check:lint script
  • Self-review completed using the steps outlined here
  • PR title follows the correct format: type(NODE-xxxx)[!]: description
    • Example: feat(NODE-1234)!: rewriting everything in coffeescript
  • Changes are covered by tests
  • New TODOs have a related JIRA ticket

@W-A-James W-A-James changed the base branch from main to NODE-6090 June 4, 2024 21:05
@W-A-James W-A-James force-pushed the NODE-6090 branch 3 times, most recently from e3b7a63 to 61564cd Compare June 5, 2024 15:30
@@ -273,7 +273,7 @@ export class Server extends TypedEventEmitter<ServerEvents> {
public async command(
ns: MongoDBNamespace,
command: Document,
options?: CommandOptions
options: CommandOptions
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there are no instances in the driver where we don't pass options into this method

src/sdam/topology.ts Outdated Show resolved Hide resolved
Comment on lines 70 to 72
/** @internal */
timeoutContext!: TimeoutContext;

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NOTE: This is marked with a non-null assertion because the field will be set during execute_operation and only referenced within that function in the following places at which point, if it is not defined, this would indicate a bug in the driver. I can add an inline comment explaining this

  1. Topology.selectServer
  2. Server.command
  3. ConnectionPool.checkOut
  4. Connection.command

@W-A-James W-A-James marked this pull request as ready for review June 11, 2024 17:54
Copy link
Contributor

@baileympearson baileympearson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great so far!

src/timeout.ts Outdated
Comment on lines 113 to 118
export type TimeoutContextOptions = {
timeoutMS?: number;
serverSelectionTimeoutMS: number;
waitQueueTimeoutMS: number;
socketTimeoutMS?: number;
};
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are going to be some places, I think, where we could use a timeout context but we won't have connection checkout or server selection (I'm thinking RTT pinger, moniters, connect / handshakes, and auth). This might not be relevant here, but once we have timeouts applied to the connection layer any place that uses connection.command() should pass one in.

Could we make serverSelectionTimeoutMS and waitQueueTimeoutMS optional?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we were to make serverSelectionTimeoutMS optional, would we want to return a null/0 timeout when requested for the serverSelectionTimeout?

@@ -118,6 +118,14 @@ export async function executeOperation<
);
}

const timeoutContext = TimeoutContext.create({
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just out of curiosity - why are we instantiating a timeout here and not in resolveOptions?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My intention was that we construct exactly one TimeoutContext per operation with a lifetime equal to that of the operation. The first convergent place we have access to both the client and operation options would be in executeOperation, but the absolute first place is in each of our operation helper methods (insertOne, updateOne, etc).

I am open to updating our AbstractOperation constructor to accept the client options we'd need to construct the TimeoutContext and updating these methods since that would be cleaner and make clear that each operation "owns" its TimeoutContext.

Wouldn't constructing the TimeoutContext in resolveOptions lead to multiple TimeoutContext objects being constructed that likely wouldn't be used.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am open to updating our AbstractOperation constructor to accept the client options we'd need to construct the TimeoutContext and updating these methods since that would be cleaner and make clear that each operation "owns" its TimeoutContext.

Yeah, you're right that resolveOptions would lead to instantiation of timeouts we don't need, specifically when we construct dbs and collections. I'm not sure an operation owning a timeout context is the right choice either - I think that cursors / change streams will need to "own" their timeout context and they'll perform multiple operations inside the same context. Maybe we can leave it as-is for now and reconsider once we've implemented CSOT for cursors.

src/timeout.ts Outdated Show resolved Hide resolved
src/timeout.ts Outdated Show resolved Hide resolved
src/timeout.ts Show resolved Hide resolved
src/sdam/topology.ts Outdated Show resolved Hide resolved
src/sdam/topology.ts Show resolved Hide resolved
test/unit/timeout.test.ts Outdated Show resolved Hide resolved
test/unit/operations/get_more.test.ts Outdated Show resolved Hide resolved
src/cmap/connection_pool.ts Show resolved Hide resolved
@W-A-James
Copy link
Contributor Author

I think we can consider this more fully later, but wouldn't that mean that in the case that an operation auto-connects it's possible for the operation to take up to 2x timeoutMS?

Yes, but more specifically it would be that it could take up to client.timeoutMS + operation.timeoutMS.

@W-A-James
Copy link
Contributor Author

W-A-James commented Jun 12, 2024

which part in particular is dead code? The else block?

On closer inspection, the only place where this isn't dead code is when change streams are selecting the server on encountering an error (see ChangeStream._processErrorIteratorMode and ChangeStream._processErrorStreamMode), since they don't have a TimeoutContext to pass in. Should we address this once we start working on change streams in NODE-5685 or do we want to clean this up here?

Addressing this would also let us make timeoutContext a required field on SelectServerOptions

@W-A-James W-A-James requested a review from baileympearson June 12, 2024 20:41
@baileympearson
Copy link
Contributor

On closer inspection, the only place where this isn't dead code is when change streams are selecting the server on encountering an error (see ChangeStream._processErrorIteratorMode and ChangeStream._processErrorStreamMode), since they don't have a TimeoutContext to pass in. Should we address this once we start working on change streams in NODE-5685 or do we want to clean this up here?

Maybe address it later? We don't actually provide a timeout to selectServer when resuming a change stream, I guess it would default to whatever was set on the client 🤔

test/unit/timeout.test.ts Outdated Show resolved Hide resolved
src/timeout.ts Outdated Show resolved Hide resolved
@W-A-James W-A-James requested a review from durran June 20, 2024 20:48
@baileympearson baileympearson merged commit a0604d0 into NODE-6090 Jun 21, 2024
19 of 26 checks passed
@baileympearson baileympearson deleted the NODE-6187 branch June 21, 2024 16:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants