Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Possible port lock situation when trying to connect to a local, not running, NATS server #847

Open
LaurensVergote opened this issue Nov 28, 2023 · 4 comments · May be fixed by #850
Open
Labels
needs info Additional info is needed

Comments

@LaurensVergote
Copy link
Contributor

LaurensVergote commented Nov 28, 2023

Observed behavior

When creating a connection to a local hostname, it's possible that the TcpClient created will grab the target port as its local port. This seems to result in a connection successfully being established and the client application grabbing the port, making it impossible for the nats server to ever start up later on using that port.

output of netstat (table format for readability):

netstat -abon | findstr 4222

Proto Local Address Foreign Address State PID
TCP 10.3.1.54:4222 10.3.1.54:4222 ESTABLISHED 4708

Possible solutions:

  • Use the TcpClient constructor with IPEndpoint input (TcpClient)
  • After constructing the TcpClient, modify the internal socket (TcpClient.Client) to exclude the target port from being used as the local endpoint's port

Comments:
While I'm not 100% certain on the root cause of this, the above description is my best educated guess on why this would happen.
I've only noticed it on dev machines that run the host process, which tries to connect every 1 second to NATS, and while the target local nats-server, is not running.

This results in not being able to start the local nats-server anymore.

I know it's very niche but it's better if its noted down at least, even if there's no intent to fix it or spend time on it.

Expected behavior

The NATS client does not use the target endpoint's port as the local endpoint's when connecting with a local endpoint

Server and client version

Server version not relevant.
client version: 1.1.1

Host environment

Running on Windows 10 Enterprise, loading the NATS client library as .netstandard 2.0 from a .NET 6.0 host process.

Steps to reproduce

Continually call Connect until you randomly get the target's endpoint port as your local endpoint's port.

@LaurensVergote LaurensVergote added the defect Suspected defect such as a bug or regression label Nov 28, 2023
@scottf
Copy link
Collaborator

scottf commented Nov 28, 2023

I don't see it. Why would Connection.cs#L386 touch any port at all, and why would the ensuing code touch any port other than the specified port.

If it does, and it's inside the TcpClient code, this would affect every C# project ever.

Do you have any local name resolution in the environment?
Are you possibly starting a second server somehow?
Can you post your "Continually call Connect" code?

@scottf scottf added needs info Additional info is needed and removed defect Suspected defect such as a bug or regression labels Nov 28, 2023
@LaurensVergote
Copy link
Contributor Author

Hi @scottf, I created a small console app that simulates this problem
https://gist.github.com/LaurensVergote/f4270f3a22ea4960eb4dfadc2ed63dc9
It's not pretty code since it's cobbled together, but should demonstrate what I mean is happening.
You can put a breakpoint on the Console.WriteLine to inspect the objects.
In normal circumstances, the ConnectAsync() call will throw an exception saying there is nothing running on the target port.

@LaurensVergote
Copy link
Contributor Author

After some further investigation, I noticed that this issue does not happen on an initial connect, only on a reconnect.
I currently have a fix that works in a test application, will put it in the NATS.Client code to test if it resolves the issue.

@LaurensVergote
Copy link
Contributor Author

LaurensVergote commented Dec 19, 2023

Reproduction using just the NATS.Client library: https://gist.github.com/LaurensVergote/ba4b4ea1825f2ff5fed3d1aee9214dd8
Just make sure a nats-server.exe is available next to the compiled binary

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
needs info Additional info is needed
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants