Hotspots going relayed into after 2021-09-03~ update multiple users

[MMOStars]

Hello, there are at least 10-15 people on Sensecap discord reporting same issue that after 2021-09-03~ update

100% this change broke something for us: f31e45f

This is what we get via peerbook:

| address | name |listen_add|connectio| nat |last_updat|
+----------------------------------------------+-----------+----------+---------+-------+----------+
|/p2p/113gAgf8DTFi4YQFrQYMRhN7WUj25rTwS2mkTM5uj|odd-clay-wo| 2 | 5 |symmetr|15306.244s|
+----------------------------------------------+-----------+----------+---------+-------+----------+

+--------------------------------------------------------------------------------------------------+
| listen_addrs (prioritized) |
+--------------------------------------------------------------------------------------------------+
| /ip4/xxxxxxxxxxxxxx/tcp/44158 |
|/p2p/11KgYQR9h3Rz9cmzVZLwtZiHxThMs731uFTraLiyPmfHcxp7DYx/p2p-circuit/p2p/113gAgf8DTFi4YQFrQYMRhN7W|
+--------------------------------------------------------------------------------------------------+

+---------------------------------------------------------+
| connections |
+---------------------------------------------------------+
|/p2p/11KgYQR9h3Rz9cmzVZLwtZiHxThMs731uFTraLiyPmfHcxp7DYx |
|/p2p/1128k7YK3UfSh5qaAp37qe2jw3LaG6ycQUAAqD33L9v9DoRHdrkK|
|/p2p/112BwJXdn117Y9pNcte1VKpZWFViakbi9tuUMrDsmpnZsM6DEfPb|
|/p2p/11GxM19uKgCpoArMGBUDGeNLxRYM8L9ferCCd8sHkgFdZtVU86P |
|/p2p/11XG5Dw6L1jmu9XE61b1LJ6TdgWm7An1a1Nut34XwMbUyiUWoQV |
+---------------------------------------------------------+

Sensecap dashboard reporting this:

Listen IP(Based on Helium API):
/ip4/xxxxxxxx/tcp/44158,/p2p/1121Gc63SGenk9emVaEe3TsQejLgQHp193ptawGmtp7q34M4X9yQ/p2p-circuit/p2p/113gAgf8DTFi4YQFrQYMRhN7WUj25rTwS2mkTM5ujXP68mCGNw8
NAT Type: Symmetric
Relayed: Yes

Port 44158 has been confirmed to be open as per https://www.yougetsignal.com/tools/open-ports/

The hotspot WAS NOT relayed until last update!
The NAT type on Router is COG (In other words I presume CONE), NOT SYMMETRIC! And this change somehow changes our nat type to symmetric, but it's not symmetric!
https://explorer.helium.com/hotspots/113gAgf8DTFi4YQFrQYMRhN7WUj25rTwS2mkTM5ujXP68mCGNw8/activity
HELIUM EXPLORER DOES NOT REPORT THAT HOTSPOT IS RELAYED!

After we reboot the hotspot, the relay is reporting is NONE and Relayed status is: No.
After 2-4 hours, poof, we become relayed and start listing on both IP + P2P, which should not happen.
I own 3 hotspots, 1 hotspot on same ISP, does not have this type of issue, which is super odd, but this can't because of our router settings as technically we are not choosing to somehow assign 2 listening IPs to API, which should not be happening.

Please help solve this issue ASAP!

[ranezinjo]

I am having THE SAME issue!

[sullyjman]

Been seeing a lot of reports of this in the helium discord

[Vagabond]

This won't actually harm you, although it is an issue. This is likely fallout from the IP changing issue fixes over at #374 . I know the app might scream at you but if you have a valid external IP as the first row in the listen addrs you're fine, they're tried in the order they appear.

I will take a look as to how this is happening again. If you have an affected hotspot you can email me the name/serial number/manufacturer and I can try to work with the manufacturer to pull logs.

[MMOStars]

This won't actually harm you, although it is an issue. This is likely fallout from the IP changing issue fixes over at #374 . I know the app might scream at you but if you have a valid external IP as the first row in the listen addrs you're fine, they're tried in the order they appear.

I will take a look as to how this is happening again. If you have an affected hotspot you can email me the name/serial number/manufacturer and I can try to work with the manufacturer to pull logs.

Good day, my person is odd-clay-worm (https://explorer.helium.com/hotspots/113gAgf8DTFi4YQFrQYMRhN7WUj25rTwS2mkTM5ujXP68mCGNw8)

If you need logs, Jerry (dev) from Sensecap would be willing to work on this 1 with you.

I would like to encourage other posters to share their addresses for this issue to be fixed ASAP.

[Vanlushai]

I have the same issue. . .please let me know how to fix it

[trillion95]

I have the same issue 🙃

[arnoudius]

I have the same issue. Rebooting the device does indeed temporarily fix the issue, but then a few hours later the issue occurs again. I have had this since the 2021-09-03 firmware update. Before that, it was working fine / not switching to relayed after a few hours.

[ArnoutVerbeken]

So MMOstars was able to compile a list of all hotspots affected by this issue. He could not access Github and asked me to post this json file here. See attached.
Maybe it can help you troubleshooting or prioritizing this issue.
As an overview, here are the total numbers:
"{"helium": 1, "calchip": 922, "nebra": 137, "syncro": 21, "bobcat": 1800, "LongAP": 38, "mimic": 0, "rak": 4, "kerlink": 0, "dewi": 0, "sensecap": 281, "unknown": 0}"
Which boils down to the following numbers in percentage (of the total fleet of that brand):
Bobcat ~3%
calchip ~1.6%
Sensecap ~2.4%
Nebra ~2.1%
LongAP ~7.6%
bugged_miners.zip

[arnoudius]

It also seems to happen more frequently now. I just rebooted the miner an hour ago (sensecap m1), and now (+/- one hour later) it's already saying it's relayed again. It seems the service can still be reached though. If I for example use canyouseeme.org, it does report the port is open / can see the service, but on the Sensecap dashboard it reports it as being relayed / p2p status unhealthy. I also don't receive any rewards while it's reporting that it is being relayed. As soon as I restart the device, I start receiving rewards again, until it gets relayed again (after a few hours).

[thatsaint]

This same issue is happening to 2 of my Bobcats

[trillion95]

They need to fix this ASAP!!! I shouldn’t have to restart my hs every time!!!!!

[Vagabond]

If you read my earlier response, you will know that this bug is not actually going to harm you. I will work on fixing it but the relay status is not going to cause you any problems itself.

[kzkz0]

This is happening to my SenseCap M1 Hot Spot after I switched it from WiFi to ethernet cable. The dashboard says it's not relayed but the Helium app says it is and I have a really long string of numbers and characters for my Listen IP.

I have confirmed port 44158 is open and I have created challenges and a few beacons but haven't seen any witnesses since this has happened. If you need more details please let me know.

[GoviCoin]

having same issue , i switched from WiFI to Ethernet, still says relayed...checked API. says p2p
https://explorer.helium.com/hotspots/11MwQFbLUPso73YZQY2mjdbiCBVNjTkCVX2yB8asTFtcBsk7Knk
any help?

[kzkz0]

Update - My Listen IP is now correct and I am no longer relayed I guess it just needed time.

[rockthereelstape]

Same thing just happened to 2 of my hotspots , sensecap

[MMOStars]

Hi devs, was this patch pushed with the latest release 2021.09.14.0? Seems to be still happening. Etc:

https://explorer.helium.com/hotspots/113gAgf8DTFi4YQFrQYMRhN7WUj25rTwS2mkTM5ujXP68mCGNw8

address | name |listen_add|connectio| nat |last_updat|
+----------------------------------------------+-----------+----------+---------+-------+----------+
|/p2p/113gAgf8DTFi4YQFrQYMRhN7WUj25rTwS2mkTM5uj|odd-clay-wo| 2 | 8 |symmetr| 188.172s |
+----------------------------------------------+-----------+----------+---------+-------+----------+

+--------------------------------------------------------------------------------------------------+
| listen_addrs (prioritized) |
+--------------------------------------------------------------------------------------------------+
| /ip4/xxxxxxxxxxxxx/tcp/44158 |
|/p2p/11M9iUMErdCpgx3HvvJoeLBos3vnbTBBg7Pb36922ECYA1Tigwv/p2p-circuit/p2p/113gAgf8DTFi4YQFrQYMRhN7W|
+--------------------------------------------------------------------------------------------------+

+---------------------------------------------------------+
| connections |
+---------------------------------------------------------+
|/p2p/11M9iUMErdCpgx3HvvJoeLBos3vnbTBBg7Pb36922ECYA1Tigwv |
|/p2p/11N6nbmHR81LcWZvxyfJQG5wFxXVqGAgg3qJeEjtKpGGB5muVTt |
|/p2p/112SwEwjkPxkLPqzHr5AodiZovnk4EbvR6BsVpRomJ96ggvx7BZV|
|/p2p/13xhTvtw9bFKgN6zbzKXVptuXzs1cb1bJFjaLP3oWLULviAjktM |
|/p2p/11G9UnvWtaQQsxWKFX9fgk2iqDksqTEndKqKyg4dL5aEG8y1QpQ |
|/p2p/112cPFo5r9UW4uHWfEDCVa5JH4mufmDhQr6SRagdsJPi2RXgY4e9|
|/p2p/112VDXfCV1iTe1GufSkEKhoCRDJa7J6LZYYAp8VVQYiXpDvCGaFE|
|/p2p/1197AexoUhZozLqpagTGw4959faXWcHT699KsVEaoLwdNpoq7Ub |
+---------------------------------------------------------+

[arnoudius]

Doesn't seem like it indeed. I just received the update (2021.09.14.0), but after a few hours it's now saying 'relayed' again on the dashboard (sensecap).

[ArnoutVerbeken]

Same here, new FW 2019.09.14, relayed according to dashboard and peerbook gives this:

|                   address                   |    name     |listen_add|connectio|  nat  |last_updat|
+---------------------------------------------+-------------+----------+---------+-------+----------+
|/p2p/112kt1425oAGpqCPBQS8qyYK3qeKyTv1W1Wzos5J|dizzy-pecan-y|    2     |    9    |symmetr|  85.3s   |
+---------------------------------------------+-------------+----------+---------+-------+----------+

+--------------------------------------------------------------------------------------------------+
|                                    listen_addrs (prioritized)                                    |
+--------------------------------------------------------------------------------------------------+
|                                  /ip4/xx.xx.xx.xx/tcp/44158                                   |
|/p2p/1183NZjePCqisgHEcyHNkrfEMRTQW3Lc3j9VE1GDr8V1PryNQuU/p2p-circuit/p2p/112kt1425oAGpqCPBQS8qyYK3|
+--------------------------------------------------------------------------------------------------+

+---------------------------------------------------------+
|                       connections                       |
+---------------------------------------------------------+
|/p2p/112dADs1gLzPckMkkjDdk9ihp1EdEQmVw5i7hJztf6wyWArvrL6b|
|/p2p/11rkWQQptJzyFKhqTvZRt9W1zBW6Jd1v3FLXg5fGP6K3C3Sj68w |
|/p2p/11tRi5ouPAuodT7j7bPJ36cKZot6txhre6XgaEtcfgpiPEFY3BS |
|/p2p/112BwJXdn117Y9pNcte1VKpZWFViakbi9tuUMrDsmpnZsM6DEfPb|
|/p2p/11D9oVhjTAMZmfgnUn1d3PMtrV8xSNXrRf51fGFnSjWbzovtctW |
|/p2p/11rVv72FaqA3wmWgT1dH7XYRB21GGKm5p9BSzSCniFHbussXKw5 |
|/p2p/112ua5DtuAsH9FuqvWQqvSZbsBXXVjaQ2enJCeDtzsijPs8zctx5|
|/p2p/112ke7UfbGA41DPYEmnt1FZHh4BE9wfL3hCbqy5MxqZRq2i8BmYm|
|/p2p/1183NZjePCqisgHEcyHNkrfEMRTQW3Lc3j9VE1GDr8V1PryNQuU |
+---------------------------------------------------------+

[chowdarynaresh111]

I also have the same issue after restarting the Device it started creating challenges and dashboard shows ok now.
From morning almost 6 hours once this issue started no activity recorded .

[chowdarynaresh111]

I got a solution for this, I have connected to different Wifi network 1 day and re-connected after 1 day with Ethernet.
Now I went to my wifi list and remove the one I have added before by click on FORGET resolved my relayed issue.
I still see two line at API but mining is started back online.

[Vanlushai]

I got a solution for this, I have connected to different Wifi network 1 day and re-connected after 1 day with Ethernet.
Now I went to my wifi list and remove the one I have added before by click on FORGET resolved my relayed issue.
I still see two line at API but mining is started back online.

Is it a permanent fix? is it still out of relay?

[chowdarynaresh111]

It is out of relay now and double IP it took some time to reflect at API.

…

Sent from my iPhone

On Sep 17, 2021, at 12:33 PM, Vanlushai ***@***.***> wrote:  I got a solution for this, I have connected to different Wifi network 1 day and re-connected after 1 day with Ethernet. Now I went to my wifi list and remove the one I have added before by click on FORGET resolved my relayed issue. I still see two line at API but mining is started back online. Is it a permanent fix? is it still out of relay? — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.

[Vanlushai]

It is out of relay now and double IP it took some time to reflect at API.
…
Sent from my iPhone
On Sep 17, 2021, at 12:33 PM, Vanlushai @.***> wrote:  I got a solution for this, I have connected to different Wifi network 1 day and re-connected after 1 day with Ethernet. Now I went to my wifi list and remove the one I have added before by click on FORGET resolved my relayed issue. I still see two line at API but mining is started back online. Is it a permanent fix? is it still out of relay? — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.

Oh. .ok. .but your miner is showing as relayed on the API ? my miners are showing as relayed only on the sensecap dashboard. .

[chowdarynaresh111]

Mine also shows relayed only on Dashboard and API has two lines for Lisner IPV4 and P2P. AFter I removed the wifi connection, relayed status was gone on Dashboard and it took few hours to remove one line from API.

…

On Fri, Sep 17, 2021 at 10:58 PM Vanlushai ***@***.***> wrote: It is out of relay now and double IP it took some time to reflect at API. … <#m_-2580854958049372941_> Sent from my iPhone On Sep 17, 2021, at 12:33 PM, Vanlushai *@*.***> wrote:  I got a solution for this, I have connected to different Wifi network 1 day and re-connected after 1 day with Ethernet. Now I went to my wifi list and remove the one I have added before by click on FORGET resolved my relayed issue. I still see two line at API but mining is started back online. Is it a permanent fix? is it still out of relay? — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android. Oh. .ok. .but your miner is showing as relayed on the API ? my miners are showing as relayed only on the sensecap dashboard. . — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#381 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AVVG5KSY6KF2NYSVVDCN5RLUCP545ANCNFSM5DSNUOXQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

[Vanlushai]

Mine also shows relayed only on Dashboard and API has two lines for Lisner IPV4 and P2P. AFter I removed the wifi connection, relayed status was gone on Dashboard and it took few hours to remove one line from API.
…
On Fri, Sep 17, 2021 at 10:58 PM Vanlushai @.> wrote: It is out of relay now and double IP it took some time to reflect at API. … <#m_-2580854958049372941_> Sent from my iPhone On Sep 17, 2021, at 12:33 PM, Vanlushai @.> wrote:  I got a solution for this, I have connected to different Wifi network 1 day and re-connected after 1 day with Ethernet. Now I went to my wifi list and remove the one I have added before by click on FORGET resolved my relayed issue. I still see two line at API but mining is started back online. Is it a permanent fix? is it still out of relay? — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android. Oh. .ok. .but your miner is showing as relayed on the API ? my miners are showing as relayed only on the sensecap dashboard. . — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#381 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AVVG5KSY6KF2NYSVVDCN5RLUCP545ANCNFSM5DSNUOXQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

👍👍i shall try the same thing. Can you please tell me what miner are you using?

[MMOStars]

Update from me, after exactly 6 days of uptime now my 2/4 units experience this bug?

The unit was untouched based on logs all that time.

https://explorer.helium.com/hotspots/11iRTRDKEQ77etK8EwbB3deLEpK43o3m4NeUwFDJXgr6EKj8xDa

This bug also prevents us from running discovery mode with is plain tilting!

[jakub2907]

Just got the same issue. Hot Spot running fine for weeks on Public IP with ports open and all of a sudden it got Relayed.
I also have two Listen Ips when looking into a Peerbook on SenseCap dashboard.
Here's my Hot Spot that is experiencing this
https://explorer.helium.com/hotspots/11vyHF7YcEVu4xgXxxGAGsZCoQE9bXsV7qLmM8QFdh2q9G31mch/activity

[trillion95]

I’m getting the same problems with latest firmware again! 😕

[trillion95]

[Vagabond]

Has anyone tried enabling uPNP/PMP instead?

Also, if you have two listen IPs, and one is the public IP you expect and one is a relay, you're actually fine as the public IP is preferred. I know the app/explorer makes a big fuss about "relayed" but in this case it is not an actual problem.

[steel171]

Hello everyone! I have the same issue as everyone here.. explorer is fine while the api shows two listening address, one is from my public IP and the other is a p2p address.

i was on wifi and then moved to ethernet (i'm connected to wifi repeater)
do you guys think this has something to do with my miner's low activity?

[PaulVMo]

In case it wasn't obvious, this issue is also impacting validators. At present, approximately 1400 validators are showing an IP4 and P2P address at the same time. While annoying and causing repeat questions on Discord, there is no indication that it causes an performance impact at this point.