Powerwall 3 - TEDAPI: Access Denied Check Gateway Password on FW 24.44.2

[JohnJ9ml]

I believe I saw a similar comment from someone in passing on another issue, but after an hour of searching I can't locate it again:

Last night I suspect my installation (3 PW3 plus Solar) did a firmware upgrade around 22:20. I'm basing that on the Alerts that were raised from 22:21 to 22:23 my time. From that point forward, I'm unable to log in and process with pypowerwall:

jjames@crewchief:pywall$ python -m pypowerwall tedapi
pyPowerwall - Powerwall Gateway TEDAPI Reader [v0.12.1]

• Connecting to https://192.168.91.1... SUCCESS

Enter Powerwall Gateway Password: XXXXXXXXX (same old Wifi PW tail)

Connecting to Powerwall Gateway 192.168.91.1
Access Denied: Check your Gateway Password
Failed to connect to Powerwall Gateway

ERROR: Unable to connect to Powerwall Gateway. Check your password and try again

Setting debug to True reflects all the normal things for such a problem: 403 responses.

Since I can't locate the other postings in another Issue, I thought maybe I'd get something that calls out the issue directly if this is going to be a ripple effect issue as tokens go stale.

[jasonacox]

Hi @JohnJ9ml

There are quite a few similar reports on the Dashboard project but most have been resolved: https://github.com/jasonacox/Powerwall-Dashboard/issues?q=is%3Aissue%20powerwall%203%20login

Some suggestions and notes:

• Try to log in to the WiFi access point with that same password to verify it still works - And run the Tesla One app to ensure it works
• The password you will use for local TEDAPI setup is the one on the sticker on the PW3 itself, not the one on the Gateway.
• Some other notes here: https://github.com/jasonacox/Powerwall-Dashboard?tab=readme-ov-file#powerwall-3
• I think this only applies to PW2 systems but putting it out there: try using the last 5 characters of the password.
• If nothing else works, try switching to cloud mode to get basic metrics. Also, that will give you the Firmware version (please let us know what it is, especially if it is not 24.44.0 510adfd5):

# set up cloud
python3 -m pypowerwall cloud

# test
python3 -m pypowerwall get

# get version

import pypowerwall
pw = pypowerwall.Powerwall(email='[email protected]')
pw.version()

[jasonacox]

Just noticed you posted the firmware version in the subject. 😊 24.44.2

[JohnJ9ml]

This is what happens when your wife thinks you'll sleep in for a change on a weekend morning, you peek at the iPad on your nightstand in the dark just because you're curious whether the system ran on batteries all night, blah blah blah, find everything is offline, and spend an hour searching for related problems - all before even firing off the coffeemaker, let alone had a cupper yet.

This is not an issue. This should have been a discussion. That bad aim is just the results of a pre-caffeine response.

I'll go follow these links, but this is not an issue. Sorry for the badly-placed noise.

[jasonacox]

I'll go follow these links, but this is not an issue.

Actually, if the firmware upgrade (nobody else has reported 24.44.2 yet) changed the access and broke pypowerwall's ability to log in, this would be an issue. Let us know what you discover and if you get it working again. I created a new firmware observation post here: jasonacox/Powerwall-Dashboard#109 (comment)

[JohnJ9ml]

An update:

I can get on with the Tesla One app.

A get from the cloud works okay.

Trying the last five as the password does not work.

The version we already worked out.

I thought I'd seen from someone else on some other post that once they'd connected on the One app things reverted to working, but I didn't have any such luck.

I throttled back to cloud mode for the time being, but I"m very much going to miss the string data, among other things. It helped me discover a rather significant installation "issue".

I'll keep poking.

[jasonacox]

We do have some others reporting an upgrade to the new firmware, but that it is still working for them, so there is hope.

I can get on with the Tesla One app.

Hum... so now makes me think of connection.

my installation (3 PW3 plus Solar)

I know that the PW3 setups basically have a "leader" and "followers". I have to wonder if that changed in your setup during the upgrade. Please forgive my ignorance as I only know about PW2's, but do each of the PW3's have their own password or endpoint? If so, perhaps it moved and you would need to rebuild the route to 192.168.91.1 to the new "leader". I would also try rebooting your system running pypowerwall to see if it was an ARP issue pointing you to the wrong one. Then I would try each of the other PW3 (if they have different passwords? I don't know) and see if that helps.

[JohnJ9ml]

Yes, one is a leader and the other two are followers. I did a restart of the pi earlier today to see if there was a stale route or something. The role of leader is not something that can move: the first one connected to the gateway stays put, and there is daisy chaining via CAT6 to each subsequent PW.

I can ping and I can connect, I just can't authenticate.

BTW, another data point: I had a browser tab just still running the dashboard. That never missed a tick. It's still running. I was thinking about that and concerned that its credentials would eventually expire and that would be the end there too, but it has thus far continued to run.

I'm willing to do any and all, but I never learned about networking at the ARP level (by the time I was messing with that, networking finally got more solid to where I didn't have to), and the only thing I know about Docker is that it came up on a pristine pi when I set it all up to run the dashboard, but I'm happy to dig through any and all that I can to continue on.

Later: I went down and checked, and the IP addresses are still the same on the primary PW. It's still connected through the ethernet connection, still running 192.168.90.2, and even though it's not being used, the IP address of the Wifi connection still looks as I remember it.

[jasonacox]

Ok, that's good to know (how the PW3's are chained together).

BTW, another data point: I had a browser tab just still running the dashboard

Ah! Ok, so that means you are still running the pypowerwall container. Can you take a look at the logs?

docker logs pypowerwall -f

I want to know if it is showing the login issue. If it is, stop it. It could be that we hit some API protection that is rejecting new connections. Leave it offline for 10m or so and then try the python trick to see if you can log in again:

python3 -m pypowerwall.tedapi

If you use the PW3 password (all caps which I assume you know), hopefully you get something like...

pyPowerwall - Powerwall Gateway TEDAPI Reader [v0.12.3]
 - Connecting to https://192.168.91.1... SUCCESS

Enter Powerwall Gateway Password: XXXXXXXX

Connecting to Powerwall Gateway 192.168.91.1

 - Configuration:
   - Site Name: Cox Energy Gateway
   - Battery Commission Date: 2021-09-25T16:05:08-07:00
   - VIN: 1232100-00-E--TG123456789
   - Number of Powerwalls: 2

 - Power Data:
   - Battery Charge: 56.37% (14448Wh of 25629Wh)
   - Battery: -3850W
   - Site: 1738W
   - Load: 782W
   - Solar: 2880W
   - Solar_Rgm: 2913W
   - Generator: 0W
   - Conductor: 0W

 - Configuration and Status saved to config.json and status.json

[JohnJ9ml]

I looked through the docker pypowerwall logs, and there were 403s returned shortly after the firmware upgrade restarted, but then it's just persistent complaints about a TypeError, but functions otherwise.

Curious anyway, I shut down the container for ten minutes.

Before restarting the container, I gave the tedapi test a try from one of my other boxes. It still failed with access denied.

I then restarted the container, but had no problems at all. The Proxy started up, and now it continues to run.

One little note: the proxy server is running pypowerwall 0.11.1.

After restarting the container, I still can't successfully run the pypowerwall.tedapi test from any of the other boxes I use for this.

[jasonacox]

Interesting. Can you paste the startup log or run verify.sh so I can see what mode pypowerwall is using to access your data? It sounds more like it has fallen back to cloud access.

Also, what do you see in the pypowerwall.env file? Does the PW_GW_PWD setting match what you are trying using the pypowerwall.tedapi test?

[JohnJ9ml]

Here is the verify.sh output:

`Verify Powerwall-Dashboard 4.5.5 on Linux - Timezone: America/Los_Angeles

This script will attempt to verify all the services needed to run
Powerwall-Dashboard. Use this output when you open an issue for help:
https://github.com/jasonacox/Powerwall-Dashboard/issues/new

Checking pypowerwall

• Config File pypowerwall.env: GOOD
• Container (pypowerwall): GOOD
• Service (port 8675): GOOD
• Version: 0.11.1 Proxy t64
• Powerwall State: CONNECTED - Firmware: 24.44.2
• Site Name: Blue Rancho
• Gateway TEDAPI: Available (192.168.91.1)
• TEDAPI Vitals: Connected - Mode: full
• Cloud Mode: NO

Checking telegraf

• Config File telegraf.conf: GOOD
• Local Config File telegraf.local: GOOD
• Container (telegraf): GOOD
• Version: Telegraf 1.28.2 (git: HEAD@8d9cf395)

Checking influxdb

• Config File influxdb.conf: GOOD
• Environment File influxdb.env: GOOD
• Container (influxdb): GOOD
• Service (port 8086): GOOD
• Filesystem (./influxdb): GOOD
• Version: InfluxDB shell version: 1.8.10

Checking grafana

• Config File grafana.env: GOOD
• Container (grafana): GOOD
• Service (port 9000): GOOD
• Filesystem (./grafana): GOOD
• Version: Grafana CLI version 9.1.2

Checking weather411

• Container (weather411): GOOD
• Service (port 8676): GOOD
• Weather: {"temperature": 58.19}
• Version: 0.2.3

All tests succeeded.
`

Yes, password in the pypowerwall.env file matches.

[jasonacox]

TEDAPI Vitals: Connected - Mode: full

That's awesome. I assume going to http://localhost:8675/vitals (where localhost is the address of your dashboard) produces valid response? Do you see all the right data on your dashboard?

I get mixed up sometimes because of all the concurrent discussions. Is everything working now?

[JohnJ9ml]

Yeah, it's all a jumble after a while, isn't it?

Yes, vitals works on the dashboard host.

No, the pypowerwall.tedapi call talking from the other machines that worked fine right up until the FW upgrade do not work.

Both other machines can still ping, can still make a connection to 192.168.91.1, but I fail on authentication.

Not to put too much focus on this, but the one little bit I notice along the way while we experiment with the Dashboard notion is this: the dashboard connection / docker image is running against v0.11.1, whereas the other machines are working with 0.12.1. I don't suppose something in the PW side might have gone backward there? For example, is it possible the Bearer vs Basic bit is somehow coming into play, for example? This is a total swag, but right now this is a Sesame Street problem, and the one thing not like the others is the one that's working.

[jasonacox]

the pypowerwall.tedapi call talking from the other machines that worked fine right up until the FW upgrade do not work.

To help eliminate some variables, would you mind trying to run the pypowerwall.tedapi on the host running the Dashboard? Also, you can upgrade the version of the python library without impacting the docker container:

pip install pypowerwall==0.12.1

python3 -m pypowerwall.tedapi

On your other hosts, to verify that it isn't the 0.12.1 version, you can downgrade the library:

pip install pypowerwall==0.11.1

python3 -m pypowerwall.tedapi

For example, is it possible the Bearer vs Basic bit is somehow coming into play

The changes between 0.11.1 and 0.12.1 related to TEDAPI are the cache reduction and addition of the components payload. But as you say, worth exploring just in case. The diff: v0.11.1...v0.12.1

The other difference in your case is that it is a different host which is why I asked to test on the Dashboard host. Also, the other reason for access denied is if you are hitting anything but the 192.168.91.1 endpoint. However unlikely, if after the upgrade somehow the other machines are routing to the wrong mac address, it could trigger the deny based on that. Long shot, but just trying to uncover differences.