Nexarian/re work proxy #130

Nexarian · 2025-01-10T04:37:19Z

No description provided.

jasonacox · 2025-01-14T05:33:45Z

proxy/server.py

+#    and always use those if available (required for Docker)
+# Configuration - Environment variables
+type PROXY_CONFIG = Dict[CONFIG_TYPE, str | int | bool | None]
+type PROXY_STATS = Dict[PROXY_STATS_TYPE, str | int | bool | None | PROXY_CONFIG | Dict[str, int]]


The type constructs make this Python 3.12+ only. How difficult would it be to make it backward compatible for Python 3? I can update the library to be >= 3.12 but that seems like a big lift. Thoughts?

I could, but I think if we do that we should move this to a class. Look at how unwieldy the type is... A dataclass type would be better and still backwards compatible.

jasonacox · 2025-01-14T05:44:06Z

proxy/server.py

+                content = b"Error during proxy"
+                content_type = "text/plain"
+
+        if self.configuration['PW_BROWSER_CACHE'] > 0 and content_type in ['text/css', 'application/javascript', 'image/png']:


I believe this should be:

if self.configuration[CONFIG_TYPE.PW_BROWSER_CACHE] > 0 and content_type in ['text/css', 'application/javascript', 'image/png']:

jasonacox · 2025-01-14T05:58:31Z

proxy/server.py

+                VERSION=status.get("version", ""),
+                HASH=status.get("git_hash", ""),
+                EMAIL=self.configuration[CONFIG_TYPE.PW_EMAIL],
+                STYLE=self.configuration[CONFIG_TYPE.PW_STYLE],


I'm not able to get the proxy to render the animation e.g. http://localhost:9999/example.html and it is throwing an error here:

File "/Users/jason/Code/pypowerwall/proxy/server.py", line 824, in handle_static_content content = content.decode(UTF_8).format( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ KeyError: '\n "configuration"' ----------------------------------------

Format doesn't work, but replace does? I'm not sure why. I've switched it back.

@jasonacox Is this correct?

No, it should be this:

I think you are not sending the local assets. I'll see if I can figure out why.

I notice that the opentype files are not getting served - need to debug a bit more....

Old / Existing server.py

01/14/2025 08:02:32 PM [proxy] [DEBUG] Served from local web root: /favicon.ico type text/plain 01/14/2025 08:02:32 PM [proxy] [DEBUG] Served from local web root: /example.html type text/html 01/14/2025 08:02:32 PM [proxy] [DEBUG] Served from local web root: /index.html type text/html 01/14/2025 08:02:32 PM [proxy] [DEBUG] Served from local web root: /app.css type text/css 01/14/2025 08:02:32 PM [proxy] [DEBUG] Served from local web root: /vendor.js type application/javascript 01/14/2025 08:02:32 PM [proxy] [DEBUG] Served from local web root: /app.js type application/javascript 01/14/2025 08:02:32 PM [proxy] [DEBUG] Served from local web root: /clear.js type application/javascript 01/14/2025 08:02:32 PM [proxy] [DEBUG] Served from local web root: /40.17c71172308436a079d1.js type application/javascript 01/14/2025 08:02:32 PM [proxy] [DEBUG] Served from local web root: /39.17c71172308436a079d1.js type application/javascript 01/14/2025 08:02:32 PM [proxy] [DEBUG] Served from local web root: /1.17c71172308436a079d1.js type application/javascript 01/14/2025 08:02:32 PM [proxy] [DEBUG] Served from local web root: /eca1317ee8a99162d0d0e2df77330cec.otf type font/opentype 01/14/2025 08:02:32 PM [proxy] [DEBUG] Served from local web root: /230aeae00823cd3b622d093948d9c433.png type image/png 01/14/2025 08:02:32 PM [proxy] [DEBUG] Served from local web root: /012955c70685614a5639d326f41890bd.png type image/png 01/14/2025 08:02:32 PM [proxy] [DEBUG] Served from local web root: /4e28cc8f2bdf3ba640331daa2c453341.png type image/png 01/14/2025 08:02:32 PM [proxy] [DEBUG] Served from local web root: /cb0da8a8999c06735455bf5056a5cd78.png type image/png 01/14/2025 08:02:32 PM [proxy] [DEBUG] Served from local web root: /e19c20e966bde501f94e41cd0322dbe8.otf type font/opentype

New server.py

01/14/2025 08:07:11 PM [proxy] [DEBUG] Served from local web root: /favicon.ico type text/plain 01/14/2025 08:07:13 PM [proxy] [DEBUG] Served from local web root: /example.html type text/html 01/14/2025 08:07:14 PM [proxy] [DEBUG] Served from local web root: /index.html type text/html 01/14/2025 08:07:14 PM [proxy] [DEBUG] Served from local web root: /app.css type text/css 01/14/2025 08:07:14 PM [proxy] [DEBUG] Served from local web root: /vendor.js type application/javascript 01/14/2025 08:07:14 PM [proxy] [DEBUG] Served from local web root: /app.js type application/javascript 01/14/2025 08:07:14 PM [proxy] [DEBUG] Served from local web root: /clear.js type application/javascript 01/14/2025 08:07:14 PM [proxy] [DEBUG] Served from local web root: /40.17c71172308436a079d1.js type application/javascript 01/14/2025 08:07:14 PM [proxy] [DEBUG] Served from local web root: /1.17c71172308436a079d1.js type application/javascript 01/14/2025 08:07:14 PM [proxy] [DEBUG] Served from local web root: /39.17c71172308436a079d1.js type application/javascript 01/14/2025 08:07:14 PM [proxy] [DEBUG] Served from local web root: /012955c70685614a5639d326f41890bd.png type image/png 01/14/2025 08:07:14 PM [proxy] [DEBUG] Served from local web root: /230aeae00823cd3b622d093948d9c433.png type image/png 01/14/2025 08:07:14 PM [proxy] [DEBUG] Served from local web root: /4e28cc8f2bdf3ba640331daa2c453341.png type image/png 01/14/2025 08:07:14 PM [proxy] [DEBUG] Served from local web root: /cb0da8a8999c06735455bf5056a5cd78.png type image/png

jasonacox · 2025-01-15T04:22:23Z

The logging behavior is a bit odd... seems like the way we have the initializer, it is creating a new powerwall instance with each call:

01/14/2025 08:15:13 PM [proxy] [INFO] pyPowerwall Proxy Server - Local Mode
01/14/2025 08:15:13 PM [proxy] [INFO] Connected to Energy Gateway 192.168.91.1 (Cox Energy Gateway)
01/14/2025 08:15:13 PM [proxy] [INFO] TEDAPI Mode Enabled for Device Vitals (full)
01/14/2025 08:15:13 PM [proxy] [DEBUG] 127.0.0.1 "GET /help HTTP/1.1" 200 -
01/14/2025 08:15:13 PM [pypowerwall.tedapi.pypowerwall_tedapi] [DEBUG]  -- tedapi: Request for /api/site_info/site_name
01/14/2025 08:15:13 PM [pypowerwall.tedapi] [DEBUG] Using Cached Payload
01/14/2025 08:15:18 PM [pypowerwall.tedapi.pypowerwall_tedapi] [DEBUG]  -- tedapi: Request for /api/site_info/site_name
01/14/2025 08:15:18 PM [pypowerwall.tedapi] [DEBUG] Using Cached Payload
01/14/2025 08:15:18 PM [proxy] [INFO] pyPowerwall Proxy Server - Local Mode
01/14/2025 08:15:18 PM [proxy] [INFO] Connected to Energy Gateway 192.168.91.1 (Cox Energy Gateway)
01/14/2025 08:15:18 PM [proxy] [INFO] TEDAPI Mode Enabled for Device Vitals (full)
01/14/2025 08:15:18 PM [proxy] [DEBUG] 127.0.0.1 "GET /help HTTP/1.1" 200 -
01/14/2025 08:15:18 PM [pypowerwall.tedapi.pypowerwall_tedapi] [DEBUG]  -- tedapi: Request for /api/site_info/site_name
01/14/2025 08:15:18 PM [pypowerwall.tedapi] [DEBUG] Using Cached Payload
01/14/2025 08:15:18 PM [pypowerwall.tedapi.pypowerwall_tedapi] [DEBUG]  -- tedapi: Request for /api/site_info/site_name
01/14/2025 08:15:18 PM [pypowerwall.tedapi] [DEBUG] Using Cached Payload
01/14/2025 08:15:18 PM [proxy] [INFO] pyPowerwall Proxy Server - Local Mode
01/14/2025 08:15:18 PM [proxy] [INFO] Connected to Energy Gateway 192.168.91.1 (Cox Energy Gateway)
01/14/2025 08:15:18 PM [proxy] [INFO] TEDAPI Mode Enabled for Device Vitals (full)

Nexarian · 2025-01-15T04:38:24Z

It's not creating a new powerwall instance, but init for the handler object is being called repeatedly... That is very weird.

Nexarian · 2025-01-15T04:43:34Z

The problem appears to be this line: with ThreadingHTTPServer((host, port), lambda *args: Handler(*args, configuration, pw, pw_control)) as server:

It's not doing what I expect, which is create one server and serve forever. Instead, it's constantly creating new threads.

Nexarian · 2025-01-15T05:25:34Z

I pushed some changes that I think help, but they still don't fix the static content render. I'll keep digging. Feel free to look more, too if you'd like.

jasonacox · 2025-01-15T05:47:56Z

The power flow animation page is throwing some errors in the Javascript console on the browser that might help:

I did notice that you added a 404 response code to the DISABLED APIs. I know it doesn't like that and needs to be a 200 response so I'll suggest that change.

jasonacox · 2025-01-15T05:49:51Z

proxy/server.py

+        elif path in DISABLED:
+            result = self.send_json_response(
+                {"status": "404 Response - API Disabled"},
+                status_code=HTTPStatus.NOT_FOUND


Status code needs to be 200 for this API. The Powerflow animation code looks for it. The payload can be as it shows.

That's a bit counterintuitive that DISABLED still returns 200. However, I think I sorta understand. It's not NOT FOUND in the proxy itself, it's simply disabled for use in when being called THROUGH the proxy.

jasonacox · 2025-01-15T05:57:38Z

Ah... maybe this is the issue:

$ curl -i http://localhost:9999/app.css
HTTP/1.0 200 OK
Server: BaseHTTP/0.6 Python/3.12.4
Date: Wed, 15 Jan 2025 05:54:44 GMT
Content-type: text/html
Set-Cookie: AuthCookie=local;path=/;
Set-Cookie: UserRecord=local;path=/;
Cache-Control: no-cache, no-store
Content-type: text/css

Notice that two Content-type's are being set. I'll see if I can find where it is doing that in the code.

jasonacox · 2025-01-15T06:00:22Z

Yes, that was it. I'll flag where you need to remove that.

jasonacox · 2025-01-15T06:01:46Z

proxy/server.py

+    def handle_static_content(self, path) -> str:
+        self.proxystats[PROXY_STATS_TYPE.GETS] += 1
+        self.send_response(HTTPStatus.OK)
+        self.send_header('Content-type', 'text/html')


You can't sent this here. The static content could be of a differen type. Just remove this and allow the content type to be set later on after the static content is determined.

jasonacox · 2025-01-15T06:10:44Z

We still have something off that is throwing a JS error in the browser console. It's going to be harder to trace because it is coming from the animation code:

EDIT: nevermind - This is a existing bug when using TEDAPI to connect. It seems that the power flow animation wants to load /networks and it doesn't find it. The fix is to add /networks to the DISABLED list so it doesn't try. I'll add that note. :)

jasonacox · 2025-01-15T06:36:54Z

proxy/server.py

-# Logging
+DISABLED: Final[Set[str]] = set([
+    '/api/customer/registration',
+])


Add '/networks' here

Nexarian · 2025-01-16T00:16:35Z

@jasonacox What else should we test?

Nexarian · 2025-01-16T04:09:42Z

I'm trying to get Powerwall Dashboard working and unfortunately it looks like port 8676, which I was using for my second inverter is allocated to Weather411. I think we may need to pick a different port range for multiple TEGs, or just move Weather411 to something different.

I also have had no luck getting InfluxDB to start up.

jasonacox · 2025-01-16T05:35:00Z

I'm trying to get Powerwall Dashboard working and unfortunately it looks like port 8676, which I was using for my second inverter is allocated to Weather411. I think we may need to pick a different port range for multiple TEGs, or just move Weather411 to something different.

They don't need to be consecutive, but a pattern would make sense. I suggest a 10+ pattern like:

8675, 8685, 8695, 8705, 8715, 8725, 8735, 8745, 8755, 8765, ...

It occurs to me that we should verify that there isn't a user provided overlap. We could assign those automatically if no port is provided.

Did you get "Powerwall Dashboard" working? In the current configuration, it only pulls data for 8675. I think the next act will be to figure out how to combine all the gateway instances into the Dashboard in a meaningful way. Possibly a custom telegraf.local.conf with name override for the separate powerwalls.

Nexarian added 9 commits January 7, 2025 23:41

Beginning work on refactoring the proxy.

0fffceb

WIP

f527031

WIP

290da83

Create proxy stats type

e52bfaf

Continue to refactor

29319d2

Continuing to work on refactoring

9a90606

Continuing to refactor

bd1fb0c

Nearly done with initial refactor

ce4b80d

More refactoring

5ddf1d9

Nexarian mentioned this pull request Jan 10, 2025

Support multiple local Tesla Energy Gateways (Powerwall/Inverters) using the server for local or cloud access #119

Open

Nexarian added 16 commits January 10, 2025 22:49

More refactoring

5c7b27b

Working on multiple pypowerwalls

d0ed440

A bit more refactoring

46c5672

Continuing to refactor

8c4e473

Restore missing functions

5d43a77

WIP on configuration loading

9dc5d50

Add configuration file.

55ef8a4

Add ability to load multiple configurations from environment variables

449201c

Sort

6beda4f

Finishing configuration

5de1c83

Minor fixes

b48c26c

Fixing proxystats

ae196cb

Startup tweaks

0b88120

Fixing startup bugs

900b652

Working proxy

099141f

Removing normalization

a04be08

jasonacox reviewed Jan 14, 2025

View reviewed changes

Fix missing config types

5751d31

Fixing some of the static content

d2241b6

Fixes to URL parsing

4344252

jasonacox reviewed Jan 15, 2025

View reviewed changes

jasonacox and others added 6 commits January 14, 2025 22:39

Remove hardcoded port in URL

b8628fe

Remove hardcoded port from version URL

19fd3b2

Update static content type handling and disable /networks URL

1904725

Move DISABLED to status code 200

84543b3

Remove double set content type

86d82e7

Add networks to disabled

a57bcb3

Separate out strings

4fd8e5e

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Nexarian/re work proxy #130

Nexarian/re work proxy #130

Nexarian commented Jan 10, 2025

jasonacox Jan 14, 2025

Nexarian Jan 15, 2025

jasonacox Jan 14, 2025

Nexarian Jan 15, 2025

jasonacox Jan 14, 2025

Nexarian Jan 15, 2025

Nexarian Jan 15, 2025

Nexarian Jan 15, 2025

jasonacox Jan 15, 2025

jasonacox Jan 15, 2025

jasonacox commented Jan 15, 2025

Nexarian commented Jan 15, 2025

Nexarian commented Jan 15, 2025

Nexarian commented Jan 15, 2025

jasonacox commented Jan 15, 2025

jasonacox Jan 15, 2025

Nexarian Jan 16, 2025

jasonacox commented Jan 15, 2025

jasonacox commented Jan 15, 2025

jasonacox Jan 15, 2025 •

edited

Loading

Nexarian Jan 16, 2025

jasonacox commented Jan 15, 2025 •

edited

Loading

jasonacox Jan 15, 2025

Nexarian Jan 16, 2025

Nexarian commented Jan 16, 2025

Nexarian commented Jan 16, 2025 •

edited

Loading

jasonacox commented Jan 16, 2025

Nexarian/re work proxy #130

Are you sure you want to change the base?

Nexarian/re work proxy #130

Conversation

Nexarian commented Jan 10, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jasonacox commented Jan 15, 2025

Nexarian commented Jan 15, 2025

Nexarian commented Jan 15, 2025

Nexarian commented Jan 15, 2025

jasonacox commented Jan 15, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jasonacox commented Jan 15, 2025

jasonacox commented Jan 15, 2025

jasonacox Jan 15, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jasonacox commented Jan 15, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Nexarian commented Jan 16, 2025

Nexarian commented Jan 16, 2025 • edited Loading

jasonacox commented Jan 16, 2025

jasonacox Jan 15, 2025 •

edited

Loading

jasonacox commented Jan 15, 2025 •

edited

Loading

Nexarian commented Jan 16, 2025 •

edited

Loading