Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

some working sites don't work correctly in spoofdpi even when not whitelisted #216

Open
Endy3032 opened this issue Aug 29, 2024 · 11 comments
Open

some working sites don't work correctly in spoofdpi even when not whitelisted #216

Endy3032 opened this issue Aug 29, 2024 · 11 comments
Assignees
@xvzc
Labels
bug

Comments

@Endy3032
Copy link

Version

v0.10.12

Operating System

macOS 14.5

How are you running SpoofDPI?

Using a launchagent

spoofdpi -addr 0.0.0.0 -dns-addr 1.1.1.1 -pattern store.steampowered.com -pattern cs.rin.ru -enable-doh

Description

Some ISPs in vietnam are more aggressive in blocking steam (specifically just store.steampowered.com), so i only need steam requests go through spoofdpi

On spoof-dpi v0.10.5, the -url arg works as expected by running

spoof-dpi -addr 0.0.0.0 -dns-addr 1.1.1.1 -url store.steampowered.com -enable-doh -debug

and cs.rin.ru (rin) also loads correctly even if i whitelist it or not

On v0.10.7, both steam and rin still loads just fine when switching to the -pattern arg, however rin won't load anymore if its not whitelisted (while it loads just fine on v0.10.5 without being in the whitelist)

However from v0.10.10 onwards, rin won't load ~99% of the time, regardless if its in the whitelist or not

behavior table

SpoofDPI steam loads rin loads
Not running No Yes
0.10.5 If whitelisted Yes
0.10.7 - 0.10.8 If whitelisted If whitelisted
>0.10.10 If whitelisted No
@Endy3032 Endy3032 added the bug label Aug 29, 2024
@Endy3032 Endy3032 changed the title some sites don't work correctly even when they're not in whitelist some working sites don't work correctly in spoofdpi even when not whitelisted Aug 29, 2024
@xvzc xvzc self-assigned this Aug 29, 2024
@xvzc
Copy link
Owner

xvzc commented Aug 29, 2024

I think i know what causes this issue. I'm sorry to say this, but I'm not very sure if i can fix this because fixing this issue will cause another issue. This may take some time. Sorry.

@xvzc
Copy link
Owner

xvzc commented Aug 29, 2024

Can you also attach a debug log about cs.rin.ru ?

@Endy3032
Copy link
Author

Endy3032 commented Aug 29, 2024 •
edited
Loading

Adding on to this, seems like from v0.10.7 onwards, if i add any -pattern arg when launching, trying a fast.com speedtest will result in ~12mbps (from ~300+mbps) so im sticking to v0.10.5 for the moment

Here are the debug logs for cs.rin.ru, all launched with -dns-addr 1.1.1.1 -enable-doh -pattern cs.rin.ru -debug

v0.10.12 
CONNECT cs.rin.ru:443 HTTP/1.1
Host: cs.rin.ru:443
Proxy-Connection: keep-alive
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/128.0.0.0 Safari/537.36


2024-08-29T23:12:47+07:00 DBG 8477d73b-b507cc06-54aaec22-a4178934 [DNS] resolving cs.rin.ru using doh resolver(https://1.1.1.1/dns-query)
2024-08-29T23:12:47+07:00 DBG 3d3c9afc-50c0f74f-a2f0b98e-221377f6 [DNS] resolved 2404:6800:4005:809::200a from optimizationguide-pa.googleapis.com in 35 ms
2024-08-29T23:12:47+07:00 DBG 3d3c9afc-50c0f74f-a2f0b98e-221377f6 [HTTPS] new connection to the server [2405:4803:c660:14ff:7c00:ef83:8644:4252]:54700 -> optimizationguide-pa.googleapis.com
2024-08-29T23:12:47+07:00 DBG 3d3c9afc-50c0f74f-a2f0b98e-221377f6 [HTTPS] sent connection estabalished to 127.0.0.1:54694
2024-08-29T23:12:47+07:00 DBG 3d3c9afc-50c0f74f-a2f0b98e-221377f6 [HTTPS] client sent hello 2316 bytes
2024-08-29T23:12:47+07:00 DBG 3d3c9afc-50c0f74f-a2f0b98e-221377f6 [HTTPS] writing plain client hello to optimizationguide-pa.googleapis.com
2024-08-29T23:12:47+07:00 DBG 8477d73b-b507cc06-54aaec22-a4178934 [DNS] resolved 2a06:1700:0:3a:43:5352:494e:1337 from cs.rin.ru in 132 ms
2024-08-29T23:12:47+07:00 DBG [PROXY] request from 127.0.0.1:54702
v0.10.10 
CONNECT cs.rin.ru:443 HTTP/1.1
Host: cs.rin.ru:443
Proxy-Connection: keep-alive
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/128.0.0.0 Safari/537.36

DEBU[2024-08-29T23:13:25+07:00] [DNS] resolving cs.rin.ru using doh resolver(https://1.1.1.1/dns-query)
DEBU[2024-08-29T23:13:25+07:00] [DNS] resolved 2404:6800:4005:807::200a from optimizationguide-pa.googleapis.com in 37 ms
DEBU[2024-08-29T23:13:25+07:00] [HTTPS] new connection to the server [2405:4803:c660:14ff:7c00:ef83:8644:4252]:54742 -> optimizationguide-pa.googleapis.com
DEBU[2024-08-29T23:13:25+07:00] [HTTPS] sent connection estabalished to 127.0.0.1:54736
DEBU[2024-08-29T23:13:25+07:00] [HTTPS] client sent hello 2252 bytes
DEBU[2024-08-29T23:13:25+07:00] [HTTPS] writing plain client hello to optimizationguide-pa.googleapis.com
DEBU[2024-08-29T23:13:25+07:00] [DNS] resolved 2a06:1700:0:3a:43:5352:494e:1337 from cs.rin.ru in 117 ms
DEBU[2024-08-29T23:13:26+07:00] [PROXY] request from 127.0.0.1:54745
v0.10.8 
CONNECT cs.rin.ru:443 HTTP/1.1
Host: cs.rin.ru:443
Proxy-Connection: keep-alive
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/128.0.0.0 Safari/537.36

DEBU[2024-08-29T23:13:49+07:00] [DNS] cs.rin.ru resolving with dns over https
DEBU[2024-08-29T23:13:49+07:00] [PROXY] Start HTTPS
DEBU[2024-08-29T23:13:49+07:00] [HTTPS] New connection to the server optimizationguide-pa.googleapis.com [2405:4803:c660:14ff:7c00:ef83:8644:4252]:54776
DEBU[2024-08-29T23:13:49+07:00] [HTTPS] Sent 200 Connection Estabalished to 127.0.0.1:54773
DEBU[2024-08-29T23:13:49+07:00] [HTTPS] Client sent hello 2316bytes
DEBU[2024-08-29T23:13:49+07:00] [HTTPS] Writing plain client hello to optimizationguide-pa.googleapis.com
DEBU[2024-08-29T23:13:49+07:00] [PROXY] Start HTTPS
DEBU[2024-08-29T23:13:49+07:00] [HTTPS] New connection to the server cs.rin.ru 192.168.102.2:54777
DEBU[2024-08-29T23:13:49+07:00] [HTTPS] Sent 200 Connection Estabalished to 127.0.0.1:54774
DEBU[2024-08-29T23:13:49+07:00] [HTTPS] Client sent hello 1830bytes
DEBU[2024-08-29T23:13:49+07:00] [HTTPS] Writing chunked client hello to cs.rin.ru
DEBU[2024-08-29T23:13:49+07:00] [HTTPS] window-size: 0
DEBU[2024-08-29T23:13:49+07:00] [HTTPS] Using legacy fragmentation.
v0.10.7 
CONNECT cs.rin.ru:443 HTTP/1.1
Host: cs.rin.ru:443
Proxy-Connection: keep-alive
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/128.0.0.0 Safari/537.36

DEBU[2024-08-29T23:14:20+07:00] [DNS] cs.rin.ru resolving with dns over https
DEBU[2024-08-29T23:14:20+07:00] [PROXY] Start HTTPS
DEBU[2024-08-29T23:14:20+07:00] [HTTPS] New connection to the server optimizationguide-pa.googleapis.com [2405:4803:c660:14ff:7c00:ef83:8644:4252]:54835
DEBU[2024-08-29T23:14:20+07:00] [HTTPS] Sent 200 Connection Estabalished to 127.0.0.1:54832
DEBU[2024-08-29T23:14:20+07:00] [HTTPS] Client sent hello 2316bytes
DEBU[2024-08-29T23:14:20+07:00] [HTTPS] Writing plain client hello to optimizationguide-pa.googleapis.com
DEBU[2024-08-29T23:14:20+07:00] [PROXY] Start HTTPS
DEBU[2024-08-29T23:14:21+07:00] [PROXY] Request from 127.0.0.1:54837
v0.10.5 
CONNECT cs.rin.ru:443 HTTP/1.1
Host: cs.rin.ru:443
Proxy-Connection: keep-alive
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/128.0.0.0 Safari/537.36

DEBU[2024-08-29T23:14:37+07:00] [PROXY] Request from 127.0.0.1:54852

@Endy3032
Copy link
Author

I'm sorry to say this, but I'm not very sure if i can fix this because fixing this issue will cause another issue. This may take some time. Sorry.

It's fine, no worries :D as v0.10.5 has been working great for me for the past few weeks, i guess ill stick to it for the time being

Btw surprisingly seems like v0.11.1 fixed this? No sure if its a fluke or not but i just downloaded the binary and tried restarting it a few times and rin loaded fine on all tries. The slow speedtest problem still persisted tho

v0.11.0 
CONNECT cs.rin.ru:443 HTTP/1.1
Host: cs.rin.ru:443
Proxy-Connection: keep-alive
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/128.0.0.0 Safari/537.36


2024-08-29T23:31:04+07:00 DBG 847c9569-4c1b2a94-1f7f9763-042adad2 [DNS] resolving cs.rin.ru using doh resolver(https://1.1.1.1/dns-query)
2024-08-29T23:31:05+07:00 DBG 847c9569-4c1b2a94-1f7f9763-042adad2 [DNS] resolved 2a06:1700:0:3a:43:5352:494e:1337 from cs.rin.ru in 325 ms
2024-08-29T23:31:09+07:00 DBG 8f96d7df-aaa0e922-ac6a9db3-54a6bb02 [HTTP] dial tcp [2a06:1700:0:3a:43:5352:494e:1337]:80: connect: operation timed out
2024-08-29T23:31:11+07:00 DBG [PROXY] request from 127.0.0.1:56199
v0.11.1 
CONNECT cs.rin.ru:443 HTTP/1.1
Host: cs.rin.ru:443
Proxy-Connection: keep-alive
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/128.0.0.0 Safari/537.36


DBG 2024-08-29T23:31:34+07:00 dfb9aecb-e686f162-8af83ba8-3ed5edf3 [DNS] resolving cs.rin.ru using doh resolver(https://1.1.1.1/dns-query)
DBG 2024-08-29T23:31:34+07:00 a48b5bfd-5d4e2cc6-6c9de3b6-789bb065 [DNS] resolved 2404:6800:4005:821::200a from optimizationguide-pa.googleapis.com in 56 ms
DBG 2024-08-29T23:31:34+07:00 a48b5bfd-5d4e2cc6-6c9de3b6-789bb065 [HTTPS] new connection to the server [2405:4803:c660:14ff:7c00:ef83:8644:4252]:56220 -> optimizationguide-pa.googleapis.com
DBG 2024-08-29T23:31:34+07:00 a48b5bfd-5d4e2cc6-6c9de3b6-789bb065 [HTTPS] sent connection estabalished to 127.0.0.1:56214
DBG 2024-08-29T23:31:34+07:00 a48b5bfd-5d4e2cc6-6c9de3b6-789bb065 [HTTPS] client sent hello 2220 bytes
DBG 2024-08-29T23:31:34+07:00 a48b5bfd-5d4e2cc6-6c9de3b6-789bb065 [HTTPS] writing plain client hello to optimizationguide-pa.googleapis.com
DBG 2024-08-29T23:31:34+07:00 dfb9aecb-e686f162-8af83ba8-3ed5edf3 [DNS] resolved 185.100.87.208 from cs.rin.ru in 121 ms
DBG 2024-08-29T23:31:34+07:00 dfb9aecb-e686f162-8af83ba8-3ed5edf3 [HTTPS] new connection to the server 192.168.102.2:56221 -> cs.rin.ru
DBG 2024-08-29T23:31:34+07:00 dfb9aecb-e686f162-8af83ba8-3ed5edf3 [HTTPS] sent connection estabalished to 127.0.0.1:56215
DBG 2024-08-29T23:31:34+07:00 dfb9aecb-e686f162-8af83ba8-3ed5edf3 [HTTPS] client sent hello 1830 bytes
DBG 2024-08-29T23:31:34+07:00 dfb9aecb-e686f162-8af83ba8-3ed5edf3 [HTTPS] writing chunked client hello to cs.rin.ru
DBG 2024-08-29T23:31:34+07:00 dfb9aecb-e686f162-8af83ba8-3ed5edf3 [HTTPS] window-size: 0
DBG 2024-08-29T23:31:34+07:00 dfb9aecb-e686f162-8af83ba8-3ed5edf3 [HTTPS] using legacy fragmentation
DBG 2024-08-29T23:31:37+07:00 11c81012-d4862c7c-df9fc824-726fb979 [PROXY] request from 127.0.0.1:56222

@Endy3032
Copy link
Author

Heres a table with the speedtest results, all launched with -dns-addr 1.1.1.1 -enable-doh
Only patterns provided are cs.rin.ru and store.steampowered.com

SpoofDPI version Speed (with -pattern) Speed (without -pattern)
v0.10.5 Fast Fast
v0.10.7/8 Slow Fast
>v0.10.10 Slow Slow

@xvzc
Copy link
Owner

xvzc commented Aug 29, 2024

How much slow is it?

@Endy3032
Copy link
Author

Endy3032 commented Aug 30, 2024 •
edited
Loading

it went from ~310mbps down to 12mbps (according to fast.com)
for other sites the base speed fluctuates but when using any version that's labeled as slow, the speed plummets to ~12mbps everywhere

@Ledorub
Copy link
Collaborator

Ledorub commented Aug 30, 2024

@Endy3032, have you tried without DoH? Is there any difference?

@Endy3032
Copy link
Author

Just did some further testing (this time only enabling DoH) and every version from v0.10.10 got the slow speed of ~12mbps

Updated table (again fast is ~310mbps while slow is ~12mbps)

SpoofDPI version no args pattern only doh only pattern + doh
v0.10.5 Fast Fast Fast Fast
v0.10.7/8 Fast Slow Fast Slow
>v0.10.10 Slow Slow Slow Slow

seems like v0.10.7 slowed down pattern checking and v0.10.10 slowed down DoH

@Ledorub
Copy link
Collaborator

Ledorub commented Sep 17, 2024 •
edited
Loading

I performed four tests and got the following results:

Test Download, Mb/s Upload, Mb/s Ping, ms
w/ proxy 1 766.33 655.47 5
w/o proxy 1 756.40 462.48 5
w/ proxy 2 764.77 548.30 5
w/o proxy 2 766.73 739.83 7

Looks like the SpoofDPI does not affect the speed much. The upload speed seems to be less stable even without proxy.

I ran the SpoofDPI v0.12.0 without DoH and with 1.1.1.1 as a DNS server, -system-proxy=false.
The tests were performed sequentially, in the same order they appear in the table.
I used Speedtest via Firefox 129.0.2 on macOS Sonoma 14.6.1. My plan speed is 800 Mb/s.

@Endy3032
Copy link
Author

I ran the SpoofDPI v0.12.0 without DoH and with 1.1.1.1 as a DNS server, -system-proxy=false.

seems like it is indeed the system proxy thats causing the slowdown on the host from some light testing

however one new super weird problem i noticed is that now some sites refuse to even load at all on 0.10.7 onwards if i let system proxy run despite working normally back when i tested it above. and since my phone connects to my mac hotspot with the mac as a proxy i couldnt use this

yet again 0.10.5 still works flawlessly for me (-addr 0.0.0.0 -dns-addr 1.1.1.1 -enable-doh -debug -url store.steampowered.com -url cs.rin.ru) so im quite confused lol

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@xvzc xvzc

Labels
bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@Ledorub @xvzc @Endy3032