Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] Signing in with magic links redirects to sign-in page, with 401 Unauthorized for GET /api/user/workspace #20

Closed
laefk opened this issue Jan 12, 2025 · 6 comments
Assignees

Comments

@laefk
Copy link

laefk commented Jan 12, 2025

Bug Description

I have AppFlowy-Cloud running on https://appflowy.example.com and AppFlowy-Web running on https://notes.example.com. When I click a magic link to sign into AppFlowy-Web in the browser, however, I am redirected back to the log-in page.

In my most recent attempt, the magic link I received was https://appflowy.example.com/gotrue/verify?token=ad03366c827ba23c0431905a1f3c2a9a74e17133572c2549eeec88e8&type=magiclink&redirect_to=https://notes.example.com/.

After clicking that link, the URL I finally landed on was https://notes.example.com/login?redirectTo=https%3A%2F%2Fnotes.example.com%2Fapp%23access_token%3D[jwt]%26expires_at%3D1736658783%26expires_in%3D7200%26refresh_token%3D[token]%26token_type%3Dbearer%26type%3Dmagiclink.

Steps to Reproduce

  1. Start AppFlowy-Cloud and AppFlowy-Web on separate subdomains, both working on :443.
  2. Visit the AppFlowy-Web interface in your web browser.
  3. Enter your email address for sign in.
  4. Click "Continue".
  5. Click the magic sign-in link received via email.

Expected Behavior

The web interface should sign me in.

Browser and Version

Firefox 133.0

AppFlowy Version(s)

4c71e62

Screenshots

No response

Logs and Console Output

I observe that when I visit the magic link, appflowy.example.com places two cookies (sb-access-token and sb-refresh-token) in my browser:

Set-Cookie sb-access-token=[jwt]; Path=/; Expires=Mon, 13 Jan 2025 03:13:03 GMT; Max-Age=86400; HttpOnly; Secure
Set-Cookie sb-refresh-token=[token]; Path=/; Expires=Mon, 13 Jan 2025 03:13:03 GMT; Max-Age=86400; HttpOnly; Secure

While it's not specified in the Set-Cookie header, Firefox considers these cookies to be SameSite=None.

After the redirect, the following errors are reported in the console:

XHRGET https://appflowy.example.com/api/user/workspace [HTTP/1.1 401 Unauthorized 74ms]
Object { stack: "ee@https://notes.example.com/static/js/index-DdrFhRaK.js:250:61156\nOh@https://notes.example.com/static/js/index-DdrFhRaK.js:252:1045\nf@https://notes.example.com/static/js/index-DdrFhRaK.js:252:5894\nEventHandlerNonNull*sC</<@https://notes.example.com/static/js/index-DdrFhRaK.js:252:5963\nsC<@https://notes.example.com/static/js/index-DdrFhRaK.js:252:5355\nrd@https://notes.example.com/static/js/index-DdrFhRaK.js:254:512\npromise callback*_request@https://notes.example.com/static/js/index-DdrFhRaK.js:255:1079\nrequest@https://notes.example.com/static/js/index-DdrFhRaK.js:254:1879\nNa.prototype[t]@https://notes.example.com/static/js/index-DdrFhRaK.js:255:1520\nhh/<@https://notes.example.com/static/js/index-DdrFhRaK.js:250:55753\nik@https://notes.example.com/static/js/index-DdrFhRaK.js:257:83565\ngetUserWorkspaceInfo@https://notes.example.com/static/js/index-DdrFhRaK.js:257:103616\npt/T<@https://notes.example.com/static/js/app.hooks-DoynRcsi.js:7:5473\npt/<@https://notes.example.com/static/js/app.hooks-DoynRcsi.js:7:6998\nId@https://cdn.jsdelivr.net/npm/[email protected]/umd/react-dom.production.min.js:165:137\nXb@https://cdn.jsdelivr.net/npm/[email protected]/umd/react-dom.production.min.js:200:286\nMi@https://cdn.jsdelivr.net/npm/[email protected]/umd/react-dom.production.min.js:189:119\ndb@https://cdn.jsdelivr.net/npm/[email protected]/umd/react-dom.production.min.js:79:182\nSk@https://cdn.jsdelivr.net/npm/[email protected]/umd/react-dom.production.min.js:198:175\nyb@https://cdn.jsdelivr.net/npm/[email protected]/umd/react-dom.production.min.js:196:166\nOi@https://cdn.jsdelivr.net/npm/[email protected]/umd/react-dom.production.min.js:187:248\nS@https://cdn.jsdelivr.net/npm/[email protected]/umd/react.production.min.js:17:26\nU@https://cdn.jsdelivr.net/npm/[email protected]/umd/react.production.min.js:21:60\nEventHandlerNonNull*@https://cdn.jsdelivr.net/npm/[email protected]/umd/react.production.min.js:21:239\n@https://cdn.jsdelivr.net/npm/[email protected]/umd/react.production.min.js:10:179\n@https://cdn.jsdelivr.net/npm/[email protected]/umd/react.production.min.js:10:195\n@https://cdn.jsdelivr.net/npm/[email protected]/umd/react.production.min.js:31:3\n", message: "Request failed with status code 401", name: "AxiosError", code: "ERR_BAD_REQUEST", config: {…}, request: XMLHttpRequest, response: {…} }
app.hooks-DoynRcsi.js:7:5527

The body returned for GET https://appflowy.example.com/api/user/workspace is as follows:

No Authorization header

When that request is made, no cookies are sent with it. Here's the full request:

GET /api/user/workspace HTTP/1.1
Host: appflowy.example.com
User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:133.0) Gecko/20100101 Firefox/133.0
Accept: application/json, text/plain, */*
Accept-Language: en-CA,en;q=0.8,fr;q=0.5,fr-FR;q=0.3
Accept-Encoding: gzip, deflate, br, zstd
Origin: https://notes.example.com
DNT: 1
Sec-GPC: 1
Connection: keep-alive
Referer: https://notes.example.com/
Sec-Fetch-Dest: empty
Sec-Fetch-Mode: cors
Sec-Fetch-Site: same-site
Pragma: no-cache
Cache-Control: no-cache

And the full response:

HTTP/1.1 401 Unauthorized
Server: nginx/1.26.0 (Ubuntu)
Date: Sun, 12 Jan 2025 03:13:07 GMT
Content-Type: text/plain; charset=utf-8
Content-Length: 23
Connection: keep-alive
x-request-id: 28ef515de2dfa5b82dfade1135ce3283
Access-Control-Allow-Origin: https://notes.example.com
Access-Control-Allow-Methods: GET, POST, PUT, DELETE, PATCH, OPTIONS
Access-Control-Allow-Headers: Content-Type, Authorization, Accept, Client-Version
Access-Control-Max-Age: 3600

Additional Context

My AppFlowy-Cloud is on commit 505b4ca8fce2a2c0f99cd48ddc44892ff4b7911b.

@khorshuheng
Copy link
Collaborator

khorshuheng commented Jan 13, 2025

I wasn't able to reproduce this with AppFlowy Web and AppFlowy Cloud deployed on localhost, using the default deploy.env with only GOTRUE_SMTP_* env variables changed. Neither was I able to reproduce this with our official appflowy web (appflowy.com). Will try and see if i can reproduce this issue on a fresh installation on a VM.

I would like to check a few things:

  1. Does this happens only with magic link sign in?
  2. Are you able to login via magic link on our official AppFlowy Web page?

If I can't reproduce this, will you be able to join our discord, where i can contact you directly to get more information?

@laefk
Copy link
Author

laefk commented Jan 13, 2025

Out of interest: How is authorisation for GET https://appflowy.example.com/api/user/workspace supposed to work? Is it supposed to send a cookie, or send an Authorization header? If it's the latter, where is that header's value supposed to be pulled from? I'm wondering if my subdomain setup is causing a problem with the SameSite=None cookies or similar.

Does this happens only with magic link sign in?

I've only tried it with magic link sign-in, as I haven't seen a place to enter a password. Is there a different sign-in method that you'd like me to try?

Are you able to login via magic link on our official AppFlowy Web page?

Yes, I was able to sign in using a magic link on the official AppFlowy Web page. (Unrelated, but after that first success, I tried again to see what differences there were in the network requests to my own instance, and the official AppFlowy Web is now refusing to generate a magic link for me. I imagine I've tripped some sort of rate limit?)

If I can't reproduce this, will you be able to join our discord, where i can contact you directly to get more information?

Absolutely!

And, if it helps, here are my config files:

My .env file for AppFlowy-Cloud:

# Fully qualified domain name for the deployment. Replace localhost with your domain,
# such as http://mydomain.com.
FQDN=https://appflowy.example.com

# PostgreSQL Settings
POSTGRES_HOST=postgres
POSTGRES_USER=appflowy
POSTGRES_PASSWORD=[postgres_password]
POSTGRES_PORT=5432
POSTGRES_DB=postgres

# Postgres credential for supabase_auth_admin
SUPABASE_PASSWORD=[supabase_password]

# Redis Settings
REDIS_HOST=redis
REDIS_PORT=6379

# Minio Host
MINIO_HOST=minio
MINIO_PORT=9000

AWS_ACCESS_KEY=minioadmin
AWS_SECRET=minioadmin

# AppFlowy Cloud
## URL that connects to the gotrue docker container
APPFLOWY_GOTRUE_BASE_URL=http://gotrue:9999
## URL that connects to the postgres docker container. If your password contains special characters, instead of using ${POSTGRES_PASSWORD},
## you will need to convert them into url encoded format. For example, `p@ssword` will become `p%40ssword`.
APPFLOWY_DATABASE_URL=postgres://${POSTGRES_USER}:${POSTGRES_PASSWORD}@${POSTGRES_HOST}:${POSTGRES_PORT}/${POSTGRES_DB}
APPFLOWY_ACCESS_CONTROL=true
APPFLOWY_WEBSOCKET_MAILBOX_SIZE=6000
APPFLOWY_DATABASE_MAX_CONNECTIONS=40
## URL that connects to the redis docker container
APPFLOWY_REDIS_URI=redis://${REDIS_HOST}:${REDIS_PORT}

# admin frontend
## URL that connects to redis docker container
ADMIN_FRONTEND_REDIS_URL=redis://${REDIS_HOST}:${REDIS_PORT}
## URL that connects to gotrue docker container
ADMIN_FRONTEND_GOTRUE_URL=http://gotrue:9999
## URL that connects to the cloud docker container
ADMIN_FRONTEND_APPFLOWY_CLOUD_URL=http://appflowy_cloud:8000

# authentication key, change this and keep the key safe and secret
# self defined key, you can use any string
GOTRUE_JWT_SECRET=[jwt_secret]
# Expiration time in seconds for the JWT token
GOTRUE_JWT_EXP=7200

# User sign up will automatically be confirmed if this is set to true.
# If you have OAuth2 set up or smtp configured, you can set this to false
# to enforce email confirmation or OAuth2 login instead.
# If you set this to false, you need to either set up SMTP
GOTRUE_MAILER_AUTOCONFIRM=false
# Number of emails that can be per minute
GOTRUE_RATE_LIMIT_EMAIL_SENT=100

# If you intend to use mail confirmation, you need to set the SMTP configuration below
# You would then need to set GOTRUE_MAILER_AUTOCONFIRM=false
# Check for logs in gotrue service if there are any issues with email confirmation
# Note that smtps will be used for port 465, otherwise plain smtp with optional STARTTLS
GOTRUE_SMTP_HOST=example.com
GOTRUE_SMTP_PORT=587
GOTRUE_SMTP_USER=[email protected]
GOTRUE_SMTP_PASS=[smtp_password]
GOTRUE_SMTP_ADMIN_EMAIL=[email protected]

# This user will be created when GoTrue starts successfully
# You can use this user to login to the admin panel
GOTRUE_ADMIN_EMAIL=[email protected]
GOTRUE_ADMIN_PASSWORD=[admin_passowrd]

# Set this to true if users can only join by invite
GOTRUE_DISABLE_SIGNUP=false

# External URL where the GoTrue service is exposed. Replace `your-host` with your domain.
# For example, if your host is `appflowy.home.com`, API_EXTERNAL_URL should be set to `http://appflowy.home.com/gotrue`
API_EXTERNAL_URL=${FQDN}/gotrue

# GoTrue connect to postgres using this url. If your password contains special characters,
# replace ${SUPABASE_PASSWORD} with the url encoded version. For example, `p@ssword` will become `p%40ssword`
GOTRUE_DATABASE_URL=postgres://supabase_auth_admin:${SUPABASE_PASSWORD}@${POSTGRES_HOST}:${POSTGRES_PORT}/${POSTGRES_DB}

# Refer to this for details: https://github.com/AppFlowy-IO/AppFlowy-Cloud/blob/main/doc/AUTHENTICATION.md
# Google OAuth2
GOTRUE_EXTERNAL_GOOGLE_ENABLED=false
GOTRUE_EXTERNAL_GOOGLE_CLIENT_ID=
GOTRUE_EXTERNAL_GOOGLE_SECRET=
GOTRUE_EXTERNAL_GOOGLE_REDIRECT_URI=${API_EXTERNAL_URL}/callback
# GitHub OAuth2
GOTRUE_EXTERNAL_GITHUB_ENABLED=false
GOTRUE_EXTERNAL_GITHUB_CLIENT_ID=
GOTRUE_EXTERNAL_GITHUB_SECRET=
GOTRUE_EXTERNAL_GITHUB_REDIRECT_URI=${API_EXTERNAL_URL}/callback
# Discord OAuth2
GOTRUE_EXTERNAL_DISCORD_ENABLED=false
GOTRUE_EXTERNAL_DISCORD_CLIENT_ID=
GOTRUE_EXTERNAL_DISCORD_SECRET=
GOTRUE_EXTERNAL_DISCORD_REDIRECT_URI=${API_EXTERNAL_URL}/callback
# Apple OAuth2
GOTRUE_EXTERNAL_APPLE_ENABLED=false
GOTRUE_EXTERNAL_APPLE_CLIENT_ID=
GOTRUE_EXTERNAL_APPLE_SECRET=
GOTRUE_EXTERNAL_APPLE_REDIRECT_URI=${API_EXTERNAL_URL}/callback

# File Storage
# Create the bucket if not exists on AppFlowy Cloud start up.
# Set this to false if the bucket has been created externally.
APPFLOWY_S3_CREATE_BUCKET=true
# This is where storage like images, files, etc. will be stored.
# By default, Minio is used as the default file storage which uses host's file system.
# Keep this as true if you are using other S3 compatible storage provider other than AWS.
APPFLOWY_S3_USE_MINIO=true
APPFLOWY_S3_MINIO_URL=http://${MINIO_HOST}:${MINIO_PORT} # change this if you are using a different address for minio
APPFLOWY_S3_ACCESS_KEY=${AWS_ACCESS_KEY}
APPFLOWY_S3_SECRET_KEY=${AWS_SECRET}
APPFLOWY_S3_BUCKET=appflowy
#APPFLOWY_S3_REGION=us-east-1

# AppFlowy Cloud Mailer
# Note that smtps (TLS) is always required, even for ports other than 465
APPFLOWY_MAILER_SMTP_HOST=example.com
APPFLOWY_MAILER_SMTP_PORT=587
APPFLOWY_MAILER_SMTP_USERNAME=[email protected]
APPFLOWY_MAILER_SMTP_EMAIL=[email protected]
APPFLOWY_MAILER_SMTP_PASSWORD=[smtp_password]
APPFLOWY_MAILER_SMTP_TLS_KIND=opportunistic # "none" "wrapper" "required" "opportunistic"

# Log level for the appflowy-cloud service
RUST_LOG=info

# PgAdmin
# Optional module to manage the postgres database
# You can access the pgadmin at http://your-host/pgadmin
# Refer to the APPFLOWY_DATABASE_URL for password when connecting to the database
PGADMIN_DEFAULT_EMAIL=[email protected]
PGADMIN_DEFAULT_PASSWORD=[admin_password]

# Portainer (username: admin)
PORTAINER_PASSWORD=[admin_password]

# Cloudflare tunnel token
CLOUDFLARE_TUNNEL_TOKEN=

# NGINX
# Optional, change this if you want to use custom ports to expose AppFlowy
NGINX_PORT=3080
NGINX_TLS_PORT=3443

# AppFlowy AI
AI_OPENAI_API_KEY=
AI_SERVER_PORT=5001
AI_SERVER_HOST=ai
AI_DATABASE_URL=postgresql+psycopg://${POSTGRES_USER}:${POSTGRES_PASSWORD}@${POSTGRES_HOST}:${POSTGRES_PORT}/${POSTGRES_DB}
AI_REDIS_URL=redis://${REDIS_HOST}:${REDIS_PORT}
LOCAL_AI_TEST_ENABLED=false
AI_APPFLOWY_BUCKET_NAME=appflowy
AI_APPFLOWY_HOST=http://your-host
AI_AWS_ACCESS_KEY_ID=${AWS_ACCESS_KEY}
AI_AWS_SECRET_ACCESS_KEY=${AWS_SECRET}

# AppFlowy Indexer
APPFLOWY_INDEXER_ENABLED=true
APPFLOWY_INDEXER_DATABASE_URL=postgres://${POSTGRES_USER}:${POSTGRES_PASSWORD}@${POSTGRES_HOST}:${POSTGRES_PORT}/${POSTGRES_DB}
APPFLOWY_INDEXER_REDIS_URL=redis://${REDIS_HOST}:${REDIS_PORT}
APPFLOWY_INDEXER_EMBEDDING_BUFFER_SIZE=5000

# AppFlowy Collaborate
APPFLOWY_COLLABORATE_MULTI_THREAD=false
APPFLOWY_COLLABORATE_REMOVE_BATCH_SIZE=100

# AppFlowy Worker
APPFLOWY_WORKER_REDIS_URL=redis://${REDIS_HOST}:${REDIS_PORT}
APPFLOWY_WORKER_DATABASE_URL=postgres://${POSTGRES_USER}:${POSTGRES_PASSWORD}@${POSTGRES_HOST}:${POSTGRES_PORT}/${POSTGRES_DB}

# AppFlowy Web
APPFLOWY_WEB_URL=https://notes.example.com

My changes to nginx/nginx.conf for AppFlowy-Cloud:

     map $http_origin $cors_origin {
         # AppFlowy Web origin
-        "~^http://localhost:3000$" $http_origin;
+        "~^https://notes.example.com$" $http_origin;
         default "null";
     }

My .env for AppFlowy-Web:

AF_BASE_URL=https://appflowy.example.com
AF_GOTRUE_URL=https://appflowy.example.com/gotrue
#AF_WS_URL=ws://your-domain/ws/v1
# If you are using HTTPS, use wss instead of ws.
AF_WS_URL=wss://appflowy.example.com/ws/v1

@laefk
Copy link
Author

laefk commented Jan 13, 2025

I've confirmed that if I edit the GET https://appflowy.example.com/api/user/workspace request to add the header Authorization: Bearer [jwt], using the JWT that the magic link set as a cookie, the request completes successfully instead of 401ing.

Looking at src/application/services/js-services/http/http_api.ts:93, it appears that the Authorization header gets set using a value pulled from local storage:

return localStorage.getItem('token');

However, as far as I can tell, that value is not stored in my local storage, even after visiting the magic link:
image

@khorshuheng
Copy link
Collaborator

@laefk Could you join the discord https://discord.com/invite/appflowy-903549834160635914 , then post this in the self-host channel? We can troubleshoot this over there.

@Rafal-Hacus
Copy link

Hey,

After a good conversation with one of the app flow backend engineers, please set the following things in the .env file to:

      - GOTRUE_SITE_URL=appflowy-flutter://                           # redirected to AppFlowy application
      - GOTRUE_URI_ALLOW_LIST=https://Appflowyweb.domain.here/** # adjust restrict if necessary

And also make sure that you aren't running it behind yet another proxy [there are some headers that are passed via Appflowy-web which require a certain config of nginx - for me this turned out to be the issue as I had another reverse proxy after the built-in appflowy one that wasn't forwarding those headers]

Cheers!

@laefk
Copy link
Author

laefk commented Jan 17, 2025

We discussed this more in the Discord, and I did a bunch of problem-solving offline. I'm posting my findings back here just in case anyone else comes across the same problem.

When AppFlowy-Web requests a magic link to be generated for an email address, it passes a redirect_to header along with the API request. In my case, that header is set to https://notes.example.com/auth/callback. GoTrue includes that URL as the value for the redirect_to= parameter in the magic link that it generates. If the header isn't found in the request, GoTrue defaults to just using the domain (perhaps the request origin?).

When a user clicks on a magic link, AppFlowy-Web generates an OAuth2 token and redirects the user to the redirect_to URL with the token included in the fragment. If the user is being redirected to /auth/callback, that endpoint takes the fragment and stores the access token for future use by the client. If the user is being redirected to /, that endpoint has no idea what to do with it.

My setup uses an NGINX reverse proxy to serve both notes.example.com and appflowy.example.com. AppFlowy-Cloud also ships with an NGINX Docker container, which in turn reverse proxies into GoTrue, the AppFlowy API, etc.

Effectively, requests to GoTrue are going:

My NGINX -> AppFlowy-Cloud's NGINX Container -> GoTrue

That means that if the first NGINX is neglecting to forward a header when reverse proxying to the NGINX container, the NGINX container won't have the headers it needs for forwarding to other containers. That was what was happening in this case. The Redirect_to header was getting stripped out by my NGINX before the request was being passed to the NGINX container and eventually GoTrue.

I made the following changes to my NGINX config, which fixed the issue:

 map $http_upgrade $connection_upgrade {
         default upgrade;
         ''      close;
 }

 server {

         server_name appflowy.example.com;

+        underscores_in_headers on;

         proxy_http_version 1.1;
         proxy_set_header Upgrade $http_upgrade;
         proxy_set_header Connection $connection_upgrade;

         proxy_set_header Origin $http_origin;

         location / {
                 proxy_pass http://127.0.0.1:3080;
+                proxy_pass_request_headers on;
         }

 }

This makes sure that headers get passed onto the NGINX container, and allows headers with underscores (like Redirect_to).

@laefk laefk closed this as completed Jan 17, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants