Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update the remote_target_path config docs to reflect what all files are uploaded to remote #1291

Open
2 tasks
pankajkoti opened this issue Oct 30, 2024 · 0 comments
Assignees
Labels
area:docs Relating to documentation, changes, fixes, improvement
Milestone

Comments

@pankajkoti
Copy link
Contributor

It’s currently observed that users may assume, after reading the documentation, that all files from the target directory of the dbt project are uploaded to the remote target directory. However, this isn’t yet the case. The remote_target_path configuration, introduced in PR #1224, currently only uploads files from the compiled directory within the target directory of the dbt project—and only when the execution mode is set to ExecutionMode.AIRFLOW_ASYNC. We need to update the documentation for this configuration to clarify this behavior and indicate that additional files will become available as we expand support for uploading them within the operators.

Reference where users assumed https://apache-airflow.slack.com/archives/C059CC42E9W/p1730206906312459.
The question in that Slack conversation said

Hi, folks! I'm trying to configure [remote_target_path](https://astronomer.github.io/astronomer-cosmos/configuration/cosmos-conf.html#remote-target-path) but I doesn't send any file to the s3 location that I've set. What else is needed to configure it?

Acceptance Criteria

  • Docs says what all target directory files are currently uploaded in the remote target path
  • Mention the future work in the docs for supporting more files to be uploaded to remote target path
@pankajkoti pankajkoti self-assigned this Oct 30, 2024
@pankajkoti pankajkoti added this to the Cosmos 1.8.0 milestone Oct 30, 2024
@dosubot dosubot bot added the area:docs Relating to documentation, changes, fixes, improvement label Oct 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:docs Relating to documentation, changes, fixes, improvement
Projects
None yet
Development

No branches or pull requests

1 participant