-
Notifications
You must be signed in to change notification settings - Fork 171
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
chore: merge master into ai-video #3103
Conversation
* eth/watchers: Create PriceFeed watcher Makefile: Use mockgen binary from tool dependencies eth/contracts: Add chainlink interfaces source Makefile: Generate Chainlink contracts ABI tools: Add abigen tool to repo eth/contracts: Generate chainlink bindings Makefile: Fix abigen bindings generation Revert everything abigen Turns out there's already bindings exported from the Chainlink lib. go.mod: Add chainlink library eth/watchers: Add pricefeed watcher eth/watchers: Clean-up event watching code eth/watchers: Improve price tracking Revert "go.mod: Add chainlink library" This reverts commit ac415bd. Revert "Revert everything abigen" This reverts commit b7c40b1. eth/contracts: Gen bindings for proxy iface eth/watchers: Use local bindings for contracts eth/watchers: Simplify event subs logic eth/watchers: Simplify&optimize truncated ticker eth/watchers: Update decimals on fetch eth/watchers: Improve handling of decimals eth/watchers: Fix price rat creation eth/watchers: Make sure we use UTC on truncated timer eth/contracts/chainlink: Generate only V3 contract bindings eth/watchers: Watch PriceFeed only with polling eth/watchers: Add a retry logic on price update eth/watchers: Use clog instead of fmt.Printf * eth: Create separate pricefeed client unit This will make the code more testable. * eth: Add tests for pricefeed client * eth/watchers: Add tests to the truncated ticker Gosh that was much harder than I thought * eth/watchers: Add tests for pricefeedwatcher * eth: Add comments to the new components * go fmt * eth: Address minor review comments * eth,eth/watchers: Improve pricefeed watcher interface * eth/watchers: Remove truncated ticker tests
* eth/watchers: Create PriceFeed watcher Makefile: Use mockgen binary from tool dependencies eth/contracts: Add chainlink interfaces source Makefile: Generate Chainlink contracts ABI tools: Add abigen tool to repo eth/contracts: Generate chainlink bindings Makefile: Fix abigen bindings generation Revert everything abigen Turns out there's already bindings exported from the Chainlink lib. go.mod: Add chainlink library eth/watchers: Add pricefeed watcher eth/watchers: Clean-up event watching code eth/watchers: Improve price tracking Revert "go.mod: Add chainlink library" This reverts commit ac415bd. Revert "Revert everything abigen" This reverts commit b7c40b1. eth/contracts: Gen bindings for proxy iface eth/watchers: Use local bindings for contracts eth/watchers: Simplify event subs logic eth/watchers: Simplify&optimize truncated ticker eth/watchers: Update decimals on fetch eth/watchers: Improve handling of decimals eth/watchers: Fix price rat creation eth/watchers: Make sure we use UTC on truncated timer eth/contracts/chainlink: Generate only V3 contract bindings eth/watchers: Watch PriceFeed only with polling eth/watchers: Add a retry logic on price update eth/watchers: Use clog instead of fmt.Printf * eth: Create separate pricefeed client unit This will make the code more testable. * eth: Add tests for pricefeed client * eth/watchers: Add tests to the truncated ticker Gosh that was much harder than I thought * eth/watchers: Add tests for pricefeedwatcher * eth: Add comments to the new components * go fmt * cmd: make pricePerUnit flags strings * cmd: Allow price per unit to be speficied with a currency Currently ignoring the currency value. * cmd: Add logic to start price update loop * cmd: Add flag for specifying price feed address * cmd: Add a lil test to priceDataToWei * TODO: Reminder for something I noticed is missing * cmd/starter: Support currencies for custom broadcaster prices * eth: Address minor review comments * eth,eth/watchers: Improve pricefeed watcher interface * eth/watchers: Fix pricefeed watcher after merge * cmd,core,server: Support dynamic updates to price in USD * eth/watchers: Remove truncated ticker tests * eth/watchers: Finalize pricefeedwatcher docs/tests * cmd: Address review comment * core: Create tests for autoconvertedprice * cmd,core: Move wei default to AutoConvertedPrice * Address review comments * cmd: Fix the e2e flow for setting/updating configs * CHANGELOG * cmd: Make sure pricePerPixel can be specified with e notation Parse it directlty as a big.Rat from a raw string, like I was doing for pricePerUnit in some places. * Fix tests Turns out tests were not running on my branch due to base branch * go fmt * core: Fix typo in comment * cmd,server: Use 3 decimal points when logging PPP Found out that's officially supported precision on the discovery logic, so let's reflect that here.
* release v0.7.3 * release v0.7.3
* Revert "Bump ffmpeg version and nv-codec-headers (#2973)" This reverts commit cad6713. * Revert "Update CUDA build version from 11.7.1 to 12.0.0 (#2978)" This reverts commit 6c09a9f. * Reapply "Update CUDA build version from 11.7.1 to 12.0.0 (#2978)" This reverts commit ebbf210. * Force ffmpeg reinstall * Revert "Force ffmpeg reinstall" This reverts commit 5adb9a5. --------- Co-authored-by: Victor Elias <[email protected]>
* server: Remove maxPrice check mid-session * server: Fix tests * server: Fix erroneous usage of assert.EqualErrorf When I was writing the tests for validatePrice I found out we were using that function wrong in a couple places and never checking any error. We were sending err.Error() to check the error from err. * server: Fix error checks after fixing assertion * CHANGELOG
* server: Allow dynamic (and sometimes >max) prices for Os * CHANGELOG
…2999) * discovery: Ignore maxPrice on db_discovery queries Still kept the feature on the db as it had all the tests already implemented and could still be useful in the future. I can remove it if preferred though. * server: Get prices as big rats for selection While this may not seem useful now since we just convert them to floats on the probability calculation, it will be useful later when comparing prices to max price. * server: Add maxPrice filter logic on selection algorithm * CHANGELOG * server: Break filter in 2 functions
This commit introduces two new AI-specific issue templates, aiming to streamline the routing of AI subnet-related issues and feature requests to the appropriate team.
This commit adds a pull request labeler action that automatically attaches the `ai` label when a pull request is created to the `ai-video` branch.
This commit introduces two new AI-specific issue templates, aiming to streamline the routing of AI subnet-related issues and feature requests to the appropriate team. It also adds a pull request labeler action that automatically attaches the `ai` label when a pull request is created to the `ai-video` branch.
* ci: change issue template order This commit ensures that the main branch issue templates are put above the AI related issue templates. * ci(ai): add PR labeler config file This commmit adds a https://github.com/actions/labeler configuration file so that all PRs on the `ai-video` branch will be correctly labeled with the `ai` label.
* ci(ai): fix incorrect labels This commit fixed the labels that were specified in the Issue Templates to the one found in the repository. * ci: rename labeler and remove trailing whitespace
* ci(ai): fix pull request config warning (#3018) This commit gets rid of the Pull request labeler configuration file warning. * ci(ai): auto assign AI issues This commit auto assigns the AI issues to the right member of the AI team. * ci(ai): cleanup labeler actions This commit cleans up the labeler actions and ensure they run on the right triggers.
* Fix CI * Fix CI
Signed-off-by: kevincatty <[email protected]> Co-authored-by: Rafał Leszko <[email protected]>
* refactor: add -gateway and deprecate -broadcaster This commit adds the `gateway` flag and deprecates the `broadcaster` flag per core team decision (details: https://discord.com/channels/423160867534929930/1051963444598943784/1210356864643109004). * chore: update pending changelog --------- Co-authored-by: John | Elite Encoder <[email protected]>
* refactor(census): rename Broadcaster metrics to Gateway This commit renames the metrics related to Broadcaster to Gateway, following a team decision. More details can be found in the discussion here: [Team Discussion Link](.com/channels/423160867534929930/1051963444598943784/1210356864643109004). * chore: update pending changelog
) * refactor: add -pricePerGateway and deprecate -pricePerBroadcaster This commit adds the `pricePerGateway` flag and deprecates the `pricePerBroadcaster` flag per core team decision (details: https://discord.com/channels/423160867534929930/1051963444598943784/1210356864643109004). * chore: update pending changelog * refactor: remove redundant deprecation comment This commit removes the `PricePerBroadcaster` deprecation comment since this is already clear from the glog warning below. * fix: correct the `pricePerBroadcaster` flag check This commit ensures that the deprecation condition for the `pricePerBroadcaster` flag properly handles the default empty string value. * fix: ensure 'pricePerGateway' is used This commit ensures that the `pricePerGateway` is correctly used instead of the `pricePerBroadcaster` when it is set. * refactor: deprecate 'pricePerGateway' broadcasters property in favor of 'gateways' This commit updates the configuration to replace the `broadcasters` property specified under the `pricePerGateway` flag with `gateways`. Additionally, it ensures that a warning is issued when the deprecated property is still used. * test: fix TestParseGetBroadcasterPrices test This commit ensures that the TestParseGetBroadcasterPrices function uses the new getGatewayPrices function. * test: rename TestParseGetBroadcasterPrices to reflect Gateway naming This commit updates the `TestParseGetBroadcasterPrices` function to `TestParseGetGatewayPrices` to align with the new node naming convention.
This commit introduces a safeguard to ensure that the Docker image tagged as 'stable' is only pushed when a new tag is created on the stable branch. This prevents unintended updates to the stable Docker image, ensuring consistency and reliability for users relying on the stable tag.
* Return appropriate errors when auth fails, not just 5xx * Fix unit tests for new signature * Fix remaining tests * Fix another test * Refactor Forbidden to error type * Refactor Forbidden to error type --------- Co-authored-by: Rafał Leszko <[email protected]>
* ci: fix syntax error in Docker action tags This commit addresses a syntax error in the Docker image tag creation step. * ci(docker): ensure stable tag is created on master branch This commit ensures that the stable tag is created on the master branch.
Signed-off-by: linghuying <[email protected]>
* Add logging to selection_algorithm.go * Add even more logging
3b440ad
to
4d54872
Compare
This commit adds a new `currency` field to the `AIModelConfig` to specify the currency. Additionally, it improves the AI startup code in `starter.go` to correctly handle parsing of this currency while ensuring compatibility with 'offchain' mode. Further improvements to the AI startup code are deferred to avoid conflicts with existing pull requests.
4d54872
to
8e654d7
Compare
@victorges I'm currently testing if the pricing still works correctly. |
@victorges if you agree with my last commit I think we can merge this and handle the following things in seperate pull requests:
|
@victorges looks like the docker container build ci fails most likely due to incompatible system requirements. The new ffmpeg requires Cuda 12.4 and ubuntu 22.04 or 24.04. I can take a look later 👍🏻. |
This commit disables the linux/arm64 for now as we haven't yet added support it. This prevent the Docker CI from failing.
@victorges this pull request could possibly contain a racing condition #3115 (comment). Will look at it next week. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM! Fix the build and it's good to go
@@ -1095,6 +1095,16 @@ func StartLivepeer(ctx context.Context, cfg LivepeerConfig) { | |||
return | |||
} | |||
|
|||
// Get base pixels per unit. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not sure if it's allowed to be missing, but makes sense. Is this how the per gateway configs work as well? (dont need to be the same, just to know if it's a new thing or a fix)
This commit aims to synchronize the ai-video branch with the master branch. It addresses an issue where PR #3103 was not correctly recognized by GitHub as being in sync with the master branch. * eth,eth/watcher: Create Chainlink price feed watcher (#2972) * eth/watchers: Create PriceFeed watcher Makefile: Use mockgen binary from tool dependencies eth/contracts: Add chainlink interfaces source Makefile: Generate Chainlink contracts ABI tools: Add abigen tool to repo eth/contracts: Generate chainlink bindings Makefile: Fix abigen bindings generation Revert everything abigen Turns out there's already bindings exported from the Chainlink lib. go.mod: Add chainlink library eth/watchers: Add pricefeed watcher eth/watchers: Clean-up event watching code eth/watchers: Improve price tracking Revert "go.mod: Add chainlink library" This reverts commit ac415bd. Revert "Revert everything abigen" This reverts commit b7c40b1. eth/contracts: Gen bindings for proxy iface eth/watchers: Use local bindings for contracts eth/watchers: Simplify event subs logic eth/watchers: Simplify&optimize truncated ticker eth/watchers: Update decimals on fetch eth/watchers: Improve handling of decimals eth/watchers: Fix price rat creation eth/watchers: Make sure we use UTC on truncated timer eth/contracts/chainlink: Generate only V3 contract bindings eth/watchers: Watch PriceFeed only with polling eth/watchers: Add a retry logic on price update eth/watchers: Use clog instead of fmt.Printf * eth: Create separate pricefeed client unit This will make the code more testable. * eth: Add tests for pricefeed client * eth/watchers: Add tests to the truncated ticker Gosh that was much harder than I thought * eth/watchers: Add tests for pricefeedwatcher * eth: Add comments to the new components * go fmt * eth: Address minor review comments * eth,eth/watchers: Improve pricefeed watcher interface * eth/watchers: Remove truncated ticker tests * cmd/livepeer: Use price feed watcher for dynamic pricePerPixel (#2981) * eth/watchers: Create PriceFeed watcher Makefile: Use mockgen binary from tool dependencies eth/contracts: Add chainlink interfaces source Makefile: Generate Chainlink contracts ABI tools: Add abigen tool to repo eth/contracts: Generate chainlink bindings Makefile: Fix abigen bindings generation Revert everything abigen Turns out there's already bindings exported from the Chainlink lib. go.mod: Add chainlink library eth/watchers: Add pricefeed watcher eth/watchers: Clean-up event watching code eth/watchers: Improve price tracking Revert "go.mod: Add chainlink library" This reverts commit ac415bd. Revert "Revert everything abigen" This reverts commit b7c40b1. eth/contracts: Gen bindings for proxy iface eth/watchers: Use local bindings for contracts eth/watchers: Simplify event subs logic eth/watchers: Simplify&optimize truncated ticker eth/watchers: Update decimals on fetch eth/watchers: Improve handling of decimals eth/watchers: Fix price rat creation eth/watchers: Make sure we use UTC on truncated timer eth/contracts/chainlink: Generate only V3 contract bindings eth/watchers: Watch PriceFeed only with polling eth/watchers: Add a retry logic on price update eth/watchers: Use clog instead of fmt.Printf * eth: Create separate pricefeed client unit This will make the code more testable. * eth: Add tests for pricefeed client * eth/watchers: Add tests to the truncated ticker Gosh that was much harder than I thought * eth/watchers: Add tests for pricefeedwatcher * eth: Add comments to the new components * go fmt * cmd: make pricePerUnit flags strings * cmd: Allow price per unit to be speficied with a currency Currently ignoring the currency value. * cmd: Add logic to start price update loop * cmd: Add flag for specifying price feed address * cmd: Add a lil test to priceDataToWei * TODO: Reminder for something I noticed is missing * cmd/starter: Support currencies for custom broadcaster prices * eth: Address minor review comments * eth,eth/watchers: Improve pricefeed watcher interface * eth/watchers: Fix pricefeed watcher after merge * cmd,core,server: Support dynamic updates to price in USD * eth/watchers: Remove truncated ticker tests * eth/watchers: Finalize pricefeedwatcher docs/tests * cmd: Address review comment * core: Create tests for autoconvertedprice * cmd,core: Move wei default to AutoConvertedPrice * Address review comments * cmd: Fix the e2e flow for setting/updating configs * CHANGELOG * cmd: Make sure pricePerPixel can be specified with e notation Parse it directlty as a big.Rat from a raw string, like I was doing for pricePerUnit in some places. * Fix tests Turns out tests were not running on my branch due to base branch * go fmt * core: Fix typo in comment * cmd,server: Use 3 decimal points when logging PPP Found out that's officially supported precision on the discovery logic, so let's reflect that here. * Release 0.7.3 (#2988) * release v0.7.3 * release v0.7.3 * Revert "Bump ffmpeg version and nv-codec-headers" (#2989) * Revert "Bump ffmpeg version and nv-codec-headers (#2973)" This reverts commit cad6713. * Revert "Update CUDA build version from 11.7.1 to 12.0.0 (#2978)" This reverts commit 6c09a9f. * Reapply "Update CUDA build version from 11.7.1 to 12.0.0 (#2978)" This reverts commit ebbf210. * Force ffmpeg reinstall * Revert "Force ffmpeg reinstall" This reverts commit 5adb9a5. --------- Co-authored-by: Victor Elias <[email protected]> * Bump LPMS (#2992) * release v0.7.4 (#2993) * server: Skip redundant maxPrice check in ongoing session (#2994) * server: Remove maxPrice check mid-session * server: Fix tests * server: Fix erroneous usage of assert.EqualErrorf When I was writing the tests for validatePrice I found out we were using that function wrong in a couple places and never checking any error. We were sending err.Error() to check the error from err. * server: Fix error checks after fixing assertion * CHANGELOG * server: Allow Os price to increase up to 2x mid-session (#2995) * server: Allow dynamic (and sometimes >max) prices for Os * CHANGELOG * server,discovery: Allow B to use any O in case none match maxPrice (#2999) * discovery: Ignore maxPrice on db_discovery queries Still kept the feature on the db as it had all the tests already implemented and could still be useful in the future. I can remove it if preferred though. * server: Get prices as big rats for selection While this may not seem useful now since we just convert them to floats on the probability calculation, it will be useful later when comparing prices to max price. * server: Add maxPrice filter logic on selection algorithm * CHANGELOG * server: Break filter in 2 functions * Fix transcoding price metrics (#3001) * ci(ai): add AI issue templates This commit introduces two new AI-specific issue templates, aiming to streamline the routing of AI subnet-related issues and feature requests to the appropriate team. * ci(ai): add AI pull request labeler This commit adds a pull request labeler action that automatically attaches the `ai` label when a pull request is created to the `ai-video` branch. * ci: change issue template order and add PR labeler config (#3006) * ci: change issue template order This commit ensures that the main branch issue templates are put above the AI related issue templates. * ci(ai): add PR labeler config file This commmit adds a https://github.com/actions/labeler configuration file so that all PRs on the `ai-video` branch will be correctly labeled with the `ai` label. * ci(ai): fix incorrect labels (#3012) * ci(ai): fix incorrect labels This commit fixed the labels that were specified in the Issue Templates to the one found in the repository. * ci: rename labeler and remove trailing whitespace * ci(ai): fix pull request config warning (#3018) (#3019) * ci(ai): fix pull request config warning (#3018) This commit gets rid of the Pull request labeler configuration file warning. * ci(ai): auto assign AI issues This commit auto assigns the AI issues to the right member of the AI team. * ci(ai): cleanup labeler actions This commit cleans up the labeler actions and ensure they run on the right triggers. * Initialize round by any B/O who has the initializeRound flag set to true (#3029) * Fix CI Darwin Build (#3049) * Fix CI * Fix CI * chore: fix function names (#3040) Signed-off-by: kevincatty <[email protected]> Co-authored-by: Rafał Leszko <[email protected]> * Create option to filter Os by min livepeer version used (#3050) * Update LPMS with the mobile transcoding fix (#3003) * Release 0.7.5 (#3051) * refactor: add -gateway and deprecate -broadcaster (#3053) * refactor: add -gateway and deprecate -broadcaster This commit adds the `gateway` flag and deprecates the `broadcaster` flag per core team decision (details: https://discord.com/channels/423160867534929930/1051963444598943784/1210356864643109004). * chore: update pending changelog --------- Co-authored-by: John | Elite Encoder <[email protected]> * refactor(census): rename Broadcaster metrics to Gateway (#3055) * refactor(census): rename Broadcaster metrics to Gateway This commit renames the metrics related to Broadcaster to Gateway, following a team decision. More details can be found in the discussion here: [Team Discussion Link](.com/channels/423160867534929930/1051963444598943784/1210356864643109004). * chore: update pending changelog * refactor: add -pricePerGateway and deprecate -pricePerBroadcaster (#3056) * refactor: add -pricePerGateway and deprecate -pricePerBroadcaster This commit adds the `pricePerGateway` flag and deprecates the `pricePerBroadcaster` flag per core team decision (details: https://discord.com/channels/423160867534929930/1051963444598943784/1210356864643109004). * chore: update pending changelog * refactor: remove redundant deprecation comment This commit removes the `PricePerBroadcaster` deprecation comment since this is already clear from the glog warning below. * fix: correct the `pricePerBroadcaster` flag check This commit ensures that the deprecation condition for the `pricePerBroadcaster` flag properly handles the default empty string value. * fix: ensure 'pricePerGateway' is used This commit ensures that the `pricePerGateway` is correctly used instead of the `pricePerBroadcaster` when it is set. * refactor: deprecate 'pricePerGateway' broadcasters property in favor of 'gateways' This commit updates the configuration to replace the `broadcasters` property specified under the `pricePerGateway` flag with `gateways`. Additionally, it ensures that a warning is issued when the deprecated property is still used. * test: fix TestParseGetBroadcasterPrices test This commit ensures that the TestParseGetBroadcasterPrices function uses the new getGatewayPrices function. * test: rename TestParseGetBroadcasterPrices to reflect Gateway naming This commit updates the `TestParseGetBroadcasterPrices` function to `TestParseGetGatewayPrices` to align with the new node naming convention. * ci: protect Docker 'stable' tag (#3062) This commit introduces a safeguard to ensure that the Docker image tagged as 'stable' is only pushed when a new tag is created on the stable branch. This prevents unintended updates to the stable Docker image, ensuring consistency and reliability for users relying on the stable tag. * Return appropriate errors when auth fails, not just 5xx (#3065) * Return appropriate errors when auth fails, not just 5xx * Fix unit tests for new signature * Fix remaining tests * Fix another test * Refactor Forbidden to error type * Refactor Forbidden to error type --------- Co-authored-by: Rafał Leszko <[email protected]> * ci: fix syntax error in Docker action tags (#3068) * ci: fix syntax error in Docker action tags This commit addresses a syntax error in the Docker image tag creation step. * ci(docker): ensure stable tag is created on master branch This commit ensures that the stable tag is created on the master branch. * chore: fix some comments (#3070) Signed-off-by: linghuying <[email protected]> * Add logging to selection_algorithm.go (#3076) * Add logging to selection_algorithm.go * Add even more logging * Fix typo in logs (#3079) * Add ctx to logging for selection algorithm (#3080) * Add ctx to logging for selection algorithm * Reorg imports * Fix unit tests * chore: make function comment match function name (#3081) Signed-off-by: tongjicoder <[email protected]> * refactor: rename internal references from Broadcaster to Gateway (#3060) * refactor: rename internal references from Broadcaster to Gateway This commit updates internal references from 'Broadcaster' to 'Gateway' in accordance with the core team’s decision. For more details, refer to the discussion: [Discord Link](https://discord.com/channels/423160867534929930/1051963444598943784/1210356864643109004). * chore: update pending changelog * Add logging to the session refresh (#3083) * Add `/healthz` endpoint (#3095) * Update LPMS to ffmpeg 7 (#3096) * install_ffmpeg: point to LPMS * Update to use ffmpeg7 LPMS * release v0.7.6 * cmd: Use `-gateway` consistently Avoid references to deprecated CLI flags. * core: Rename 'InArray' to 'HasCapability' (#3118) --------- Signed-off-by: kevincatty <[email protected]> Signed-off-by: linghuying <[email protected]> Signed-off-by: tongjicoder <[email protected]> Co-authored-by: Victor Elias <[email protected]> Co-authored-by: Rafał Leszko <[email protected]> Co-authored-by: Thom Shutt <[email protected]> Co-authored-by: Victor Elias <[email protected]> Co-authored-by: kevincatty <[email protected]> Co-authored-by: John | Elite Encoder <[email protected]> Co-authored-by: linghuying <[email protected]> Co-authored-by: tongjicoder <[email protected]> Co-authored-by: Josh Allmann <[email protected]>
This commit releases the new rebased `ai-video` branch so that it can be deployed by orchestrators and Gateways (see #3103).
What does this pull request do? Explain your changes. (required)
This pull request updates the
ai-video
branch with the latest changes from themaster
branch. Initially, I attempted a rebase, but due to significant divergence between the two branches, it became too complex to resolve. Instead, I performed a simple merge and addressed all the resulting merge conflicts.Specific updates (required)
How did you test each of these updates (required)
make
worked.Does this pull request close any open issues?
Checklist:
make
runs successfully./test.sh
pass - @victorges I still need to fix the tests on theai-video
branch.