Skip to content
This repository has been archived by the owner on Dec 14, 2023. It is now read-only.

Update from upstream Sep 20, 2023 #294

Closed
wants to merge 39 commits into from
Closed

Commits on May 18, 2022

  1. Bump 3rd_party/marian-dev from 199201e to e88c1aa (#416)

    Bumps [3rd_party/marian-dev](https://github.com/browsermt/marian-dev) from `199201e` to `e88c1aa`.
    - [Release notes](https://github.com/browsermt/marian-dev/releases)
    - [Commits](browsermt/marian-dev@199201e...e88c1aa)
    
    ---
    updated-dependencies:
    - dependency-name: 3rd_party/marian-dev
      dependency-type: direct:production
    ...
    
    Signed-off-by: dependabot[bot] <[email protected]>
    
    Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
    dependabot[bot] authored May 18, 2022
    Configuration menu
    Copy the full SHA
    ad78165 View commit details
    Browse the repository at this point in the history

Commits on Jun 20, 2022

  1. Set up python packaging for pypi distribution (#424)

    Old GitHub CI using Ubuntu and MacOS explicitly and building wheels have
    been removed in favour of the more portable pypa specified builds. These
    wheels should work just as well across a wider range of distributions.
    
    pybind11:CMakeLists.txt requires Development.Module instead of
    Development.* to avoid Embed from getting in the way of manylinux
    builds.
    
    manylinux_x86_64 builds are added for cp3.6 - 3.10. The linux build
    uses an old image via docker.  Since the docker images are able to use
    shared ccache folder, builds quite fast on warm starts.
    
    ccache usage in setup.py is now triggered by an environment variable.
    This allows for builds not to fail if ccache not present.
    
    On tag pushes corresponding to versions, CI is configured to deliver
    built wheels to PyPI, reading from repository secrets.
    
    Improves setup.py including documentation and some formatting, and
    additional links to source.
    
    Fixes: #315
    jerinphilip authored Jun 20, 2022
    Configuration menu
    Copy the full SHA
    61d2c35 View commit details
    Browse the repository at this point in the history

Commits on Jun 21, 2022

  1. Basic HTML property testing for WebAssembly (#425)

    Import
    https://gist.github.com/jelmervdl/a4c8b6b92ad88a885e1cbd51c6ad4902 and
    attach it to CI.  NodeJS-14 is failing on trying to use the WebAssembly
    binary. So we use node-16 independently setup.  This paves way for more
    complicated testing for WebAssembly bindings in the future.
    jerinphilip authored Jun 21, 2022
    Configuration menu
    Copy the full SHA
    8771078 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    05a8778 View commit details
    Browse the repository at this point in the history

Commits on Jun 24, 2022

  1. Python package: pyyaml >= 5.1 (#429)

    Fixes issue on Colab which says vanilla YAML intall (3.x) does not have
    yaml.FullLoader (https://stackoverflow.com/a/55553392/4565794).
    
    Fix a broken link for presentation in PyPI.
    jerinphilip authored Jun 24, 2022
    Configuration menu
    Copy the full SHA
    3ef85e1 View commit details
    Browse the repository at this point in the history

Commits on Jun 25, 2022

  1. Python: Work offline if models are available (#431)

    Try to check if models.json is downloaded first, if it is use it. 
    If not, fall back to attempting to fetch it from the network.
    
    Fixes: #430
    jerinphilip authored Jun 25, 2022
    Configuration menu
    Copy the full SHA
    84c761b View commit details
    Browse the repository at this point in the history

Commits on Jun 29, 2022

  1. MacOS Wheels (#432)

    * Remove trailing whitespace
    * Additional MacOS wheels: Wheels for python 3.6 to 3.10 with a 
       minimum target of MacOS 10.9
    * Install bergamot package from wheel directory
    * Remove no-index as we need dependencies
    graemenail authored Jun 29, 2022
    Configuration menu
    Copy the full SHA
    7f79128 View commit details
    Browse the repository at this point in the history

Commits on Jan 17, 2023

  1. update download path

    XapaJIaMnu authored Jan 17, 2023
    Configuration menu
    Copy the full SHA
    06c31af View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    21eff44 View commit details
    Browse the repository at this point in the history

Commits on Jan 18, 2023

  1. Configuration menu
    Copy the full SHA
    6cefc43 View commit details
    Browse the repository at this point in the history
  2. Bump qs and express in /wasm/test_page (#444)

    Bumps [qs](https://github.com/ljharb/qs) to 6.11.0 and updates ancestor dependency [express](https://github.com/expressjs/express). These dependencies need to be updated together.
    
    
    Updates `qs` from 6.7.0 to 6.11.0
    - [Release notes](https://github.com/ljharb/qs/releases)
    - [Changelog](https://github.com/ljharb/qs/blob/main/CHANGELOG.md)
    - [Commits](ljharb/qs@v6.7.0...v6.11.0)
    
    Updates `express` from 4.17.1 to 4.18.2
    - [Release notes](https://github.com/expressjs/express/releases)
    - [Changelog](https://github.com/expressjs/express/blob/master/History.md)
    - [Commits](expressjs/express@4.17.1...4.18.2)
    
    ---
    updated-dependencies:
    - dependency-name: qs
      dependency-type: indirect
    - dependency-name: express
      dependency-type: direct:production
    ...
    
    Signed-off-by: dependabot[bot] <[email protected]>
    
    Signed-off-by: dependabot[bot] <[email protected]>
    Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
    dependabot[bot] authored Jan 18, 2023
    Configuration menu
    Copy the full SHA
    620c8b0 View commit details
    Browse the repository at this point in the history
  3. Arm updated (#443)

    * ARM Support using ruy and simd_utils
    
    * Adding ARM build on GitHub CI
    
    * Add workflow and successful build
    
    ssplit-cpp modified to get cross compiled android on GitHub CI working.
    
    * Client side fixes for int8 no shift on ARM [python]
    
    * Revert "Client side fixes for int8 no shift on ARM [python]"
    
    This reverts commit 020af05.
    
    * moving int8shift no-op inside the library
    
    * Bump 3rd-party/marian-dev
    
    * update the marian branch test
    
    * arm backend works
    
    * Latest and greatest clang-format
    
    Co-authored-by: Jerin Philip <[email protected]>
    XapaJIaMnu and jerinphilip authored Jan 18, 2023
    Configuration menu
    Copy the full SHA
    6f2659f View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    7d24908 View commit details
    Browse the repository at this point in the history
  5. Expand the node-test.js example code with documentation (#434)

    * Expand the node-test.js example code with documentation
    
    Is there a better way to document code than by providing an annotated & working example of it? Just listing all the exposed methods feels like giving people a box of bricks and expecting them to build a house with it.
    
    * Use @jerin's feedback to simplify node-test.js explanations
    
    * Use native `console.assert` instead
    
    See #426 for an explanation
    
    * Fix comment
    
    Co-authored-by: Nikolay Bogoychev <[email protected]>
    jelmervdl and XapaJIaMnu authored Jan 18, 2023
    Configuration menu
    Copy the full SHA
    2834f04 View commit details
    Browse the repository at this point in the history
  6. More portable WASM demo (#437)

    * Replace most of the wasm demo page with code from the firefox extension
    
    This code should be more generic and copy/pastable into other projects. Maybe one day it will be an npm package?
    
    * Fix Ukrainian model support
    
    * Add quality estimation output
    
    Automatically enabled when the model(s) support it
    
    * Little "Translating…" indicator
    
    * Don't make Safari fail on something tiny
    
    * Rewire lots of async state to be able to predictably know when the translator is working or not
    
    Previously so much was lazy loaded that it was not easy to catch lack of SIMD support. Now I can just enable the interface only after it has properly loaded.
    
    * No need for a two-stage setup for the worker. Just promise to call `initialize()`!
    
    * More (correct) types and comments for code
    
    * Keyboard shortcuts for input area for bold, italic and underline.
    
    Enough to demo mark-up translation
    
    * Fix `delete()`
    
    * Move javascript glue code into its own npm package
    
    * Add nodejs support and test to package
    
    * More stand-alone build command
    
    …for now, not really used by anything I think
    
    * Ignore build packages
    
    * Use local filesystem for build so it is automatically cached
    
    * fix overflow on demo page
    
    But this might break the mobile demo? I'll have to check into that
    
    * Bring back integrity check, except for NodeJS for now
    
    * Make `build` part of `prepare` so we always make sure we build a complete package
    
    * Move worker code into its own folder
    
    This way I can mark it as a commonjs module which will help cause nodejs treat the files the same as WebWorkers do right now. Firefox doesn't implement `{type: 'module'}` yet for WebWorkers.
    
    * Add README
    
    * Fix paths
    
    * Add npm publish automation
    
    * Make sure webpack ignores node compatibility code
    
    * Add missing webpack:ignore around a worker
    
    * Default to getting models from S3
    
    * Separate "loading" and "translating" indicators
    
    * Bump npm package version
    
    * Add credits
    
    * Don't block on the worker loading
    
    * Not just Mozilla, but Bergamot!
    
    * Make individual translation requests cancelable
    
    * Swap button turns vertically when in skyscraper mode
    
    * Make it easier to debug errors from inside the worker
    
    * Don't bork on deleting a failed worker
    
    * Don't bork on calling translate() with a failed worker
    
    * Handle compilation error with more grace
    
    * `contenteditable=true` seems to work better with some browser extensions
    
    Looking at you, Vimium!
    
    * Clean up abort promise
    
    * Bump npm package version
    
    * Remove `workerUrl` option in favour of better webpack support
    
    With that option it was hard for Webpack to figure out dependencies, and it did not enter my worker script for rewriting. With the hardcoded url it does, and with a bit of `new webpack.DefinePlugin({'typeof self': JSON.stringify('object')}),` we can have webpack remove node-specific code on build!
    
    * Bump version
    
    Minor API change hehe
    
    Co-authored-by: Nikolay Bogoychev <[email protected]>
    jelmervdl and XapaJIaMnu authored Jan 18, 2023
    Configuration menu
    Copy the full SHA
    8d5f877 View commit details
    Browse the repository at this point in the history

Commits on Jan 19, 2023

  1. Fix compilation on x86

    XapaJIaMnu committed Jan 19, 2023
    Configuration menu
    Copy the full SHA
    1ba7461 View commit details
    Browse the repository at this point in the history

Commits on Mar 1, 2023

  1. Fix path to example program

    kpu committed Mar 1, 2023
    Configuration menu
    Copy the full SHA
    82c276a View commit details
    Browse the repository at this point in the history

Commits on May 4, 2023

  1. Bump 3rd_party/marian-dev from 69e27d2 to 8ceb051 (#446)

    Bumps [3rd_party/marian-dev](https://github.com/browsermt/marian-dev) from `69e27d2` to `8ceb051`.
    - [Release notes](https://github.com/browsermt/marian-dev/releases)
    - [Commits](browsermt/marian-dev@69e27d2...8ceb051)
    
    ---
    updated-dependencies:
    - dependency-name: 3rd_party/marian-dev
      dependency-type: direct:production
    ...
    
    Signed-off-by: dependabot[bot] <[email protected]>
    Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
    dependabot[bot] authored May 4, 2023
    Configuration menu
    Copy the full SHA
    eb0fe1b View commit details
    Browse the repository at this point in the history
  2. Update workflows

    XapaJIaMnu authored May 4, 2023
    Configuration menu
    Copy the full SHA
    fceb713 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    3c2a667 View commit details
    Browse the repository at this point in the history

Commits on May 10, 2023

  1. Bump 3rd_party/marian-dev from 8ceb051 to bb65f47 (#447)

    Bumps [3rd_party/marian-dev](https://github.com/browsermt/marian-dev) from `8ceb051` to `bb65f47`.
    - [Commits](browsermt/marian-dev@8ceb051...bb65f47)
    
    ---
    updated-dependencies:
    - dependency-name: 3rd_party/marian-dev
      dependency-type: direct:production
    ...
    
    Signed-off-by: dependabot[bot] <[email protected]>
    Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
    dependabot[bot] authored May 10, 2023
    Configuration menu
    Copy the full SHA
    b3d36bc View commit details
    Browse the repository at this point in the history

Commits on Jun 6, 2023

  1. Configuration menu
    Copy the full SHA
    ada8c39 View commit details
    Browse the repository at this point in the history

Commits on Jul 12, 2023

  1. Configuration menu
    Copy the full SHA
    eaa2562 View commit details
    Browse the repository at this point in the history

Commits on Jul 31, 2023

  1. Bump 3rd_party/marian-dev from 6a6bbb6 to aa0221e (#452)

    Bumps [3rd_party/marian-dev](https://github.com/browsermt/marian-dev) from `6a6bbb6` to `aa0221e`.
    - [Commits](browsermt/marian-dev@6a6bbb6...aa0221e)
    
    ---
    updated-dependencies:
    - dependency-name: 3rd_party/marian-dev
      dependency-type: direct:production
    ...
    
    Signed-off-by: dependabot[bot] <[email protected]>
    Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
    dependabot[bot] authored Jul 31, 2023
    Configuration menu
    Copy the full SHA
    e333208 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    becb6e2 View commit details
    Browse the repository at this point in the history
  3. Fix CI (#454)

    * Use ubuntu-latest, macos-latest in GitHub Actions for cibuildwheel
    
    * Update deprecated ubuntu-18.04 to ubuntu-latest for docs in GH actions
    graemenail authored Jul 31, 2023
    Configuration menu
    Copy the full SHA
    cbfa839 View commit details
    Browse the repository at this point in the history
  4. Bump bergamot-translator-tests from 7984d14 to a04432d (#455)

    Bumps [bergamot-translator-tests](https://github.com/browsermt/bergamot-translator-tests) from `7984d14` to `a04432d`.
    - [Commits](browsermt/bergamot-translator-tests@7984d14...a04432d)
    
    ---
    updated-dependencies:
    - dependency-name: bergamot-translator-tests
      dependency-type: direct:production
    ...
    
    Signed-off-by: dependabot[bot] <[email protected]>
    Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
    dependabot[bot] authored Jul 31, 2023
    Configuration menu
    Copy the full SHA
    8011f9c View commit details
    Browse the repository at this point in the history

Commits on Aug 1, 2023

  1. Enables model ensembles (#450)

    * Enables model ensembles
    
    Adds the ability to use ensembles of models. This supports ensembles of
    binary- or npz-format models, as well as mixtures of both.
    
    When all models in the ensembles are of binary format, the load from
    memory path is used. Otherwise, they are loaded via the file system.
    Enable log-level debug for output related to this.
    
    * Fix formatting
    
    * Fix WASM bindings for MemoryBundle
    
    For now, this does not support ensembles.
    
    * Remove shared_ptr wrapping the AlignedMemory of models.
    
    * Fix formatting
    graemenail authored Aug 1, 2023
    Configuration menu
    Copy the full SHA
    4b0da8d View commit details
    Browse the repository at this point in the history

Commits on Aug 8, 2023

  1. Bump 3rd_party/ssplit-cpp from ad2c5a5 to a311f98 (#456)

    Bumps [3rd_party/ssplit-cpp](https://github.com/browsermt/ssplit-cpp) from `ad2c5a5` to `a311f98`.
    - [Commits](browsermt/ssplit-cpp@ad2c5a5...a311f98)
    
    ---
    updated-dependencies:
    - dependency-name: 3rd_party/ssplit-cpp
      dependency-type: direct:production
    ...
    
    Signed-off-by: dependabot[bot] <[email protected]>
    Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
    dependabot[bot] authored Aug 8, 2023
    Configuration menu
    Copy the full SHA
    2bdc493 View commit details
    Browse the repository at this point in the history

Commits on Aug 11, 2023

  1. Bump 3rd_party/marian-dev from aa0221e to 8dbde0f (#458)

    Bumps [3rd_party/marian-dev](https://github.com/browsermt/marian-dev) from `aa0221e` to `8dbde0f`.
    - [Commits](browsermt/marian-dev@aa0221e...8dbde0f)
    
    ---
    updated-dependencies:
    - dependency-name: 3rd_party/marian-dev
      dependency-type: direct:production
    ...
    
    Signed-off-by: dependabot[bot] <[email protected]>
    Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
    dependabot[bot] authored Aug 11, 2023
    Configuration menu
    Copy the full SHA
    ca95467 View commit details
    Browse the repository at this point in the history

Commits on Aug 14, 2023

  1. Remove wormhole references (#459)

    * Remove warmhole references
    
    * Remove more references to the WORMHOLE
    
    * Update marian to wormhole removed marian
    
    * Whoops
    
    ---------
    
    Co-authored-by: Jelmer van der Linde <[email protected]>
    XapaJIaMnu and jelmervdl authored Aug 14, 2023
    Configuration menu
    Copy the full SHA
    534ed37 View commit details
    Browse the repository at this point in the history

Commits on Aug 16, 2023

  1. Configuration menu
    Copy the full SHA
    47024ec View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    62770bb View commit details
    Browse the repository at this point in the history

Commits on Aug 17, 2023

  1. Configuration menu
    Copy the full SHA
    db38262 View commit details
    Browse the repository at this point in the history

Commits on Sep 11, 2023

  1. Bump 3rd_party/marian-dev from 300a50f to 780df27 (#464)

    Bumps [3rd_party/marian-dev](https://github.com/browsermt/marian-dev) from `300a50f` to `780df27`.
    - [Commits](browsermt/marian-dev@300a50f...780df27)
    
    ---
    updated-dependencies:
    - dependency-name: 3rd_party/marian-dev
      dependency-type: direct:production
    ...
    
    Signed-off-by: dependabot[bot] <[email protected]>
    Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
    dependabot[bot] authored Sep 11, 2023
    Configuration menu
    Copy the full SHA
    0b069ac View commit details
    Browse the repository at this point in the history

Commits on Sep 20, 2023

  1. Bump 3rd_party/marian-dev from 780df27 to 11c6ae7 (#466)

    Bumps [3rd_party/marian-dev](https://github.com/browsermt/marian-dev) from `780df27` to `11c6ae7`.
    - [Commits](browsermt/marian-dev@780df27...11c6ae7)
    
    ---
    updated-dependencies:
    - dependency-name: 3rd_party/marian-dev
      dependency-type: direct:production
    ...
    
    Signed-off-by: dependabot[bot] <[email protected]>
    Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
    dependabot[bot] authored Sep 20, 2023
    Configuration menu
    Copy the full SHA
    321be8a View commit details
    Browse the repository at this point in the history

Commits on Dec 7, 2023

  1. Configuration menu
    Copy the full SHA
    73182d4 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    7774029 View commit details
    Browse the repository at this point in the history
  3. Fix MKL key URL

    kpu committed Dec 7, 2023
    Configuration menu
    Copy the full SHA
    0367ae0 View commit details
    Browse the repository at this point in the history