Llama2 7b model C++ example #3666
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ototh-htec
requested review from
turneram,
attila-dusnoki-htec and
gyulaz-htec
gyulaz-htec
reviewed
Nov 29, 2024
src/api/api.cpp
Outdated
| @@ -1522,6 +1522,21 @@ extern "C" migraphx_status migraphx_module_add_instruction(migraphx_instruction_ | |||
| return api_error_result; | |||
| } | |||
|
|
|||
| extern "C" migraphx_status migraphx_module_get_last_instruction(migraphx_instruction_t* out, | |||
There was a problem hiding this comment.
We can remove these changes (commits) which are related to exposing this API part. Because we ended up not needing these changes.
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## develop #3666 +/- ##
========================================
Coverage 92.23% 92.23%
========================================
Files 514 514
Lines 21746 21746
========================================
Hits 20057 20057
Misses 1689 1689 ☔ View full report in Codecov by Sentry. |
…_last_instruction" This reverts commit d970b30.
ototh-htec
changed the title
This build is not recommended to merge 🔴 |
🔴bert_large_uncased_fp16: FAILED: MIGraphX is not within tolerance - check verbose output |
turneram
reviewed
Dec 7, 2024
| make -j | ||
|
|
||
| # Running the example | ||
| export MIOPEN_FIND_ENFORCE=3 |
There was a problem hiding this comment.
Is this needed? I don't believe we make any MIOpen calls for this model.
spolifroni-amd
approved these changes
Dec 10, 2024
causten
reviewed
Jan 23, 2025
| ##################################################################################### | ||
| # The MIT License (MIT) | ||
| # | ||
| # Copyright (c) 2015-2024 Advanced Micro Devices, Inc. All rights reserved. |
There was a problem hiding this comment.
Suggested change
| # Copyright (c) 2015-2024 Advanced Micro Devices, Inc. All rights reserved. | |
| # Copyright (c) 2015-2025 Advanced Micro Devices, Inc. All rights reserved. |
| make -j | ||
|
|
||
| # Running the example | ||
| export MIOPEN_FIND_ENFORCE=3 |
There was a problem hiding this comment.
Suggested change
| export MIOPEN_FIND_ENFORCE=3 |
| ##################################################################################### | ||
| # The MIT License (MIT) | ||
| # | ||
| # Copyright (c) 2015-2024 Advanced Micro Devices, Inc. All rights reserved. |
There was a problem hiding this comment.
Suggested change
| # Copyright (c) 2015-2024 Advanced Micro Devices, Inc. All rights reserved. | |
| # Copyright (c) 2015-2025 Advanced Micro Devices, Inc. All rights reserved. |
| ##################################################################################### | ||
| # The MIT License (MIT) | ||
| # | ||
| # Copyright (c) 2015-2024 Advanced Micro Devices, Inc. All rights reserved. |
There was a problem hiding this comment.
Suggested change
| # Copyright (c) 2015-2024 Advanced Micro Devices, Inc. All rights reserved. | |
| # Copyright (c) 2015-2025 Advanced Micro Devices, Inc. All rights reserved. |
| /* | ||
| * The MIT License (MIT) | ||
| * | ||
| * Copyright (c) 2015-2024 Advanced Micro Devices, Inc. All rights reserved. |
There was a problem hiding this comment.
Suggested change
| * Copyright (c) 2015-2024 Advanced Micro Devices, Inc. All rights reserved. | |
| * Copyright (c) 2015-2025 Advanced Micro Devices, Inc. All rights reserved. |
| /* | ||
| * The MIT License (MIT) | ||
| * | ||
| * Copyright (c) 2015-2024 Advanced Micro Devices, Inc. All rights reserved. |
There was a problem hiding this comment.
Suggested change
| * Copyright (c) 2015-2024 Advanced Micro Devices, Inc. All rights reserved. | |
| * Copyright (c) 2015-2025 Advanced Micro Devices, Inc. All rights reserved. |
| /* | ||
| * The MIT License (MIT) | ||
| * | ||
| * Copyright (c) 2015-2024 Advanced Micro Devices, Inc. All rights reserved. |
There was a problem hiding this comment.
Suggested change
| * Copyright (c) 2015-2024 Advanced Micro Devices, Inc. All rights reserved. | |
| * Copyright (c) 2015-2025 Advanced Micro Devices, Inc. All rights reserved. |
| /* | ||
| * The MIT License (MIT) | ||
| * | ||
| * Copyright (c) 2015-2024 Advanced Micro Devices, Inc. All rights reserved. |
There was a problem hiding this comment.
Suggested change
| * Copyright (c) 2015-2024 Advanced Micro Devices, Inc. All rights reserved. | |
| * Copyright (c) 2015-2025 Advanced Micro Devices, Inc. All rights reserved. |
| ##################################################################################### | ||
| # The MIT License (MIT) | ||
| # | ||
| # Copyright (c) 2015-2024 Advanced Micro Devices, Inc. All rights reserved. |
There was a problem hiding this comment.
Suggested change
| # Copyright (c) 2015-2024 Advanced Micro Devices, Inc. All rights reserved. | |
| # Copyright (c) 2015-2025 Advanced Micro Devices, Inc. All rights reserved. |
| ##################################################################################### | ||
| # The MIT License (MIT) | ||
| # | ||
| # Copyright (c) 2015-2024 Advanced Micro Devices, Inc. All rights reserved. |
There was a problem hiding this comment.
Suggested change
| # Copyright (c) 2015-2024 Advanced Micro Devices, Inc. All rights reserved. | |
| # Copyright (c) 2015-2025 Advanced Micro Devices, Inc. All rights reserved. |
causten
requested changes
Jan 23, 2025
| ### Getting the pre-quantized model from HuggingFace | ||
| ```bash | ||
| pip install -U "huggingface_hub[cli]" | ||
| huggingface-cli login YOUR_HF_TOKEN |
There was a problem hiding this comment.
Suggested change
| huggingface-cli login YOUR_HF_TOKEN | |
| huggingface-cli login login --token <YOUR_HF_TOKEN> |
| ```bash | ||
| pip install -U "huggingface_hub[cli]" | ||
| huggingface-cli login YOUR_HF_TOKEN | ||
| hugginggface-cli download https://huggingface.co/amd/Llama-2-7b-chat-hf-awq-int4-asym-gs128-onnx |
There was a problem hiding this comment.
Suggested change
| hugginggface-cli download https://huggingface.co/amd/Llama-2-7b-chat-hf-awq-int4-asym-gs128-onnx | |
| huggingface-cli download amd/Llama-2-7b-chat-hf-awq-int4-asym-gs128-onnx |
|
|
||
| ```bash | ||
| # Convert dataset to numpy format | ||
| ./prepocess_dataset.py |
There was a problem hiding this comment.
I think a step is missing. I performed the steps listed to use the pre-quantized model
Suggested change
| ./prepocess_dataset.py | |
| gunzip /mgx_llama2/open_orca/open_orca_gpt4_tokenized_llama.calibration_1000.pkl.gz | |
| python3 preprocess_dataset.py |
|
|
||
| # Builidng the example | ||
| cd mgx_llama2 | ||
| mkdir build && cd build |
There was a problem hiding this comment.
there was already a build dir so the command failed.
|
|
||
| # Running the example | ||
| export MIOPEN_FIND_ENFORCE=3 | ||
| ./mgxllama2 |
There was a problem hiding this comment.
root@12320f9edbf9:/mgx_llama2/build# ./mgxllama2
terminate called after throwing an instance of 'std::runtime_error'
what(): hip error (101): invalid device ordinal
Aborted (core dumped)| ./mgxllama2 | ||
|
|
||
| # Test the accuracy of the output | ||
| python3 eval_accuracy.py |
There was a problem hiding this comment.
root@12320f9edbf9:/mgx_llama2# python3 eval_accuracy.py
None of PyTorch, TensorFlow >= 2.0, or Flax have been found. Models won't be available and only tokenizers, configuration and file/data utilities can be used.
None of PyTorch, TensorFlow >= 2.0, or Flax have been found. Models won't be available and only tokenizers, configuration and file/data utilities can be used.
Traceback (most recent call last):
File "/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_http.py", line 406, in hf_raise_for_status
response.raise_for_status()
File "/usr/local/lib/python3.10/dist-packages/requests/models.py", line 1024, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://huggingface.co/meta-llama/Llama-2-7b-chat-hf/resolve/main/config.json| #### Building and running the example | ||
|
|
||
| ```bash | ||
| # Convert dataset to numpy format |
There was a problem hiding this comment.
when I launched the docker via run_docker.sh I landed in the /mgx_llama2/build directory so the preprocess_dataset.py command's path is wrong
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Implemented an example for https://huggingface.co/amd/Llama-2-7b-chat-hf-awq-int4-asym-gs128-onnx/tree/main Llama2 7b model with MIGraphX.
Details about the example and description for running is available in README (https://github.com/ROCm/AMDMIGraphX/tree/htec/mgx-llama2-7b-example/examples/transformers/mgx_llama2)