Skip to content

Commit

Permalink
Port all changes from 'master' into develop branch
Browse files Browse the repository at this point in the history
  • Loading branch information
ashahba committed Apr 27, 2020
1 parent 5992a50 commit 31e91eb
Show file tree
Hide file tree
Showing 533 changed files with 12,309 additions and 26,608 deletions.
2 changes: 0 additions & 2 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,6 @@
# See the License for the specific language governing permissions and
# limitations under the License.
#
# SPDX-License-Identifier: EPL-2.0
#


PY_VERSION := 2
Expand Down
26 changes: 7 additions & 19 deletions benchmarks/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,23 +7,18 @@ Training and inference scripts with Intel-optimized MKL
The model scripts can be run on Linux and require the following
dependencies to be installed:
* [Docker](https://docs.docker.com/install/)
* [Python](https://www.python.org/downloads/) 2.7 or later
* [Python](https://www.python.org/downloads/) 3.5 or later
* [git](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git)
* `wget` for downloading pre-trained models

## TensorFlow Use Cases

| Use Case | Framework | Model | Mode | Instructions |
| -----------------------| --------------| ------------------- | --------- |------------------------------|
| Adversarial Networks | TensorFlow | [DCGAN](https://arxiv.org/pdf/1511.06434.pdf) | Inference | [FP32](adversarial_networks/tensorflow/dcgan/README.md#fp32-inference-instructions) |
| Content Creation | TensorFlow | [DRAW](https://arxiv.org/pdf/1502.04623.pdf) | Inference | [FP32](content_creation/tensorflow/draw/README.md#fp32-inference-instructions) |
| Face Detection and Alignment | Tensorflow | [FaceNet](https://arxiv.org/pdf/1503.03832.pdf) | Inference | [FP32](face_detection_and_alignment/tensorflow/facenet/README.md#fp32-inference-instructions) |
| Face Detection and Alignment | TensorFlow | [MTCC](https://arxiv.org/pdf/1604.02878.pdf) | Inference | [FP32](face_detection_and_alignment/tensorflow/mtcc/README.md#fp32-inference-instructions) |
| Image Recognition | TensorFlow | [DenseNet169](https://arxiv.org/pdf/1608.06993.pdf) | Inference | [FP32](image_recognition/tensorflow/densenet169/README.md#fp32-inference-instructions) |
| Image Recognition | TensorFlow | [Inception ResNet V2](https://arxiv.org/pdf/1602.07261.pdf) | Inference | [Int8](image_recognition/tensorflow/inception_resnet_v2/README.md#int8-inference-instructions) [FP32](image_recognition/tensorflow/inception_resnet_v2/README.md#fp32-inference-instructions) |
| Image Recognition | TensorFlow | [Inception V3](https://arxiv.org/pdf/1512.00567.pdf) | Inference | [Int8](image_recognition/tensorflow/inceptionv3/README.md#int8-inference-instructions) [FP32](image_recognition/tensorflow/inceptionv3/README.md#fp32-inference-instructions) |
| Image Recognition | TensorFlow | [Inception V4](https://arxiv.org/pdf/1602.07261.pdf) | Inference | [Int8](image_recognition/tensorflow/inceptionv4/README.md#int8-inference-instructions) [FP32](image_recognition/tensorflow/inceptionv4/README.md#fp32-inference-instructions) |
| Image Recognition | TensorFlow | [MobileNet V1](https://arxiv.org/pdf/1704.04861.pdf) | Inference | [Int8](image_recognition/tensorflow/mobilenet_v1/README.md#int8-inference-instructions) [FP32](image_recognition/tensorflow/mobilenet_v1/README.md#fp32-inference-instructions) |
| Image Recognition | TensorFlow | [MobileNet V1*](https://arxiv.org/pdf/1704.04861.pdf) | Inference | [Int8](image_recognition/tensorflow/mobilenet_v1/README.md#int8-inference-instructions) [FP32](image_recognition/tensorflow/mobilenet_v1/README.md#fp32-inference-instructions) |
| Image Recognition | TensorFlow | [ResNet 101](https://arxiv.org/pdf/1512.03385.pdf) | Inference | [Int8](image_recognition/tensorflow/resnet101/README.md#int8-inference-instructions) [FP32](image_recognition/tensorflow/resnet101/README.md#fp32-inference-instructions) |
| Image Recognition | TensorFlow | [ResNet 50](https://arxiv.org/pdf/1512.03385.pdf) | Training & Inference | [Int8 Inference,](image_recognition/tensorflow/resnet50/README.md#int8-inference-instructions) [FP32 Inference,](image_recognition/tensorflow/resnet50/README.md#fp32-inference-instructions) [FP32 Training,](image_recognition/tensorflow/resnet50/README.md#fp32-training-instructions) [BFloat16 Training,](image_recognition/tensorflow/resnet50/README.md#bfloat16-training-instructions)|
| Image Recognition | TensorFlow | [ResNet 50v1.5](https://github.com/tensorflow/models/tree/master/official/resnet) | Training & Inference | [Int8 Inference,](image_recognition/tensorflow/resnet50v1_5/README.md#int8-inference-instructions) [FP32 Inference,](image_recognition/tensorflow/resnet50v1_5/README.md#fp32-inference-instructions) [BFloat16 inference,](image_recognition/tensorflow/resnet50v1_5/README.md#bfloat16-inference-instructions))[FP32 Training,](image_recognition/tensorflow/resnet50v1_5/README.md#fp32-training-instructions) [BFloat16 Training](image_recognition/tensorflow/resnet50v1_5/README.md#bfloat16-training-instructions)|
Expand All @@ -33,26 +28,19 @@ dependencies to be installed:
| Language Modeling | TensorFlow | [LM-1B](https://arxiv.org/pdf/1602.02410.pdf) | Inference | [FP32](language_modeling/tensorflow/lm-1b/README.md#fp32-inference-instructions) |
| Language Modeling | TensorFlow | [BERT](https://arxiv.org/pdf/1810.04805.pdf) | Training | [FP32& BFloat16](language_modeling/tensorflow/bert_large/README.md) |
| Language Translation | TensorFlow | [BERT](https://arxiv.org/pdf/1810.04805.pdf) | Inference | [FP32](language_translation/tensorflow/bert/README.md#fp32-inference-instructions) |
| Language Translation | TensorFlow | [GNMT](https://arxiv.org/pdf/1609.08144.pdf) | Inference | [FP32](language_translation/tensorflow/gnmt/README.md#fp32-inference-instructions) |
| Language Translation | TensorFlow | [GNMT*](https://arxiv.org/pdf/1609.08144.pdf) | Inference | [FP32](language_translation/tensorflow/mlperf_gnmt/README.md#fp32-inference-instructions) |
| Language Translation | TensorFlow | [Transformer Language](https://arxiv.org/pdf/1706.03762.pdf)| Inference | [FP32](language_translation/tensorflow/transformer_language/README.md#fp32-inference-instructions) |
| Reinforcement | TensorFlow | [MiniGo](https://arxiv.org/abs/1712.01815.pdf) | Training | [FP32](reinforcement/tensorflow/minigo/README.md#fp32-training-instructions)|
| Language Translation | TensorFlow | [Transformer_LT_Official ](https://arxiv.org/pdf/1706.03762.pdf)| Inference | [FP32](language_translation/tensorflow/transformer_lt_official/README.md#fp32-inference-instructions) |
| Language Translation | TensorFlow | [Transformer_LT_mlperf ](https://arxiv.org/pdf/1706.03762.pdf)| Training | [FP32 Training,](language_translation/tensorflow/transformer_mlperf/README.md#fp32-training-instructions) [ BFloat16 Training](language_translation/tensorflow/transformer_mlperf/README.md#bfloat16-training-instructions) |
| Object Detection | TensorFlow | [R-FCN](https://arxiv.org/pdf/1605.06409.pdf) | Inference | [Int8](object_detection/tensorflow/rfcn/README.md#int8-inference-instructions) [FP32](object_detection/tensorflow/rfcn/README.md#fp32-inference-instructions) |
| Object Detection | TensorFlow | [Faster R-CNN](https://arxiv.org/pdf/1506.01497.pdf) | Inference | [Int8](object_detection/tensorflow/faster_rcnn/README.md#int8-inference-instructions) [FP32](object_detection/tensorflow/faster_rcnn/README.md#fp32-inference-instructions) |
| Object Detection | TensorFlow | [SSD-MobileNet](https://arxiv.org/pdf/1704.04861.pdf) | Inference | [Int8](object_detection/tensorflow/ssd-mobilenet/README.md#int8-inference-instructions) [FP32](object_detection/tensorflow/ssd-mobilenet/README.md#fp32-inference-instructions) |
| Object Detection | TensorFlow | [SSD-ResNet34](https://arxiv.org/pdf/1512.02325.pdf) | Inference | [Int8](object_detection/tensorflow/ssd-resnet34/README.md#int8-inference-instructions) [FP32](object_detection/tensorflow/ssd-resnet34/README.md#fp32-inference-instructions) |
| Object Detection | TensorFlow | [SSD-MobileNet*](https://arxiv.org/pdf/1704.04861.pdf) | Inference | [Int8](object_detection/tensorflow/ssd-mobilenet/README.md#int8-inference-instructions) [FP32](object_detection/tensorflow/ssd-mobilenet/README.md#fp32-inference-instructions) |
| Object Detection | TensorFlow | [SSD-ResNet34*](https://arxiv.org/pdf/1512.02325.pdf) | Inference | [Int8](object_detection/tensorflow/ssd-resnet34/README.md#int8-inference-instructions) [FP32](object_detection/tensorflow/ssd-resnet34/README.md#fp32-inference-instructions) |
| Object Detection | TensorFlow | [SSD-ResNet34](https://arxiv.org/pdf/1512.02325.pdf) | Training | [FP32](object_detection/tensorflow/ssd-resnet34/README.md#fp32-training-instructions) [BF16](object_detection/tensorflow/ssd-resnet34/README.md#bf16-training-instructions) |
| Object Detection | TensorFlow | [SSD-VGG16](https://arxiv.org/pdf/1512.02325.pdf) | Inference | [Int8](object_detection/tensorflow/ssd_vgg16/README.md#int8-inference-instructions) [FP32](object_detection/tensorflow/ssd_vgg16/README.md#fp32-inference-instructions) |
| Recommendation | TensorFlow | [NCF](https://arxiv.org/pdf/1708.05031.pdf) | Inference | [FP32](recommendation/tensorflow/ncf/README.md#fp32-inference-instructions) |
| Recommendation | TensorFlow | [Wide & Deep Large Dataset](https://arxiv.org/pdf/1606.07792.pdf) | Inference | [Int8](recommendation/tensorflow/wide_deep_large_ds/README.md#int8-inference-instructions) [FP32](recommendation/tensorflow/wide_deep_large_ds/README.md#fp32-inference-instructions) |
| Recommendation | TensorFlow | [Wide & Deep](https://arxiv.org/pdf/1606.07792.pdf) | Inference | [FP32](recommendation/tensorflow/wide_deep/README.md#fp32-inference-instructions) |
| Text-to-Speech | TensorFlow | [WaveNet](https://arxiv.org/pdf/1609.03499.pdf) | Inference | [FP32](text_to_speech/tensorflow/wavenet/README.md#fp32-inference-instructions) |


## TensorFlow Serving Use Cases


| Use Case | Framework | Model | Mode | Instructions |
| -----------------------| --------------| ------------------- | --------- |------------------------------|
| Image Recognition | TensorFlow Serving | [Inception V3](https://arxiv.org/pdf/1512.00567.pdf) | Inference | [FP32](image_recognition/tensorflow_serving/inceptionv3/README.md#fp32-inference-instructions) |

*Means the model is belong to [MLPerf](https://mlperf.org/) models, will long term support.
2 changes: 1 addition & 1 deletion benchmarks/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,5 +15,5 @@
# See the License for the specific language governing permissions and
# limitations under the License.
#
# SPDX-License-Identifier: EPL-2.0

#
76 changes: 0 additions & 76 deletions benchmarks/adversarial_networks/tensorflow/dcgan/README.md

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

2 changes: 1 addition & 1 deletion benchmarks/common/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,5 +15,5 @@
# See the License for the specific language governing permissions and
# limitations under the License.
#
# SPDX-License-Identifier: EPL-2.0

#
6 changes: 3 additions & 3 deletions benchmarks/common/base_benchmark_util.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
# See the License for the specific language governing permissions and
# limitations under the License.
#
# SPDX-License-Identifier: EPL-2.0

#

from __future__ import absolute_import
Expand Down Expand Up @@ -67,8 +67,8 @@ def _define_args(self):

self._common_arg_parser.add_argument(
"-p", "--precision",
help="Specify the model precision to use: fp32, int8, or bfloat16",
required=required_arg, choices=["fp32", "int8", "bfloat16"],
help="Specify the model precision to use: fp32, int8",
required=required_arg, choices=["fp32", "int8"],
dest="precision")

self._common_arg_parser.add_argument(
Expand Down
2 changes: 1 addition & 1 deletion benchmarks/common/base_model_init.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
# See the License for the specific language governing permissions and
# limitations under the License.
#
# SPDX-License-Identifier: EPL-2.0

#

import glob
Expand Down
17 changes: 3 additions & 14 deletions benchmarks/common/platform_util.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
# See the License for the specific language governing permissions and
# limitations under the License.
#
# SPDX-License-Identifier: EPL-2.0

#

from __future__ import absolute_import
Expand Down Expand Up @@ -199,20 +199,9 @@ def __init__(self, args):
raise ValueError("Unable to determine Operating system type.")

def linux_init(self):
# check to see if the lscpu command is present
lscpu_path = ''
lscpu_path_cmd = "command -v lscpu"
lscpu_cmd = "lscpu"
try:
lscpu_path = subprocess.check_output(lscpu_path_cmd, shell=True,
stderr=subprocess.STDOUT).\
strip()
if self.args.verbose:
print("lscpu_path_cmd = {}\nlscpu located here: {}".format(lscpu_path_cmd, lscpu_path))
if not os.access(lscpu_path, os.F_OK | os.X_OK):
raise ValueError("{} does not exist or is not executable.".
format(lscpu_path))

lscpu_output = subprocess.check_output([lscpu_path],
lscpu_output = subprocess.check_output([lscpu_cmd],
stderr=subprocess.STDOUT)
# handle python2 vs 3 (bytes vs str type)
if isinstance(lscpu_output, bytes):
Expand Down
2 changes: 1 addition & 1 deletion benchmarks/common/tensorflow/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,5 +15,5 @@
# See the License for the specific language governing permissions and
# limitations under the License.
#
# SPDX-License-Identifier: EPL-2.0

#
2 changes: 1 addition & 1 deletion benchmarks/common/tensorflow/run_tf_benchmark.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
# See the License for the specific language governing permissions and
# limitations under the License.
#
# SPDX-License-Identifier: EPL-2.0

#

from __future__ import absolute_import
Expand Down
Loading

0 comments on commit 31e91eb

Please sign in to comment.