Skip to content

Rock-paper-scissors game example which uses CustomVision to recognize user's gestures

License

Notifications You must be signed in to change notification settings

ionicastefan20/rock-paper-scissors-customvision

 
 

Repository files navigation

description languages page_type products urlFragment
Code and tutorial for Rock-paper-scissors game used at Azure Custom Vision hands-on workshop
javascript
sample
azure
azure-cognitive-services
rock-paper-scissors-customvision

Rock-paper-scissors

Code and tutorial for "Rock-paper-scissors" game used at Azure Custom Vision hands-on workshop.

Contents

Outline the file contents of the repository. It helps users navigate the codebase, build configuration and any related assets.

File/folder Description
Step{1-7} Sample source code.
.gitignore Define what to ignore at commit time.
README.md This README file.
LICENSE The license for the sample.

Prerequisites

Azure setup

Create a new resource group

  1. Open Azure Portal
  2. Click on Create a resource, search for Resource group in the Marketplace and click on it
  3. Click on Create to create new resource group

Add new resource group

  1. Fill-in the form the following information for your resource group:
    • Subscription: <your_subscription>
    • Resource group RPS_ResourceGroup
    • Region: West Europe

Fill-in the form

  1. Click Review + create
  2. Verify the values and click on Create

Verify the values and click create

  1. Wait until the deployment of the resource group is complete and click on Go to resource group

Create a new Custom Vision resource

  1. Open the resource group you have created. The resource group is empty
  2. Click on Create resources or + Create to create a new resource

Create new resource

  1. Search for Custom Vision in the Marketplace and click on it

Search for "Custom Vision" in the Marketplace and click on it

  1. Click Create
  2. Fill-in the form the following information for your Custom Vision resource:
    • Create options: Both
    • Subscription: <your_subscription>
    • Resource group: RPS_ResourceGroup
    • Region: West Europe
    • Name: CustomVisionRPS (or any available name)
    • Training pricing tier: Free F0
    • Prediction pricing tier: Free F0

Fill-in the form and click create

  1. Click Review + create
  2. Verify the values and click on Create
  3. Wait until the deployment of the resource is complete

Create a new AppService resource

  1. Open your resource group, verify that you now have two resources with Custom Vision type and click on + Create to add another resource

Open your resource group and verify that you now have two resources with "Cognitive Service" type and click "Add" to add another resource

  1. Search for Web App in the Marketplace and click on it

Search for "Web App" in the Marketplace and click on it

  1. Click Create

  2. Fill-in the form the following information for your AppService resource:

    • Subscription: <your_subscription>
    • Resource group: RPS_ResourceGroup
    • Name: <choose_unique_name>
    • Publish: Code
    • Runtime stack: Node 14 LTS (or higher)
    • Operating system: Windows
    • Region: West Europe
  3. Click on Change size for App Service Plan

Fill-in the form and and click "Change size" for App Service Plan

  1. Select Dev / Test plan and F1 pricing tier, then click Apply

Select "Dev / Test" plan and "F1" pricing tier, then click "Apply" 7. Click Review + create

Click "Review and create"

  1. Verify values and click Create
  2. Wait until the deployment of the Web App is complete and click on Go to resource
  3. In your new Web App, on the left panel go to Settings -> Configuration -> Application Settings
  4. Click on + New application setting

Click New application setting

  1. Fill-in the following information:
    • Setting name: SCM_DO_BUILD_DURING_DEPLOYMENT
    • Value: true

Setting name is SCM_DO_BUILD_DURING_DEPLOYMENT and value true

  1. Click OK
  2. Click Save. This will enable build step when we deploy our NodeJS application later

Click Save

  1. Open your resource group and verify that you have the following resources: App Service Plan, App Service, Application Insights and two Custom Vision

Open your resource group and verify that you have the following resources: App Service Plan, App Service and two Cognitive Services

Train a model in Custom Vision

  1. Sign in Custom Vision using your Azure account

Sign in https://www.customvision.ai/ using created Azure account

  1. Click New Project

Click "New Project"

  1. Fill-in the form

    • Name: RPS
    • Resource: RPSCustomVision[F0]
    • Project Types: Classification
    • Classification Types: Multiclass
    • Domains: General
  2. Click Create project

Fill-in the form and click "Create project"

Upload and tag images

  1. In Custom Vision project click Add images

In Custom Vision project click "Add images"

  1. Select the images and click Open to add them
  2. Write an appropriate Tag for your images (for example, paper), press Enter and click Upload files

Add images, select appropriate tag (e.g. paper) and click "Upload files"

  1. Wait until upload is finished

Wait until upload is finished

  1. Repeat the process to add other group of images

Train a model

  1. In Custom Vision project click on Train

In Custom Vision project click "Train"

  1. Select Quick Training and click Train

Select "Quick Training" and click "Train"

  1. Wait for training to finish. This process can take a few minutes

Wait for training to finish

Manual validation

  1. Click on Quick Test

Click on "Quick Test"

  1. Upload a test image

Upload test image

If the prediction of the test image is incorrect, you can correct it and select the right tag.

  1. Open Predictions tab

Click on "Predictions"

  1. Select the incorrectly predicted image

Select incorrectly predicted image

  1. Assign correct tag and click Save and close

Select incorrectly predicted image

(Optional) Advanced training

  1. In Custom Vision project click Train

In Custom Vision project click "Train"

  1. Select Advanced Training, choose 1 hour for training budget, and click Train

Select "Advanced Training" for 1 hour and click "Train"

  1. Wait for training to finish

Wait for training to finish

Publish iteration

  1. Open Performance tab, select the finished iteration and click Publish

Open Performance tab, select the finished iteration and click Publish

  1. Fill-in Model name, select RPSCustomVision_Prediction as prediction resource and click Publish

Fill-in model name, select "RPSCustomVision_Prediction" as prediction resource and click "Publish"

  1. Click on Prediction URL to see your credentials

Click on Prediction URL to see your credentials

Web application code

Application based on code from NodeJS app on Azure example.

Structure

  • index.js - server-side logic
  • public/index.html - WebApp HTML template
  • public/css/app.css - WebApp CSS styles
  • public/js/app.js - WebApp Javascript logic

Local development

Local development is the same for all the steps:

  1. Open "StepN" folder
  2. Run npm install
  3. Run npm start
  4. Open http://localhost:1337

Azure deployment

Azure deployment is the same for all the steps:

  1. Open "StepN" folder
  2. Create zip archive with contents of "StepN" folder, except for node_modules folder
  3. Open <choose_unique_name>.scm.azurewebsites.net
  4. navigate to Tools -> Zip Push Deploy
  5. Drag-and-drop zip archive to /wwwroot
  6. Open <choose_unique_name>.azurewebsites.net and verify that site is running

Code changes are described in respective steps:

  • Step 1: After cloning the repository, you can either choose to develop locally or deploy to Azure

  • Step 2: Initalize camera, take a picture, and store it in canvas element

  • Step 3: Submit and receive a picture to index.js. Then, submit picture from index.js to Custom Vision API. Finally, receive predictions from Custom Vision

  • Step 4: Return best match prediction to client-side and render result to HTML page

  • Step 5: Add AI opponent and show opponents move

  • Step 6: Visualize counter and show results

  • Step 7: Download the Custom Vision model and use it locally with tensorflow.js

Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com.

When you submit a pull request, a CLA bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.

About

Rock-paper-scissors game example which uses CustomVision to recognize user's gestures

Resources

License

Code of conduct

Security policy

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • JavaScript 66.2%
  • CSS 22.1%
  • HTML 11.7%