eCloudtureeCloudture
  • Training
    • Cloud Map
    • Cloud Training
    • Cloud Certification
  • Career
    • University Cloud Seeding Program
  • Resource
    • Blog
    • Examinations Center
    • Venue Rental
  • eCloudture
    • About eCloudture
    • Sharing Story
    • Contact
  • chinese
    • Training
      • Cloud Map
      • Cloud Training
      • Cloud Certification
    • Career
      • University Cloud Seeding Program
    • Resource
      • Blog
      • Examinations Center
      • Venue Rental
    • eCloudture
      • About eCloudture
      • Sharing Story
      • Contact
    • chinese
    • Home
    • AI/ML
    • Create a Virtual assistant app with Amazon Lex and AWS Amplify

    Create a Virtual assistant app with Amazon Lex and AWS Amplify

    • Posted by Shelly Yu
    • Categories AI/ML, All Post
    • Date 09/08/2019
    • Comments 0 comment

    Overview



    AWS Amplify recently added support for augmented reality (AR) and virtual reality (VR) content within your applications. The XR category has built-in support for Amazon Sumerian.

    Amazon Lex is a service for building conversational interfaces into any application using voice and text. As a fully managed service, you don’t need to worry about managing infrastructure.

    Amazon Sumerian lets you create and run virtual reality (VR), augmented reality (AR), and 3D applications quickly and easily without requiring any specialized programming or 3D graphics expertise.

    AWS Amplify makes it easy to create, configure, and implement scalable mobile and web apps powered by AWS.

    Scenario

    We can design Amazon Lex chatbot according to different situations and cooperate with AWS Amplify to automatically deploy the application. We don’t worry about the backend program and hardware load problem at all, just focus on the design of the main code. The Amazon Sumerian scene can be quickly deployed in various installations, suitable for interactive receptionists of various large-scale exhibition events or various simulation teaching materials.

    Preparation

    • Use Windows for example
    • Region in US East (N. Virginia)
    • Recommend using VScode for this lab, because it can edit the project and use the command line at the same time.

    Create an Amazon Lex Chatbot

    First, we used the Amazon Lex default “BookTrip” template to simplify the process to create the theme of this chatbot. The robot will detect the intent to judge the user’s intentions, and reply to the relevant dialogue or make corresponding actions. For example, the robot will continuously ask the user around the theme of “BookTrip” and finally make a reservation. Or the action of renting a car.

    • On the Service menu, click Amazon Lex, choose Create.

    • Choose BookTrip template, type Bot name : yourbotname, and then Create.


    002-Amazon Lex_create-01.jpg

    • Choose the bot created before, click Settings, choose Aliases.

      • Setting as following :

        • Alias name : test

        • Bot version : latest

          It will give a different alias name for the different versions for each time.

    • Click PLUS icon to add new alias.



    • Slect Publish.

      • Choose an alias : test, and Publish.



    Waiting for publishing, and you can close the page.

    Design AWS Amplify project

    Open Visual Studio Code’s terminal, Using the command line to install AWS Amplify CLI.

    • AWS Amplify CLI install

    • Create react app by using create-react-app, npx create-react-app sumerian-amplify-app.

    The file will default create in C:\Users\USER\sumerian-amplify-app.

    • Change to react project with cd sumerian-amplify-app.

    • Install the aws-amplify and aws-amplify-react libaries with npm install aws-amplify aws-amplify-react --save.

    • Open sumerian-amplify-app project with Visual Studio Code

    • Update the App.js.

      Replace your_scene_region in line 13.

      For eaxample : us-east-1

      Replace your_scene_name in line 15, 28.

    • Insert the following content in Index.css.

      body { height: 600px; }

    Add an Authentication Service to the Amplify Project

    We use AWS Amplify to build the identity verification capabilities of the application, which is just one of the features in the AWS Amplify tools, if you need to add more features to the application, please go to AWS Amplify.

    • Get into your sumerian-amplify-app folder in terminal.

    • Type amplify add auth in the terminal, select Yes to use the default configuration.



    It will add an identity pool in AWS Cognito automatically.

    • Type amplify push to update CloudFormation stack.

    NOTES: It will take three to five minutes to update the stack.

    • Extend the amplify folder, open the amplify-meta.jason.



    • Note the UnauthRoleName and AuthRoleName and IdentityPoolName, we will use it later.





    • Back to AWS console click Services, select IAM.

    • Choose Roles, search your UnAuthRoleName and click in.

    • Select Add inline policy.



    • Select JSON, and paste the content.

    • Click Review policy, name your policy and then Create policy.

    This policy will allow your Sumerian scene to use Amazon Lex and allow your app to load the Amazon Sumerian scene.

    • Attach the same Inline policy to AuthRolename with the same steps as UnauthRolename.

      Add a role

    • Click Services, select Cognito.

    • Select Manage Identity Pools.

    • Search for your IdentityPoolName and click Edit identity pool on the upper right corner.

      AWS Cognito console doesn’t have the search bar, please look for the identity pool name by yourself.

    • Extend the Unauthenticated identities, check the Enable access to unauthenticated identities option.


    010-identity pool_check-01.jpg

    • Copy the Identity pool ID, we will use it later.


    011-identity pool_copy-01.jpg

    Create an Amazon Sumerian Scene

    We will create a new scene and add the host to talk (Maya), and then connect to the previously established Amazon Lex chatbot by setting Dialogue Component to give the host (Maya) the function of a voice conversation. Finally, Then publish a private Amazon Sumerian scene for our application to use.

    • On the Service menu, click Amazon Sumerian, you will get into the Amazon Sumerian Dashboard.

    • Select Create new scene, and type your Scene name.

    • Click the Entities of yourscenename on the upper left corner, and then extend the AWS Configuration on the right side.

      You will see the config list after click your “yourscenename”


    012-Sumerian_update-01.jpg

    • Extend the AWS Configuration, update the Cognito Identity Pool ID your copy before.

      This step is to enable this scene to use the permissions just set in AWS Cognito


    013-Sumerian_ID-01.jpg

    • Click Import Assets, choose Maya and select Add to add a host.


    014-Sumerian host_import-01.jpg

    • Maya will show in the Assets on your left side.


    015-Sumerian host_import-02.jpg

    • Drop the Maya in your scene, you will see Maya.


    016-Sumerian host_import-03.jpg

    Note: You may need to zoom in to see Maya.

    • Select Maya entities, and click Add component, choose Dialogue.


    017-Sumerian config_add dialogue-01.jpg

    • Insert the values for Name and Alias, these inputs reference the Amazon Lex chatbot what you created.

      Bot name is created by yourself, Alias name is a test

      This step gives Maya the ability of Amazon Lex chatbot


    018-Sumerian config_add dialogue-02.jpg

    • Select Maya entities, and click Add component, choose State Machine.


    019-Sumerian config_add state machine-01.jpg

    • Open the State Machine, click PLUS button to add new behavior.


    020-Sumerian config_add state machine-02.jpg

    • Rename the name : ChatBot.


    021-Sumerian config_add state machine-03.jpg

    Next, we will add different states in this behavior component.

    • Rename State 1 : Start, and choose Add Action.


    022-Sumerian config_add state machine-04.jpg

    • Search for and add AWS SDK Ready.


    023-Sumerian config_add state machine-05.jpg

    • Click Add State for five times to create five states.


    024-state action_add-01.jpg

    • Choose the new state, and rename it.

      State 1 : Wait for Input

      State 2 : Start Recording

      State 3 : Stop Recording

      State 4 : Process with Lex

      State 5 : Play Response


    025-state action_add-02.jpg

    • Click state Wait for Input, add a Key Down action.


    026-state action_add-03.jpg

    • Change the key what you want.

      I use T in this lab


    027-state action_add-04.jpg

    • Click state Start Recording, add Start Microphone Recording and Key Up action.

      It must the same with Key down


    028-state action_add-05.jpg

    • Click state Stop Recording, add a Stop Microphone Recording action.


    029-state action_add-06.jpg

    • Click state Process with Lex, add a Send Audio Input to Dialogue Bot action.


    030-state action_add-07.jpg

    Do not select Log user input or Log bot response.

    • Click state Play Response, add a Start Speech action and select Use Lex Response.


    031-state action_add-08.jpg

    • Adding transitions by clicking the state and dragging the arrow to another state

    On the Process With Lex state, drag the transition from the “On Response Ready output”.


    032-state action_add-09.jpg

    The final graph will look like the following.


    033-state action_final-10.jpg

    • Select Publish on the upper right corner, choose Host privately and Publish.


    034-Sumerian scene_publish-01.jpg

    If you are already published, click unpublish.


    035-Sumerian scene_publish-02.jpg

    • You will see a JSON file, Download JSON configuration.

    • Drag the JSON file in src folder and rename it to sumerian-exports.js.


    036-Sumerian scene_publish-03.jpg

    • Update the content of sumerian-export.js as fowling.


    037-JSON file_update-01.jpg

    Running on a VR or AR Device

    • Type amplify add hosting to set up.

    • Choose DEV to create a S3 hosting bucket.


    038-add hosting_set up-01.jpg

    • Insert your hosting bucket name, and choose the following option.


    039-add hosting_set up-02.jpg

    Once set up, AWS Amplify will automatically create an S3 Bucket

    • Type amplify publish to publish your Amplify project.


    040-add hosting_publish-01.jpg

    This publishes the app to a live URL, you can open the URL in any browser or VR headset.


    041-run app_start-01.jpg


    042-run app_start-02.jpg

    • Create account and Login, you will see the Sumerian scene.

    • Press the key you set up before to talk with her.

    You can say “Book a hotel” or “Book a car” to start the dialogue.


    043-lex_flow-01.png


    044-lex_flow-02.png

    Conlusion

    Now you have learned how to install and set up the AWS Amplify CLI and create applications with AWS Amplify. Also learn how to create an Amazon Sumerian scene, add the host to connect with Amazon Lex chatbot, and publish the Amazon Sumerian scenes privately, integrate them into your application or other devices, and create your own virtual assistant.

    Reference

    • Using the Dialogue Component and Amazon Lex to Build a Chatbot

    • AWS Amplify

    • AWS Amplify Tutorial

    • Amazon Lex Programming Model

    Tag:Amplify, Chatbot, Lex, Sumerian

    • Share:
    Shelly Yu

    Previous post

    Set S3 CORS Configuration to complete cross-domain request
    09/08/2019

    Next post

    Amazon Personalize
    09/08/2019

    You may also like

    reInvent_facebook_post_werner_Eng
    AWS re:Invent 2021 – Werner Vogels Keynote
    6 December, 2021
    reInvent-Adam-Eng_工作區域 1
    AWS re:Invent 2021 – Adam Selipsky Keynote
    1 December, 2021
    shutterstock_134102588
    AutoScaling – Health check v.s. Status Check
    25 November, 2021

    Leave A Reply Cancel reply

    Your email address will not be published. Required fields are marked *

    搜尋文章

    Categories

    • AI/ML
    • All Post
    • AWS re:Invent
    • Big Data
    • Container
    • Foundation
    • News
    • Security
    • Serverless
    Phone : +886 2 22801777
    Mail : info@ecloudture.com

    Training

    • Cloud Training
    • Cloud Map
    • Cloud Certification

    Talent

    • University Cloud Seeding Program

    Resource

    • Blog
    • Examinations Center
    • Venue Rental

    eCloudture

    • About eCloudture
    • Sharing Story
    • Contact

    • Privacy
    • Terms
    • Sitemap