Building the “Alexa, write my quiz” Skill Part 1
Voice assistants, such as Amazon’s Alexa, make it feel like we’re already living in a future that was once only science fiction. Today, the most effective use of voice assistants is to answer questions that people ask regularly. This makes for a perfect match with many issues in the education space, since the learning process presents uniquely recurring and time-intensive challenges.
One regular and particularly burdensome task for instructors is writing quizzes. If a computer could take over this time-intensive process, then instructors could use this time as they see fit, such as for instruction or personalized feedback for students. Additionally, once the initial task of automating quiz generation is completed, machine learning could be used to improve this process and improve the quality of the quizzes, a goal that I will discuss in the next blog post in this series.
To start on the task of automating quiz generation, I’ve created an Alexa skill as shown in Figure 1 that can either pull questions from an existing test bank or pull a glossary file from Canvas to convert into a test bank and ultimately return those questions to students either as an interactive quiz inside Alexa or as a quiz file in Canvas for the professor. To accomplish this, I slightly modified the textbook parser used by another Unicon colleague's work, added to and updated their Lex chatbot, integrated with Canvas, and then converted it to an Alexa skill.
Figure 1: Diagram of AWS pipeline for Alexa skill
alexa-stream-handlerLambda retrieves the slot data listed inside the gray box from the user through the Alexa skill. It then triggers the
write-quiz-lambdaLambda and sends the data there. This Lambda can retrieve a pdf from Canvas, put it in the canvas-source S3 bucket, convert it to the appropriate csv files, retrieve those via Athena, create a quiz, then upload that to Canvas. If the user requests that the quiz source is from an existing test bank, then the steps for retrieving and converting the pdf files from Canvas are skipped.
Classroom Lex Chatbot Updates
First I needed software that could take in a pdf and output a test bank csv file that could be uploaded to S3 and used as a database via AWS Athena. My colleagues had already created a
pdf-parserjar that could be pointed to a pdf file on the host and converted it into a glossary csv file using Apache PDFBox. The glossary csv file was a table with columns
definition; these files could be queried with Athena as the glossary database as shown in Figure 2a. They also had an Athena quiz database consisting of csv files that acted as a table with columns
quizData was json indicating the quiz type, answer choices, and answer as shown in Figure 2b. Using these two databases, they created a Lex Chatbot that included a
QuizService asked the user to fill in a
quizName, and an
answer slot. It would first prompt for the subject and quiz name, then use that data to query the Athena quiz database table and the Athena glossary database to create a vocabulary quiz for the user and prompt for the answer.
Figure 2a: Athena glossary database table
Figure 2b: Original Athena quiz database table
Since the existing files in the Athena quiz database had been created manually, I updated the code in the
pdf-parser jar to add the ability to parse multiple choice test bank pdfs into csv files for this database as shown in Figure 3. To do this, I added the json field “question” inside the
quizData column since I didn’t want to have a separate database to hold the multiple-choice questions, even though the separate Athena glossary database held the definitions that served as the questions for the existing
QuizService intent. In order to distinguish between the two setups, I added the
quizType “multiple-choice” to indicate that there would be a “question” json field, leaving the
quizType “definition” to indicate that the Athena glossary database would need to be queried for the definition/question. With extensive testing, I coded to cover multiple edge cases found in the test bank pdf files. This allowed me to create a pretty robust implementation for known input formats with the surety that it could always be improved to be robust against additional input formats.
Figure 3: Updated Athena quiz database table
After uploading the output quiz csv files to the S3 bucket for the Athena quiz database, I updated the existing
QuizService intent to check for the
quizType so that it would query the glossary database to create the question only if needed and otherwise use the question provided in the
Then I updated the
QuizService to ask the user for the number of questions they wanted to be quizzed over via the numQuest slot. To ensure that the user would always be asked different questions, I further modified the
QuizService to randomly select the requested number of questions from the Athena quiz database output. I then added a counter to the session that would wait for that number of questions to be asked prior to clearing all of the slot data.
Now, I had all of the pieces in place for writing quizzes; I just had to write output to a text file and upload it to S3 instead of returning it to the user. For these tweaks, I created a new intent called
Then I updated the
pdf-parser jar to be able to take pdf glossaries from OpenStax textbooks and convert those into quiz csv files, still using Apache PDFBox. In order to get my jar to correctly parse pdf files that consisted of two columns of text, I had to fork the PDFBox repository and change the writeLine method and WordWithTextPositions class to be protected instead of private so that I could override them with code to separate the columns. Aside from that hiccup, all that was needed was implementation of an additional Java Writer that formatted the glossary data according to my existing csv Athena database schema.
Next, I integrated my Lex intent with the Canvas API so that a glossary uploaded to Canvas could be pulled down by my Lambda function, converted to a quiz, and then uploaded back to Canvas. All Canvas files for a course can be retrieved in a json format by performing a GET at
/api/v1/courses/:course_id/files. From that json output, each file’s block of json indicates a “folder_id” that it’s contained in. With that value, all of the files in a specific folder in Canvas can be retrieved in a json format by performing a GET at
/api/v1/folders/:folder_id/files. Each file’s block of json has a “url” field with the url that can download the specific file. I used Apache’s HttpClient to go through this process in my Java code to download the file. Then I sent the file through my
pdf-parser described in the section above to create a csv quiz from the file. Next it went back through my Lex Lambda to format the data as a quiz in a text file.
Finally, the text file quiz needed to be uploaded to Canvas. Still using Apache’s HttpClient, I updated my Java code to perform a POST on
/api/v1/courses/:course_id/files. This POST returned a json response containing the element “upload_url.” Next, I performed a POST to the “upload_url” with a body that contained the text file that I wanted to upload to Canvas.
Lex Chatbot to Alexa Skill
Following this example, it appeared that converting between the Lex Chatbot and Alexa skill would be straightforward. As the example instructed, in the AWS Lex Console, I clicked on Actions > Export to download a json file containing the Lex bot intent and slot schema. Next, I went to the Alexa Developer Console and clicked on the “Create Skill” button. Then I entered a name for my skill, selected a Custom model, and selected to provision my own backend resources (meaning I’d be providing my own Lambda in AWS). Next, I clicked on the “Interaction Model” tab and then the “JSON Editor”. On the JSON Editor screen shown in Figure 4, I uploaded the schema file that I had downloaded from the Lex bot. Then I clicked on “Endpoint” and pasted my ARN to hook up my Lambda, and I added the Alexa Skills Kit as a trigger for my Lambda in AWS using the Skill ID. Finally, I clicked on the “Test” tab in the Alexa Developer Console, and tried out my skill.
Figure 4: Importing Lex Chatbot schema to Alexa skill
Unfortunately, it didn’t work at all. Because the request and response json for Alexa is in a completely different format than for Lex, little to none of the code in my Lex Lambda could be used for my Alexa skill Lambda, so I had to rewrite all of it. I added Amazon’s ask-sdk to my POM file, wrote a handler with HandlerInput and IntentRequest objects as arguments, and then rewrote all of my services to deal with those objects instead of the request and context objects that my Lex handler used. Once that was finished, my Alexa Lambda was ready, so I pasted its ARN as the Endpoint for my Alexa skill, and I was finally able to successfully test out my skill as shown in Figure 5.
Figure 5: Testing Alexa skill
Voice interfaces are becoming increasingly common in our lives, propelling us into an incredible future in which computers act as our personal assistants to do repetitive or time-intensive tasks for us so that we have more time to spend on what matters most. With computers and voice interfaces taking the labor out of teaching where possible, instructors can have more time to spend on the highly valuable personal interactions of teaching. I have outlined a simple case of how to start down the path towards using Alexa in the education space, specifically to write quizzes from test banks and upload them to Canvas. I look forward to starting to implement machine learning to parse the pdf files that are turned into test banks such that this technology can be usable without having to customize the pdf parser any time the pdf files are in a slightly different format. What other labor-intensive education tasks would you like us to solve with Alexa?
- Github Repo: https://github.com/Unicon/alexa-skill-quiz-generator
- Building the “Alexa, write my quiz” Skill Part 2: Review & Application of Machine Learning Technologies for Glossary Text Extraction