I clarify tips on how to construct an app that generates a number of alternative questions (MCQs) on any user-defined topic. The app is extracting Wikipedia articles which can be associated to the consumer’s request and makes use of RAG to question a chat mannequin to generate the questions.
I’ll show how the app works, clarify how Wikipedia articles are retrieved, and present how these are used to invoke a chat mannequin. Subsequent, I clarify the important thing elements of this app in additional element. The code of the app is obtainable right here.
App Demo
The gif above reveals the consumer getting into the educational context, the generated MCQ and the suggestions after the consumer submitted a solution.
On the first display screen the consumer describes the context of the MCQs that ought to be generated. After urgent “Submit Context” the app searches for Wikipedia articles which content material matches the consumer question.
The app splits every Wikipedia web page into sections and scores them primarily based on how intently they match the consumer question. These scores are used to pattern the context of the subsequent query which is displayed within the subsequent display screen with 4 selections to reply. The consumer can choose a alternative and submit it by “Submit Reply”. It’s also doable to skip this query by way of “Subsequent Query”. On this case it’s thought-about that the query didn’t meet the consumer’s expectation. Will probably be averted to make use of the context of this query for the era of following questions. To finish the session the consumer can select “Finish MCQ”.
The subsequent display screen after the consumer submitted a solution reveals if the reply was appropriate and supplies an extra clarification. Following, the consumer can both get a brand new query by way of “Subsequent Query” or finish the session with “Finish MCQ”.
The top session display screen reveals what number of questions have been accurately and wrongly answered. Moreover, it additionally comprises the variety of questions the consumer rejected by way of “Subsequent Query”. If the consumer selects “Begin New Session” the beginning display screen will probably be displayed the place a brand new context for the subsequent session might be offered.
Idea
The goal of this app is to provide prime quality and up-to-date questions on any user-defined matter. Thereby consumer suggestions is taken into account to make sure that the generated questions are assembly the consumer’s expectations.
To retrieve high-quality and up-to-date context, Wikipedia articles are chosen with respect to the consumer’s question. Every article is cut up into sections whereas each part is scored primarily based on its similarity with the consumer question. If the consumer rejects a query the respective part rating will probably be downgraded to scale back the probability of sampling this part once more.
This course of might be separated into two workflows:
- Context Retrieval
- Query Era
That are described under.
Context Retrieval
The workflow how the context of the MCQs is derived from Wikipedia primarily based on the consumer question is proven under.
The consumer inserts the question that describes the context of the MCQs at the beginning display screen. An instance of the consumer question might be: “Ask me something about stars and planets”.
To effectively seek for Wikipedia articles this question is transformed into key phrases. The key phrases of the question above are: “Stars”, “Planets”, “Astronomy”, “Photo voltaic System”, and “Galaxy”.
For every key phrase a Wikipedia search is executed of which the highest three pages are chosen. Not every of those 15 pages are match to the question offered by the consumer. To take away irrelevant pages on the earliest doable stage the vector similarity of the embedded consumer question and web page excerpt is calculated. Pages which similarity is under a threshold are filtered out. In our instance 3 of 15 pages have been eliminated.
The remaining pages are learn and divided into sections. As not the complete web page content material could also be associated to the consumer question, splitting the pages into sections permits to pick out elements of the web page that match particularly properly to the consumer question. Therefore, for every part the vector similarity in opposition to the consumer question is calculated and sections with low similarity are filtered out. The remaining 12 pages contained 305 sections of which 244 have been stored after filtering.
The final step of the retrieval workflow is to assign a rating to every part with respect to the vector similarity. This rating will later be used to pattern sections for the query era.
Query Era
The workflow to generate a brand new MCQ is proven under:
Step one is to pattern one part with respect to the part scores. The textual content of this part is inserted along with the consumer question right into a immediate to invoke a chat mannequin. The chat mannequin returns a json formatted response that comprises the query, reply selections, and an evidence of the proper reply. In case the context offered will not be appropriate to generate a MCQ that addresses the consumer question the chat mannequin is instructed to return a key phrase to establish that the query era was not profitable.
If the query era was profitable, the questions and the reply selections are exhibited to the consumer. As soon as the consumer submits a solution it’s evaluated if the reply was appropriate, and the reason of the proper reply is proven. To generate a brand new query the identical workflow is repeated.
In case the query era was not profitable, or the consumer rejected the query by clicking on “Subsequent Query” the rating of the part that was chosen to generate the immediate is downgraded. Therefore, it’s much less probably that this part will probably be chosen once more.
Key Elements
Subsequent, I’ll clarify some key elements of the workflows in additional element.
Extracting Wiki Articles
Wikipedia articles are extracted in two steps: First a search is run to search out appropriate pages. After filtering the search outcomes, the pages separated by sections are learn.
Search requests are despatched to this URL. Moreover, a header containing the requestor’s contact info and a parameter dictionary with the search question and the variety of pages to be returned. The output is in json format that may be transformed to a dictionary. The code under reveals tips on how to run the request:
headers = {'Person-Agent': os.getenv('WIKI_USER_AGENT')}
parameters = {'q': search_query, 'restrict': number_of_results}
response = requests.get(WIKI_SEARCH_URL, headers=headers, params=parameters)
page_info = response.json()['pages']After filtering the search outcomes primarily based on the pages’ excerpts the textual content of the remaining pages is imported utilizing wikipediaapi:
import wikipediaapi
def get_wiki_page_sections_as_dict(page_title, sections_exclude=SECTIONS_EXCLUDE):
wiki_wiki = wikipediaapi.Wikipedia(user_agent=os.getenv('WIKI_USER_AGENT'), language='en')
web page = wiki_wiki.web page(page_title)
if not web page.exists():
return None
def sections_to_dict(sections, parent_titles=[]):
consequence = {'Abstract': web page.abstract}
for part in sections:
if part.title in sections_exclude: proceed
section_title = ": ".be a part of(parent_titles + [section.title])
if part.textual content:
consequence[section_title] = part.textual content
consequence.replace(sections_to_dict(part.sections, parent_titles + [section.title]))
return consequence
return sections_to_dict(web page.sections)To entry Wikipedia articles, the app makes use of wikipediaapi.Wikipedia, which requires a user-agent string for identification. It returns a WikipediaPage object which comprises a abstract of the web page, web page sections with the title and the textual content of every part. Sections are hierarchically organized which means every part is one other WikipediaPage object with one other checklist of sections which can be the subsections of the respective part. The operate above reads all sections of a web page and returns a dictionary that maps a concatenation of all part and subsection titles to the respective textual content.
Context Scoring
Sections that match higher to the consumer question ought to get the next likelihood of being chosen. That is achieved by assigning a rating to every part which is used as weight for sampling the sections. This rating is calculated as follows:
[s_{section}=w_{rejection}s_{rejection}+(1-w_{rejection})s_{sim}]
Every part receives a rating primarily based on two elements: how typically it has been rejected, and the way intently its content material matches the consumer question. These scores are mixed right into a weighted sum. The part rejection rating consists of two elements: the variety of how typically the part’s web page has been rejected over the best variety of web page rejections and the variety of this part’s rejections over the best variety of part rejections:
[s_{rejection}=1-frac{1}{2}left( frac{n_{page(s)}}{max_{page}n_{page}} + frac{n_s}{max_{s}n_s} right)]
Immediate Engineering
Immediate engineering is a vital side of the Studying App’s performance. This app is utilizing two prompts to:
- Get key phrases for the wikipedia web page search
- Generate MCQs for sampled context
The template of the key phrase era immediate is proven under:
KEYWORDS_TEMPLATE = """
You are an assistant to generate key phrases to seek for Wikipedia articles that include content material the consumer needs to be taught.
For a given consumer question return at most {n_keywords} key phrases. Ensure each key phrase is an effective match to the consumer question.
Fairly present fewer key phrases than key phrases which can be much less related.
Directions:
- Return the key phrases separated by commas
- Don't return the rest
"""This technique message is concatenated with a human message containing the consumer question to invoke the Llm mannequin.
The parameter n_keywords set the utmost variety of key phrases to be generated. The directions be sure that the response might be simply transformed to an inventory of key phrases. Regardless of these directions, the LLM typically returns the utmost variety of key phrases, together with some much less related ones.
The MCQ immediate comprises the sampled part and invokes the chat mannequin to reply with a query, reply selections, and an evidence of the proper reply in a machine-readable format.
MCQ_TEMPLATE = """
You're a studying app that generates multiple-choice questions primarily based on instructional content material. The consumer offered the
following request to outline the educational content material:
"{user_query}"
Primarily based on the consumer request, following context was retrieved:
"{context}"
Generate a multiple-choice query instantly primarily based on the offered context. The proper reply should be explicitly said
within the context and will all the time be the primary choice within the selections checklist. Moreover, present an evidence for why
the proper reply is appropriate.
Variety of reply selections: {n_choices}
{previous_questions}{rejected_questions}
The JSON output ought to observe this construction (for variety of selections = 4):
{{"query": "Your generated query primarily based on the context", "selections": ["Correct answer (this must be the first choice)","Distractor 1","Distractor 2","Distractor 3"], "clarification": "A short clarification of why the proper reply is appropriate."}}
Directions:
- Generate one multiple-choice query strictly primarily based on the context.
- Present precisely {n_choices} reply selections, making certain the primary one is the proper reply.
- Embody a concise clarification of why the proper reply is appropriate.
- Don't return the rest than the json output.
- The offered clarification shouldn't assume the consumer is conscious of the context. Keep away from formulations like "As said within the textual content...".
- The response should be machine readable and never include line breaks.
- Test whether it is doable to generate a query primarily based on the offered context that's aligned with the consumer request. If it isn't doable set the generated query to "{fail_keyword}".
"""
The inserted parameters are:
user_query: textual content of consumer questioncontext: textual content of sampled partn_choices: variety of reply selectionsprevious_questions: instruction to not repeat earlier questions with checklist of all earlier questionsrejected_questions: instruction to keep away from questions of comparable nature or context with checklist of rejected questionsfail_keyword: key phrase that signifies that query couldn’t be generated
Together with earlier questions reduces the possibility that the chat mannequin repeats questions. Moreover, by offering rejected questions, the consumer’s suggestions is taken into account when producing new questions. The instance ought to be sure that the generated output is within the appropriate format in order that it may be simply transformed to a dictionary. Setting the proper reply as the primary alternative avoids requiring an extra output that signifies the proper reply. When displaying the alternatives to the consumer the order of selections is shuffled. The final instruction defines what output ought to be offered in case it isn’t doable to generate a query matching the consumer question. Utilizing a standardized key phrase makes it straightforward to establish when the query era has failed.
Streamlit App
The app is constructed utilizing Streamlit, an open-source app framework in Python. Streamlit has many features that permit so as to add web page components with just one line of code. Like for instance the aspect by which the consumer can write the question is created by way of:
context_text = st.text_area("Enter the context for MCQ questions:")the place context_text comprises the string, the consumer has written. Buttons are created with st.button or st.radio the place the returned variable comprises the data if the button has been pressed or what worth has been chosen.
The web page is generated top-down by a script that defines every aspect sequentially. Each time the consumer is interacting with the web page, e.g. by clicking on a button the script might be re-run with st.rerun(). When re-running the script, it is very important carry over info from the earlier run. That is carried out by st.session_state which may include any objects. For instance, the MCQ generator occasion is assigned to session states as:
st.session_state.mcq_generator = MCQGenerator()in order that when the context retrieval workflow has been executed, the discovered context is obtainable to generate a MCQ on the subsequent web page.
Enhancements
There are numerous choices to boost this app. Past Wikipedia, customers might additionally add their very own PDFs to generate questions from customized supplies—reminiscent of lecture slides or textbooks. This is able to allow the consumer to generate questions on any context, for instance it might be used to organize for exams by importing course supplies.
One other side that might be improved is to optimize the context choice to attenuate the variety of rejected questions by the consumer. As a substitute of updating scores, additionally a ML mannequin might be skilled to foretell how probably it’s {that a} query will probably be rejected with respect to options like similarity to accepted and rejected questions. Each time one other query is rejected this mannequin might be retrained.
Additionally, the generated query might be saved in order that when a consumer needs to repeat the educational train these questions might be used once more. An algorithm might be utilized to pick out beforehand wrongly answered questions extra incessantly to give attention to enhancing the learner’s weaknesses.
Abstract
This text showcases how retrieval-augmented era (RAG) can be utilized to construct an interactive studying app that generates high-quality, context-specific multiple-choice questions from Wikipedia articles. By combining keyword-based search, semantic filtering, immediate engineering, and a feedback-driven scoring system, the app dynamically adapts to consumer preferences and studying objectives. Leveraging instruments like Streamlit permits speedy prototyping and deployment, making this an accessible framework for educators, college students, and builders alike. With additional enhancements—reminiscent of customized doc uploads, adaptive query sequencing, and machine learning-based rejection prediction—the app holds robust potential as a flexible platform for personalised studying and self-assessment.
Additional Studying
To be taught extra about RAGs I can advocate these articles from Shaw Talebi and Avishek Biswas. Harrison Hoffman wrote two wonderful tutorials on embeddings and vector databases and constructing an LLM RAG Chatbot. How one can handle states in streamlit might be present in Baertschi’s article.
If not said in any other case, all photos have been created by the writer.
