• 2 min read
UIUC built an AI tutor for ECE 120
An open-source UIUC project uses 11 models in parallel to answer electrical engineering questions in about 2 seconds.
Image: Hacker News
A UIUC AI Teaching Assistant project aims to deliver what its creators call “Better than Google” multimedia search and question answering for electrical engineering coursework. The system is already live on Hugging Face at https://huggingface.co/spaces/kastan/ai-teaching-assistant and is designed around material for UIUC course ECE 120, an introduction to electrical engineering.
According to the project page, the system runs 11 separate models in parallel for text and image retrieval, generation, moderation, and ranking, while still hitting a median 2-second response time.
The assistant pulls from textbooks, lecture videos, and student Q&A forums, listed by the team in that order of importance. That source data is not publicly available, because the project was not granted distribution rights by the original authors. Still, the broader codebase is fully open source, aside from commercial textbook content, and the team encourages others to swap in their own Pinecone document database.
One of the project’s more unusual elements is its RLHF workflow. The author highlights a method that combines semantic search retrieval during RLHF, built using a dataset created iteratively with the help of five electrical engineering students. That dataset is publicly available on Hugging Face at https://huggingface.co/datasets/kastan/rlhf-qa-comparisons.
For evaluation, the team uses an in-house dataset of Q&A pairs written by expert electrical engineers, then compares model outputs against human-written answers. The final judge is GPT-3, which labels answers as “better” or “worse” than the ground truth. The project page openly acknowledges the weakness here: GPT-3 is evaluating itself, and it tends to rate GPT-3 highly.
Recommended reading
Fandango Drops 'at Home' in Free Streaming Push
The repository includes:
- main.py for aggregating model implementations
- TA_gradio_ux.py for the Gradio interface
- prompting.py for prompt engineering
- evaluation.py for GPT-3-based system evaluation
- feedback.json for user feedback collected through the app
- run_ta_gradio.sh as the launch script
Setup is straightforward: install Python requirements, add the API keys listed in run_ta_gradio.sh, build a Pinecone database from your own PDFs, transcripts, or slide images, and launch the web app with bash run_ta_gradio.sh.
Computing Editor
Tomas lives in the terminal. He covers chips, laptops, and operating systems with a focus on performance and efficiency. He reads kernel changelogs the way other people read fiction, and he's always on the hunt for the perfect mechanical keyboard switch. If it processes data, Tomas has an opinion on it.
via Hacker News
