Apply Makes Passing
in pc science was something however simple. I vividly bear in mind reaching a breaking level across the finish of the tenth week of my first semester. With only a few weeks till my first remaining, I sat gazing Calc 1 follow issues, spiraling into despair. I’d all the time been good at math. I did all of the homework and paid consideration in all of the lectures. So how might it’s that I didn’t even know the place to begin? Why wasn’t something clicking?
I typically joked with associates about dropping out of this system, even effectively into my remaining semester. Week 10 of Semester 1 was the one time I very critically thought-about it.
It was January 2022, proper on the heels of the COVID tech hiring increase. I’d tried my hand at frontend growth and had a reasonably good grasp of React. Not one of the introductory math programs I used to be taking made any sense. Loads of acquaintances and associates of associates had gotten soft tech jobs with out levels, so why couldn’t I? What use was figuring out learn how to show a operate was steady out in the true world?
On reflection, I understood that that was precisely what I used to be speculated to really feel. That was once I truly determined to pursue my diploma, not once I utilized a yr earlier. That feeling of impending doom was what lit a fireplace below me and drove me to check like a person possessed for the subsequent few months.
To at the present time, I’ve by no means been happier to get again a grade than once I opened the scan of my graded Calc 1 examination to see “61/100” staring me again within the face: a passing grade with a cool margin of two factors above failing. However all that mattered was that it was a passing grade, particularly when virtually half the scholars had failed the category, many for the second or third time.
By all accounts, my first semester of undergrad was tough. Sure, this was by design, and sure, I discovered lots from it, each by way of the fabric itself and (largely) about resilience and perseverance. However it took shifting to Germany and beginning my grasp’s for me to grasp how good I actually had it again then, at the very least in a single explicit regard.
The Human Coaching Knowledge Drawback
One of many greatest surprises to me at my new college was that previous exams are a lot much less of a factor right here. For all of the stress and nervousness I had throughout my bachelor’s, one factor I knew I might all the time rely on was the existence of plentiful and easily-accessible scans of previous exams and exam-relevant drawback units, particularly for introductory programs.
For Discrete Math, I solved all the handfuls of previous exams going again virtually a decade. I distinctly bear in mind warming up for Linear Algebra 1 with questions from the Nineteen Nineties. This was so ingrained within the tradition of my program that I fully took it with no consideration. The one purpose I managed to cross Calc 1 (by the pores and skin of my tooth) was as a result of I had spent hours on finish fixing a whole lot of questions from exams.
I used to be so accustomed to exams from previous years being available that skimming over them had turn out to be a part of my course of for vetting courses I used to be contemplating taking. This meant that my impolite awakening got here pretty early on in my first semester of grad college, whereas attempting to determine my schedule.
So surprising was the revelation that I can map my response to the 5 phases of grief. At first, I used to be in denial, completely satisfied that there should be some secret platform the place all of the previous exams have been hiding. Anger, bargaining, and melancholy quickly adopted. Acceptance didn’t actually, however I used to be keen to postpone my issues till finals got here nearer on the finish of the semester.
As my first two finals (on back-to-back days, no much less) approached in a rush, I discovered myself confronted with what I wish to name the Human Coaching Knowledge Drawback. Granted, the human mind and machines are (very!) considerably totally different. However I couldn’t assist however liken my scenario to that of a machine studying mannequin with inadequate coaching knowledge. I used to be fully stumped on learn how to bridge the hole between lecture notes and potential examination questions.
My undergrad expertise had granted me the perception of what human underfitting appears to be like like, each at coaching time (finding out) and take a look at time (on examination day). I vividly bear in mind a couple of class the place, for one purpose or one other, I most popular extra in-depth evaluation of lecture slides or notes to fixing follow issues.
This was an method I rapidly dropped throughout my freshman yr, and for good purpose: even in theory-heavy courses, it yielded disastrous outcomes. Figuring out the proofs for all 40 theorems the professor required was a lot much less assist in passing Linear Algebra 2 than practising making use of them to unravel issues would have been. That’s to not say an ample grasp of the speculation isn’t obligatory; it completely is. However with the ability to recite the lecture notes by coronary heart gained’t prevent in the event you can’t reply questions like those on the ultimate.
And so, armed with a whole lot of slides and a imprecise thought of the construction of every examination, I racked my mind for methods to keep away from the pitfall of getting into blind with none follow issues. Denial crept again in, and I desperately looked for previous exams I knew didn’t exist. Finally, I shifted my consideration from discovering the Holy Grail to turning my drawback into one an LLM may have the ability to resolve.
Artificial Coaching Knowledge for People
Researchers at IBM outline artificial knowledge as “info that’s been generated on a pc to reinforce or substitute actual knowledge to enhance AI fashions” [1]. It has many advantages, from mitigating privateness issues to slicing prices, resulting in its widespread adoption for makes use of as diversified as tooling for monetary establishments [1] and 3D content material technology [2].
In my case, the motivation was easy: the real-world (human) coaching knowledge I wanted to check simply wasn’t obtainable within the wild.
In fact, utilizing artificial knowledge solely is smart if that knowledge precisely imitates the information our educated mannequin will encounter in the true world. I knew I needed to be very intentional about how I generated the mock exams I wished to make use of. Simply telling Claude to put in writing a follow take a look at or two wouldn’t reduce it, even when I gave it all of the slides and materials I needed to work with. Solely when getting down to write an examination does one notice what number of choices there are to be made, effectively past what’s in and what’s out by way of the fabric.
Fortunately, I wasn’t flying fully blind on that entrance. For one class, I had details about the examination’s construction and the sorts of questions there have been on it from college students who had taken it the yr prior. For the opposite, the professor supplied a breakdown of the examination into sections and a small handful of open-ended evaluation questions.
Each courses had Q&A periods after their respective remaining lectures. I paid particular consideration to something that appeared like a touch as to what they may ask, which later proved to be very useful.
Straightforward Mode: Replicating a Template
The primary examination was simple since I had rather more to work with. It additionally had a repute for being comparatively formulaic. I gave Claude the instance questions and construction I had and requested it to stay to the identical type.
Most of the questions lent themselves properly to slight adjustments that made them novel sufficient to be value fixing for follow with out straying too removed from what was typical for the precise examination. Aside from a couple of LaTeX formatting hiccups, which have been pretty simply resolved, it was easy crusing.
To insure myself in opposition to any surprises, I additionally had it generate some trickier questions based mostly on the lecture slides and my notes from the Q&A session. Regardless that nothing sudden was requested ultimately, doing a little focused evaluation tailor-made to my very own private blind spots was an awesome confidence booster.
Though I undoubtedly would have been in a position to research for the primary examination with out the assistance of LLMs, I nonetheless felt like I gained lots by utilizing Claude. I might completely think about how useful it might have been for a few of the newer or extra superior programs I took in undergrad, the place there have been solely a small handful of previous exams obtainable.
Arduous Mode: Development from Scratch
The second examination was a a lot more durable nut to crack. To begin with, the breadth of the fabric was a lot wider. Secondly, the slides solely very loosely mirrored what was mentioned in school. Most significantly, there was far much less info obtainable on what the examination would appear like. What particulars there have been have been exhausting to seek out and imprecise.
The primary two issues have been at the very least partially mitigated by the truth that I made an effort to take complete notes all through the semester. As for hints on the construction and magnificence of the examination, I scoured each attainable platform and picked up something that appeared even remotely related. In that vein, the Q&A session ended up being a godsend. Transcribing the professor’s solutions and feedback left me with a significantly better (albeit nonetheless incomplete) thought of what to anticipate.
Admittedly, I used to be initially pessimistic concerning the prospect of Claude with the ability to generate mock exams of a lot worth. Although I had used it pretty extensively for guided materials evaluation, I had my doubts about how it might fare with the uncertainty at play. Nonetheless, I gave it every little thing I knew concerning the examination and hoped for the very best.
I used to be pleasantly stunned on the outcomes. Though the primary few makes an attempt produced exams that didn’t really feel fairly proper, the core did appear promising. They did seem to adequately cowl the fabric and to be difficult sufficient. After some forwards and backwards, Claude began producing exams that I might have been satisfied have been actual.
I solved the improved exams and requested Claude to appropriate my options. The very act of fixing follow exams made me really feel nice about my grasp of the fabric. Claude’s standard sycophancy was the cherry on prime. (It did level out errors, however was exceptionally comfortable on deducting factors and overly-excited about appropriate solutions.) Finally, nonetheless, I wouldn’t know the way effectively Claude had completed coaching me till take a look at time. With the fateful day quick approaching, I hoped for the very best.
Generalizing to Check Knowledge and Stopping Dataset Air pollution
When Artificial Knowledge Alone Doesn’t Reduce It
Whereas artificial knowledge definitely has its advantages, it has a important downside. What a mannequin learns based mostly on artificial knowledge will, at finest, mannequin the simulated world from which that knowledge is drawn. That simulated world might diverge from actuality in methods we’re fully unaware of till it’s too late [3].
As Dani Shanley places it in “Artificial knowledge, actual hurt,“
“… simply as generative AI fashions can produce believable (however false) textual content or photos, artificial knowledge mills could create datasets that seem statistically legitimate, whereas introducing refined, hard-to-catch distortions and synthetic patterns, or lacking essential real-world complexities.” [3]
Shanley additionally attracts consideration to the hidden and disproportionate affect of the people tasked with synthesizing knowledge on how fashions finally behave. Largely arbitrary choices on their half might have important, presumably dangerous, downstream results [3].
I noticed this affect in motion whereas finding out for my second examination. Slowly however absolutely, I had unintentionally skewed Claude’s outputs based mostly on my private interpretation of what the professor had mentioned. My intestine feeling on what the examination ought to appear like turned the arbiter of which questions have been related and which weren’t.
It additionally turned clearer as time went on that my coaching dataset was veering ever additional right into a biased tackle actuality. After the sixth mock examination, it was apparent that Claude had simply settled on a set set of a number of dozen questions.
Even when prompted to introduce extra selection, each output from there on out was just a few cobbling collectively of questions I had already seen. Granted, these did embody many key questions it was closely implied would seem on the precise examination.
On take a look at day, I used to be shocked at how a lot the examination resembled those I had solved for follow. The gimmes the professor had hinted at have been certainly there, however so have been a powerful variety of non-trivial questions I had solved whereas finding out. Roughly 60% of the questions have been similar or similar to ones I had practiced. Most of the relaxation have been on matters I had at the very least touched on.
Nonetheless, one a part of the examination ended up being a major blind spot. It was a bit on matters we had mentioned solely briefly originally of the semester. Whereas finding out, I used to be unreasonably assured in swiftly dismissing sure varieties of questions, be it as a result of they appeared uncharacteristic (e.g., too mathematical) or as a result of they have been about issues I had deemed too insignificant to incorporate within the notes I took in school.
Sadly, these turned out to the precise varieties of questions that have been requested in that part. Some have been about matters that solely appeared on a single slide all semester. Others have been deeply technical in a approach I simply didn’t anticipate. Although I did my finest to reply them, I hadn’t educated my psychological mannequin on knowledge that may allow it to generalize to those questions effectively sufficient.
The capsule was all of the extra bitter to swallow because the sorts of questions I struggled with have been ones Claude included in its first makes an attempt at mock exams. These have been exactly those I did away with early on based mostly on little greater than hunches.
On this case, the slip up was removed from catastrophic. For my part, it wasn’t even near undoing the advantages of finding out utilizing artificial mock exams. Even so, it serves as a cautionary story that hearkens again to Shanley’s warnings about how artificial knowledge can insidiously exacerbate mannequin subjectivity and bias [3].
Overcoming Overfitting: How you can Make the Better of Artificial Human Coaching Knowledge
For a lot of real-world purposes, an artificial dataset that yields a mannequin with solely 60% accuracy would most likely be thought-about subsequent to ineffective. With adequate real-world knowledge (i.e., precise previous exams), there isn’t any doubt in my thoughts that 90%+ accuracy can be achievable.
To be honest, although, the (human) mannequin into account has flaws that machines don’t and is, in some ways, a lot more durable to coach. I can say with confidence that that 60% would virtually definitely surpass the accuracy of another technique I might have tried.
I’ll completely stick with this technique for future exams, with three key takeaways I plan to implement:
- Separate chats are the way in which to go. The suggestions loop that led Claude to converge on particular questions undoubtedly had lots to do with me operating the complete cycle of producing exams and checking solutions in a single huge, lengthy context. This meant any new mock examination was immediately based mostly on the entire earlier ones. Past that, Claude tried to be useful by tailoring the inquiries to what it thought have been my weak spots, main it to turn out to be much more entrenched in what it thought must be requested. Common context rot(1) was additionally most likely an necessary issue.
- Maintain an open thoughts. As talked about above, the main blind spot I developed was largely the results of placing an excessive amount of inventory in my subjective evaluation of what materials would or ought to make the reduce. As an alternative of difficult my assumptions and devoting a while to masking minor matters that appeared like lengthy pictures, I leaned into my biases.
- Increase with real-world coaching knowledge! That is, in fact, simpler mentioned than completed. It considerably contradicts the very premise of this text. However what you are able to do as a scholar (or as an educator) is enrich the financial institution of identified questions for future college students. I managed to recollect a lot of the questions that have been on my second examination and doc them for future college students to make use of when finding out.
Afterword: My Ideas on LLMs as a Studying Support
The elephant within the room is that not one of the examination preparation workflow I described would have been even remotely possible once I began my bachelor’s in late 2021. Perhaps that is what made the method really feel virtually magical to me.
I bear in mind wishing I had a solution to robotically examine and proper my solutions on mock exams when finding out in my freshman yr.In case you would have instructed me again then that an AI software, not to mention a free one, would have the ability to do this (nonetheless imperfectly) in 2026, I’d have thought you have been loopy.
A lot has been written concerning the new issues LLMs have caused. Most of the factors which have been made are particularly related to college students. And certainly, I can’t argue that claims like “AI is making folks dumber” are fully unfounded. I’ve seen firsthand how these instruments let an individual outsource pondering and remove any mental discomfort. For an ever-growing vary of advanced duties, they signify the final word shortcut [4].
Concerningly, I imagine individuals who resist the temptation to take these shortcuts are more and more being penalized, at the very least within the quick run. A pal who was the one one to not vibe-code assignments in a sure class involves thoughts. Others cruised to good grades on their homework regardless of threats about how AI-generated submissions would supposedly be rejected. He put within the work and ended up being docked important factors for minor errors, with little in the way in which of constructive suggestions or recourse.
Nonetheless, in the long term, it’s a well-established incontrovertible fact that progress, in its myriad types, entails some sort of stress. A kind of types is studying, and the required stress comes within the type of lively engagement with the fabric. Few issues are extra rewarding in my view than the lightbulb second of lastly understanding a tough idea after combating it for hours or days. Experiencing such moments with Fourier sequence, reductions, metric areas, and plenty of different ideas was a significant a part of what led me to decide on to pursue a grasp’s diploma within the discipline.
LLMs undoubtedly allow would-be learners to deprive themselves of this stress and, in flip, of precise studying. Usually, although, I feel too little consideration is paid to the opposite facet of the coin: with the precise method, they’ll personalize and democratize studying like no invention because the web has.
Having skilled increased schooling each pre- and post-ChatGPT, I really feel enormously lucky to have instruments like Claude and Gemini at my fingertips. Their utility for examination preparation was simply the tip of the iceberg. It felt like my productiveness was boosted tenfold all through the semester. Issues clicked a lot quicker than they ever would have in any other case. LLMs have been a recreation changer for every little thing from technique (when and learn how to research what) to reviewing slides and notes to creating real curiosity and curiosity within the materials.
To summarize with a platitude: “With nice energy comes nice accountability.” LLMs are what you make of them. With the precise method, they’ll coach you to tackle the heavy lifting as a substitute of doing it for you.
In case you loved this text, please contemplate following me on LinkedIn to maintain up with future articles and initiatives.
Footnotes
(1) Engineering at Anthropic defines context rot as a phenomenon the place “because the variety of tokens within the context window will increase, the mannequin’s means to precisely recall info from that context decreases.” [5]
References
[1] Okay. Martineau and R. Feris, “What’s artificial knowledge?,” IBM Analysis Weblog, Feb. 7, 2023. https://analysis.ibm.com/weblog/what-is-synthetic-data.
[2] Y. Shi, P. Wang, J. Ye, M. Lengthy, Okay. Li, and X. Yang, “MVDream: Multi-view diffusion for 3D technology,” arXiv preprint arXiv:2308.16512, 2023. https://doi.org/10.48550/arXiv.2308.16512.
[3] D. Shanley, “Artificial knowledge, actual hurt,” Ada Lovelace Institute Weblog, Sep. 18, 2025. https://www.adalovelaceinstitute.org/weblog/synthetic-data-real-harm/.
[4] S. Bogdanov, “In the long term, LLMs make us dumber,” @desunit (Sergey Bogdanov), Aug. 12, 2025. https://desunit.com/weblog/in-the-long-run-llms-make-us-dumber/.
[5] P. Rajasekaran, E. Dixon, C. Ryan, and J. Hadfield, “Efficient context engineering for AI brokers,” Engineering at Anthropic, Sep. 29, 2025. https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents.
