, My Day Seemed Utterly Completely different
Imagine it or not, two years in the past, I used to be nonetheless writing and debugging code every day. Line by line. Virtually slamming my head in opposition to the desk after 2 hours of debugging to no avail. I do know, it sounds sort of loopy, proper?
A standard day for me:
- Writing (and debugging) each SQL question and Python script from scratch
- Constructing slide decks bullet by bullet
- Writing documentation no one would learn till one thing broke (and even then, they hardly would)
I wrote about my day within the life as a knowledge scientist again in 2024. And nearly none of that’s what my precise Tuesday appears to be like like now.
I gained’t faux that’s purely a great factor. Some days it feels much less like my job bought simpler and extra prefer it quietly was a unique job. One I needed to be taught on the fly, whereas I used to be nonetheless doing the outdated one.
(And sure, I do know there are many knowledge scientists nonetheless doing lots of the issues I did two years in the past, however that is my expertise in addition to the expertise of many knowledge scientists I do know these days).
Immediate Engineering Is a Main A part of the Job
Sure, we had ChatGPT in 2024. We had immediate engineering. However I used to be not doing it as a result of ChatGPT usually pissed off me. It was extra work to elucidate the context behind what I used to be doing earlier than feeding ChatGPT my code, and even then it will nonetheless not appear to have the ability to discover the bug.
Developments in AI, particularly these with Claude, have modified lots of that perspective. Issues like tasks and abilities have made it a lot simpler to debate your venture with an AI that already is aware of the context and historical past behind it.
So a significant chunk of my day now goes into writing and refining prompts. Early on, my prompts have been lazy. One thing like:
Summarize the forecast accuracy for this mannequin.
Which will get you a obscure paragraph that usually doesn’t comprise the insights you really need. Now I write prompts nearer to:
Summarize this mannequin’s forecast accuracy over the past 14 days. Report the precise MAPE and RMSE for every day, flag any day the place MAPE exceeded 5%, and state whether or not the pattern is bettering or degrading week-over-week. Don’t spherical error metrics, report them to 2 decimal locations.
The distinction in output high quality is gigantic, and truthfully, that’s now a talent I’ve to actively strengthen.
A number of issues that at the moment are a part of my common workflow:
- Double checking LLM mannequin outputs
- Testing immediate variants in opposition to the identical job and evaluating outputs aspect by aspect
- Writing constraints immediately into the immediate (models, decimal precision, what not to guess at) as an alternative of correcting the output after the actual fact
Discovering Value-Efficient LLM Options (& Reducing Token Utilization)
LLMs are costly. Rather more so than XGBoost fashions. Which means that much more consideration wants to enter using LLMs to investigate massive datasets.
The identical knowledge science rules nonetheless apply, although:
- When an easier heuristic or mannequin can carry out the duty, all the time go together with that first.
- At all times clear your knowledge earlier than feeding it right into a mannequin. Rubbish in=rubbish out
- Carry out characteristic choice and choose solely significant options earlier than coaching an ML mannequin so that you don’t shove lots of of random options and trigger overfitting or an excessive amount of noise.
These pillars map over very effectively to LLMs. Not each job wants the most important, most costly mannequin accessible. Classifying a assist ticket or extracting a date from a doc doesn’t want the identical horsepower as summarizing a 40-page contract. Routing the straightforward stuff to a smaller, cheaper mannequin and reserving the costly one for duties that want it was an actual value lever.
Listed here are some examples of how I work on limiting prices:
- Knowledge cleansing to chop down enter sizes (for instance, eradicating hyperlinks, pictures, and different characters not related to the mannequin from an electronic mail chain)
- Caching repeated calls as an alternative of re-running the identical immediate in opposition to the identical enter
- Utilizing conventional ML when applicable as an alternative of an LLM for every part
- Monitoring token spend per job
- Researching finest practices for decreased token utilization
Stakeholder Communication and Displays
Right here’s the place lots of saved time goes: conferences, slides, and translating what a mannequin did into one thing a non-technical stakeholder can act on.
I used to spend hours constructing a deck from scratch. Now I can generate a tough draft of a stakeholder-ready dashboard or slide define in minutes, which sounds prefer it ought to liberate my afternoon. In apply, it simply means I spend that freed-up time in additional conferences, strolling folks by means of what the mannequin discovered and why it issues, as a result of the turnaround is quick sufficient that stakeholders count on check-ins extra usually.
The precise talent that issues right here hasn’t modified: taking one thing technically true and making it one thing a product supervisor or govt can decide from. AI can draft the slide. It may possibly’t resolve what the level of the slide is (that’s nonetheless me.)
Conclusion
Even with all of this, most of my job continues to be the identical beneath. I nonetheless have conferences and must collaborate with my group members. I nonetheless must resolve what’s value modeling within the first place. I nonetheless must catch when an AI-generated abstract confidently states one thing that isn’t true. I nonetheless must know the area effectively sufficient to note when a quantity appears to be like barely fallacious as an alternative of clearly fallacious. And I nonetheless use conventional ML when crucial.
If something, that judgment issues extra now, not much less as a result of it’s the one a part of the day that by no means bought automated.
My day in 2026 isn’t shorter than it was in 2024. It’s simply formed in a different way. Much less time doing the mechanical components, extra time on the components that require me to suppose deeper about enterprise issues.
Thanks for studying
- I constructed a 30 day social media content material calendar generator utilizing AI: Get it right here.
- Join with me on LinkedIn
- Purchase me a espresso to assist my work!
