Organizing Code, Experiments, and Analysis for Kaggle Competitions

and I neglect. Train me and I bear in mind. Contain me and I study.

holds true, and studying by doing is among the most instructive processes to amass a brand new talent. Within the area of information science and machine studying, taking part in competitions is among the simplest methods to achieve hands-on expertise and improve your abilities and talents.

Kaggle is the world’s largest knowledge science neighborhood, and its competitions are extremely revered within the business. Most of the world’s main ML conferences (e.g., NeurIPS), organizations (e.g., Google), and universities (e.g., Stanford) host competitions on Kaggle.

The featured Kaggle Competitions award medals to prime performers on the personal leaderboard. Just lately, I’ve participated in my very first medal-awarding Kaggle competitors, and I used to be lucky sufficient to earn a Silver Medal. This was the NeurIPS – Ariel Knowledge Problem 2025. I don’t intend to share my answer right here. Should you’re , you may take a look at my answer right here.

What I didn’t notice previous to participation is how a lot Kaggle exams apart from simply ML abilities.

Kaggle exams one’s coding and software program engineering abilities. It harassed one’s capacity to correctly set up their codebase in an effort to shortly iterate and check out new concepts. It additionally examined the power to trace experiments and ends in a transparent method.

Being a part of the NeurIPS 2025 Competitors Observe, a analysis convention, additionally examined the power to analysis and study a brand new area shortly and successfully.

All in all, this competitors humbled me quite a bit and taught me many classes apart from ML.

The aim of this text is to share a few of these non-ML classes with you. All of them revolve round one precept: group, group, group.

First, I’ll persuade you why clear code structuring and course of group isn’t time losing or good to have, however quite important for competing in Kaggle particularly and any profitable knowledge science undertaking generally. Then, I’ll share with you a few of the strategies I used and classes realized relating to code structuring and the experimentation course of.

I need to begin with a word of humility. Under no circumstances am I an knowledgeable on this area. I’m nonetheless within the outset of my journey. All I hope for is that some readers will discover a few of these classes useful and can study from my pitfalls. In case you have every other suggestions or solutions, I urge you to share them in order that all of us can study collectively.

1 Science Golden Tip: Manage

1.1 However Velocity is Essential!

1.2 Prices of Lack of Group

Drivendata (2022) mentions an excellent instance of the prices of an unorganized knowledge science undertaking. It mentions the story of a failed knowledge science undertaking that took months to finish and value thousands and thousands of {dollars}. The failure got here all the way down to an incorrect conclusion found early within the undertaking. A code bug within the knowledge cleansing polluted the information and led to improper insights. If the crew had higher tracked the information sources and transformations, they’d have caught the bug earlier, and cash would have been saved.

If there may be one lesson to remove from this part, it’s that group shouldn’t be a nice-to-have, however quite an important a part of any knowledge science undertaking. With out a clear code construction and course of group, we’re sure to make errors, waste time, and produce irreproducible work.

1.3 What to trace and set up?

2 The Codebase

2.1 Repo Construction

I wish to initialize my work with the broadly in style Cookiecutter Knowledge Science (ccds) template. While you initialize a undertaking with ccds, it creates a folder with the next construction. ¹

├── LICENSE            <- Open-source license if one is chosen
├── Makefile           <- Makefile with comfort instructions like `make knowledge` or `make prepare`
├── README.md          <- The highest-level README for builders utilizing this undertaking.
├── knowledge
│   ├── exterior       <- Knowledge from third social gathering sources.
│   ├── interim        <- Intermediate knowledge that has been remodeled.
│   ├── processed      <- The ultimate, canonical knowledge units for modeling.
│   └── uncooked            <- The unique, immutable knowledge dump.
│
├── docs               <- A default mkdocs undertaking; see www.mkdocs.org for particulars
│
├── fashions             <- Educated and serialized fashions, mannequin predictions, or mannequin summaries
│
├── notebooks          <- Jupyter notebooks. Naming conference is a quantity (for ordering),
│                         the creator's initials, and a brief `-` delimited description, e.g.
│                         `1.0-jqp-initial-data-exploration`.
│
├── pyproject.toml     <- Undertaking configuration file with package deal metadata for 
│                         {{ cookiecutter.module_name }} and configuration for instruments like black
│
├── references         <- Knowledge dictionaries, manuals, and all different explanatory supplies.
│
├── studies            <- Generated evaluation as HTML, PDF, LaTeX, and so on.
│   └── figures        <- Generated graphics and figures for use in reporting
│
├── necessities.txt   <- The necessities file for reproducing the evaluation atmosphere, e.g.
│                         generated with `pip freeze > necessities.txt`
│
├── setup.cfg          <- Configuration file for flake8
│
└── {{ cookiecutter.module_name }}   <- Supply code to be used on this undertaking.
    │
    ├── __init__.py             <- Makes {{ cookiecutter.module_name }} a Python module
    │
    ├── config.py               <- Retailer helpful variables and configuration
    │
    ├── dataset.py              <- Scripts to obtain or generate knowledge
    │
    ├── options.py             <- Code to create options for modeling
    │
    ├── modeling                
    │   ├── __init__.py 
    │   ├── predict.py          <- Code to run mannequin inference with educated fashions          
    │   └── prepare.py            <- Code to coach fashions
    │
    └── plots.py                <- Code to create visualizations 

2.1.1 Surroundings Administration

While you use ccds, you’re prompted to pick out an atmosphere supervisor. I personally want uv by Astral. It information all of the used packages within the pyproject.toml file and permits us to recreate the identical atmosphere by merely utilizing uv sync.

Underneath the hood, uv makes use of venv. I discover utilizing uv a lot easier than immediately managing digital environments as a result of managing and studying pyproject.toml is way easier than necessities.txt.

Furthermore, I discover uv a lot easier than conda. uv is constructed particularly for python whereas conda is way more generic.

2.1.2 The Generated Module

I discover the utilization of the package deal fairly useful, and in Part 2.3, I’ll talk about what to put right here and what to put in Jupyter Notebooks.

2.1.3 Staying Versatile

2.2 Model Management

2.3 The Three Code Varieties

2.3.1 The Module

2.3.2 Scripts

The utilization of those scripts helps make our outcomes reproducible. To breed an older outcome (prepare the identical mannequin, for instance), one solely has to clone the identical model of the repo and run the script used to make the previous outcomes 2.

For the reason that scripts are run from the CLI, utilizing a library to handle CLI arguments simplifies the code. I like utilizing typer for easy scripts that don’t have many config choices and utilizing hydra for advanced ones (I’ll talk about hydra in additional depth later).

2.3.3 Notebooks

3 Working the Codebase on Kaggle

3.1 Cloning The Repo

Almost certainly, you’ll be utilizing a non-public repo. The only option to clone a non-public repo on Kaggle is to make use of a private entry token (PAT). You possibly can create a PAT on GitHub by following this information. An awesome apply is to create a PAT particularly for Kaggle with the minimal required permissions.

Within the cloning pocket book, you should use the next code to clone your repo:

from kaggle_secrets import UserSecretsClient
user_secrets = UserSecretsClient()
github_token = user_secrets.get_secret("GITHUB_TOKEN")
person = "YOUR_GITHUB_USERNAME"
CLONE_URL = f"https://oauth2:{github_token}@github.com/{person}/YOUR_REPO_NAME.git"
get_ipython().system(f"git clone {CLONE_URL}")

This code downloads your repo into the working listing of the present pocket book. It assumes that you’ve saved your PAT in a Kaggle secret named GITHUB_TOKEN. Just remember to activate the key within the pocket book settings earlier than working it.

3.2 Putting in Required Packages

Within the cloning pocket book, it’s also possible to set up the required packages. If you’re utilizing uv, you may construct your customized module, set up it, and set up its dependencies by working the next instructions: 3.

cd ariel-2025 && uv construct

This creates a wheel file within the dist/ listing to your module. You possibly can then set up it and all its dependencies in a customized listing by working: ⁴.

pip set up /path/to/wheel/file --target /path/to/customized/dir

Be certain that to switch /path/to/wheel/file and /path/to/customized/dir with the precise paths. /path/to/wheel/file would be the path to the .whl file contained in the REPO_NAME/dist/ listing. The /path/to/customized/dir may be any listing you want. Bear in mind the customized listing path as a result of subsequent notebooks will depend on it to import your module and your undertaking dependencies.

I wish to each obtain the repo and set up the packages in a single pocket book. I identify this pocket book the identical identify because the repo to simplify importing it later.

3.3 Working One of many Scripts

The very first thing to do in any subsequent pocket book is to import the pocket book containing the cloned repo and put in packages. While you do that, Kaggle shops the contents of /kaggle/working/ from the imported pocket book right into a listing named /kaggle/enter/REPO_NAME/, the place REPO_NAME is the identify of the repo 5.

Many occasions, your scripts will create outputs (e.g., submission information) relative to their places. By default, your code will reside on /kaggle/enter/REPO_NAME/, which is read-only. Subsequently, you must copy the contents of the repo to /kaggle/working/, which is the present working listing and is read-write. Whereas this can be pointless, it’s a good apply that causes no hurt and prevents foolish points.

cp -r /kaggle/enter/REPO_NAME/REPO_NAME/ /kaggle/working/

Should you immediately run your scripts from /kaggle/working/scripts/, you’re going to get import errors as a result of Python can’t discover the put in packages and your module. This will simply be solved by updating the PYTHONPATH atmosphere variable. I exploit the next command to replace it after which run my scripts:

! export PYTHONPATH=/kaggle/enter/REPO_NAME/custom_dir:$PYTHONPATH && cd /kaggle/working/REPO_NAME/scripts && python your_script.py --arg1 val1 --arg2 val2

I normally identify any pocket book working a script with the script identify for simplicity. Furthermore, after I re-run the pocket book on Kaggle, I identify the model with the hash of the present Git commit to maintain observe of which model of the code was used to generate the outcomes. ⁶.

3.4 Gathering All the things Collectively

3.5 Is all this Effort Price it?

4 Recording Learnings and Analysis

4.1 Readings Monitoring

Rajpurkar (2023) suggests having a listing of all of the papers and articles you learn. This lets you shortly overview what you will have learn and refer again to it when wanted.

Professor Rajpurkar additionally suggests annotating every paper with one, two, or three stars. One-star papers are irrelevant papers, however you didn’t know that earlier than studying them. Two-star papers are related. Three-star papers are extremely related. This lets you shortly filter your readings in a while.

You also needs to take notes on every paper you learn. These notes ought to give attention to how the paper pertains to your undertaking. They need to be quick to be reviewed simply, however have sufficient particulars to know the principle concepts. Within the papers checklist, you must hyperlink studying notes to every paper for simple entry.

I additionally like conserving notes on the papers themselves, equivalent to highlights. Should you’re utilizing a PDF reader or an e-Ink system, you must retailer the annotated model of the paper for future reference and hyperlink it in your notes. Should you want studying on paper, you may scan the annotated model and retailer it digitally.

4.2 Instruments

Zotero is a good instrument for managing analysis papers. It’s nice at storing and organizing papers. You possibly can create a set for every undertaking and retailer all of the related papers there. Importing papers may be very simple utilizing the browser extension, and exporting citations in BibTeX format is easy.

5 Experiment Monitoring

5.1 Wandb

Weights and Biases (wandb), pronounced “w-and-b” (for weights and biases) or “wand-b” (for being magical like a wand) or “wan-db” (for being a database), is a good instrument for monitoring experiments. It permits us to run a number of experiments and save all their configurations and ends in a central place.

Wandb supplies us with a dashboard to match the outcomes of various experiments, the hyperparameters used, and the coaching curves. It additionally tracks system metrics equivalent to GPU and CPU utilization.

Wandb additionally integrates with Hugging Face libraries, making it simple to trace experiments when utilizing transformers.

When you begin utilizing a number of experiments, wandb turns into an indispensable instrument.

5.2 Hydra

Hydra is a instrument constructed by Meta that simplifies configuration administration. It permits you to outline all of your configuration in YAML information and simply override them from the CLI.

It’s a very versatile instrument and matches a number of use circumstances. This information discusses use Hydra for experiment configuration.

6 The Finish-to-Finish Course of

Determine 2 summarizes the method mentioned on this article. First, we analysis concepts and file our learnings. Then, we experiment with these concepts on our native machines in Jupyter Notebooks. As soon as we’ve got a working thought, we refactor the code into our module and create scripts to run the experiments. We run the brand new experiment(s) on Kaggle. Lastly, we observe the outcomes of the brand new experiments.

As a result of every thing is rigorously tracked, we’re in a position to predict our shortcomings and shortly head again to the analysis or improvement phases to repair them.

7 Conclusion

On this article, we mentioned a method to arrange our codebase, tricks to observe analysis and learnings, and instruments to trace experiments. Determine 2 summarizes the proposed approach.

I hope this text was useful to you. In case you have every other suggestions or solutions, please share them within the feedback part beneath.

Better of luck in your subsequent competitors!

7.1 References

Rajpurkar, P. (2023). Harvard CS197: AI Analysis Experiences. https://www.cs197.seas.harvard.edu