TIME-MOE: Billion-Scale Time Sequence Basis Mannequin with Combination-of-Consultants | by Nikos Kafritsas | Oct, 2024

And open-source as nicely!

The Combination-of-Consultants (MOE) structure has surged in reputation with the rise of enormous language fashions (LLMs).

As time-series fashions undertake cutting-edge strategies, Combination-of-Consultants has naturally discovered its place within the time-series basis area.

This text discusses Time-MOE, a time-series basis mannequin that makes use of MOE to enhance forecasting accuracy whereas lowering computational prices. Key contributions embody:

1. Time-300B Dataset: The biggest open time-series dataset, with 300 billion time factors throughout 9 domains, and a scalable data-cleaning pipeline.
2. Scaling Legal guidelines for Time Sequence: Insights into how scaling legal guidelines have an effect on massive time-series fashions.
3. Time-MOE structure: A household of open-source time-series fashions leveraging MOE to reinforce efficiency.

Let’s get began

✅ Discover the hands-on challenge for Time-MOE within the AI Initiatives folder, together with different cool initiatives!

Time-MOE is a 2.4B parameter open-source time-series basis mannequin utilizing Combination-of-Consultants (MOE) for zero-shot forecasting