My field is primarily audio/vision ML so I don't have any particular domain expertise, but I don't get how you can lump all "time-series" together in one big bucket. My naive understanding is that the way to improve the accuracy is to gain as much joint information as possible. Like:
> Known-future covariates. Some drivers of sensor behaviour are known in advance, such as production schedules, planned shutdowns, weather forecasts. Timestral can condition on them to improve forecast quality, so a scheduled load increase is expected rather than flagged as a surprise.
To me this should be the entire basis of the system, not a condition to improve forecast quality? Like if you're modelling electricity consumption, if all you have is a window of N previous samples, you're going to get completely beaten by a comparatively simple system that knows the time of day, that there's holidays coming up, the weather forecast, etc.
I would also think that these time-series forecasting models would gain from a sophisticated understanding at least the labels of the covariates? A 5M model has no idea what a "bank holiday" is, it just sees an arbitrary class go from 0 to 1, and won't have any useful inductive biases as to what it means or how to link them to other pieces of information.
E.g., wouldn't you predict €/MWh of electricity far more accurately with access to price of natural gas (itself influenced by many geopolitical factors), breakdown in composition of production of each energy source, predictions about roll-out (requiring a sophisticated LLM capable of understanding news and their implications), etc. What's the use in trying to predict it from 2 variables and a narrow window looking back at just 8192 or 32768 samples?
Or perhaps I misunderstand the use-case and this is more for extremely short term movements?
For some kinds of forecasting it doesn't need to be exactly accurate - eg say you are forecasting traffic for how to scale a load balancer, it can be a bit off it will just cost you a little and you can adjust after - a lot of feedback systems dont need full accuracy. You wouldnt use this to forecast whether to buy shares. It basically replaces various traditional curve fitting tools. If you need a really good forecast yes you are going to build a domain model. At least this is my understanding...
> say you are forecasting traffic for how to scale a load balancer, it can be a bit off
Problem is you really don't even need ML for the curve fitting tools. Just ARIMA or Holt Winters is enough and likely faster. Heck, Kubernetes HPA doesn't even implement full PID, just P.
It seems like the main challenge is going to be finding domains where accuracy is important enough to spend inference token money on, but not so important that you'd hire a team to build a custom model. Or maybe that team borrows Timestral weights to fine tune to the domain, but they probably already have a different model customized to the problem?
so If I understand correctly classical ML would beat this generic approach, classical ML being feature engineering/selection and curve fitting?
My next question would be that wouldn't it be smarter to take an auto research agent or even just a classic grid search of sorts to find those features and best curve to fit your problem using backtesting to evaluate and iterate.
Agree on the share-buying point. I keep a side project that tracks about twenty published bitcoin price models side by side and grades each one on its own forecast record. Two lessons carried over to any time-series benchmark for me. First, freeze every forecast as it stood the day it was made; refitting on later data flatters everything. Second, put the naive baseline (tomorrow looks like today) in the same table. The best-known model in my set, stock-to-flow, sits more than 100% off over the past year, and it looked brilliant in backtests. A 0.03 gain means little to me until I see it next to persistence.
You're right but, if the time series looks like music then you can apply your music domain knowledge. You don't have to only apply knowledge that's general across every domain, part of the magic of large neural nets is that you can do recognition and prediction at the same time thereby using (statistically) things known from training about how to identify a sequence and then also how to continue it.
Sure adding outside info helps even more, but with enough data it should become unnecessary (or so goes the theory).
But why would you throw away that data at all? These are tiny models compared to domain-specific audio/music/image models, and any capacity spent on recognition is capacity that could have been used for prediction.
Is there a reason you don't throw in something like cross-attention on an embedding of the sequence description/data labels? I've trained numerous generative models for audio and images and it's always the case that the unconditional model performs worse, trains slower, etc. than the conditional one (with the best performance from a hybrid using classifier-free guidance or similar scheme).
Again I don't have experience with the domain of time-series forecasting directly, but I would think there are generalities between autoregressive generative models and time-series forecasting, given that both involve predicting the next sample(s) given N regularly sampled past inputs.
Do you actually find that for some use cases foundation models work better than AutoARIMA or etc? I do a lot of ecom projections and it’s never been the case for me, most of my cases are best with seasonal naive.
Would love to know actual time series foundation model use cases!
Exactly this. It's a zero-shot model that can understand what a time series is with very little a-priori information. I would assume they've trained across a wide diversity of real world signals.
I interviewed for a company which used time series modelling to predict engine failure on ships. Though they didn't call it that, they called it just "AI" and weren't familiar with the time series literature at all. (Neither was I, but I did at least know that the time series literature was a thing.)
They just used a bunch of sensors attached to the engine.
That seems like a scenario where there wouldn't be external sources of information capable of improving your predictions. It also seemed to me like a scenario where good old fashioned statistical methods would outperform LSTMs, which was what they used.
(I didn't get the job - I was offered it, but they rescinded their offer at the last moment, supposedly because their investors had insisted they should hire a web developer instead.)
I'm doing time series forecasting for a biological system. I haven't used these foundation models but my experience is that knowledge of how things are related is baked into what variables you put into the model (which can be more than raw data, you can make any derived product you think would be useful)
but it's up to the model to figure out how they are related (if I knew exactly how they were related I wouldn't be training a forecasting model).
I'm sure someone who actually knows this stuff has a better easy to view it but my impression was that these time series foundation models aren't trained for a specific dataset but more they are trained on a procedure for how to look at datasets and make predictions. It doesn't seem to lump things into a bucket much more than any other model type (which also are just consistent procedures to look at datasets). It's not like you have radically different model types across different fields.
Forecasting is just trying to identify recurring and long-time trends in data, and you're absolutely right that with a narrow observation window it is more difficult to do so, however if you have enough historical data (i.e. several years), these models are remarkably good at picking up on patterns leading up to e.g. bank holidays, the information is contained in the data.
That being said, more information is of course better, assuming it doesn't introduce bias, like focusing on specific events like holidays too much, when the information is already contained in the historical data, without having to make assumptions about what an upcoming holiday means.
The biggest challenge last time I worked with time series forecasting (which is ~5 years ago) was learning long-term shifts in the baseline of the data (e.g., the general use of electricity always goes up, a lot more solar panels and EVs influence the baseline, etc.).
I can only assume (hope) that there has been some substantial progress in that area though.
(Edited the formatting for a nicer reading experience)
100% agree. Time series forecasting is really bad and there are some theorems that say it is *impossible for large class of series to be forecasted by more than a small number of steps..turning into a straight-line, meanwhile if you have say a handful of important day indicators of the same signal that are proxies, you can recover pretty much exact curve. Better to spend compute in my opinion or variable learning approximate Xt -> yt mapping than the X_t-4..n -> X_t mapping...
I can see another use for time-series generative models; to create the right shape of noisy data.
To improve the robustness of machine learning models, artificial noise is often introduced into actual training data. Something like Timestral could simplify the process of generating it.
I appreciate this analysis, but I wonder, setting aside unpredictable current events for the moment, aren’t seasonal and scheduled events also covariates? eg forecasting baseball scores year round?
A lot of misconception around the utility of foundational time series models on HN.
Sure, you could train a XGboost model on a time series with holiday or event covariates, but what if you have 10k different series to forecast for?
You could train 10k different models, one for each series, but then they have very little history to learn from. Alternatively, you could train a single generic model to cover all 10k, but then you have to deal with scale mismatch between series in addition to the feature dev overhead between series. A TSFM saves a ton of work in modeling each of these series and could potentially generalize better to produce more accurate forecasts.
But why is the question, I run about 400k models every 30 minutes at work. I have tried all other approaches, but xgboost model per timeseries has never been beaten before. Even if I wasn't targetting accuracy, Xgboost loading a pretrained model and predicting with it and still faster.
Alternatively, this is the exact ideal application for hierarchical Bayesian modeling. It’s the middle ground between the traditional stratification/aggregation decision.
What's the input like? Does it incorporate language understanding, so that each time series has a natural language explanation of the series and all the data fields? If it takes in really descriptive text like that I can see the opportunity for generalization. If it's just a time series of non-descript numbers I see much much less opportunity for cross-series learning.
All the businesses with product catalogs for example, every product can have several time series associated with it.
If a business has 10k employees, you have, at minimum, 10k time series to perform employee analyst operations over. There are several hundred companies in the US with 10k+ employees.
There are many more than 10k postal zipcodes in the US to perform forecasts over.
The difference is the same, whether you have 10k series representing the same variables or 10k series representing different variables decomposes into the same forecasting problem that you can either over-generalize across all time-series, producing worse accuracy at the tails, or train a model for each individual forecast, producing worse accuracy in general due to limited data input.
What kind of infrastructure do you use for that scale? Are these computationally light ARIMA models?
We're running 150k models per day and while I'm sure I could scale this to running them every 30 minutes, it would mostly be me throwing money at the problem to parallelize the compute.
To meaningfully test a forecasting model, use a data-generating process with a high signal-to-noise ratio where a classical method (empirical conditional distributions, i.e., counting) can capture most of the signal. Then check whether the model captures that signal too. If you only test on high-noise processes, any model looks comparable to SOTA because there's little signal to differentiate them. Also testing on a range of signals with different signal-to-noise ratio would allow us to know how much of the signal the model captures in different scenarios.
Very nice. It’s mind-blowing to me that it used to take me months to build and deploy a forecasting model five years ago. Now in minutes I can use something like this, implemented with help from an LLM, and get apparently quite good results (and fine-tune on my data, it seems).
Speed looks nice too. I want an open weight model with this performance now.
Any 'vibe check' on these; I don't have benchmarks but I tried out Le Chonk for general coding work that they claim it works really well at and I was very disappointed, it seemed to make all kinds of obvious errors of the kind that I haven't seen from opus, sol, deepseek flash etc in over a year
I've been using Mistral Large 4 and some of my other friends have as well.
Something is going on with the performance (as in speed of the service, not quality). Some days it's quite fast, some days it's so slow I interrupt it because I thought it was hanging.
I think they're shifting hosting capacity around, or otherwise messing with the model. Note that it's still a "preview" version, meaning they're still training it further for the final release and they are likely still figuring out the optimal hosting configuration as well.
It wouldn't surprise me to see further changes in both quality of the model and speed before the final release.
I'm looking forward to the weights release as well, has anything about the architecture and possibly quantization resilience been shared publicly so far?
I updated my vibe config to point to Mistral 4 and it charges my subscription capacity, not my API allowance. So it is included in subscriptions, you just need to do a little bit of legwork.
Odd, I am in France too. Progressive rollout maybe? I find it weird to announce a model with fanfare but not make it immediately available to customers, let alone paid ones.
It was for me too, and it's only become slow very recently. I can't say with certainty, but I think it was around the time Mistral 4 launched that it seemed to slow down.
ChatGPT became massive because of it's generalisability without needing training a new model for every domain and for its zero-shot learning inference/learning capabilities. If this model can be scaled or easily fine-tuned, it can have huge implications on forecasting in domains where custom models are too hard or too expensive! I am somewhat sceptical and wonder if the big quant firms already have versions of this internally or if not, why no one has achieved these kinds of results until now.
You are not authorized to exercise any rights under this license if the global consolidated monthly revenue of your company (or that of your employer) exceeds $20 million (or its equivalent in another currency) for the preceding month.
What troubles me about time series foundation models is that it's impossible to get a quick answer of whether these models work out of sample for many domains which don't generate new data super quickly.
So, yeah, they gave an example of power price forecasting (my domain), so I would love to built a nice model with fancy covariates and all and backtest over the last couple of years -- but I have no way of knowing whether Mistral would be leaking data form its training into my backtest, no?
backtest on data collected after the day of release. Unless they have a time machine, or are themselves consuming the power you are measuring, it won't leak any data.
My question: What kind of dynamical system observability does Timestral provide? Is it a blackbox y = F(x, params) model for offline use? Can it provide the actual dynamical system equations? Or are you stuck forever querying the Timestral server (which is bad, think about if you're grid system operator doing day ahead, month ahead forecasts, and Timestral goes down)
Typically, in linear, non-linear, and NN models, we fit some noisy data to A) An underlying model with unknown parameters or B) A model free search which attempts to construct combination of "basis" functions or C) A purely black box model which we only care about Input/Output error.
A) is useful because often we know an underlying model we want to match, but don't know the parameter values. If we can determine the parameters, we get insights about the real system. E.g, how are these two variables coupled and how strongly -> We can make business decisions about what happens if we adjust some parameters.
So what exactly does this timeseries foundation model do and provide?
Maybe, the observability is that you have a forecast, and you adjust the covariates and see what happens. But one would want much more insights than a "executive summary" of the forecast.
The important parts for me as a signals processing R&D engineer.
> Classical models such as ARIMA and linear regression remain well-established tools for forecasting time series. These models are usually fitted to a single series and learn only from that series' history.
> Deep learning models such as PatchTST add the capacity to learn nonlinear patterns across many related series, but they are still typically trained from scratch on each dataset, so what they learn doesn't carry over to new domains.
> Time series foundation models bring the pre-training approach to time series. Trained on a large and diverse corpus, our model learns patterns that transfer across series, forecasting zero-shot on data it has never seen, even in domains where data is scarce.
Edit: Looks like it's a pure black box which you can run on premise.
> Output: Probabilistic forecasts, 21 quantiles
> Inputs: Multivariate targets, past-only and known-future covariates
While not fully tested in court, I am kf the opinion the mkdel wieghts are nkt copywritabke, thus I am lersonally at leave to ignore their stupid not open license and use the wieghts as I see fit. Anyone else doing this should be advised to consult with an attoruney and be prepared to defend themselves in court.
exactly, a $100B public company can have new business units that are low revenue but are treated differently than VC funded seed startups with the same revenue.
perhaps this is wrong, but to me at some abstract level LLMs feel like a more general class of time-series models. like a general purpose sequence model
Did some on our internal datasets. It did not beat our AutoARIMA baselines, they are pretty noisy series though. Custom ML models only do a little better.
You can't easily evaluate the success of any single forecast, but as soon as you have a collection of forecasts there are a lot of ways to evaluate which forecasters are more accurate and which are well calibrated in their predictions.
For example you can look at the calibration of the forecasts: Take all the outcomes where you estimated ~10% probability. How many of them actually happened? If you are an well-calibrated forecaster, the actual number should be fairly close to 10%.
This is the pseudo-intellectual version of your boomer relative ranting about the weatherman on TV, “how does this guy still have a job when he’s always wrong!”
Or the poll-bashing that happened after the 2016 election when an outcome with a 29% chance per one model was observed.
Any interesting forecasting problem is underconstrained. Monte Carlo methods are the classic way to mitigate that one-universe problem.
Nobody can predict the future with certainty. All models are wrong, but some are useful.
Monte Carlo methods are used to estimate the risk associated with a prediction (and presumably an action taken against that prediction). Does that answer your question? I can elaborate if you like.
It estimates risk based on your current understanding of the universe.
1. No guarantee that the universe tomorrow will behave as the distributions you used in your Monte Carlo today.
2. If the universe stats serving you 10 unlikely events in a row, is your model model wrong, or just 10 unlucky draws? If your model was correct and you start "correcting it" based on these 10 draws, you now made it wrong.
Like the other person said, all models are wrong but some are useful (a quote they did not make up btw). It sounds like you understand how MC works. Do you feel like you've answered your own question regarding how MC helps? I'm trying to be helpful since you asked above.
Meh....time series data is fundamentally different from textual data. I see why a time series "foundation model" seems appealing, especially to the marketing folks at Mistral. Real-world usability is very limited, which is probably the reason why the blog post features an imaginary customer rather than an actual reference.
Time based data is extremely important, and there are tons of techniques out there to train super fast super small models, but they are a pain in the ass to tune. I could see a foundation model that is kind of good at a lot of things, that could be fine tuned or used when you want something more straight forward to use to analyze a bunch of data.
Honestly this might be very good for feature selection to refine a much more specialized model.
I mean their latest model is gated behind API pricing that's more than enough to prevent any pupil from using it. I did not try ML4 (I'll wait till the end of the preview at least) so I can't tell anything about its performance outside of benchmarks, but on paper it's basically at the level of Muse Spark Open and I've not seen nearly as many people spitting on that model.
My field is primarily audio/vision ML so I don't have any particular domain expertise, but I don't get how you can lump all "time-series" together in one big bucket. My naive understanding is that the way to improve the accuracy is to gain as much joint information as possible. Like:
> Known-future covariates. Some drivers of sensor behaviour are known in advance, such as production schedules, planned shutdowns, weather forecasts. Timestral can condition on them to improve forecast quality, so a scheduled load increase is expected rather than flagged as a surprise.
To me this should be the entire basis of the system, not a condition to improve forecast quality? Like if you're modelling electricity consumption, if all you have is a window of N previous samples, you're going to get completely beaten by a comparatively simple system that knows the time of day, that there's holidays coming up, the weather forecast, etc.
I would also think that these time-series forecasting models would gain from a sophisticated understanding at least the labels of the covariates? A 5M model has no idea what a "bank holiday" is, it just sees an arbitrary class go from 0 to 1, and won't have any useful inductive biases as to what it means or how to link them to other pieces of information.
E.g., wouldn't you predict €/MWh of electricity far more accurately with access to price of natural gas (itself influenced by many geopolitical factors), breakdown in composition of production of each energy source, predictions about roll-out (requiring a sophisticated LLM capable of understanding news and their implications), etc. What's the use in trying to predict it from 2 variables and a narrow window looking back at just 8192 or 32768 samples?
Or perhaps I misunderstand the use-case and this is more for extremely short term movements?
For some kinds of forecasting it doesn't need to be exactly accurate - eg say you are forecasting traffic for how to scale a load balancer, it can be a bit off it will just cost you a little and you can adjust after - a lot of feedback systems dont need full accuracy. You wouldnt use this to forecast whether to buy shares. It basically replaces various traditional curve fitting tools. If you need a really good forecast yes you are going to build a domain model. At least this is my understanding...
> say you are forecasting traffic for how to scale a load balancer, it can be a bit off
Problem is you really don't even need ML for the curve fitting tools. Just ARIMA or Holt Winters is enough and likely faster. Heck, Kubernetes HPA doesn't even implement full PID, just P.
It seems like the main challenge is going to be finding domains where accuracy is important enough to spend inference token money on, but not so important that you'd hire a team to build a custom model. Or maybe that team borrows Timestral weights to fine tune to the domain, but they probably already have a different model customized to the problem?
so If I understand correctly classical ML would beat this generic approach, classical ML being feature engineering/selection and curve fitting? My next question would be that wouldn't it be smarter to take an auto research agent or even just a classic grid search of sorts to find those features and best curve to fit your problem using backtesting to evaluate and iterate.
Agree on the share-buying point. I keep a side project that tracks about twenty published bitcoin price models side by side and grades each one on its own forecast record. Two lessons carried over to any time-series benchmark for me. First, freeze every forecast as it stood the day it was made; refitting on later data flatters everything. Second, put the naive baseline (tomorrow looks like today) in the same table. The best-known model in my set, stock-to-flow, sits more than 100% off over the past year, and it looked brilliant in backtests. A 0.03 gain means little to me until I see it next to persistence.
Forecaster of the Confounding Company here :)
You're right but, if the time series looks like music then you can apply your music domain knowledge. You don't have to only apply knowledge that's general across every domain, part of the magic of large neural nets is that you can do recognition and prediction at the same time thereby using (statistically) things known from training about how to identify a sequence and then also how to continue it.
Sure adding outside info helps even more, but with enough data it should become unnecessary (or so goes the theory).
But why would you throw away that data at all? These are tiny models compared to domain-specific audio/music/image models, and any capacity spent on recognition is capacity that could have been used for prediction.
Is there a reason you don't throw in something like cross-attention on an embedding of the sequence description/data labels? I've trained numerous generative models for audio and images and it's always the case that the unconditional model performs worse, trains slower, etc. than the conditional one (with the best performance from a hybrid using classifier-free guidance or similar scheme).
Again I don't have experience with the domain of time-series forecasting directly, but I would think there are generalities between autoregressive generative models and time-series forecasting, given that both involve predicting the next sample(s) given N regularly sampled past inputs.
>"Is there a reason you don't throw in something like cross-attention on an embedding of the sequence description/data labels?"
Won't you need to attribute a weight to the data coming from the model and the weight for your data?
Do you actually find that for some use cases foundation models work better than AutoARIMA or etc? I do a lot of ecom projections and it’s never been the case for me, most of my cases are best with seasonal naive.
Would love to know actual time series foundation model use cases!
You’re not going to get venture capital funding by using auto.arima
Exactly this. It's a zero-shot model that can understand what a time series is with very little a-priori information. I would assume they've trained across a wide diversity of real world signals.
I interviewed for a company which used time series modelling to predict engine failure on ships. Though they didn't call it that, they called it just "AI" and weren't familiar with the time series literature at all. (Neither was I, but I did at least know that the time series literature was a thing.) They just used a bunch of sensors attached to the engine.
That seems like a scenario where there wouldn't be external sources of information capable of improving your predictions. It also seemed to me like a scenario where good old fashioned statistical methods would outperform LSTMs, which was what they used.
(I didn't get the job - I was offered it, but they rescinded their offer at the last moment, supposedly because their investors had insisted they should hire a web developer instead.)
I'm doing time series forecasting for a biological system. I haven't used these foundation models but my experience is that knowledge of how things are related is baked into what variables you put into the model (which can be more than raw data, you can make any derived product you think would be useful) but it's up to the model to figure out how they are related (if I knew exactly how they were related I wouldn't be training a forecasting model).
I'm sure someone who actually knows this stuff has a better easy to view it but my impression was that these time series foundation models aren't trained for a specific dataset but more they are trained on a procedure for how to look at datasets and make predictions. It doesn't seem to lump things into a bucket much more than any other model type (which also are just consistent procedures to look at datasets). It's not like you have radically different model types across different fields.
Forecasting is just trying to identify recurring and long-time trends in data, and you're absolutely right that with a narrow observation window it is more difficult to do so, however if you have enough historical data (i.e. several years), these models are remarkably good at picking up on patterns leading up to e.g. bank holidays, the information is contained in the data.
That being said, more information is of course better, assuming it doesn't introduce bias, like focusing on specific events like holidays too much, when the information is already contained in the historical data, without having to make assumptions about what an upcoming holiday means.
The biggest challenge last time I worked with time series forecasting (which is ~5 years ago) was learning long-term shifts in the baseline of the data (e.g., the general use of electricity always goes up, a lot more solar panels and EVs influence the baseline, etc.).
I can only assume (hope) that there has been some substantial progress in that area though.
(Edited the formatting for a nicer reading experience)
100% agree. Time series forecasting is really bad and there are some theorems that say it is *impossible for large class of series to be forecasted by more than a small number of steps..turning into a straight-line, meanwhile if you have say a handful of important day indicators of the same signal that are proxies, you can recover pretty much exact curve. Better to spend compute in my opinion or variable learning approximate Xt -> yt mapping than the X_t-4..n -> X_t mapping...
I can see another use for time-series generative models; to create the right shape of noisy data.
To improve the robustness of machine learning models, artificial noise is often introduced into actual training data. Something like Timestral could simplify the process of generating it.
I appreciate this analysis, but I wonder, setting aside unpredictable current events for the moment, aren’t seasonal and scheduled events also covariates? eg forecasting baseball scores year round?
A lot of misconception around the utility of foundational time series models on HN.
Sure, you could train a XGboost model on a time series with holiday or event covariates, but what if you have 10k different series to forecast for? You could train 10k different models, one for each series, but then they have very little history to learn from. Alternatively, you could train a single generic model to cover all 10k, but then you have to deal with scale mismatch between series in addition to the feature dev overhead between series. A TSFM saves a ton of work in modeling each of these series and could potentially generalize better to produce more accurate forecasts.
But why is the question, I run about 400k models every 30 minutes at work. I have tried all other approaches, but xgboost model per timeseries has never been beaten before. Even if I wasn't targetting accuracy, Xgboost loading a pretrained model and predicting with it and still faster.
Have you tried the latest TSFMs? Research (even in 2025) shows they outperform XGBoost models in certain scenarios. https://www.sciencedirect.com/science/article/pii/S266682702...
Are they very stationary time series with not very changing dynamics over time?
Alternatively, this is the exact ideal application for hierarchical Bayesian modeling. It’s the middle ground between the traditional stratification/aggregation decision.
This is a pretty approachable overview: https://casrai.org/guides/bayesian-hierarchical-models
Andrew Gelman has several books covering this topic in great depth as well.
What's the input like? Does it incorporate language understanding, so that each time series has a natural language explanation of the series and all the data fields? If it takes in really descriptive text like that I can see the opportunity for generalization. If it's just a time series of non-descript numbers I see much much less opportunity for cross-series learning.
> 10k different series
who has these though? i cannot imagine somoone making sense of 10k timeseries.
All the businesses with product catalogs for example, every product can have several time series associated with it. If a business has 10k employees, you have, at minimum, 10k time series to perform employee analyst operations over. There are several hundred companies in the US with 10k+ employees. There are many more than 10k postal zipcodes in the US to perform forecasts over.
sorry i meant 10k types of timeseries data not individual items. I assume thats what gp meant.
The difference is the same, whether you have 10k series representing the same variables or 10k series representing different variables decomposes into the same forecasting problem that you can either over-generalize across all time-series, producing worse accuracy at the tails, or train a model for each individual forecast, producing worse accuracy in general due to limited data input.
As an energy retailer, I have 10k+ timeseries.
Though you could perhaps classify them first and use much fewer models.
No reason to spend time classifying and worrying about cross-series validation when you use a TSFM.
I forecast 400k timeseries every 30minutes at work.
What kind of infrastructure do you use for that scale? Are these computationally light ARIMA models?
We're running 150k models per day and while I'm sure I could scale this to running them every 30 minutes, it would mostly be me throwing money at the problem to parallelize the compute.
To meaningfully test a forecasting model, use a data-generating process with a high signal-to-noise ratio where a classical method (empirical conditional distributions, i.e., counting) can capture most of the signal. Then check whether the model captures that signal too. If you only test on high-noise processes, any model looks comparable to SOTA because there's little signal to differentiate them. Also testing on a range of signals with different signal-to-noise ratio would allow us to know how much of the signal the model captures in different scenarios.
Very nice. It’s mind-blowing to me that it used to take me months to build and deploy a forecasting model five years ago. Now in minutes I can use something like this, implemented with help from an LLM, and get apparently quite good results (and fine-tune on my data, it seems).
Speed looks nice too. I want an open weight model with this performance now.
Perhaps I'm misunderstanding you, but you should be able to download the models at https://huggingface.co/collections/mistralai/timestral
Any 'vibe check' on these; I don't have benchmarks but I tried out Le Chonk for general coding work that they claim it works really well at and I was very disappointed, it seemed to make all kinds of obvious errors of the kind that I haven't seen from opus, sol, deepseek flash etc in over a year
Just tested Mistral 4 too in Vibe and it's slow. It works well, but is way too slow. Switched back to Mistral's GLM5.3, which is awesome.
Edit : working with GLM5.3 now, it's also slow. Maybe Mistral's under heavy load.
Disclaimer : Not Cofounder of The Forecasting Company here
I've been using Mistral Large 4 and some of my other friends have as well.
Something is going on with the performance (as in speed of the service, not quality). Some days it's quite fast, some days it's so slow I interrupt it because I thought it was hanging.
I think they're shifting hosting capacity around, or otherwise messing with the model. Note that it's still a "preview" version, meaning they're still training it further for the final release and they are likely still figuring out the optimal hosting configuration as well.
It wouldn't surprise me to see further changes in both quality of the model and speed before the final release.
We are improving all layers of the stack (model, inference, harness) so do expect improvements in quality in perf over the following weeks.
<mistral vibe pm here>
Cool!
I'm looking forward to the weights release as well, has anything about the architecture and possibly quantization resilience been shared publicly so far?
How are you accessing Mistral 4 from Vibe? The only ones available on my end are Medium 3.5 and GLM-5.3 despite having a Pro subscription.
We are rolling it out via A/B experiments and improving the experience as we roll-out. 100% of the users should see it in the following 7 days.
<mistral vibe pm here>
Nice, looking forward to it.
I updated my vibe config to point to Mistral 4 and it charges my subscription capacity, not my API allowance. So it is included in subscriptions, you just need to do a little bit of legwork.
Location based maybe ? I'm in France.
Odd, I am in France too. Progressive rollout maybe? I find it weird to announce a model with fanfare but not make it immediately available to customers, let alone paid ones.
Thanks for feedback. With future model releases we will aim to have it available to all paid users on day one.
Glad to hear :)
I've been using Mistral 4 as well, but from OMP with a Vibe subscription. Not location-based, I'm in the US.
I agree with you though. It's slow as hell, and so is their GLM offering.
The GLM offering in the last months was blazingly fast for me.
It was for me too, and it's only become slow very recently. I can't say with certainty, but I think it was around the time Mistral 4 launched that it seemed to slow down.
Maybe their Mistral 4 usage spiked with the news.
ChatGPT became massive because of it's generalisability without needing training a new model for every domain and for its zero-shot learning inference/learning capabilities. If this model can be scaled or easily fine-tuned, it can have huge implications on forecasting in domains where custom models are too hard or too expensive! I am somewhat sceptical and wonder if the big quant firms already have versions of this internally or if not, why no one has achieved these kinds of results until now.
It's quite impressive that they get performance so close to the leader t0-gamma but unfortunate about the restrictive license
huh indeed
To be fair, they let you build a $200M yearly revenue company on top of it for free, seems quite generous
I find rather disgusting and crude to jam this into the MIT license. And by the way, it's terrible; it says '$', but doesn't even say WHICH '$'.
They're French so they can't say "it was implied the US Dollar". This is so weird and broken I can't believe a legal team worked on this.
What troubles me about time series foundation models is that it's impossible to get a quick answer of whether these models work out of sample for many domains which don't generate new data super quickly. So, yeah, they gave an example of power price forecasting (my domain), so I would love to built a nice model with fancy covariates and all and backtest over the last couple of years -- but I have no way of knowing whether Mistral would be leaking data form its training into my backtest, no?
backtest on data collected after the day of release. Unless they have a time machine, or are themselves consuming the power you are measuring, it won't leak any data.
yes, that's why i mentioned *for many domains which don't generate new data super quickly* ...
crazy how they make an improvement of 0.03 look like a 5x difference
0.03 in what units?
My question: What kind of dynamical system observability does Timestral provide? Is it a blackbox y = F(x, params) model for offline use? Can it provide the actual dynamical system equations? Or are you stuck forever querying the Timestral server (which is bad, think about if you're grid system operator doing day ahead, month ahead forecasts, and Timestral goes down)
Typically, in linear, non-linear, and NN models, we fit some noisy data to A) An underlying model with unknown parameters or B) A model free search which attempts to construct combination of "basis" functions or C) A purely black box model which we only care about Input/Output error.
A) is useful because often we know an underlying model we want to match, but don't know the parameter values. If we can determine the parameters, we get insights about the real system. E.g, how are these two variables coupled and how strongly -> We can make business decisions about what happens if we adjust some parameters.
So what exactly does this timeseries foundation model do and provide?
Maybe, the observability is that you have a forecast, and you adjust the covariates and see what happens. But one would want much more insights than a "executive summary" of the forecast.
The important parts for me as a signals processing R&D engineer.
> Classical models such as ARIMA and linear regression remain well-established tools for forecasting time series. These models are usually fitted to a single series and learn only from that series' history.
> Deep learning models such as PatchTST add the capacity to learn nonlinear patterns across many related series, but they are still typically trained from scratch on each dataset, so what they learn doesn't carry over to new domains.
> Time series foundation models bring the pre-training approach to time series. Trained on a large and diverse corpus, our model learns patterns that transfer across series, forecasting zero-shot on data it has never seen, even in domains where data is scarce.
Edit: Looks like it's a pure black box which you can run on premise.
> Output: Probabilistic forecasts, 21 quantiles
> Inputs: Multivariate targets, past-only and known-future covariates
> Or are you stuck forever querying the Timestral server
"We have published a GitHub repository so you can run Timestral on your own data."
[1] https://github.com/mistralai/timestral
How does it compare to nixtla TimeGPT?
https://github.com/Nixtla/nixtla
Timestral is free and open weights.
No need to pay for proprietary TimeGPT.
Timestral is not free - it has a "Modified" MIT, which requires you to buy from Mistral if your global business makes more than $20m a month.
Calling it "Modified" MIT is just misleading.
While not fully tested in court, I am kf the opinion the mkdel wieghts are nkt copywritabke, thus I am lersonally at leave to ignore their stupid not open license and use the wieghts as I see fit. Anyone else doing this should be advised to consult with an attoruney and be prepared to defend themselves in court.
Gotta pay if i make more than 20 mil a month?
Sounds like a great problem to have.
It does make it difficult to deploy if you are part of an enterprise, even if your own use-case is tiny.
exactly, a $100B public company can have new business units that are low revenue but are treated differently than VC funded seed startups with the same revenue.
https://www.nixtla.io/docs/introduction/introduction I remember using that and was quite good so happy that there is more open competition now in the space
perhaps this is wrong, but to me at some abstract level LLMs feel like a more general class of time-series models. like a general purpose sequence model
I like that it’s something different but I’m struggling to think of a way to use this.
When you are behind in a field, make a new field and be the leader in it
Head of Macro here -- this is a great time series forecasting model, we are migrating to this in our pod!
What is macro
Macrodata Refinery is mysterious but important work
What IRL company is the closest to lumon right now ?
All of them? It’s basically a parody of knowledge work.
FWIW you have urls in your profile but nothing about whatever Macro is
> Head of Macro here
lol. so what?
Any benchmarks against AutoARIMA?
Did some on our internal datasets. It did not beat our AutoARIMA baselines, they are pretty noisy series though. Custom ML models only do a little better.
I dont know how one can objectively evaluate forecasting. We observe just one universe. We overfit our models to this one observation we have.
Which is why at the end of the day why nobody trusts or blames the forecasting models, specially in all or nothing types of decisions.
You can't easily evaluate the success of any single forecast, but as soon as you have a collection of forecasts there are a lot of ways to evaluate which forecasters are more accurate and which are well calibrated in their predictions.
For example you can look at the calibration of the forecasts: Take all the outcomes where you estimated ~10% probability. How many of them actually happened? If you are an well-calibrated forecaster, the actual number should be fairly close to 10%.
Brier Score:
This is the pseudo-intellectual version of your boomer relative ranting about the weatherman on TV, “how does this guy still have a job when he’s always wrong!”
Or the poll-bashing that happened after the 2016 election when an outcome with a 29% chance per one model was observed.
Any interesting forecasting problem is underconstrained. Monte Carlo methods are the classic way to mitigate that one-universe problem.
Nobody can predict the future with certainty. All models are wrong, but some are useful.
How does Monte Carlo help mitigation?
I fully agree that all models are wrong.
Their main usage is just replaying the past or encoding the modelers assumptions for the future.
But there is no way to know if a model is good or bad.
There would be if we had infinite universes.
Monte Carlo methods are used to estimate the risk associated with a prediction (and presumably an action taken against that prediction). Does that answer your question? I can elaborate if you like.
It estimates risk based on your current understanding of the universe.
1. No guarantee that the universe tomorrow will behave as the distributions you used in your Monte Carlo today.
2. If the universe stats serving you 10 unlikely events in a row, is your model model wrong, or just 10 unlucky draws? If your model was correct and you start "correcting it" based on these 10 draws, you now made it wrong.
Like the other person said, all models are wrong but some are useful (a quote they did not make up btw). It sounds like you understand how MC works. Do you feel like you've answered your own question regarding how MC helps? I'm trying to be helpful since you asked above.
Meh....time series data is fundamentally different from textual data. I see why a time series "foundation model" seems appealing, especially to the marketing folks at Mistral. Real-world usability is very limited, which is probably the reason why the blog post features an imaginary customer rather than an actual reference.
Time based data is extremely important, and there are tons of techniques out there to train super fast super small models, but they are a pain in the ass to tune. I could see a foundation model that is kind of good at a lot of things, that could be fine tuned or used when you want something more straight forward to use to analyze a bunch of data.
Honestly this might be very good for feature selection to refine a much more specialized model.
Agered. These are essentially numerology models.
I’ve been bullish on time series forecasting since 2006 and always thought transformers were perfectly designed to implement this
Spacetimeformer was imo a huge leap forward that got overshadowed by all the language model progress
Excited to see more effort put into time series forecasting
Focounder of the timecasting company here - this is some great european sovereignty right there!
Mistral is an embarrassment and clear proof that Europe needs to review its mechanisms for growing and retaining tech talent.
With such strong human capital, both in quantity and quality, Europe should in theory be competitive with US and China in AI.
Which US or Chinese time-series foundation models do you prefer over Mistral's?
Care to explain why Mistral is an embarrassment?
I was going to write a technical reply but my high schooler son just happened to summarize it in a great one liner:
"Mistral is so bad, it's the only AI my school doesn't bother to block"
I mean their latest model is gated behind API pricing that's more than enough to prevent any pupil from using it. I did not try ML4 (I'll wait till the end of the preview at least) so I can't tell anything about its performance outside of benchmarks, but on paper it's basically at the level of Muse Spark Open and I've not seen nearly as many people spitting on that model.
Still not an explanation.