26 points | by psoulos 5 hours ago ago
3 comments
Wait, I don't get it. Is the model not open? If not, why are they comparing it to Open Weight models instead of the state of the art?
Because SOTA Closed Base Models are not available to compare to.
Very similar to Kimi K3 it seems, but fewer active params (80b vs 104b). I don't think they mentioned what form of attention they use
Wait, I don't get it. Is the model not open? If not, why are they comparing it to Open Weight models instead of the state of the art?
Because SOTA Closed Base Models are not available to compare to.
Very similar to Kimi K3 it seems, but fewer active params (80b vs 104b). I don't think they mentioned what form of attention they use