At this stage to me Go is the new C compiler. Quite honestly I spent 10 years or more writing Go code. At times 18 hours a day. And now with AI, I get models to write my Go code. I STILL want it to be Go code. Is that strange? No because I understand the performance and safety of Go. But I rarely write it directly anymore. Probably not at all in 6-9 months. Yet I still occasionally look at the code to see what the output looks like.
So yea hopefully a nice new addition but only my prompts now matter. So need to make sure they pickup on the new API.
Yes. It's that thing — the fact that a language helps you read what a single function does really easily, but in so doing makes the entire codebase quite dense, is not optimal for us, but is good for them.
And the things that we do to manage this, that are symbolic or based on visceral real-world analogy, are good for us and bad for them.
Go is kind of in the middle, in the way that many vaguely Wirth-like and CSP-like languages possibly are, I guess.
If I want simplicity and maintainability, I’d pick Kotlin over Go. If I want speed, I’m picking Rust over Go. Why would I ever pick Go? It’s the best at nothing.
As I do more agentic coding, I am starting to feel strongly that I need to set aside a project -- and time -- for strictly hand-coding, using AI (if at all) only to explain things, never to write any code.
First, I like programming and don't want to stop doing it just because a machine can do it too. Second, I'm pretty sure atrophied coding skills will make me worse at directing the machines, thus worse at what seems to be "the job" these days.
>First, I like programming and don't want to stop doing it just because a machine can do it too. Second, I'm pretty sure atrophied coding skills will make me worse at directing the machines, thus worse at what seems to be "the job" these days.
In theory wouldn’t atrophied coding skills not matter if models are sufficiently good at coding? I mean isn’t whats left just design and architecture and testing?
Sure, if LLMs are producing shit code you need to have an understanding of what it should do instead. But I think that day is already gone.
Its hard to think of a skill that you’d need to build with an LLM that requires writing meaningful amounts of code by oneself. From my end all the complexity is business logic and system design.
> In theory wouldn’t atrophied coding skills not matter if models are sufficiently good at coding? I mean isn’t whats left just design and architecture and testing?
No, because good understanding of design comes from programming experience. Atrophied skills mean you lose touch with the actual nuts and bolts of what you are building.
This is also the worst kind of lock-in you can imagine, by outsourcing all of your understanding of a codebase and your skills you are now completely at the mercy of LLM providers. Even if you switch to open weight models you're at the mercy of graphics cards producers and scalpers.
> Sure, if LLMs are producing shit code you need to have an understanding of what it should do instead. But I think that day is already gone.
Maybe your standards are too low. By my standard, they still produce shit code and require lots of iterations to get reasonable designs (I often end up just rewriting the bits that are obviously wrong rather than trying to figure out how to "prompt engineer" the right output).
They do often produce passable code but my experience is the longer you spend reading the code the more things you find wrong with it, almost like an uncanny valley.
At least that's my experience with Opus and Fable. Maybe it's too old-fashioned at this point to complain about code quality, but I find it very strange how many people have started proclaiming that their code quality is good.
>No, because good understanding of design comes from programming experience.
“Programming” is a bit loaded. Do you mean thinking through and typing everything out yourself?
What design skills specifically can you only develop from continually programming in this way? I fully grant that you need to know how to code yourself to build things with models but what specific ability to use models atrophies alongside your no AI coding abilities?
Personally I’ve found the opposite to be true. As writing out code myself has shrank to nearly 0, I now have a much shorter feedback loop on design. You get a lot more repetition and feedback when that’s basically all you are doing.
Read my comment again. And the one I replied to. He claimed that his ability to code through LLMs has decreased as his coding ability atrophied. That is what im questioning. Im not questioning coding for the love of the game.
I haven't actually tried it yet, but I think it'd be good to write a dsl for whatever you're doing. Then have ai make an interpreter to run it. Sort of as a way to stay... interested? Engaged?
I still prefer Go over the alternatives because I'm reviewing the code carefully and trying to make it all look as good as if I'd written it very carefully by hand.
The few times I've needed a language without a GC and used Rust and C as target languages, I've reviewed the code less and enjoyed it less.
People are sleeping on Go as a LLM language of choice. Everyone trying to rewrite in rust which is fine and rust is fine. However its compile times are several orders of magnitude greater than Go. You can have LLMs write and compile 20 apps in the time one rust app compiles. I've got a Go personal project that would take hours for rust to compile, I can't imagine making the same velocity in Rust. Though I do see the workplace appeal of bringing back the xkcd "compiling" meme.
However it kind of makes me feel like my professional life was wasted and I don't really know what kind of future I have. One of Go's main points was you can pick it up in a weekend. What career moat do people like us have?
That is fighting the last war on the early days of 5 GLs, and taken care by agentic runtimes.
This isn't futurology, besides a few projects that already came up on Phoronix and around here, with Assembly generation, there are enough SIGPLAN papers on ML driven compilers already.
Some languages have a culture of doing things correctly and conforming strictly to the spec. These languages have indeed sorted it out years ago. Haskell for example. Any language that doesn’t have a special number type for JSON is a half-assed implementation, because 1e1000 is a perfectly valid JSON number that is beyond the typical IEEE754 double precision floating point numbers; does your language handle that correctly? Haskell does. This is but a small example of a strong culture bias towards correctness.
The spec (RFC 8259) leaves significant leeway to implementations:
A JSON parser MAY accept non-JSON forms or extensions.
An implementation may set limits on the size of texts that it accepts. An implementation may set limits on the maximum depth of nesting. An implementation may set limits on the range and precision of numbers. An implementation may set limits on the length and character contents of strings.
Add to that things like duplicate keys being allowed by the spec. There’s a lot of implementation choices to decide on.
And then things can get really complex with JSON Schema.
I generally like `encoding/json/v2` and I'm glad they did it.
I wish they'd left `MarshalIndent` in there though. The equivalent code is super complicated and non-obvious, so it's a pain to migrate, and it's a pain when you're banging out some quick code and want to dump some readable JSON. I suppose these days the LLMs can write the complicated version for us, of course.
I'm working on better SIMD support in Dart and one of the motivations for that work is specializing parser specifications based on schema specifications of that language.
Or more concretely, if we had a parser generator + json parser specification that can generate efficient simdjson style parsers, then we should be able to parse json restricted to a subset of json based on a schema at least as efficiently as simdjson can parse general purpose json.
That way we could have the best of both worlds, a (potentially lazy) statically type-safe model and simdjson performance!
I still think that this is definitely doable, it's just that a lot of infrastructure needs to be built for it.
I remember reading that Meta is doing something similar with binary formats. I find it weird that there's nothing like that for json yet.
I was working on metaparsing earlier this year (specifically over grpc because of its better support for types and better serialization format).
I think most people who use JSON don't want strong typing or to use schemas. If you do, you just use a different serialization format: JSON is designed to be a generic, human-readable way to basically represent any kind of data structure (since it deserializes to JS objects). So everything about JSON is working against you if you do try to enforce type/schemas on it.
A grpc-like protocol with more prescriptive type registries/discovery (over the network) and better interop with http (like grpc-gateway) would be the best approach IMO. Definitely not JSON schema
> I think most people who use JSON don't want strong typing or to use schemas.
Not necessarily. It's biggest advantage is probably that it has very good interoperability. Basically every language has a readily accessible library for json, and even if you don't, it isn't that hard to write a parser and/or generator for it. And being human readable isn't something to sneeze at either.
I used the jsontext package at my company when we needed to make a parser for challenging clients (specific errors for each kind of mistake) while still keeping performance, and it works like a charm.
There is no introspection and speed is superb. I would definitely recommend jsontext as a parser for anything non-straightforward.
I'm just getting the neuron-band back together where Golang is concerned. I have done some in the past for little system tasks, but I am switching away from interpreted languages entirely for web stuff, and initially to Go, as I switch away from hand-coding to at least LLM-supported coding.
So I am a bit out of date on the pain points (I never really felt pain points when I last used it, because the jobs I was doing had very small, quite Go-friendly shapes).
The fact that this took 6 years to land stood out to me.
I'm not entirely surprised, and I'm glad the Go team is carefully considering their interfaces before launching. But 6 years seems like a lot of work to land a JSON API.
I think in the case of JSON, it was “good enough”. People who found this to be a problem would use one of the many 3rd party options.
I kind of like that the standard library doesn’t move fast. It makes it easier to fit the ecosystem into my head (unlike, for example, the JavaScript ecosystem where I find code doesn’t survive 2 years before failing to compile).
But I do agree that slow and steady produces its own problems. Your example of generics is a good one. But think the example of Go modules is another, perhaps even better, example of how sometimes the Go team really should have accelerated their efforts on.
lol, I don't mean to be contrary, but I looked into dependency management (partly because of the modules implementation, partly because of early teething issues, and partly out of pure curiousity.
Dependency management is a freaking nightmare, nobody has managed to create something that is bulletproof, not cargo, npm, apt, yum, nix, homebrew, nothing.
The key issue is that you're managing 3rd party software that may or may not adhere to the prescribed lifecycle/versioning
The Linux variants do it by each having dedicated teams watching upstream sources, taking what they believe to be whatever they feel is right, building for their nominated target(s), then testing, before deploying (on top of that, all of these steps vary in importance and rigor depending on numerous variables)
For programming libraries (which is what depmod, goose, vgo, and modules are managing) it's just as bad, if not harder, because their very nature is to provide a base upon which other developers can turn into whatever their imagination.
Add to that, modules was (initially) written by people who were dealing with a slightly different ecosystem; Google rather famously has a monorepo, making their lives a little easier when it comes to managing dependencies.
So, with those things in mind, dependency management is always going to be a hard problem that is opinionated (eg. I have a firm opinion that there should be an easier way to "pin" a dependency, other than vendoring - note that this isn't an invitation to every junior Go developer to come and claim go.sum does that. It doesn't. go.sum is a log of what dependencies were used for the last build by whomever wrote to it)
For like 99% of use cases encoding/json was fine. Of course improvements are welcomed. But personally I've lead development of the backend of part one of the biggest sites of its kind in the world, and we just used encoding/json and net/http. Most of our use cases are just building APIs, client facing ones, internal ones, trusted third-parties. These systems have pretty high throughput, more than most people will ever work on.
Within our company I saw people in other teams writing their own optimised JSON libraries for whatever hellish reason. I always reiterated the point that as soon as that app made a single IO request all those nanosecond savings are invisible. Yes, it affects your total CPU time for processing a request but compute was never ever a limitation for us and it's likely not a limitation for most people. And if the trade off is your app needs scale slightly larger versus writing custom JSON code or pulling in a third party dependency, then I'm accepting the larger scaling.
The biggest benefit of encoding/json/v2 isn't the speed, it's the improved API.
> generics (something that I still haven't used beyond the tutorials) took an age as well
Only about a year from when the Haskell guys lent their expertise in designing it and it landing in a release. That's actually fairly quick given that it was a pretty significant change under the hood and constrained by only two releases annually.
The Go team was always clear that they didn't have the necessary capabilities internally, despite a lot of failed efforts trying anyway, and were relying on the open source community to lend a hand. Unfortunately, open source doesn't mean open willingness to contribute. It turns out that for all the experts on HN, posting "go doesn't have generics lol" was preferable to helping out.
I think it'll bite some people, but probably is the better default for JSON specifically. Most of the nil-maps-become-null thing is just an extension of Go defaulting to nil for empty arrays and maps, which is usually not what you expect when you're writing a JSON schema like
type ListUsersResponse struct {
Users []User `json:"users"`
}
in Go. It's pretty common have code like
userRows := db.Query(...)
var users []api.User
for _, user := range userRows {
users = append(users, api.User{...})
}
where you don't _really_ intend for the default nil-ness to leak out. (Apologies for any broken syntax, I'm realizing in real time that it's been a while since I've written Go)
In Go, it’s a question of who owns the allocation. If you return a pointer type allocated within the function, the answer is “no one,” so the pointer escapes and has to be managed by the garbage collector.
If the allocation is owned by the calling function which uses it internally and then doesn’t return it, it can be a stack variable. Or the same allocation can be reused many times in a loop. Or an array of allocations can be made all at once with foreknowledge of the total number.
`out` params in C# are params that must be set within the function. They are way to implement multiple return values and tell the caller "this is written to, not read from."
But you might be conflating that with in/out for Contra/Co-variance when specifying a generic.
The parameters named "out" here are not "output parameters" in the way you seem to be thinking. They are "input parameters" named "out", because they are the buffer or pipe you want the data written to.
the marshal api has been very annoying. why does marshal return an error? the only reason the marshal can return an error is that the input type is not marshalable. it's certainly a code bug. to me it's like int+str returning an error instead of compiler error
> why does marshal return an error? the only reason the marshal can return an error is that the input type is not marshalable
Not necessarily. Among other things, MarshalJSON methods called during marshaling can return an error for whatever reason the developer sees fit - one possibility that comes to mind is that part of the object tree being marshalled is lazily loaded, and failed to do so.
And, as others have mentioned, there are types like "any" whose marshalability can't be determined at compile time. Those types aren't necessarily at the top level, either; they could appear anywhere in the object tree.
This isn’t a type system problem. It’s a data origin problem. Ie it could be any random JSON blob a user entered. The compiler doesn’t know this ahead of time.
If you wanted type safety with JSON in Go then don’t use the marshal interface. Period.
For unmarshalling, but the original complaint was about marshalling. And for that the type system could help if the marshal API only allowed you to marshal objects of types that supported marshalling.
And even for unmarshalling you could get a compile time error if you try to unmarshall to a type that doesn't support it.
I’d already addressed that when I said the marshal API is an interface that’s meant to be reusable across generalise use cases. it can be used for a variety of data formats and not just JSON.
Go has this concept baked into the language. For example read and write interfaces so you can write data to an endpoint without having to worry about whether it’s a file system object, TCP/UDP socket, gzip achieve, and a plethora of other options too.
It’s a nice convenience feature and makes up considerably for the restrictions that the Go type system places.
And if you need higher performance code, or compile time safety, or have any other constraints that limit your ability to use these generalised interfaces, then you can use different APIs from the Go standard library to achieve the same thing.
You don’t need to call json.Marshal() if you don’t want to. It’s an option, but it’s just one API into the JSON toolbox.
First of all, the json2 Marshall API is json specific.
Secondly, other languages like rust, haskell, scala, etc. Have json APIs that are typesafe and give you compile time errors if you try to use a type that doesn't support (de) serialization . In the case of rust at least, the API is also general and can be used for other data types as well.
Even in go, you could have some more type safety if Marshall took an interface (such as MarshallerTo) rather than `any`. Although then you have the problem of how you implement that interface. So I guess you probably want some kind of "derive" functionality, or macros that can approximate it for a convenient and type-safe API.
P.S. gophers like to talk about how explicit being better than implicit, but using reflection for marshalling is very implicit.
i meant i hoped it's panic as opposed to return an error. if there was an error, how am i supposed to handle it? can i tell the user hey sorry my bad it's likely a code error? crash is better than possible eating the error siliently
It is trivial to ignore an error in Go if you know your use case cannot result in one. But at the end of the day, json.Marshal takes an `any` as a parameter, so it must return an error.
At this stage to me Go is the new C compiler. Quite honestly I spent 10 years or more writing Go code. At times 18 hours a day. And now with AI, I get models to write my Go code. I STILL want it to be Go code. Is that strange? No because I understand the performance and safety of Go. But I rarely write it directly anymore. Probably not at all in 6-9 months. Yet I still occasionally look at the code to see what the output looks like.
So yea hopefully a nice new addition but only my prompts now matter. So need to make sure they pickup on the new API.
This is mostly why I have come back to Go.
I am using local LLMs and Go has a bunch of strengths where smaller contexts are concerned; less complicated flow of control being one of them.
not a lot of overloading complexity, very verbose with a small total language structure too helps.
Yes. It's that thing — the fact that a language helps you read what a single function does really easily, but in so doing makes the entire codebase quite dense, is not optimal for us, but is good for them.
And the things that we do to manage this, that are symbolic or based on visceral real-world analogy, are good for us and bad for them.
Go is kind of in the middle, in the way that many vaguely Wirth-like and CSP-like languages possibly are, I guess.
Truly the argument can be made either way.
If you code in Go, the code is more high level, prettier, simpler.
If you code in Rust, the code is more robust, efficient, faster.
Both are important for LLM, why can't we have a high level, simple language that is also robust, efficient and fast? =)
> Both are important for LLM, why can't we have a high level, simple language that is also robust, efficient and fast? =)
I think many variants from Rust can be stripped to make it simple, especially in type system, which is way more complicated than go.
I dont think we need simpler types for agentic coding though. We need compiler speed. I guess those are probably related to some degree though.
It depends if you are full vibe-coding mode or still try to understand your codebase.
LLM can generate very cryptic rust code.
Goose?
If I want simplicity and maintainability, I’d pick Kotlin over Go. If I want speed, I’m picking Rust over Go. Why would I ever pick Go? It’s the best at nothing.
What if you want a single language that's pretty fast while simple and maintainable?
Then you pick C# which is both faster and simpler than Go.
As I do more agentic coding, I am starting to feel strongly that I need to set aside a project -- and time -- for strictly hand-coding, using AI (if at all) only to explain things, never to write any code.
First, I like programming and don't want to stop doing it just because a machine can do it too. Second, I'm pretty sure atrophied coding skills will make me worse at directing the machines, thus worse at what seems to be "the job" these days.
>First, I like programming and don't want to stop doing it just because a machine can do it too. Second, I'm pretty sure atrophied coding skills will make me worse at directing the machines, thus worse at what seems to be "the job" these days.
In theory wouldn’t atrophied coding skills not matter if models are sufficiently good at coding? I mean isn’t whats left just design and architecture and testing?
Sure, if LLMs are producing shit code you need to have an understanding of what it should do instead. But I think that day is already gone.
Its hard to think of a skill that you’d need to build with an LLM that requires writing meaningful amounts of code by oneself. From my end all the complexity is business logic and system design.
> In theory wouldn’t atrophied coding skills not matter if models are sufficiently good at coding? I mean isn’t whats left just design and architecture and testing?
No, because good understanding of design comes from programming experience. Atrophied skills mean you lose touch with the actual nuts and bolts of what you are building.
This is also the worst kind of lock-in you can imagine, by outsourcing all of your understanding of a codebase and your skills you are now completely at the mercy of LLM providers. Even if you switch to open weight models you're at the mercy of graphics cards producers and scalpers.
> Sure, if LLMs are producing shit code you need to have an understanding of what it should do instead. But I think that day is already gone.
Maybe your standards are too low. By my standard, they still produce shit code and require lots of iterations to get reasonable designs (I often end up just rewriting the bits that are obviously wrong rather than trying to figure out how to "prompt engineer" the right output).
They do often produce passable code but my experience is the longer you spend reading the code the more things you find wrong with it, almost like an uncanny valley.
At least that's my experience with Opus and Fable. Maybe it's too old-fashioned at this point to complain about code quality, but I find it very strange how many people have started proclaiming that their code quality is good.
>No, because good understanding of design comes from programming experience.
“Programming” is a bit loaded. Do you mean thinking through and typing everything out yourself?
What design skills specifically can you only develop from continually programming in this way? I fully grant that you need to know how to code yourself to build things with models but what specific ability to use models atrophies alongside your no AI coding abilities?
Personally I’ve found the opposite to be true. As writing out code myself has shrank to nearly 0, I now have a much shorter feedback loop on design. You get a lot more repetition and feedback when that’s basically all you are doing.
If it helps, imagine they were talking about something like drawing or painting or stitching or sculpting instead of coding.
You don't expect everyone to stop wanting to do those just because they can be automated more or less effectively, right?
> You don't expect everyone to stop wanting to do those just because they can be automated more or less effectively, right?
Please reread my comment and the one im replying to. That’s not what I’m addressing.
> In theory wouldn’t atrophied coding skills not matter if models are sufficiently good at coding?
I am begging HN users to understand that some people enjoy writing software themselves
Read my comment again. And the one I replied to. He claimed that his ability to code through LLMs has decreased as his coding ability atrophied. That is what im questioning. Im not questioning coding for the love of the game.
I haven't actually tried it yet, but I think it'd be good to write a dsl for whatever you're doing. Then have ai make an interpreter to run it. Sort of as a way to stay... interested? Engaged?
I still prefer Go over the alternatives because I'm reviewing the code carefully and trying to make it all look as good as if I'd written it very carefully by hand.
The few times I've needed a language without a GC and used Rust and C as target languages, I've reviewed the code less and enjoyed it less.
People are sleeping on Go as a LLM language of choice. Everyone trying to rewrite in rust which is fine and rust is fine. However its compile times are several orders of magnitude greater than Go. You can have LLMs write and compile 20 apps in the time one rust app compiles. I've got a Go personal project that would take hours for rust to compile, I can't imagine making the same velocity in Rust. Though I do see the workplace appeal of bringing back the xkcd "compiling" meme.
However it kind of makes me feel like my professional life was wasted and I don't really know what kind of future I have. One of Go's main points was you can pick it up in a weekend. What career moat do people like us have?
My only issue with it is that you can't ship a wasm go app in under 2mb. With rust you can make it under 100kb.
I would rather have LLM directly generate machine code, and best of all, macro Assemblers are blazing fast for decades.
You can't PGO that code, though. Also, I'm not sure it can generate architecture specific optimized (-march, -mtune) code as well.
Considering it just pulls and remixes blocks, this is a very tall order for today's LLMs, and maybe a couple of generations as well.
Calling GCC/clang at the back and emitting ASM is both cheating and has no advantages whatsoever as well.
That is fighting the last war on the early days of 5 GLs, and taken care by agentic runtimes.
This isn't futurology, besides a few projects that already came up on Phoronix and around here, with Assembly generation, there are enough SIGPLAN papers on ML driven compilers already.
> "JavaScript Object Notation (JSON) is a simple data interchange format [...]"
Clearly it isn't, otherwise every language would have sorted it out by... a long time ago.
Some languages have a culture of doing things correctly and conforming strictly to the spec. These languages have indeed sorted it out years ago. Haskell for example. Any language that doesn’t have a special number type for JSON is a half-assed implementation, because 1e1000 is a perfectly valid JSON number that is beyond the typical IEEE754 double precision floating point numbers; does your language handle that correctly? Haskell does. This is but a small example of a strong culture bias towards correctness.
The spec (RFC 8259) leaves significant leeway to implementations:
A JSON parser MAY accept non-JSON forms or extensions.
An implementation may set limits on the size of texts that it accepts. An implementation may set limits on the maximum depth of nesting. An implementation may set limits on the range and precision of numbers. An implementation may set limits on the length and character contents of strings.
Add to that things like duplicate keys being allowed by the spec. There’s a lot of implementation choices to decide on.
And then things can get really complex with JSON Schema.
Ironically Go has(d) json.Number which almost no one used, so it got dropped from v2: https://pkg.go.dev/encoding/json#Number
You can still use it though, and I built my tiny decimal library around it: https://github.com/ncruces/decimal
Ok so nothing is simple then?
I like it. I never really liked marshalling anyways, It was weird.
I'm happy it's getting fixed!
I think I can build a new web tools serving for this new API
I generally like `encoding/json/v2` and I'm glad they did it.
I wish they'd left `MarshalIndent` in there though. The equivalent code is super complicated and non-obvious, so it's a pain to migrate, and it's a pain when you're banging out some quick code and want to dump some readable JSON. I suppose these days the LLMs can write the complicated version for us, of course.
I'm not sure if I'm missing something, but isn't this functionality just moved into jsontext? https://pkg.go.dev/encoding/json/jsontext#Value.Indent
I'm working on better SIMD support in Dart and one of the motivations for that work is specializing parser specifications based on schema specifications of that language.
Or more concretely, if we had a parser generator + json parser specification that can generate efficient simdjson style parsers, then we should be able to parse json restricted to a subset of json based on a schema at least as efficiently as simdjson can parse general purpose json.
That way we could have the best of both worlds, a (potentially lazy) statically type-safe model and simdjson performance!
I still think that this is definitely doable, it's just that a lot of infrastructure needs to be built for it.
I remember reading that Meta is doing something similar with binary formats. I find it weird that there's nothing like that for json yet.
I was working on metaparsing earlier this year (specifically over grpc because of its better support for types and better serialization format).
I think most people who use JSON don't want strong typing or to use schemas. If you do, you just use a different serialization format: JSON is designed to be a generic, human-readable way to basically represent any kind of data structure (since it deserializes to JS objects). So everything about JSON is working against you if you do try to enforce type/schemas on it.
A grpc-like protocol with more prescriptive type registries/discovery (over the network) and better interop with http (like grpc-gateway) would be the best approach IMO. Definitely not JSON schema
> I think most people who use JSON don't want strong typing or to use schemas.
Not necessarily. It's biggest advantage is probably that it has very good interoperability. Basically every language has a readily accessible library for json, and even if you don't, it isn't that hard to write a parser and/or generator for it. And being human readable isn't something to sneeze at either.
This approach is called "staged parsing":
https://dl.acm.org/doi/pdf/10.1145/2714064.2660241
https://dl.acm.org/doi/pdf/10.1145/3314221.3314625
I used the jsontext package at my company when we needed to make a parser for challenging clients (specific errors for each kind of mistake) while still keeping performance, and it works like a charm.
There is no introspection and speed is superb. I would definitely recommend jsontext as a parser for anything non-straightforward.
I… think this is good?
I'm just getting the neuron-band back together where Golang is concerned. I have done some in the past for little system tasks, but I am switching away from interpreted languages entirely for web stuff, and initially to Go, as I switch away from hand-coding to at least LLM-supported coding.
So I am a bit out of date on the pain points (I never really felt pain points when I last used it, because the jobs I was doing had very small, quite Go-friendly shapes).
The fact that this took 6 years to land stood out to me.
I'm not entirely surprised, and I'm glad the Go team is carefully considering their interfaces before launching. But 6 years seems like a lot of work to land a JSON API.
Sometimes I wonder about Go's velocity - generics (something that I still haven't used beyond the tutorials) took an age as well
JSON processing, quite markedly, is renown in Go for being slow, so I don't really understand why it took so long
I think in the case of JSON, it was “good enough”. People who found this to be a problem would use one of the many 3rd party options.
I kind of like that the standard library doesn’t move fast. It makes it easier to fit the ecosystem into my head (unlike, for example, the JavaScript ecosystem where I find code doesn’t survive 2 years before failing to compile).
But I do agree that slow and steady produces its own problems. Your example of generics is a good one. But think the example of Go modules is another, perhaps even better, example of how sometimes the Go team really should have accelerated their efforts on.
lol, I don't mean to be contrary, but I looked into dependency management (partly because of the modules implementation, partly because of early teething issues, and partly out of pure curiousity.
Dependency management is a freaking nightmare, nobody has managed to create something that is bulletproof, not cargo, npm, apt, yum, nix, homebrew, nothing.
The key issue is that you're managing 3rd party software that may or may not adhere to the prescribed lifecycle/versioning
The Linux variants do it by each having dedicated teams watching upstream sources, taking what they believe to be whatever they feel is right, building for their nominated target(s), then testing, before deploying (on top of that, all of these steps vary in importance and rigor depending on numerous variables)
For programming libraries (which is what depmod, goose, vgo, and modules are managing) it's just as bad, if not harder, because their very nature is to provide a base upon which other developers can turn into whatever their imagination.
Add to that, modules was (initially) written by people who were dealing with a slightly different ecosystem; Google rather famously has a monorepo, making their lives a little easier when it comes to managing dependencies.
So, with those things in mind, dependency management is always going to be a hard problem that is opinionated (eg. I have a firm opinion that there should be an easier way to "pin" a dependency, other than vendoring - note that this isn't an invitation to every junior Go developer to come and claim go.sum does that. It doesn't. go.sum is a log of what dependencies were used for the last build by whomever wrote to it)
For like 99% of use cases encoding/json was fine. Of course improvements are welcomed. But personally I've lead development of the backend of part one of the biggest sites of its kind in the world, and we just used encoding/json and net/http. Most of our use cases are just building APIs, client facing ones, internal ones, trusted third-parties. These systems have pretty high throughput, more than most people will ever work on.
Within our company I saw people in other teams writing their own optimised JSON libraries for whatever hellish reason. I always reiterated the point that as soon as that app made a single IO request all those nanosecond savings are invisible. Yes, it affects your total CPU time for processing a request but compute was never ever a limitation for us and it's likely not a limitation for most people. And if the trade off is your app needs scale slightly larger versus writing custom JSON code or pulling in a third party dependency, then I'm accepting the larger scaling.
The biggest benefit of encoding/json/v2 isn't the speed, it's the improved API.
> generics (something that I still haven't used beyond the tutorials) took an age as well
Only about a year from when the Haskell guys lent their expertise in designing it and it landing in a release. That's actually fairly quick given that it was a pretty significant change under the hood and constrained by only two releases annually.
The Go team was always clear that they didn't have the necessary capabilities internally, despite a lot of failed efforts trying anyway, and were relying on the open source community to lend a hand. Unfortunately, open source doesn't mean open willingness to contribute. It turns out that for all the experts on HN, posting "go doesn't have generics lol" was preferable to helping out.
Haskell guys? Got a reference to this? That’s news to me.
I'm not sure about the haskell association, but search for "Featherweight Go"
Thanks for the pointer. The reference makes sense now, as Philip Wadler is a coauthor of that paper.
This is how you roll out features, C++. Congrats to Go team and users.
> nil slices and maps are marshaled as an empty JSON array or object by default
I feel that is the kind of broken default that will come back and bite a bunch of people, and will eventually need to be fixed in a v3.
I think it'll bite some people, but probably is the better default for JSON specifically. Most of the nil-maps-become-null thing is just an extension of Go defaulting to nil for empty arrays and maps, which is usually not what you expect when you're writing a JSON schema like
in Go. It's pretty common have code like where you don't _really_ intend for the default nil-ness to leak out. (Apologies for any broken syntax, I'm realizing in real time that it's been a while since I've written Go)Can you explain why? This seems like the appropriate expectation to me, given how JSON is constructed and consumed.
wohoo, good news indeed better later than never :D as an active Golong user I'm very happy
> in any, out any
Completely unrelated: can anyone tell me why language authors introduce in/out/inout function parameters (see also C#).
It feels like this is some rather bad workaround for something that breaks assumptions about parameters being passed into the function.
In Go, it’s a question of who owns the allocation. If you return a pointer type allocated within the function, the answer is “no one,” so the pointer escapes and has to be managed by the garbage collector.
If the allocation is owned by the calling function which uses it internally and then doesn’t return it, it can be a stack variable. Or the same allocation can be reused many times in a loop. Or an array of allocations can be made all at once with foreknowledge of the total number.
`out` params in C# are params that must be set within the function. They are way to implement multiple return values and tell the caller "this is written to, not read from."
But you might be conflating that with in/out for Contra/Co-variance when specifying a generic.
The parameters named "out" here are not "output parameters" in the way you seem to be thinking. They are "input parameters" named "out", because they are the buffer or pipe you want the data written to.
"in" and "out" are the names of the parameters, not qualifiers.
C# `out` is kinda nice because it can be `var`d and used directly.
Erm... what do you mean?
the marshal api has been very annoying. why does marshal return an error? the only reason the marshal can return an error is that the input type is not marshalable. it's certainly a code bug. to me it's like int+str returning an error instead of compiler error
> why does marshal return an error? the only reason the marshal can return an error is that the input type is not marshalable
Not necessarily. Among other things, MarshalJSON methods called during marshaling can return an error for whatever reason the developer sees fit - one possibility that comes to mind is that part of the object tree being marshalled is lazily loaded, and failed to do so.
And, as others have mentioned, there are types like "any" whose marshalability can't be determined at compile time. Those types aren't necessarily at the top level, either; they could appear anywhere in the object tree.
Types may be mostly marshallabble, have a subset of values that are unmarshallable. E.g. non-finite floats.
I wouldn’t class unmarshallable data as a compile time error.
Plus the point of the marshal interface is that it’s reusable.
It could be if go had a more powerful type system.
This isn’t a type system problem. It’s a data origin problem. Ie it could be any random JSON blob a user entered. The compiler doesn’t know this ahead of time.
If you wanted type safety with JSON in Go then don’t use the marshal interface. Period.
For unmarshalling, but the original complaint was about marshalling. And for that the type system could help if the marshal API only allowed you to marshal objects of types that supported marshalling.
And even for unmarshalling you could get a compile time error if you try to unmarshall to a type that doesn't support it.
I’d already addressed that when I said the marshal API is an interface that’s meant to be reusable across generalise use cases. it can be used for a variety of data formats and not just JSON.
Go has this concept baked into the language. For example read and write interfaces so you can write data to an endpoint without having to worry about whether it’s a file system object, TCP/UDP socket, gzip achieve, and a plethora of other options too.
It’s a nice convenience feature and makes up considerably for the restrictions that the Go type system places.
And if you need higher performance code, or compile time safety, or have any other constraints that limit your ability to use these generalised interfaces, then you can use different APIs from the Go standard library to achieve the same thing.
You don’t need to call json.Marshal() if you don’t want to. It’s an option, but it’s just one API into the JSON toolbox.
First of all, the json2 Marshall API is json specific.
Secondly, other languages like rust, haskell, scala, etc. Have json APIs that are typesafe and give you compile time errors if you try to use a type that doesn't support (de) serialization . In the case of rust at least, the API is also general and can be used for other data types as well.
Even in go, you could have some more type safety if Marshall took an interface (such as MarshallerTo) rather than `any`. Although then you have the problem of how you implement that interface. So I guess you probably want some kind of "derive" functionality, or macros that can approximate it for a convenient and type-safe API.
P.S. gophers like to talk about how explicit being better than implicit, but using reflection for marshalling is very implicit.
AFAIK it's mostly because of floats. JSON doesn't support `NaN` and `infinity` for example.
Because they are common types, makes sense to precautionary have the error return.
Then write yourself a MustMarshal wrapper that panics.
A generic Must[T] you mean ;)
The GP is talking about compile time errors. You’re discussing exceptions, which is still a runtime check
JSON is a runtime input of unknown type but in your example those are known types so the compiler can catch them.
i meant i hoped it's panic as opposed to return an error. if there was an error, how am i supposed to handle it? can i tell the user hey sorry my bad it's likely a code error? crash is better than possible eating the error siliently
do not know at compile time that input is not marshalable, every language marshal must return error for this reason, not only golang
It is trivial to ignore an error in Go if you know your use case cannot result in one. But at the end of the day, json.Marshal takes an `any` as a parameter, so it must return an error.
Now guess what will happen when they add monadic error handling. Yes, correct, there will be another JSON Go API