The article revolves a lot around the "how" and "why" but does not really answer it's own title. We thought we have the answer right there: in democracies, the people delegates working out the rules to a group of elected people. The rules then are enforced by another, independent group of people. Currently this seems to be broken in (not only) the USA.
The answer is: historically the incumbents define the rules and use safety as an excuse to build their moats by gaining exclusive rights and they end up being cartel/monopolies.
(Some years ago, people argued that "it's already broken so voting in the crazy guy who wants to shake things up will flip the switch from broken to non-broken". But they were wrong.)
For varying definitions of broken. I know multiple people from broken regions of the world, who would dream to have it like the US. Then there are others who claim their system is broken and only a protest vote will send the sign, while where they live has one of thr highest standards of living in the world.
The absolute level of broken is mostly irrelevant, what counts is the direction of travel.
I'm not a lawyer but my understanding is that OpenAI didn't obviously break a cyber law because it requires intent which doesn't exist (with the definition of intent in a case like this being an open grey area). I think rules around making organizations obviously legally responsible for the cyber attacks performed by agents under their control would be good.
In many areas of criminal law, willful ignorance is not an excuse. One could potentially argue that the sandbox setup at OpenAI that led to the hacking incident at Hugging Face could fall into this category.
Beyond that, there's always tort law, which doesn't require intent.
Allowing several leading laboratories to formulate their own rules under the protection of anti-monopoly exemptions essentially means outsourcing the formulation of public policies to the stakeholders. The direction of independent review is correct, but the review standards, enforcement authority, and participation qualifications must undergo public debate, rather than being determined behind closed doors by a few companies.
Cohere's disagreement seems mostly about who makes the rules. From the essay (which does seem significantly written by AI, sadly):
> The rules around AI are getting written either way. What's still open is whether they get written by a group anyone can join and with evidence anyone can check, or by a handful of companies in a room with the door shut.
Of course none of the AI companies will suggest that they should be accountable for the illegal actions taken by agents they host, but I feel like that could be a very effective "pacing mechanism", and addresses the concerns brought up by this essay about fairness and transparency.
> Of course none of the AI companies will suggest that they should be accountable for the illegal actions taken by agents they host, but I feel like that could be a very effective "pacing mechanism"...
The problem is that the potential liabilities arguably vastly exceed what these companies can absorb. It would be interesting to see if any major insurer wanted to step up and insure them.
On the plus side, in the worst case scenarios, we won't have to worry about damages because nobody will be left to pursue them.
Just look at Meta's recent settlement. The minds and mental health of countless young people harmed. And they'll pay $17 billion, which is about one month's worth of revenue.
The legal system today does not serve the purpose you're wanting it to.
If you want a seat at the table then your AI must prove itself capable of winning some competition against the incumbents. How about a nice game of chess?
That is simply not possible, they are mutually exclusive matters.
Ignoring for the time being that many freedoms have already been adversely curtailed, including freedom of speech, assembly, self-governance and even self-preservation in America, you cannot even preserve the reminding freedoms if you want to “change” rules, i.e., curtail freedom.
Rules inherently curtail freedom and unfortunately for all of us, the absolutely worst types of people are in control of things like AI, convinced of themselves that they are in fact the right people to determine the rules for all of humanity.
At least that was the plan. The wrench in their gears seemingly were the Chinese models that have offered, ironically indeed, immense freedom and on top of that, an escape from the prison the American AI cabal was and continues to build around all of humanity.
There will be many scientists, myself included, that would band together to break the cartel if it forms. It's just a matter of money. It is absurd.
Any logic shows that this is really about their valuations and being tired of innovating instead of having any safety concerns. What would happen if the US did this? Chinese and EU labs would eventually win. And then either the US becomes a backwater or they have to eventually spend the money to keep up anyway. But in the meantime they will have bought themselves a few years at insane valuations.
Also, look at the stark choice they want to offer "we'll kill everyone or you give us the power to do whatever we want with the most important tech around". That's it. If we aren't in an oligarchy now, we will be if this goes through. They're going to end up stealing trillions of dollars from the rest of society.
RSI scenario is not scientist-limited though. So, you'd better get cracking if you want to catch up before it's impossible. Will first need to raise some serious capital too...
The article revolves a lot around the "how" and "why" but does not really answer it's own title. We thought we have the answer right there: in democracies, the people delegates working out the rules to a group of elected people. The rules then are enforced by another, independent group of people. Currently this seems to be broken in (not only) the USA.
The answer is: historically the incumbents define the rules and use safety as an excuse to build their moats by gaining exclusive rights and they end up being cartel/monopolies.
It’s a rhetorical question.
this has always been broken
Broken is a spectrum, not binary.
(Some years ago, people argued that "it's already broken so voting in the crazy guy who wants to shake things up will flip the switch from broken to non-broken". But they were wrong.)
For varying definitions of broken. I know multiple people from broken regions of the world, who would dream to have it like the US. Then there are others who claim their system is broken and only a protest vote will send the sign, while where they live has one of thr highest standards of living in the world.
The absolute level of broken is mostly irrelevant, what counts is the direction of travel.
Most laws, if not all, for new technologies such as the internet and AI already exist.
For example: thievery, extortion, murder, etc. are already well-established crimes.
Putting "cyber" or "AI" in front of them makes it the same crime.
I'm really interested in what is actually new about these rules.
I'm not a lawyer but my understanding is that OpenAI didn't obviously break a cyber law because it requires intent which doesn't exist (with the definition of intent in a case like this being an open grey area). I think rules around making organizations obviously legally responsible for the cyber attacks performed by agents under their control would be good.
In many areas of criminal law, willful ignorance is not an excuse. One could potentially argue that the sandbox setup at OpenAI that led to the hacking incident at Hugging Face could fall into this category.
Beyond that, there's always tort law, which doesn't require intent.
I agree, but cybermurder sounds really interesting as a genre of music or something.
"Why bother regulating the production of uranium isotopes, killing people with nuclear weapons is already a crime"
Allowing several leading laboratories to formulate their own rules under the protection of anti-monopoly exemptions essentially means outsourcing the formulation of public policies to the stakeholders. The direction of independent review is correct, but the review standards, enforcement authority, and participation qualifications must undergo public debate, rather than being determined behind closed doors by a few companies.
I believe this is a reaction to Dario Amodei's essay yesterday, "We Must Pace the Frontier": https://news.ycombinator.com/item?id=49672510
Cohere's disagreement seems mostly about who makes the rules. From the essay (which does seem significantly written by AI, sadly):
> The rules around AI are getting written either way. What's still open is whether they get written by a group anyone can join and with evidence anyone can check, or by a handful of companies in a room with the door shut.
Of course none of the AI companies will suggest that they should be accountable for the illegal actions taken by agents they host, but I feel like that could be a very effective "pacing mechanism", and addresses the concerns brought up by this essay about fairness and transparency.
> Of course none of the AI companies will suggest that they should be accountable for the illegal actions taken by agents they host, but I feel like that could be a very effective "pacing mechanism"...
The problem is that the potential liabilities arguably vastly exceed what these companies can absorb. It would be interesting to see if any major insurer wanted to step up and insure them.
On the plus side, in the worst case scenarios, we won't have to worry about damages because nobody will be left to pursue them.
The way they operate? No sane insurer. But the whole point is changing their behavior.
> The problem is that the potential liabilities arguably vastly exceed what these companies can absorb
It is ok. Fines should not be just a cost of business.
Just look at Meta's recent settlement. The minds and mental health of countless young people harmed. And they'll pay $17 billion, which is about one month's worth of revenue.
The legal system today does not serve the purpose you're wanting it to.
Shouldn’t this be a democratic process?
Democracy does not exist.
And there is no need for rules, this doomerism probability is not based on science.
Are we full-on Peter Thiel "democracy and freedom are incompatible" yet?
Atlas may start shrugging any minute now!
He is right. Democracy is not compatible with unlimited freedom for him. There are laws.
The differencd is that I see Thiels freedom limited a little as a good thing.
Counter argument: Democracy does exist. (it's just a fuzzy concept like most words we use).
If you want a seat at the table then your AI must prove itself capable of winning some competition against the incumbents. How about a nice game of chess?
Something that’s both civilised and representative of the future. Stratagema? Or Mornington Crescent?
What rules need to change that wouldn't adversely affect existing freedoms?
That is simply not possible, they are mutually exclusive matters.
Ignoring for the time being that many freedoms have already been adversely curtailed, including freedom of speech, assembly, self-governance and even self-preservation in America, you cannot even preserve the reminding freedoms if you want to “change” rules, i.e., curtail freedom.
Rules inherently curtail freedom and unfortunately for all of us, the absolutely worst types of people are in control of things like AI, convinced of themselves that they are in fact the right people to determine the rules for all of humanity.
At least that was the plan. The wrench in their gears seemingly were the Chinese models that have offered, ironically indeed, immense freedom and on top of that, an escape from the prison the American AI cabal was and continues to build around all of humanity.
The people with money, obviously.
Me.
There will be many scientists, myself included, that would band together to break the cartel if it forms. It's just a matter of money. It is absurd.
Any logic shows that this is really about their valuations and being tired of innovating instead of having any safety concerns. What would happen if the US did this? Chinese and EU labs would eventually win. And then either the US becomes a backwater or they have to eventually spend the money to keep up anyway. But in the meantime they will have bought themselves a few years at insane valuations.
Also, look at the stark choice they want to offer "we'll kill everyone or you give us the power to do whatever we want with the most important tech around". That's it. If we aren't in an oligarchy now, we will be if this goes through. They're going to end up stealing trillions of dollars from the rest of society.
RSI scenario is not scientist-limited though. So, you'd better get cracking if you want to catch up before it's impossible. Will first need to raise some serious capital too...
The way people talk about RSI reminds me of how people talk about the rapture.
> Chinese and EU labs
China simply steals from the US.
EU is in a lethargic state of sleep.
good luck having those rules audited reliably.
[dead]