A marker of progress will be when Amazon converts to automated picking. They've been trying hard for almost a decade now. They had an annual competition for years. They have a decent picking robot developed in house.[1] It's not being deployed in quantity yet.
Nor does it have anything like a humanoid hand. Just a two-surface gripper.
Amazon's production robots are mostly automatic guided vehicles, not manipulators.
I'm impressed with how far legged locomotion has come. But as yet, nobody seems to be using legged robots for any commercial purpose beyond the demo level.
Is Tesla still going to produce vast numbers of humanoid robots by the end of 2026?
There's been a lot of progress on the hardware side. Motor technology from drones has produced much better robot motors. The sweet spot on gear reduction seems to have been found. (Too much reduction, and you can't back drive. Too little, and the motors have to be too big.) The volumes are now large enough to justify making robot-specific components. Robot arms are much better than a decade ago. So are robot legs. Control is better, too. It looks like a humanoid robot will cost about as much as a car.
But they're still not quite good enough to be useful.
We'll know they are real when an Amazon Prime truck drives up and a robot does the last 100 meters of the delivery.
That would only be a minor marker. A major marker would be whatever the Chinese equivalent(s) of Amazon are (Alibaba? etc.) going that route successfully. From academic / industrial conference presentations it appears Chinese services are banking on automation far more than an entity like Amazon does.
That being said, I don't know what exact state the industrial automation technology is there and I can only extrapolate (or do websearch, which didn't lead to enough details; I only found things like https://www.youtube.com/watch?v=JnUGgc8R3ng).
When something like https://www.allegrohand.com is mass produced and used industrially, I bet the last mile (meter?) would change a bit, and full automation would be easier and less finicky.
Not sure about that. Labor costs are lower in China and the CCP is strongly incentivized to keep enough jobs around for humans to maintain their own hold on power.
Labor being cheap is more of a locale dependent condition in China now, their well developed cities and industries have had rising wages for 2+ decades now and the labor discount is small enough to be more like a side-benefit to already manufacturing in China, rather than a driving force to outsource to China. China has itself started outsourcing certain productions to chase cheaper labor prices.
That labor costs are rising and are now significantly above poorer countries like Bangladesh or Myanmar doesn't mean they're anywhere near high-income countries or above the cost of automation for low-volume tasks like last-mile delivery, which drivers do for under a dollar per trip: https://archive.ph/SizjZ
Automation can make sense even with low labor costs if it enables higher volume. Like nobody is paying humans to chisel individual screws by hand, because machines can do it quicker and better, and therefore cheaper.
But you can't substantially increase the volume of last-mile delivery by replacing the human with a robot, because the robot would have to move faster, and moving faster would be dangerous. So robotic automation of low-volume tasks can be expected to happen in high-income countries first.
Labor costs are lower, but rising, and already high enough that automation can be appealing.
The Chinese government is also pretty good at finding things for people to do, so I doubt that's a factor. Go to any big park in a city and see how many people are sweeping up leaves, or guarding a building/area that isn't particularly secure.
Why do we have the assumption that industrial robots need to have legs? The largest grocery store chain uses picker robots already - for building my orders, including fruit / vegetables (from boxes). They currently have humans filter the veggies and fruit prior to box setup, but a friend is working on the models to eliminate even that.
Their robots do movement using sliding scale in 3D spaces (think poles that are left right, up and down, and the "picker" being able to glide and move. They currently have a ceiling slider that goes down and suctions things into a pneumatic tube to then end up in my grocery bag. IMHO it works pretty well - especially considering that delivery is ~$8 for me.
Ultimately we're going to end up with several different types of robots, and not with a human centric vision. The question is if bipedal is a long term dead-end, and merely a short term method to fit into the world we currently have designed.
There's a bunch of options for this task, but combining the three wheel stair-climbing dolly with the self-balancing hoverboard/segway seems like the easiest.
you don't need legs to climb stairs, you can have a rotating mechanism with different rows of wheels that have good enough grip, maybe an extruding stick to lift the whole robot up and the wheels move forward... would take a lot fewer joints than pair of legs probably.
I worked at a warehouse a few years ago. Over lunch a manager bragged to me with some secret videos of experimental facilities. Semi-automated, and fully-automated. I don't know how far along they were though.
But the intention was very clearly to get rid of all of us, as soon as physically possible.
Yes, picking is one, and package delivery is another. I wonder how many people currently get package deliveries by robot? It might be a good metric to track?
Roboticist here. All of this, and he didn’t mention compliance or online adaptation to otherwise un-sensable dynamics. Or massively complex miniature mechanisms.
Current generation tactile sensors cost a couple thousand $ PER FINGER, and have a real world MTBF of hours. The cost can be solved with economy of scale. The fragility is harder.
> Current generation tactile sensors cost a couple thousand $ PER FINGER, and have a real world MTBF of hours. The cost can be solved with economy of scale. The fragility is harder.
Are we talking about experimental laboratory ones here? What happens when the Alibaba players start getting into the game? They have plenty of humanoid robots.
Capacitive touchscreens are not tactile in any way whatsoever. They are purely electrical, and there are plenty of things (actually, most things, I'd say) which are not sufficiently capacitive to be detected by such approaches. You usually need to measure physical deformation instead, which is much more difficult to do precisely. (There are capacitive approaches to that, but they don't work like a touchscreen)
I don't think that's true. AI could work around all the physical imperfections.
Just imagine yourself controlling an imperfect robot with some joysticks or maybe a sensor suit. You'd certainly do much better than current robots. The limit is the robot's brain, not the physical actuators/sensors.
Curious on your take on this, given your domain - is the hardware the principle challenge in your opinion, or the software?
>> It will be difficult to match this scale of breadth and depth of data for physical tasks. There’s no straightforward equivalent of “just Efficient learning, generalization, and adaptability / on-the-job learning seem like requirements.
Isn’t this the idea of NVIDIA’s Isaac? Model based adaptive learning in virtual environments for robotic systems? Or is this oversold?
Yes the problem is very hard. Mainly because high DOF generalization is very difficult.
We have self driving cars because what are the control inputs? Pedal, brake, steering wheel. This already took many many years.
Now for a humanoid robot: An action space that is metaphorically Hilbert. (Physically, yes, obviously)
Also, IMO, LLM's can aid the development of robots, but do little beyond a planning, human control interface. Below that it's the domain of control and the solution will be the correct combination of classical, neural, and real time optimization based control.
All the bad-ass biped robots that actually look natural? It's PID controls wrapped with control barrier functions constraining the QPs that are being solved in real time.
But that's annoying to derive per-application. So we'll need neural methods which can be learned (while being constrained by a priori knowledge of dynamics). My hunch is that the Yann LeCunn type of jepa models will be how tasks can be learned.
> All the bad-ass biped robots that actually look natural? It's PID controls wrapped with control barrier functions constraining the QPs that are being solved in real time.
That's not entirely true. Locomotion is well addressed by RL in sim. It's true that there is still a PD layer, and the RL policy produces setpoints for it.
I don’t think the complexity scales with every additional degree of freedom like you are painting here. I think it’s just a matter of getting the right training data in sufficient quantities for an LLM to output across all degrees of freedom simultaneously without it being some exponential leap.
> So we'll need neural methods which can be learned
Data is a problem. LLMs had the advantage of the whole internet to train on. Robots don’t have that corpus of information. And real time learning seems to be something that everyone in AI is studiously ignoring.
The hope is that RL in simulation can fill the gap.
Also there’s imitating humans, via a suitable mapping from the human sensor, control and configuration space to the robot’s. Some groups have gathered video and other data from humans doing tasks, for example with a VR headset.
>Yes the problem is very hard. Mainly because high DOF generalization is very difficult.
>We have self driving cars because what are the control inputs? Pedal, brake, steering wheel. This already took many many years.
Its actually amazing to me that this hasn't been solved yet. Its really not that hard of a problem.
Modern robotics, including self driving, are famously all about end-to-end training. We are trying to replicate what humans do through muscle memory. But muscle memory is not what makes us good at operating in the physical world. The thing that matters the most is our ability to simulate the world around us in a compressed form into the future, which lets us predict how our inputs will affect the world.
A similar system in a self driving car should be able to drive perfectly without self inflicted accidents 100% of the time, especially with basic lidar to serve as an error correction mechanism to the camera 3d scene reconstruction.
Set the boundaries and let it go at midnight to quietly wander the lawn and spray the weeds. If it could spray into nearby garden beds too, so much the better.
Being infinitely patient it could apply small targeted doses to only what you don't want. A human doing it often doesn't have the patience or accuracy.
While I agree this is a technical path that makes sense, I'm not sure it's a financial path that does. Or at least not outside of very upscale, niche products for wealthy consumers or businesses.
Which doesn't mean I disagree with you. I also think that this progression is the most likely. But it implies we're decades away from broad adoption rates.
Think fifty years to hit mass adoption, not five. (Because that much more closely aligns with other structurally disruptive tech like automobiles or computers) Which is definitely not the story being pitched to investors at the moment.
> Or at least not outside of very upscale, niche products for wealthy consumers or businesses.
Products that start life as "for the rich, first adopters" and work their way down the economic classes are a thing. Whether it is the thing in this case I don't know.
> Which is definitely not the story being pitched to investors at the moment.
I would think that what is pitched is what investors want to hear....
Sometimes, I wonder why we even need humanoid robots for some tasks. For example, we had that robot over the past year that would wash the dishes...would be more efficient if it had 8 arms for that job. Or if I wanted it to clean my room, an arm that can extend up to the ceiling.
This is why im not worried about "AI" taking over the world. Robotics still has a LONG way to go. A human can balance a plate on their arm with food while holding a glass of milk in that hand and a donut in the other and still manage to open a door, step over potential floor obstacles, maneuver tight spaces, get bumped by a child or dog, and still set it all down without spilling it 99% of the time. Just the hardware with the dexterity and responsiveness to perform the same task would cost unimaginable amounts of money to produce, not to mention the control systems needed to do it smooth and gracefully enough.
Maybe in another 2 decades I could see it possibly starting to change, but even then I wouldn't bet the horse on it until I saw it. Cars only have three degrees of freedom and even that we are barely able to get working well enough to put it into limited practice. And yet one single human finger has atleast 3 degrees of freedom, and is covered in what is the equivalent of a million tiny ultra sensitive tactile sensors.
This is true, but it’s also assuming the robot has to be human shaped. A robot with 4 extendable arms and a gyroscopically balanced cabinet in its chest wouldn’t have too much trouble with that task.
How many other tasks is it not suited for now though? If we wanted a device to automate limited specific tasks then we don't need advanced highly advanced robotics. A human has strength, dexterity, high balance, very sensitive tactile touch, can move both fast and slow, is very compact, etc. A robot has to have serious tradeoffs just to hit two of those things. A robot arm that is capable of threading a needle is likely not suited to chopping some wood or carrying groceries. And if you just add a bunch of different arms for different tasks it becomes way more complex, expensive, heavy, and inefficient.
Main problem of humnoid is price. Price starts from 20,000$ and we need jetson thor(On-device) or 5090 for learning capabilities. Their ability per cost is not greater than human.
The other problem is hard to predict/simulate real world contact physics. Thus, most of company use VLA for task with contact. Even if VLA is advancing rapidly, it's still immature to be deployed in practical.
people very soon as going to look back at all of us and just think "they didn't even have robots yet! how did they even eat?" Well, future readers, it isn't easy! We either go to the store and bring back groceries ourselves or pay a helper to do it. No robot in the loop there. Then when it comes time to cook something we either have to cut it up and otherwise prepare it and then cook it ourselves, or another human can do it, or we can buy fully prepared foods that don't require cooking. And then when we're done we have to put our own dishes away into a dishwasher. We can't just say a few words and have a robot do it for us!
> I am confused at how Waymo engineering can be so robust as to yield an astonishingly good safety record, and yet so slapdash as to happily drive into deep water.
I feel this is actually somewhat straightforward. I assume deep water on roadways is not commonly in the training set, because frankly it isn't common in real life, and when it is common people do not drive and do not gather that training data. As a result the proper response has not adequately been beaten into the models. There are probably also challenges of world-sensing, since water can act as a mirror, and maybe other complications. So waymos are bad at handling deep water on roadways. However, deep water on roadways is also not common in the areas where waymos are deployed. As a result, waymo's have a great safety record, and at the same time they make mistakes that are obvious to a human.
A common criticism of AI discourse is that people act as if LLM's "think". I don't want to be a vocabulary purist, but I suspect that's related to the astonishment here -- the Waymo doesn't know what flooding is, it doesn't fear drowning, it doesn't think. So unless it's been repeatedly trained, or a special case has been hard coded by manual effort, it doesn't know that flooded roadways are dangerous.
I have made a lot of assumptions here, and I don't truthfully know what the training data looks like. Feel free to push back if you think my assumptions are wrong. I'd especially be interested if somebody can show that water on roadways _is_ in the training data
My thinking here is that Waymo has logged hundreds of millions of miles at this point (and even more in sim), and there are a lot of nines in their safety record. So even the rare edge cases should have come up. You make a good point that the deep water scenario may be not only rare, but also under-represented in the training data. On the other hand, you'd think they would have thought to explicitly train for it; it's not that obscure of a possibility.
Ya, this does strike me as pretty high on the list of the thing I'd intentionally seek out if I was doing this job, along with object-in-the-road, icy road, tornado/hurricane, wildfire, and perhaps hail.
People drive into deep water all the time - some states specifically have laws making them financially liable for the cost of rescue because it’s such a stupid thing to do. But still, they do it.
I think the fact that there are laws about it is not good evidence that it should be in the training data. Laws often cover weird edge cases, and if an edge case happens 50 times in 100 years, there's likely a law covering it. At the same time, that's probably not enough occurrences for it to naturally end up in a dataset -- the edge case would probably need to be intentionally sought out. I'm not saying that driving-in-deep water only happens 50 times in 100 years, it's certainly more common than that, I'm just saying that despite laws on the topic, it may still be too rare to be well represented in training data. For example, in real life I've only seen a car drive into deep water once. Even if we include recordings that I've seen, that would maybe bring it up to 20?
The physical world is basically an infinite amount of global state that must be perceived indirectly through imperfect sensors and acted on using imperfect motors and manipulators.
I mean, functional robotics isn't that hard, robotic vacuums have been in homes for a decade now, and industrial robotic arms. However, humanoid robots IS hard mainly because of the form factor constraints. You can't really get a lot of power out of servo motors and other actuator if it all has to be self-contained in a humanoid form instead of using hydraulics or pneumonic or even big stepper motors as for stationary industrial robots.
The videos I've seen of humanoid robot applications are basically that it can do dishes and fold laundry, but I think if household chore robots ever come to market, they would probably not look humanoid at all and probably look like semi dishwashers/washing machines with wheels and a gripper arm.
Electric motors are absurdly power dense compared to human muscle.
The form factor has been solved basically everyone is building humanoid robots and hoping a transformer with a big enough dataset is going to do the rest.
I've yet to see anyone address the more immediate problem for (humaniform) robotics, which is that it runs counter to the economic benefits of specialization at industrial scales.
A robotic warehouse that's just like a human warehouse but with robots walking the aisles will always be more expensive (and likely far less efficient) than a warehouse purpose built for automated picking using standard containers and graspers, conveyer belts or path constrained wheeled platforms--something I've seen in operation 20 years ago.
A factory of general purpose robots sewing t-shirts will always be more expensive than a factory a low wage humans sitting there doing the same.
The Fourdrinier process for making flat-sheet goods (i.e., paper, thin plastic, in massive rolls) is more than two centuries old. The idea that robots, returning to dipping a mould into the furnish to create individual sheets, could even come close to the economics of modern papermaking is insane.
> AI progress is racing along, but virtually all of the visible progress is in the realm of knowledge work, i.e. activities that can take place inside a computer.
The most visible "progress" of so-called AI is in activities that take place at the interface between computers and gullible humans.
The reason so much less progress has been made in robots is that real-world physics isn't gullible.
> Once they have context, robots will need to reason, plan, and exercise judgement and common sense. LLM-based systems like ChatGPT and Claude are making great strides in these areas
But why would I want to make AI more powerful - and disruptive - than it already is? I don't see this as a benefit but as a disadvantage. Let's also not forget that e. g. Google deliberately ruined its search engine. Now if you search something, by default, you get AI slop results that are often not truthful or only partially truthful. This is a private web. Google wants to control information.
I can say many things, but most people here will not believe me. I'd say just watch China do the 'impossible'. Then some self reflection should be in order.
A marker of progress will be when Amazon converts to automated picking. They've been trying hard for almost a decade now. They had an annual competition for years. They have a decent picking robot developed in house.[1] It's not being deployed in quantity yet. Nor does it have anything like a humanoid hand. Just a two-surface gripper. Amazon's production robots are mostly automatic guided vehicles, not manipulators.
I'm impressed with how far legged locomotion has come. But as yet, nobody seems to be using legged robots for any commercial purpose beyond the demo level.
Is Tesla still going to produce vast numbers of humanoid robots by the end of 2026?
There's been a lot of progress on the hardware side. Motor technology from drones has produced much better robot motors. The sweet spot on gear reduction seems to have been found. (Too much reduction, and you can't back drive. Too little, and the motors have to be too big.) The volumes are now large enough to justify making robot-specific components. Robot arms are much better than a decade ago. So are robot legs. Control is better, too. It looks like a humanoid robot will cost about as much as a car.
But they're still not quite good enough to be useful.
We'll know they are real when an Amazon Prime truck drives up and a robot does the last 100 meters of the delivery.
[1] https://www.youtube.com/watch?v=WStK9HNn8c8&
That would only be a minor marker. A major marker would be whatever the Chinese equivalent(s) of Amazon are (Alibaba? etc.) going that route successfully. From academic / industrial conference presentations it appears Chinese services are banking on automation far more than an entity like Amazon does.
That being said, I don't know what exact state the industrial automation technology is there and I can only extrapolate (or do websearch, which didn't lead to enough details; I only found things like https://www.youtube.com/watch?v=JnUGgc8R3ng).
When something like https://www.allegrohand.com is mass produced and used industrially, I bet the last mile (meter?) would change a bit, and full automation would be easier and less finicky.
Not sure about that. Labor costs are lower in China and the CCP is strongly incentivized to keep enough jobs around for humans to maintain their own hold on power.
Labor being cheap is more of a locale dependent condition in China now, their well developed cities and industries have had rising wages for 2+ decades now and the labor discount is small enough to be more like a side-benefit to already manufacturing in China, rather than a driving force to outsource to China. China has itself started outsourcing certain productions to chase cheaper labor prices.
That labor costs are rising and are now significantly above poorer countries like Bangladesh or Myanmar doesn't mean they're anywhere near high-income countries or above the cost of automation for low-volume tasks like last-mile delivery, which drivers do for under a dollar per trip: https://archive.ph/SizjZ
Automation can make sense even with low labor costs if it enables higher volume. Like nobody is paying humans to chisel individual screws by hand, because machines can do it quicker and better, and therefore cheaper.
But you can't substantially increase the volume of last-mile delivery by replacing the human with a robot, because the robot would have to move faster, and moving faster would be dangerous. So robotic automation of low-volume tasks can be expected to happen in high-income countries first.
Labor costs are lower, but rising, and already high enough that automation can be appealing.
The Chinese government is also pretty good at finding things for people to do, so I doubt that's a factor. Go to any big park in a city and see how many people are sweeping up leaves, or guarding a building/area that isn't particularly secure.
The CCP is also well aware that it's facing a dramatic demographic contraction so they better prepare for that.
Why do we have the assumption that industrial robots need to have legs? The largest grocery store chain uses picker robots already - for building my orders, including fruit / vegetables (from boxes). They currently have humans filter the veggies and fruit prior to box setup, but a friend is working on the models to eliminate even that.
Their robots do movement using sliding scale in 3D spaces (think poles that are left right, up and down, and the "picker" being able to glide and move. They currently have a ceiling slider that goes down and suctions things into a pneumatic tube to then end up in my grocery bag. IMHO it works pretty well - especially considering that delivery is ~$8 for me.
Ultimately we're going to end up with several different types of robots, and not with a human centric vision. The question is if bipedal is a long term dead-end, and merely a short term method to fit into the world we currently have designed.
It would be nice to have a thing that can carry a bucket of paint up a flight of stairs
There's a bunch of options for this task, but combining the three wheel stair-climbing dolly with the self-balancing hoverboard/segway seems like the easiest.
you don't need legs to climb stairs, you can have a rotating mechanism with different rows of wheels that have good enough grip, maybe an extruding stick to lift the whole robot up and the wheels move forward... would take a lot fewer joints than pair of legs probably.
If this was awesome, we would have it on wheelchairs. We very rarely do. Stairs are built for legs.
I worked at a warehouse a few years ago. Over lunch a manager bragged to me with some secret videos of experimental facilities. Semi-automated, and fully-automated. I don't know how far along they were though.
But the intention was very clearly to get rid of all of us, as soon as physically possible.
(Otherwise gran will switch to $competitor!)
Yes, picking is one, and package delivery is another. I wonder how many people currently get package deliveries by robot? It might be a good metric to track?
> [1] https://www.youtube.com/watch?v=WStK9HNn8c8&
This video is so incorrect on so many levels. Thomas only releases the robots and at the end there is Bridget.
> Is Tesla still going to produce vast numbers of humanoid robots by the end of 2026?
Sure, with FSD by 2030 - promise! /s
Roboticist here. All of this, and he didn’t mention compliance or online adaptation to otherwise un-sensable dynamics. Or massively complex miniature mechanisms.
Current generation tactile sensors cost a couple thousand $ PER FINGER, and have a real world MTBF of hours. The cost can be solved with economy of scale. The fragility is harder.
> Current generation tactile sensors cost a couple thousand $ PER FINGER, and have a real world MTBF of hours. The cost can be solved with economy of scale. The fragility is harder.
Are we talking about experimental laboratory ones here? What happens when the Alibaba players start getting into the game? They have plenty of humanoid robots.
i think you can get some good surface for a few hundred $$ / few fingers here
https://xelarobotics.com/
How much more sensitive do you need to be compared to say a standard mobile capacitive screen and those are essentially dirt cheap at this point.
Capacitive touchscreens are not tactile in any way whatsoever. They are purely electrical, and there are plenty of things (actually, most things, I'd say) which are not sufficiently capacitive to be detected by such approaches. You usually need to measure physical deformation instead, which is much more difficult to do precisely. (There are capacitive approaches to that, but they don't work like a touchscreen)
I'm picturing humanoid robots having to operate in pairs so they can constantly fix each other.
Also so they can watch each other, in case of the humans trying to shut them down.
> Also so they can watch each other, in case of the humans trying to shut them down.
That's why the human operator needs a gun. :-)
i like this
yep. and with the western funding model, how many hardware iterations can you do before you run out of runway?
I don't think that's true. AI could work around all the physical imperfections.
Just imagine yourself controlling an imperfect robot with some joysticks or maybe a sensor suit. You'd certainly do much better than current robots. The limit is the robot's brain, not the physical actuators/sensors.
(I'm not saying there couldn't be improvements in the area of the actuators)
Curious on your take on this, given your domain - is the hardware the principle challenge in your opinion, or the software?
>> It will be difficult to match this scale of breadth and depth of data for physical tasks. There’s no straightforward equivalent of “just Efficient learning, generalization, and adaptability / on-the-job learning seem like requirements.
Isn’t this the idea of NVIDIA’s Isaac? Model based adaptive learning in virtual environments for robotic systems? Or is this oversold?
Yes the problem is very hard. Mainly because high DOF generalization is very difficult.
We have self driving cars because what are the control inputs? Pedal, brake, steering wheel. This already took many many years.
Now for a humanoid robot: An action space that is metaphorically Hilbert. (Physically, yes, obviously)
Also, IMO, LLM's can aid the development of robots, but do little beyond a planning, human control interface. Below that it's the domain of control and the solution will be the correct combination of classical, neural, and real time optimization based control.
All the bad-ass biped robots that actually look natural? It's PID controls wrapped with control barrier functions constraining the QPs that are being solved in real time.
But that's annoying to derive per-application. So we'll need neural methods which can be learned (while being constrained by a priori knowledge of dynamics). My hunch is that the Yann LeCunn type of jepa models will be how tasks can be learned.
> All the bad-ass biped robots that actually look natural? It's PID controls wrapped with control barrier functions constraining the QPs that are being solved in real time.
That's not entirely true. Locomotion is well addressed by RL in sim. It's true that there is still a PD layer, and the RL policy produces setpoints for it.
I don’t think the complexity scales with every additional degree of freedom like you are painting here. I think it’s just a matter of getting the right training data in sufficient quantities for an LLM to output across all degrees of freedom simultaneously without it being some exponential leap.
> So we'll need neural methods which can be learned
Data is a problem. LLMs had the advantage of the whole internet to train on. Robots don’t have that corpus of information. And real time learning seems to be something that everyone in AI is studiously ignoring.
The hope is that RL in simulation can fill the gap.
Also there’s imitating humans, via a suitable mapping from the human sensor, control and configuration space to the robot’s. Some groups have gathered video and other data from humans doing tasks, for example with a VR headset.
>Yes the problem is very hard. Mainly because high DOF generalization is very difficult.
>We have self driving cars because what are the control inputs? Pedal, brake, steering wheel. This already took many many years.
Its actually amazing to me that this hasn't been solved yet. Its really not that hard of a problem.
Modern robotics, including self driving, are famously all about end-to-end training. We are trying to replicate what humans do through muscle memory. But muscle memory is not what makes us good at operating in the physical world. The thing that matters the most is our ability to simulate the world around us in a compressed form into the future, which lets us predict how our inputs will affect the world.
A similar system in a self driving car should be able to drive perfectly without self inflicted accidents 100% of the time, especially with basic lidar to serve as an error correction mechanism to the camera 3d scene reconstruction.
People think robots in terms of humanoid or number-5 style robots.
I think it'll be more capable appliances at first.
Like a lawn mowing device that also spots weeds and can spray them.
Next iteration has arms to rip weeds out of the garden.
Next has attachments so you can direct it to do pruning.
Next it can figure out the pruning itself and move the outcome into the woodchipper.
And so on and so on.
It's not going to be one day a humanoid robot comes into the house and does everything.
> Like a lawn mowing device that also spots weeds and can spray them.
That's available as a tractor-pulled implement for farms. Deere and some others make such things.
Honestly, I want a little one for my lawn.
Set the boundaries and let it go at midnight to quietly wander the lawn and spray the weeds. If it could spray into nearby garden beds too, so much the better.
Being infinitely patient it could apply small targeted doses to only what you don't want. A human doing it often doesn't have the patience or accuracy.
While I agree this is a technical path that makes sense, I'm not sure it's a financial path that does. Or at least not outside of very upscale, niche products for wealthy consumers or businesses.
Which doesn't mean I disagree with you. I also think that this progression is the most likely. But it implies we're decades away from broad adoption rates.
Think fifty years to hit mass adoption, not five. (Because that much more closely aligns with other structurally disruptive tech like automobiles or computers) Which is definitely not the story being pitched to investors at the moment.
> Or at least not outside of very upscale, niche products for wealthy consumers or businesses.
Products that start life as "for the rich, first adopters" and work their way down the economic classes are a thing. Whether it is the thing in this case I don't know.
> Which is definitely not the story being pitched to investors at the moment.
I would think that what is pitched is what investors want to hear....
Sometimes, I wonder why we even need humanoid robots for some tasks. For example, we had that robot over the past year that would wash the dishes...would be more efficient if it had 8 arms for that job. Or if I wanted it to clean my room, an arm that can extend up to the ceiling.
Ooh. I see. so that's how it aaall got started: https://m.youtube.com/watch?v=YVMoLvMrC8E
The Jetsons' Rosey was never seen cleaning the bathroom, was she/it ? Some things are too implausible even for childrens' science fiction.
This is why im not worried about "AI" taking over the world. Robotics still has a LONG way to go. A human can balance a plate on their arm with food while holding a glass of milk in that hand and a donut in the other and still manage to open a door, step over potential floor obstacles, maneuver tight spaces, get bumped by a child or dog, and still set it all down without spilling it 99% of the time. Just the hardware with the dexterity and responsiveness to perform the same task would cost unimaginable amounts of money to produce, not to mention the control systems needed to do it smooth and gracefully enough.
Maybe in another 2 decades I could see it possibly starting to change, but even then I wouldn't bet the horse on it until I saw it. Cars only have three degrees of freedom and even that we are barely able to get working well enough to put it into limited practice. And yet one single human finger has atleast 3 degrees of freedom, and is covered in what is the equivalent of a million tiny ultra sensitive tactile sensors.
This is true, but it’s also assuming the robot has to be human shaped. A robot with 4 extendable arms and a gyroscopically balanced cabinet in its chest wouldn’t have too much trouble with that task.
How many other tasks is it not suited for now though? If we wanted a device to automate limited specific tasks then we don't need advanced highly advanced robotics. A human has strength, dexterity, high balance, very sensitive tactile touch, can move both fast and slow, is very compact, etc. A robot has to have serious tradeoffs just to hit two of those things. A robot arm that is capable of threading a needle is likely not suited to chopping some wood or carrying groceries. And if you just add a bunch of different arms for different tasks it becomes way more complex, expensive, heavy, and inefficient.
“If a self-driving car finds itself in a situation it can’t handle or suffers a glitch, it can pull over or, in the worst case, just hit the brakes.”
Wrong. Try hitting the brakes of your self driving car on highway at 65mph or during unprotected left turn with oncoming vehicles.
Or have a glitched self-driving car hit its brakes and block the road, for emergency vehicles, and endangering other people.
Self-driving cars can also suffer a glitch without knowing they suffered a glitch, like Waymo cars driving into flooded roads.
Main problem of humnoid is price. Price starts from 20,000$ and we need jetson thor(On-device) or 5090 for learning capabilities. Their ability per cost is not greater than human. The other problem is hard to predict/simulate real world contact physics. Thus, most of company use VLA for task with contact. Even if VLA is advancing rapidly, it's still immature to be deployed in practical.
“I may not care if my household robot takes all night to tidy up and fold the laundry.”
I do. I don’t want robot vacuuming or making noise at night or doing something potentially dangerous unmonitored while people are asleep.
Small cleaning robots have existed for long enough. For floor sweeping they are more efficient than any 2-legged form.
Not folding the laundry, though.
Not for the kind of floor I have with tons of obstacles and small corners.
people very soon as going to look back at all of us and just think "they didn't even have robots yet! how did they even eat?" Well, future readers, it isn't easy! We either go to the store and bring back groceries ourselves or pay a helper to do it. No robot in the loop there. Then when it comes time to cook something we either have to cut it up and otherwise prepare it and then cook it ourselves, or another human can do it, or we can buy fully prepared foods that don't require cooking. And then when we're done we have to put our own dishes away into a dishwasher. We can't just say a few words and have a robot do it for us!
Eat? You must be living under a rock. We've been evaporating our sustenance and storing it in the cloud for years.
Perhaps nanobots will be able to carry the chemical makeup of a cheeseburger and rebuild a bite directly in our mouths, no cooking necessary!
Remember though, there is no food cloud really, just other people's fridges.
There is a business idea sitting right there.
> We either go to the store and bring back groceries ourselves or pay a helper to do it.
Or order it from Amazon, in which case there was likely a robot in the pipeline.
Robots are very widely deployed, but almost entirely invisibly to the customer yet.
Roomba is the main exception.
As if people of the future will be able to read. Maybe some future AI anthropologists will read it though.
> I am confused at how Waymo engineering can be so robust as to yield an astonishingly good safety record, and yet so slapdash as to happily drive into deep water.
I feel this is actually somewhat straightforward. I assume deep water on roadways is not commonly in the training set, because frankly it isn't common in real life, and when it is common people do not drive and do not gather that training data. As a result the proper response has not adequately been beaten into the models. There are probably also challenges of world-sensing, since water can act as a mirror, and maybe other complications. So waymos are bad at handling deep water on roadways. However, deep water on roadways is also not common in the areas where waymos are deployed. As a result, waymo's have a great safety record, and at the same time they make mistakes that are obvious to a human.
A common criticism of AI discourse is that people act as if LLM's "think". I don't want to be a vocabulary purist, but I suspect that's related to the astonishment here -- the Waymo doesn't know what flooding is, it doesn't fear drowning, it doesn't think. So unless it's been repeatedly trained, or a special case has been hard coded by manual effort, it doesn't know that flooded roadways are dangerous.
I have made a lot of assumptions here, and I don't truthfully know what the training data looks like. Feel free to push back if you think my assumptions are wrong. I'd especially be interested if somebody can show that water on roadways _is_ in the training data
(I'm the author of the blog post)
My thinking here is that Waymo has logged hundreds of millions of miles at this point (and even more in sim), and there are a lot of nines in their safety record. So even the rare edge cases should have come up. You make a good point that the deep water scenario may be not only rare, but also under-represented in the training data. On the other hand, you'd think they would have thought to explicitly train for it; it's not that obscure of a possibility.
Ya, this does strike me as pretty high on the list of the thing I'd intentionally seek out if I was doing this job, along with object-in-the-road, icy road, tornado/hurricane, wildfire, and perhaps hail.
People drive into deep water all the time - some states specifically have laws making them financially liable for the cost of rescue because it’s such a stupid thing to do. But still, they do it.
I think the fact that there are laws about it is not good evidence that it should be in the training data. Laws often cover weird edge cases, and if an edge case happens 50 times in 100 years, there's likely a law covering it. At the same time, that's probably not enough occurrences for it to naturally end up in a dataset -- the edge case would probably need to be intentionally sought out. I'm not saying that driving-in-deep water only happens 50 times in 100 years, it's certainly more common than that, I'm just saying that despite laws on the topic, it may still be too rare to be well represented in training data. For example, in real life I've only seen a car drive into deep water once. Even if we include recordings that I've seen, that would maybe bring it up to 20?
Oh yeah, I’m not saying it should be in the training data - driving into the water would be a bad data collection strategy.
My point was more that a self-driving car going into water isn’t some unreasonably bad action, as people do it all the time.
If we set the bar for self-driving to be as safe as a human driver, or even 2x as safe, this behavior would still happen.
Ahhhh, that makes sense, that is an interesting point — I think you’re saying it’s just not high risk enough for them to make sure it’s got coverage
Couple of related reads/watches to this I found useful recently, with a similar conclusion:
https://www.youtube.com/watch?v=FUUzmRH5Yi4
https://www.construction-physics.com/p/robot-dexterity-still...
The physical world is basically an infinite amount of global state that must be perceived indirectly through imperfect sensors and acted on using imperfect motors and manipulators.
I mean, functional robotics isn't that hard, robotic vacuums have been in homes for a decade now, and industrial robotic arms. However, humanoid robots IS hard mainly because of the form factor constraints. You can't really get a lot of power out of servo motors and other actuator if it all has to be self-contained in a humanoid form instead of using hydraulics or pneumonic or even big stepper motors as for stationary industrial robots.
The videos I've seen of humanoid robot applications are basically that it can do dishes and fold laundry, but I think if household chore robots ever come to market, they would probably not look humanoid at all and probably look like semi dishwashers/washing machines with wheels and a gripper arm.
Electric motors are absurdly power dense compared to human muscle.
The form factor has been solved basically everyone is building humanoid robots and hoping a transformer with a big enough dataset is going to do the rest.
I've yet to see anyone address the more immediate problem for (humaniform) robotics, which is that it runs counter to the economic benefits of specialization at industrial scales.
A robotic warehouse that's just like a human warehouse but with robots walking the aisles will always be more expensive (and likely far less efficient) than a warehouse purpose built for automated picking using standard containers and graspers, conveyer belts or path constrained wheeled platforms--something I've seen in operation 20 years ago.
A factory of general purpose robots sewing t-shirts will always be more expensive than a factory a low wage humans sitting there doing the same.
The Fourdrinier process for making flat-sheet goods (i.e., paper, thin plastic, in massive rolls) is more than two centuries old. The idea that robots, returning to dipping a mould into the furnish to create individual sheets, could even come close to the economics of modern papermaking is insane.
[0] https://en.wikipedia.org/wiki/Paper_machine
Felt obligated to repeat this linked video here for posterity:
https://youtube.com/watch?v=weXDUc5Osto
> AI progress is racing along, but virtually all of the visible progress is in the realm of knowledge work, i.e. activities that can take place inside a computer.
The most visible "progress" of so-called AI is in activities that take place at the interface between computers and gullible humans.
The reason so much less progress has been made in robots is that real-world physics isn't gullible.
hairstylists are safe. and manicurists. and outcall masseurs.
really anything that involves a bade near your body or where body contact is the point.
They've already got a kind of masseur. You put on a spandex outfit to reduce friction and lay under 2 arms apparently.
> https://www.aescape.com/
that's neither sexy nor sensual. pass.
Another example of how AI dumbs down everything.
> Once they have context, robots will need to reason, plan, and exercise judgement and common sense. LLM-based systems like ChatGPT and Claude are making great strides in these areas
But why would I want to make AI more powerful - and disruptive - than it already is? I don't see this as a benefit but as a disadvantage. Let's also not forget that e. g. Google deliberately ruined its search engine. Now if you search something, by default, you get AI slop results that are often not truthful or only partially truthful. This is a private web. Google wants to control information.
[flagged]
[dead]
I can say many things, but most people here will not believe me. I'd say just watch China do the 'impossible'. Then some self reflection should be in order.