Stanford's "Autonomous" Helicopters Learn 90
An anonymous reader writes "Stanford computer scientists have developed an artificial intelligence system that enables robotic helicopters to teach themselves to fly difficult stunts by 'watching' other helicopters perform the same maneuvers. The result is an autonomous helicopter that can perform a complete airshow of complex tricks on its own. The stunts are 'by far the most difficult aerobatic maneuvers flown by any computer controlled helicopter,' said Andrew Ng, the professor directing the research of graduate students Pieter Abbeel, Adam Coates, Timothy Hunter and Morgan Quigley. The dazzling airshow is an important demonstration of 'apprenticeship learning,' in which robots learn by observing an expert, rather than by having software engineers peck away at their keyboards in an attempt to write instructions from scratch.'" The title of the linked article uses the term "autonomous," but that's somewhat misleading. The copters can't fly on their own, but rather can duplicate complex maneuvers learned from a human pilot.
That's exactly how Terminator got started (Score:5, Funny)
Remember those robot gunships in the Terminator movies? Yes, the they shot lasers and what not. Those things had humble beginnings.
Re: (Score:2)
Who FORGOT the skynet tag? Slackers!
Re: (Score:2, Funny)
Re: (Score:1, Redundant)
In Soviet Russia, Helicopters teach YOU!!!!
Re:That's exactly how Terminator got started (Score:4, Informative)
Wrong it was from Slap Shot (Score:1)
Its privately called
the Big Tomahawk
and the students on campus are called
The Little scalps.
http://www.imdb.com/title/tt0076723/quotes [imdb.com]
Re: (Score:2, Insightful)
We stopped evolving along a "natural" biological path a long time ago; what differentiates us is our ability to store, analyze,
I for one... (Score:1, Funny)
Oh forget it.
Re: (Score:1)
Autonomous (Score:5, Insightful)
They still are autonomous. Would you call people non-autonomous because they learn from other people?
Re: (Score:1)
Re:Autonomous (Score:5, Informative)
Your blithe refusal to even acknowledge the article is an inspiration, sir.
-Peter
Re:Autonomous (Score:5, Funny)
Yeah, they need to be able to learn an abstract concept and apply it to the conditions around them rather than learning a fixed sequence of actions and repeating blindly. In slashdot terms rather than posting "I for one welcome our robot overlords" to every article and get modded "-1 Die in Fire", they learn to make a slightly original joke tailored to the article and get modded up.
Oh God, what have I become.
Re: (Score:2)
They aren't just learning a fixed sequence. I don't know the details of the project, but I own an RC heli and they are twitchy and unstable things, so there is going to be some dynamic process going on that stops it from plowing into the dirt while trying to "play back" something. Just how integrated the systems are, I'm not sure.
Re: (Score:2)
Re: (Score:1)
Did *you* even read your original post?
I can do this all day.
But seriously, it seems like the part I quoted goes to your point. Maybe I don't understand the article, or your point, or both.
Anyway, the article seems to imply that there is some underlying "flying" software, and that the computer just "learns" the stunts from the expert. Said another way, it seems like the computer was programmed to fly the helicopter "the old fashioned way", and the new thing is that the computer is inferring what the stunt
So Mr. Ng makes semi-autonomous mini-helicopters? (Score:1, Funny)
I, for one, welcome our new supersonic nuclear cyborg pit-bull terrier guard dogs.
Re:So Mr. Ng makes semi-autonomous mini-helicopter (Score:2, Informative)
It looks like someone with mod points has never read Snowcrash.
The potential for disaster staggers the mind... (Score:2)
Better hope that one of the silly things doesn't catch sight of a couple of dogs having at it when it's in "learning mode". That could lead to some very interesting flying stunts indeed.
No eyeballs involved (Score:2)
They don't use eyeballs to watch and learn, they apparently receive the r/c commands being sent to the 'teaching' helicopter and the position data from its onboard sensors.
Helicopter macro recording? (Score:2)
This sounds like any automated testing tool I've ever used. I can either take the time to "peck away" at my keyboard and script the task by hannd. Or I can put it into "record mode" and have it record my mouse cilcks / keyboard clicks. This sounds like the sa
Re:Helicopter macro recording? (Score:5, Informative)
Re:Helicopter macro recording? (Score:5, Interesting)
Exactly - that's where the AI comes in. It looks at numerous attempts of various tricks, notes the differences in the environmental variables and adjusts the controls wrt to the current conditions for the optimal execution. If anything it's more like, record 100 attempts, analyze and define a "good" routine, but also define good "exception handlers". I must say, I was pretty confused about what, if any, "watching" was involved - i.e. reflected-light-based vision. Pretty bad explanation, bordering on fail.
Re: (Score:2)
The article made no mention of visual sensors. If they used vision in this project you would have heard about it. The "watching" is watching the inertial readings in relation to the controller positioning. That is, the computer looks at the position, velocity, and acceleration of the helicopter vs. the expert input, and determines what the expert was trying to achieve.
It isn't just replaying the user inputs, but it is just replaying a filtered version of the "moves" as determined by the inertial sensors.
Re: (Score:1)
Re: (Score:2)
Perhaps if we were talking about big planes way up in the atmosphere, but not when it comes to small helis. You can't just run a macro of the last pilot, because you'd end up in the dirt. A small heli like that has to be constantly aware of it's position and continually making adjustments. You'd be better off comparing it to the Eurofighter or the 4WD system of a modern rally car.
UCAR (Score:5, Informative)
Northrop currently has an unmanned helicopter called Firescout that has autonomously landed on a Navy ship while the ship was moving [navy.mil].
My point is that this type of work is nothing new.
Re:UCAR (Score:5, Insightful)
Re: (Score:1)
Autonomous kdawson (Score:1, Insightful)
You probably didn't learn a language on your own either but we think you might be autonomous.
Jut because the robot's have learned by watching an expert doesn't make them not autonomous. People learn by watching experts as well ... ok so maybe only some of us do
If the robots are capable ( and according to the article it seems they are ) of independent flight then they are autonomous.
Duh, it doesn't mean automatic (Score:1)
Who on slashdot wouldn't be familiar with the word autonomous?
Re: (Score:2, Funny)
Who on slashdot wouldn't be familiar with the word autonomous?
Not everyone on Slashdot posts as an Autonomous Coward.
Re: (Score:1)
you failed it.
I for one welcome our new autonomous helicopter overlords.
Not Autonomous? FTNWYWCBED* (Score:5, Insightful)
From the 2nd paragraph of the article:
"The result is an autonomous helicopter than can perform a complete airshow of complex tricks on its own."
From kdawson's summary:
"The title of the linked article uses the term "autonomous," but that's somewhat misleading. The copters can't fly on their own, but rather can duplicate complex maneuvers learned from a human pilot."
How in any way do you come to that conclusion based upon the data in the story?
They CAN and DO fly by themselves. Out of the lab. In varying weather conditions. Constantly making adjustments for wind gusts, etc., none of which is being controlled by a human.
And then the wisecrack about their AI? It uses an algorithm to study commands sent to another helicopter, studies the results, figures out what the goal of the commands was, and is able to implement those goals, on its own, more accurately than the original human pilot. That's not a strong AI?
Can we please get some editors that understand what they are reading?
*From The Not What You Would Call Brilliant Editing Department
Re: (Score:1)
Re: (Score:2, Interesting)
Re: (Score:2)
It uses a Markov Decision process. There are many possible states, each with a "reward" for being in that state and other possible states than can be reached. The goal is to learn the policy (where to go from each state) to maximize the overall reward over a long period of time. The wind and other environment help determine the policy from observing the "policy" of the human example.
Re:Not Autonomous? FTNWYWCBED* (Score:5, Insightful)
Heh, the summery is just an example of ignorance. A human being does the EXACT same thing. The main difference being that when I put my aircraft into the air, I can redefine the programming very quickly without outside intervention. Currently the AI that has been developed works against a known, set collection of goals.
One could argue, that in the military, that may be more useful than an actual person in many cases. The person my have a conviction that prevents them from carrying out the mission, for instance, a child on the battle field my prevent them from firing a weapon, even though the result of not firing means that thousands of people die elsewhere.
Personally, having not served in the military, I can not comprehend how a pilot ( and I've always wanted to be a fighter pilot mind you ) or any other military personal can actually take someone elses life. I know plenty of men and women do it on it often enough to know that you can overcome that and justify the action to yourself, at least long enough to do it, but lets face it, no one that I would want in our military should be able to take a life and NOT feel something is wrong.
To be clear, I do not fault those people who serve at all, I hold them with the highest respect because they do a job that I do not believe I could do, nor do I really WANT to do, I just want to play with the aircraft :)
So ... if this AI is a little 'dumb', it may make things a whole lot better in the end. Of course, the other side of the argument, and the side that I'm on says that the human decisions made during combat help prevent us from being walking evil. It is that ability for us to keep our morals in check and consider more than just our assigned goals that keeps us different from machines and makes us, to our knowledge, unique, for good or bad. It is ALSO that ability to adjust our goals based on the situation that makes the difference in a war.
If you turn all wars into battles between machines, its simple a contest of production and resources in which case its likely you can just decide the outcome in advance and save the resources. So do you do the Warcraft Rush, or do you camp and stockpile?
Fortunately, I can't imagine that it will even end up that we have computers doing all the work for us as those in power tend to not want to give it up, even to their computerized slaves.
Re: (Score:2)
"Personally, having not served in the military, I can not comprehend how a pilot ( and I've always wanted to be a fighter pilot mind you ) or any other military personal can actually take someone elses life"
If someone is about to kill you if you don't kill them first I suspect its rather easy frankly. Having a clear conscience won't do you much good if you've just been turned in a ball of greasy smoke.
Re: (Score:2)
I'm going to try to extract your point from this collection of fantasy.
So to continue the dramatic hyperbole: what you're saying here is you prefer machine logic deciding to blow up toddlers. Got it.
Re: (Score:2)
You sir, missed the point entirely.
I did not say I am for having machines blow toddlers up, I am however saying that what robotic solder lacks is a concious, and that it can be both a benifit and problem to have a concious.
My point was exactly the same as yours. Once the resources run out, or production is unable to keep up with demand, humans will be back in the battle field.
Re: (Score:2)
Re:Not Autonomous? FTNWYWCBED* (Score:5, Interesting)
It appears that two things are going on here, both interesting.
First, the system is learning by observing a human operator as well as the 'chopper's response. This is a bit more than simple macro programming. The system observes a system state, an operator input and a subsequent state and appears to be deriving an initial set of control laws in addition to figuring out the operator's goals.
Next, when it is in autonomous flight mode, it fine tunes the control laws to optimize the chopper's performance.
The latter is actually an easier task for simple systems. Adaptive PID control systems are already in use, but a helicopter presents a problem involving multiple inputs and degrees of freedom. The former is a more interesting problem, particularly for unstable systems. Deriving parameters for a control system by watching another in action, when the observed inputs are noisy is tricky.
Re: (Score:1)
Next, when it is in autonomous flight mode, it fine tunes the control laws to optimize the chopper's performance.
I don't get the sense that there is really an autonomous flight mode. It sounded to me as if the helicopter only carries the instrumentation that needed to be on board ("accelerometers, gyroscopes and magnetometers"). All smarts seemed to be on the ground.
Unfortunately the news release was written by a layman (a PR person, yet) for laymen, so you can't trust it.
There is another "fact" that can't hardly be correct:
The piÃce de résistance may have been the "tic toc," in which the helicopter, while pointed straight up, hovers with a side-to-side motion as if it were the pendulum of an upside down clock.
I don't know much about model helicopters, but I find it hard to believe that they can hover wi
Re: (Score:2)
I'd give them a break on the claim of autonomous. The weight and power requirements of the controllers are probably beyond the capabilities of a model helicopter. So they 'stretched' the I/O links out over radio up/down links. Its still sort of autonomous if there is no human in the loop.
I doubt they'd have any trouble putting the same systems on board a full sized helicopter. A black one, at that.
Re:Not Autonomous? MOD PARENT DOWN (Score:3, Insightful)
The helicopters can NOT fly by themselves "out of the lab." They are remote control helicopters with a few extra gadgets. In particular, they would include potentiometers to measure control surface deflection, digital servos, an AHRS unit, a data acquisition system, and a small autopilot computer based on a PowerPC, XScale, ARM9, or similar low-power CPU. The autopilot computer contains the definition of the control system, which is a set of nonlinear equations defining the dynamics and flight control of
Re: (Score:2)
They fly with the help of multiple cameras on the *ground*, which makes it more of a 3-D Printer -- not an autonomous flying machine. Notice their emphasis on the "air-ground cooperation" [stanford.edu] of the original web site for their project. That's because directing flying machines from the ground is a known tractable problem. And directing
... and commentators that read the whole article (Score:1)
How in any way do you come to that conclusion based upon the data in the story?
...
Can we please get some editors that understand what they are reading?
From the article:
During a flight, some of the necessary instrumentation is mounted on the helicopter, some on the ground. Together, they continuously monitor the position, direction, orientation, velocity, acceleration and spin of the helicopter in several dimensions. A ground-based computer crunches the data, makes quick calculations and beams new flight directions to the helicopter via radio 20 times per second.
If the helicopter is being directed by a ground-based computer, it's not autonomous. Kdawson was correct this time. Yes, the article contradicts itself, but I think the most detailed information is correct. Whoever wrote that news release didn't seem to understand what they were writing.
youtube "rc helicopter". (Score:2, Informative)
and you'll see. Throw in a little wind here and there and the robot doesn't stand a chance.
http://ca.youtube.com/watch?v=gi7G-VzU2r4 [youtube.com]
Re: (Score:2)
Actually, try this link:
http://www.youtube.com/watch?v=M-QUkgk3HyE [youtube.com]
Re: (Score:3, Insightful)
Why not? The tails on pretty much all RC helis are already controlled by a computer and they work very well. You can also get auto pilot systems that will hold the heli in a steady hover despite crosswinds etc. It might be a while before they can do the chaos maneuver a couple of feet off the ground like in that video, but there's no reason it can't be done -- not reason they can't do it even better.
JBH (Score:1)
reinforcement learning vs. simulation (Score:3, Informative)
This story got a lot more attention than the other zillions of autonomous helicopters out there. The disappointment with the Stanford one is it is reinforcement learning. It's recording and playing the commands of a human pilot instead of simulating a dynamic model and deducing commands based on a genetic algorithm. The real value in ground based autopilot is having enough computing power to use biological algorithms.
Re: (Score:2, Informative)
Or, you could RTFA.
Re: (Score:1)
Re: (Score:2)
The interesting point isn't that there are autonomous helicopters out there (there have been a few spectacular ones in the past) but the use of learning via apprenticeship. Purely self-taught algorithms have been used in the past to good effect, but apprenticeship has many practical applications. A genetic helicopter may not "discover" all of the known maneuvers out there. If you make minor updates to hardware, it might be possible for the older copters to apprentice the younger ones. At a higher level,
Re: (Score:2)
Sometimes it's easier to start by copying. RC helis are quite expensive.
Re: (Score:2)
RC helis are quite expensive.
$29.95. http://www.thinkgeek.com/geektoys/rc/a75e/ [thinkgeek.com]
Re: (Score:2)
Those are only 3 channel with very different mechanics and not capable of the kind of stunts the helis in the video are capable of. It's almost like comparing golf cart to an F1 car.
Re: (Score:2)
No, it's not just recording and playing the commands of a human. If it did that, it would crash.
The algorithm is learning an optimal policy to execute in an environment (defined by wind, altitude, etc.) to reach some goal. The learns how to react to the environment based on the actions of a human.
A genetic algorithm also has an objective function that it's trying to maximize, but would have to get to the optimum from a random starting location. It would be like putting joe blow in a cockpit and saying "d
OK, I know when I'm beaten (Score:2)
Mirror neurons (Score:3, Interesting)
I'm put in mind of mirror neurons [wikipedia.org] which fire sympathetically and seem to account for the ability of animals to mimic and thus learn novel behaviour.
Re: (Score:2)
(not that this has anything to do with the article :P)
I remember I lost your post a while back about language translation, understanding the world metaphorically. I caved in and got a subscription (lol)... and dug out the post... if you got an email/etc and your still interested let me know, the original post was here, in case you've forgotten.
http://hardware.slashdot.org/article.pl?sid=08/01/16/2333204 [slashdot.org]
Re: (Score:2)
It's funny because I almost did the same thing, i.e. subscribe so I could dig out one of my old posts, I'll probably do it.
I didn't receive an email, you've got my address ^
Cheers.
Re: (Score:2)
Tried emailing you, I'm wondering if they are getting spamblocked.
GOFAI (Score:1)
Well then... (Score:1)
Have these guys not seen the Terminator movies? (Score:1, Funny)
Why don't they just start working on the Robotrons now? Maybe then they could get them finished before 2084 and just get the whole "destruction of man by his own creation" thing over with.
Here is the Video. (Score:1)
Lesson #1 (Score:1)
Re: (Score:2)
Yeah, that's pretty amazing, but when I watch something like this, the following saying pops up: "To some a 3D heli is amazing, to me it's a demented insect after being sprayed"
Anyway, I doubt that SU's helis can do the tricks good pilots can (e.g. the Szabo brothers), nevermind better.
Nice line break... (Score:3, Funny)
I read this on the main page. It was laid out with
"Stanford computer scientists have developed an artificial intelligence system that enables robotic helicopters to teach themselves to
and then a new line, which I was certain was going to start with the word "kill".
Unrelated, but funny story. (Score:2)
neat trick? (Score:1)
So the learn... (Score:1)
However, do they learn at a geometric rate?
oiram@hollywoodpt.tv (Score:1)
Obligatory video (Score:1)
Just for comparison, check out this guy doing stunts with a model helicopter [youtube.com]. Not only incredible moves, but great choreography as well.