You could study the motion of the helicopter and formulate precise equations to control the blades. Reinforcement learning is interesting because you can teach the helicopter that it has blades and it can spin them quickly or slowly; this way or that way. Then you teach the helicopter that falling or shaking a lot are bad. If it does either of these, you punish the helicopter. Over time, the helicopter can learn how to hover because it is trying to get the most "reward" that it can by its actions. There are several algorithms that are suitable to different types of problems. We might choose one of these algorithms and program the helicopter to "learn" by that method.
The approach that I learned about today is like an artificial democracy. Instead of choosing one algorithm under the rules of which the helicopter must learn, you choose several algorithms at the same time. Each algorithm will come up with its own policy of how to fly the helicopter to maximize its own interpretation of the "reward."
Next, the helicopter listens to all of the algorithms at once, either by taking some average of biasing towards some rather than others. It would be as if the helicopter had a group of policy advisers who would provide suggestions. I read a dissertation today which argued that ensembles of policies might actually be more robust than individual policies for certain problems. Perhaps ensembles could work well on problems which are "in-between" areas of specialization for the member algorithms or problems for which very little information is known.
This idea is very intuitive and seems very related to how living systems and human-made systems work (like democratic governments or ant colonies).
No comments:
Post a Comment