| Popis: |
Robots have been used to automate many tasks, and yet designing robots remains largely a manual process. This thesis seeks to “automate automation” by developing methods to automate both design of the mechanism and control of the robot. To make these automation problems tractable, we use modular robots, as their components can be readily rearranged to form a customized robot for each new task. This work addresses two problems: control over many different modular designs, and selecting the best design for each task. First, we present a control framework geared toward the case where a large number of designs are made from a set of modules, such that it would be prohibitively expensive to create a new control policy from scratch for each robot design. We introduce modular policies that learn to control a range of designs in one training process. In the policy architecture, information about the dynamics and controls is shared among modules of the same type via shared neural network parameters. We use deep reinforcement learning algorithms to train the policies on multiple designs and environments simultaneously. We then show that modular policies can transfer to new designs and environments not seen in training. With a modular control framework in place, we apply it to select the best robot designs for specific tasks. We frame design optimization as a sequential decision making problem in which modular robots are incrementally constructed one module at a time. We then implement a deep reinforcement learning approach to train a design value function, an estimate of how each module added to the design will impact performance. The design value function serves as a search heuristic to efficiently identify the best design for each new task. This method is applied to two classes of robots: designing manipulators to reach locations in a workspace, and designing mobile robots to traverse a non-flat terrain. Through these contributions, this thesis shows that leveraging modularity in learning enables the creation and transfer of robot behaviors across tasks and designs. |
