fuzzy actor-critic learning based intelligent controller for high-level motion control of serpentine

166 Pages·2005·10.12 MB·English

FUZZY ACTOR-CRITIC LEARNING BASED INTELLIGENT CONTROLLER
FOR HIGH-LEVEL MOTION CONTROL OF SERPENTINE ROBOTS

A THESIS SUBMITTED TO
THE GRADUATE SCHOOL OF NATURAL AND APPLIED SCIENCES
OF
MIDDLE EAST TECHNICAL UNIVERSITY

BY

EVR(cid:1)M ONUR ARI

IN PARTIAL FULFILLMENT OF THE REQUIREMENTS
FOR
THE DEGREE OF MASTER OF SCIENCE
IN
ELECTRICAL AND ELECTRONICS ENGINEERING

NOVEMBER 2005
Approval of the Graduate School of Natural and Applied Sciences

Prof. Dr. Canan ÖZGEN
Director

I certify that this thesis satisfies all the requirements as a thesis for the degree of
Master of Science.

Prof. Dr. (cid:1)smet ERKMEN
Head of Department

This is to certify that we have read this thesis and that in our opinion it is fully
adequate, in scope and quality, as a thesis for the degree of Master of Science

Prof. Dr. Aydan M. ERKMEN Prof. Dr. (cid:1)smet ERKMEN

Co-supervisor Supervisor

Examining Committee Members

Prof. Dr. Erol KOCAO(cid:2)LAN (METU,EE)

Prof. Dr. (cid:1)smet ERKMEN (METU,EE)

P rof. Dr. Aydan ERKMEN (METU,EE)

Prof. Dr. Kemal (cid:1)DER (METU,ME)

Günay (cid:3)im(cid:4)ek, M.Sc. (ASELSAN)
I hereby declare that all information in this document has been obtained and
presented in accordance with academic rules and ethical conduct. I also
declare that, as required by these rules and conduct, I have fully cited and
referenced all material and results that are not original to this work.

Name, Last name: Evrim Onur ARI

Signature:
iii
ABSTRACT

FUZZY-ACTOR CRITIC LEARNING BASED INTELLIGENT CONTROLLER
FOR HIGH-LEVEL MOTION CONTROL OF SERPENTINE ROBOTS

ARI, Evrim Onur
MSc., Department of Electrical and Electronics Engineering
Supervisor: Prof. Dr. (cid:1)smet ERKMEN
Co-Supervisor: Prof. Dr. Aydan M. ERKMEN

November 2005, 148 pages

In this thesis, an intelligent controller architecture for gait selection of a serpentine
robot intended to be used in search and rescue tasks is designed, developed and
simulated. The architecture is independent of the configuration of the robot and
the robot is allowed to make different kind of movements, similar to grasping.
Moreover, it is applicable to parallel processing in several aspects and it is an
implementation of a controller network on robot segment network. In the
architecture several behaviors are defined for each of the segments. Every
behavior is realized in the form of Fuzzy Actor-Critic Learning agents based on
fuzzy networks and reinforcement learning. Each segment controller determines
the next suitable position in the sensory space acquired using ultrasound sensors, a
genetic algorithm implementation then tries to find the change of the joint angles
to achieve the desired movement in a given amount of time. This allows
optimization on different criteria, during motion. Simulations are performed and
presented to introduce the efficiency of the developed controller architecture.
Moreover a simplified mathematical analysis is performed to gain insight of the
controller dynamics.

iv
Keywords: Serpentine robots, rescue robots, planning in unstructured
environment, reinforcement learning, Fuzzy Actor-Critic Learning (FACL).

v
ÖZ

YILANSI ROBOTLARIN ÜST SEV(cid:1)YE DENET(cid:1)M(cid:1) (cid:1)Ç(cid:1)N BULANIK
EYLEY(cid:1)C(cid:1)-ELE(cid:3)T(cid:1)R(cid:1)C(cid:1) Ö(cid:2)RENMEYE DAYALI AKILLI DENETLEÇ

ARI, Evrim Onur
Yüksek Lisans, Elektrik ve Elektronik Mühendisli(cid:5)i Bölümü
Tez Yöneticisi: Prof. Dr. (cid:1)smet ERKMEN
Ortak Tez Yöneticisi: Prof. Dr. Aydan M. ERKMEN

Kasım 2005, 148 sayfa

Bu tezde, arama kurtarma çalı(cid:4)malarında kullanılmak üzere tasarlanan yılansı bir
robotun üst seviye hareket denetimi için akıllı bir kontrol sistemi mimarisi
geli(cid:4)tirilmi(cid:4) ve simule edilmi(cid:4)tir. Geli(cid:4)tirilen mimari, robotun mekanik yapı
detaylarından ba(cid:5)ımsızdır ve nesne kavrama gibi farklı hareketlerin
gerçeklenmesine olanak sa(cid:5)lamaktadır. Buna ek olarak, geli(cid:4)tirilen mimari bir çok
yönden paralel i(cid:4)lenmeye uygundur ve kontrol a(cid:5)ının robotun mekanik segment
a(cid:5)ı üzerine yerle(cid:4)tirilmesiyle olu(cid:4)turulmu(cid:4)tur. Mimaride robotun her segmenti için
çe(cid:4)itli davranı(cid:4)lar tanımlanmı(cid:4)tır. Her davranı(cid:4) “Bulanık Mantık” ve
“Kuvvetlendirici Ö(cid:5)renme” kullanan “Bulanık Eyleyici-Ele(cid:4)tirici Ö(cid:5)renmeye
Dayalı Akıllı Denetleç” yapılarıyla gerçeklenmi(cid:4)tir. Robotun her segment
denetleci, durum uzayında kendisi için en uygun bir sonraki pozisyona karar verir.
Bunun ardından “genetik algoritma” kullanan bir en iyileyici belirli bir zaman
diliminde, robotu eski durumundan yeni durumuna geçirmek için yapılması
gereken segment hareketlerini belirler. Bu sayede, robot hareketinin farklı kriterler
gözetilerek en iyilenebilmesi sa(cid:5)lanmı(cid:4)tır. Geli(cid:4)tirilen mimarinin uygulama
sonuçlarını ve etkinli(cid:5)ini göstermek amacıyla simülasyonlar yapılmı(cid:4) ve sonuçları
vi
tezde sunulmu(cid:4)tur. Buna ek olarak, denetlecin dinami(cid:5)i hakkında fikir vermesi
amacıyla, basitle(cid:4)tirilmi(cid:4) matematiksel analizler yapılarak çalı(cid:4)maya eklenmi(cid:4)tir.

Anahtar Kelimeler: Yılansı robotlar, arama kurtarma robotları, yapılandırılma-
mı(cid:4) ortamlarda yol planlama, kuvvetlendirici ö(cid:5)renme, “Bulanık Eyleyici-
Ele(cid:4)tirici Ö(cid:5)renme”.

vii
To Aylin, for her endless love...

viii
ACKNOWLEDGEMENTS

I would like to express my sincere appreciation to Aydan M. Erkmen and (cid:1)smet
Erkmen for their guidance throughout this study.

I particularly thank to my mom and dad for their lifelong support.

I am grateful to my colleagues Günay (cid:3)im(cid:4)ek, Murat Ertek, Mehmet Karaka(cid:4) and
Eray Özçelik for their friendship and morale support.

Finally, I greatly appreciate Seyit Yıldırım and Bülent Bilgin since they allowed
me to use powerful workstations at ASELSAN in order to implement the
computationally heavy simulations, and showed great understanding whenever I
spent my work time on this thesis study.

ix
TABLE OF CONTENTS

PLAGIARISM........................................................................................................ iii

ABSTRACT............................................................................................................ iv

ÖZ........................................................................................................................... vi

ACKNOWLEDGEMENTS......................….......................................................... ix

TABLE OF CONTENTS......................................................................................... x

LIST OF TABLES................................................................................................xiii

LIST OF FIGURES.............................................................................................. xiv

CHAPTER

1. INTRODUCTION……………………………………………………….… 1

1.1. Search and Rescue Robotics – Main Motivation of This Study………1

1.2. Serpentine Robots for SAR Applications……………………………. 3

1.3. Intelligent Control and Learning Techniques…………………………6

1.4. Outline of the Thesis…………………………………………………..6

2. LITERATURE SURVEY………………………………………………….. 7

2.1 Snake-like Robots…………………………………………………….. 7

2.2 Serpentine Motion Gaits……………………………………………...15

2.2.1 Natural Snake Gaits…………………………………………..16

2.2.2 Unnatural Snake Gaits………………………………………..19

2.3 Intelligent Control…………………………………………………… 21

2.3.1 Artificial Neural Networks – Basics of Artificial Learning…. 22

x