Back Matter

Search in book...
Toggle Font Controls
Create new playlist

Name your new playlist

Playlist description (optional)
Sign In

Email address

Password

Forgot Password?

or

Continue with Facebook

Continue with Google
Sign Up

Full Name

Email address

Confirm Email Address

Password

or

Continue with Facebook

Continue with Google

Previous Chapter

15. Reinforcement Learning

References

1.
M.M.M. Al-Husari, B. Hendel, I.M. Jaimoukha, E.M. Kasenally, D.J.N. Limebeer, and A.Portone. Vertical stabilisation of Tokamak Plasmas. In Proceedings of the 30th Conference on Decision and Control, December 1992.
2.
Shaojie Bai, J. Zico Kolter, and Vladlen Koltun. An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling. arXiv, April 2018.
3.
D. Bertsekas. Reinforcement Learning and Optimal Control. Athena Scientific, 2019.
4.
Ilker Birbil and Shu-Chering Fang. An electromagnetism-like mechanism for global optimization. Journal of Global Optimization, 25:263–282, 03 2003.
5.
Léon Bottou, Frank E. Curtis, and Jorge Nocedal. Optimization methods for large-scale machine learning. SIAM Review, 60:223–311, 2016.MathSciNetCrossref
6.
A. Bryson and Y. Ho. Applied Optimal Control. Hemisphere Publishing Company, 1975.
7.
Barbara Cannas, Gabriele Murgia, A Fanni, Piergiorgio Sonato, Augusto Montisci, and M.K. Zedda. Dynamic Neural Networks for Prediction of Disruptions in Tokamaks. CEUR Workshop Proceedings, 284, 01 2007.
8.
Wroblewski D. and et al. Tokamak disruption alarm based on neural network model of high-beta limit. Nuclear Fusion, 37(725), 11 1997.
9.
Prafulla Dhariwal, Heewoo Jun, Christine Payne, Jong Wook Kim, Alec Radford, and Ilya Sutskever. Jukebox: A generative model for music, 2020.
10.
Dheeru Dua and Casey Graff. UCI machine learning repository, 2017.
11.
S. Dunbar. Stochastic Processes and Advanced Mathematical Finance Brief History of Mathematical Finance Rating Everyone. In Semantic Scholar, 2015.
12.
Pablo Ramon Escobal. Methods of Orbit Determination. Krieger Publishing Company, 1965.
13.
David Foster. Generative Deep Learning. O’Reilly Media, Inc., June 2019.
14.
David E. Goldberg. Genetic Algorithms in Search, Optimization, and Machine Learning. Addison-Wesley, 1988.
15.
S. Haykin. Neural Networks. Prentice-Hall, 1999.zbMATH
16.
Cheng-Zhi Anna Huang, Ashish Vaswani, Jakob Uszkoreit, Noam Shazeer, Ian Simon, Curtis Hawthorne, Andrew M. Dai, Matthew D. Hoffman, Monica Dinculescu, and Douglas Eck. MUSIC TRANSFORMER:GENERATING MUSIC WITH LONG-TERM STRUCTURE. arKiv, 2019.
17.
Guang-Bin Huang, Qin-Yu Zhu, and Chee-Kheong Siew. Extreme learning machine: Theory and applications. Neurocomputing, 70(1):489–501, 2006. Neural Networks.
18.
P. Jackson. Introduction to Expert Systems, Third Edition. Addison-Wesley, 1999.
19.
Julian Kates-Harbeck, Alexey Svyatkovskiy, and William Tang. Predicting disruptive instabilities in controlled fusion plasmas through deep learning. Nature, 568:526–531, April 2019.Crossref
20.
Diederik P. Kingma and Jimmy Lei Ba. ADAM: A METHOD FOR STOCHASTIC OPTIMIZATION. In ICLR 2015, 2015.
21.
Daniel S. Kolosa. A Reinforcement Learning Approach to Spacecraft Trajectory Optimization. Technical Report Dissertations 3542, Western Michigan University, 2019.
22.
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. ImageNet Classification with Deep Convolutional Neural Networks. Communications of the ACM, 60(6), 2017.
23.
Y. Liang and JET EFDA Contributors. Overview of Edge Localized Modes Control in Tokamak Plasma. Technical Report Preprint of Paper for Fusion Science and Technology, JET-EFDA, 2017.
24.
Alan J Lockett and Risto Miikkulainen. Temporal Convolution Machines for Sequence Learning. Technical Report AI-09-04, Department of Computer Sciences, the University of Texas at Austin, 2009.
25.
Lopez. RNN, LSTM & GRU. dProgrammer Lopez, April 2019.
26.
Jere Schenck Meserole. Detection Filters for Fault-Tolerant Control of Turbofan Engines. Phd, Massachusetts Institute of Technology, 1981.
27.
Microsoft. sentence-completion. https://drive.google.com/drive/folders/0B5eGOMdyHn2mWDYtQzlQeGNKa2s, 2019.
28.
Aaron van den Oord, Sander Dieleman, Heiga Zen, Karen Simonyan, Oriol Vinyals, Alex Graves, Nal Kalchbrenner, Andrew Senior, and Koray Kavukcuoglu. Wavenet: A generative model for raw audio, 2016. cite arxiv:1609.03499.
29.
M. Paluszek, Y. Razin, G. Pajer, J. Mueller, and S.Thomas. Spacecraft Attitude and Orbit Control: Third Edition. Princeton Satellite Systems, 2019.
30.
Michael Phi. Illustrated Guide to LSTM’s and GRU’s: A step by step explanation. Towards Data Science, September 2018.
31.
G.A. Ratta, J..Vega, A. Murari, the EUROfusion MSTTeam, and JET Contributors. AUG-JET cross-tokamak disruption predictor. In 2nd IAEA TM, 2017.
32.
L.M. Rasdi Rere, Mohamad Ivan Fanany, and Aniati MurniA rymurthy. Simulated annealing algorithm for deep learning. Procedia Computer Science, 72:137–144, 2015.
33.
Joseph Rocca. Understanding Variational Autoencoders (VAEs). Towards Data Science, September 2019.
34.
Elizabeth Rosenthal. Artificial Intelligence Approach Points to Bright Future for Fusion Energy. Oak Ridge National Laboratory, 2019.
35.
S. Russell and P. Norvig. Artificial Intelligence A Modern Approach Third Edition. Prentice-Hall, 2010.zbMATH
36.
Paul A. Samuelson. Mathematics of speculative price. SIAM Review, 15(1):1–42, 1973.MathSciNetCrossref
37.
R.O. Sayer, Y.K.M. Peng, J.C. Wesley, S.C. Jardin, CA General Atomics, San Diego, and NJ Princeton Univ. ITER disruption modeling using TSC (Tokamak Simulation Code). Technical report, Oak Ridge National Laboratory, 11 1989.
38.
Luigi. Scibile. Non-linear control of the plasma vertical position in a tokamak. PhD thesis, University of Oxford, 1997.
39.
Richard Socher. Recursive Deep Learning for Natural Language Processing and Computer Vision. PhD thesis, Stanford University, August 2014.
40.
Russell Stewart. Maximum likelihood decoding with rnns - the good, the bad, and the ugly, 2016.
41.
Stephanie Thomas and Michael Paluszek. MATLAB Machine Learning. Apress, 2017.zbMATH
42.
Stephanie Thomas and Michael Paluszek. MATLAB Machine Learning Recipes: A Problem-Solution Approach. Apress, 2019.zbMATH
43.
P. Toiviainen and T. Eerola. MIDI toolbox 1.1. https://github.com/miditoolbox/, 2016.
44.
Phillip Wang, 2019.
45.
W. E. Wiesel. Spaceflight Dynamics. McGraw-Hill, 1988.
46.
Geoffrey Zweig and Chris J.C. Burges. The microsoft research sentence completion challenge. Technical Report MSR-TR-2011-129, Microsoft, December 2011.

Index

Activation functions4–8, 10–12, 36, 44, 45, 87

“Adam” method216

Adaptive moment estimation (ADAM)55

Aerodynamic model175, 291, 292

Aircraft model290

AircraftSim.m.178–179

Aircraft simulations180, 294–297

Air turbine75–79, 81–85

AlexNet225–231

Algorithmic Deep Learning Neural Network (ADLNN)22

AirTurbineSim.m77

algorithmic filter/estimator78

deep learning system75

detection filter (see Detection filter)

dynamical equations77

numerical models76

pressure regulator75

RHSAirTurbine.m77

training75

turbine angular velocity78

Amazon Web Services (AWS)3

Artificial terrain model182

Backpropagation routine11

Ballerina.obj file133, 136

Barycenter235

Batch normalization layer44

Bidirectional Long Short-Term Memory (biLSTM)106, 165, 169, 222, 249

BluetoothTest.m122–124

Built-in demo297

Camera model188–192

cascadeforwardnet24, 245, 246

Central solenoid field coils89, 90

Classification criteria108

classificationLayer47–48

Classification methods26

Clustering26

Completing sentences, see Sentence completion

Computer Vision Toolbox27

ConicSection.m236–237

Consumption89

ControlSim.m104, 107

Convolutional neural networks (CNNs) ,1718, 225

AlexNet225, 227, 229

batch normalization layer44

classificationLayer47–48

convolution2dLayer42–44

fullyConnectedLayer46

image identification198

imageInputLayer42

maxPooling2dLayer46

reluLayer44–46

softmaxLayer47

structuring48

convolution2dLayer42–44

CreateEllipses.m52, 53

Critic network314

Cross Channel Normalization226, 227

Cross-entropy47

Cross-entropy loss47

DancerNN.m141, 142

Dancer orientation136

Data acquisition GUI145–152

Data acquisition system138

DataFromIMU121, 122

Data preprocessing functions280

Daylight detector7–9

Deep Deterministic Policy Gradient (DDPG) algorithm308

Deep learning (DL) . See also DL networks272

applications20–21

data16–17

definition1

history2–3

machine learning1

neural networks1, 2, 113

systems2

Deep Learning Toolbox15, 22, 24, 26

Default data structure177

Detection filter

function (see DetectionFilter function)

gain matrix80

linear model79

low-pass filter82

nonfailures84

pressure regulator and tachometer failures79

time constant80

DetectionFilter function

boolean logic84, 85

code81

data structure d80

detecting failures, air turbine81

DetectionFilterSim.m script82

faults characterization85

feedforwardnet86

function signature80

MATLAB script82

residuals85, 86

scale factors uF and tachF82

string, classifier labels86

training GUI87

varargin80

Diamagnetic energy90

Disruption89

DL networks

CNN18

ELMs19

generative deep learning20

GUI with hidden layers35, 36

LSTM19

multi-layer18

Recursive deep learning19

recursive neural networks19

reinforcement learning20

RNNs18

stacked autoencoder19

TCM19

Domain-specific toolboxes25

Double pirouettes113, 129, 141

DrawEllipticOrbit239

DrawTokamak89

Dynamical models75, 116

Earth sensors

attitude geometry251, 252

linear output253–256

results267

scanning251

segmented Earth sensor256–259

segmented sensor neural network263–267

static Earth sensors251, 253, 255, 256, 260

using a neural network259–263

Edge Localized Mode (ELM)91

EllipsesNeuralNetLeaky.m58

EllipsesNeuralNet.m54

EllipsesNeuralNetOneLayer.m59

Elliptical orbit237–239, 250

Epochs31, 110

Euler’s equation116, 118

Extreme Learning Machine (ELM)19

Face identification26, 199

“Feature axes”285

Filter groups226

Finding circles

classification problem41

image data generation49–53

structure (see Convolutional neural networks (CNNs))

training and testing54–61

fmincon function301, 302

fminsearch177, 178, 298

“Forget gate”272, 274

Freezing function283

FullyConnectedLayer46, 47

GarageBand application280–281

Gated recurrent unit (GRU)284

GenerateEllipses.m49

Generative deep learning20, 157

Generative machine learning (ML) models269

Generative models269–270

Geometric Brownian Motion209

GoogLeNet230–233

Gouraud135

Grammar156–157

Graphical Processing Units (GPUs)27

“Greedy” sampling284

Grouped convolution227

GUI Algorithms29

GUIPlots130–133, 146, 149

GUI progress29

Gulfstream178, 179

Handwriting analyis20

Hessian matrix37

Hidden Markov Models3

Hyperbolic activation function11

Ideal orbits235

Image Acquisition Toolbox27

Image classification23

AlexNet for image classification225–231

classification networks225

GoogLeNet230–233

network layers printout226

peppers227–228

test images228–229

imageInputLayer42

ImageNet Dataset226

ImageNet Large-Scale Visual Recognition Challenge (ILSVRC)230

Image recognition20

Independent coefficients15

Inertial Measurement Unit (IMU)115

InitialLearnRate54

InitialObservation method309

Instrument Control toolboxes25, 26

International Tokamak Experimental Reactor (ITER)89, 111

Jazz286

Kalman filter204, 207

Keplerian elements239

Keplerian propagation235, 240

Learnables60

Level flight simulations298–300

Levenberg-Marquardt training algorithm37

Lift coefficient175

Lift-to-drag ratio291

Linear activation function87

Linear output Earth sensor253–256

Linear output sensor neural network,259–263

Linear quadratic controller107

Long Short-Term Memory (LSTM)19, 106, 272–274

fullyConnectedLayer (outputSize)219

generate new music278–279

layer structure218

ltmLayer (numHiddenUnits)219

NNEarthSensor.m259–262

in orbit determination247–250

regressionLayer219

sampling278

sequenceInputLayer (inputSize)219

set training parameters277–278

stock prediction215

training window220

LPMS-B2, 115, 137, 153

Machine learning1, 2, 15, 25, 28, 269, 289

Machine translation3, 20

Magnetohydrodynamic (MHD)90

MathWorks products

Computer Vision Toolbox27

Deep Learning Toolbox26

Image Acquisition Toolbox27

Instrument Control Toolbox26

machine learning25

Parallel Computing Toolbox27

Statistics and Machine Learning Toolbox26

Text Analytics Toolbox27

MATLAB-based deep learning tools3

MATLAB bluetooth function119

MATLAB GUI29

MATLAB open source tools27

MATLAB’s pattern recognition network, patternnet63

maxPooling2dLayer46

Mean dynamic pressure302

Mean heating rate303

Mean stagnation temperature303

MIDI dataset272

Model output probabilities283

Model training curves282

Modern pop song286

Movie database generation

CreateMovieDatabase.m63

Excel and text files64

function demo65

MPAA ratings63

randn64

str2double64

Multi-layer neural networks3, 18

“Multi-output” model275

Musical event272

Music generation270–271

Music production282

N-dimensional latent space286

Network training histogram31, 33

Network training performance31

Network training regression31

Network training state32

Neural net training144

Neural networks

activation functions4–6

daylight detector7–8

linear function6, 7

machine intelligence4

multi-layer4

threshold logic1

XOR problem8–16

NNEarthSensor.m259–262

NNSegmentedEarthSensor.m264, 265

Non-uniform time distribution305

Numerical disruptions model

controller95–97

disturbances94–95

dynamics91–94

sensors94

Object recognition26

Optimal landing307

Optimal trajectory300–307

Orbit determination

algorithmic approaches250

elliptical orbit237–239, 250

ideal orbits235

LSTM implementation247–250

orbits generation

barycenter235

conic section236–237

Keplerian elements239

last test orbit242

neural network244

RHSOrbit241

training and testing243–247

OrbitLSTM.m.247, 249

OrbitNeuralNet243, 244

Orbit parameter239

Orbit period239

Orbits.m241

Oswald efficiency factor175, 291

Output space275

Parallel Computing Toolbox27

Pattern recognition157

Perceptrons (book)2

picsum.photos232, 233

Pirouette

classification140–144

dancer simulation126–130

dancer with the sensor belt138

data acquisition118–124

data acquisition GUI145–152

elastic belt manufacturing137

hardware sources153

IMU115, 137–138

measurements113

orientation124–126

physics116–118

quaternion display133–136

real-time plotting130–133

stages, dancers doing pirouettes113, 114

testing, data acquisition system138–140

trained network113, 115

Plasma dynamics model

DefaultDataStructure function98

first-order lag99

JET numbers100

MATLAB function98

RHSTokamak99

warnings99

Plasma internal inductance90

Plasma with ELM disturbances

DisruptionSim.m101

eigenvalues102

magnitude value102

open-loop simulation101

tRep102

PlotSet13

PlotStock.m212

Pooling layers44

predictAndUpdateState219

PrepareSequences163

Propulsion system289

Quadratic error11

QuaternionToMatrix125, 126

QuaternionVisualization134, 136, 149, 151

Raw audio271

Recurrent neural networks (RNNs)18, 273

Recursive deep learning19

Recursive/online training111

Regression formulation275

Regression methods26

Reinforcement learning20, 289

critic network314

time history316

training window315

reluLayer44–46

ReLU problem45

rgb2gray51

RHSDancer126–128

RHS2DTitan function299

RHS2DTitan models294–297

RHSOrbit240–242

RHSPointMassAircraft176–179

rlDDPGAgent311

Root-mean-squared error (RMSE)219

Root mean square propagation (RMSProp)55

Satellites251

Scanning251

Segmented Earth sensor256–259

Segmented sensor neural network263–267

Sensors251

Sentence completion

conversion, sentences163–164

deep learning system155

grammar156–157

mapping sentences to numbers161–162

numeric dictionary160–161

by pattern recognition157

ReadDatabase.m158–159

sentence generation157

training and testing

NLP172

SentenceCompletionNN- Fitted.m165–166

SentenceCompletionNN.m169–171

testing code168

training progress168

Simulation2DTitan function303

“Single-output” model275

softmaxLayer47, 48

Speech recognition20

splitEachLabel54

Stacked autoencoder19

Static Earth sensors251–256, 259, 260

Statistical neural networks17

Statistics and Machine Learning Toolbox26

Step method309

Stochastic gradient descent with momentum (sgdm)55

Stock market model209

StockMarketNeuralNet.m215, 216

Stock prediction23

artificial stock market, creation209–212

BiLSTM set221, 222

creating a stock market212–214

LSTM layer215–221

PlotStock.m212

stochastic differential equation209

stock market with a hundred stocks214

StockPrice210–214

Support-vector machines (SVM)3, 26

tansig33

Targeting20

Temporal Convolutional Machine (TCM)19

Temporal convolutional neural networks (TCNs)284

TensorFlow3

Terrain-based navigation23, 289

close-up terrain186–188

CNN, training and testing198–203

default data structure177

generating terrain182–186

lift coefficient175

numerical integration function176

plotting the trajectory192–195

point-mass aircraft equations of motion173, 174

simulation203–207

simulation outputs179, 180

three-dimensional aircraft model173

training images, creation195–198

velocity vector173, 174

Terrain tiles182, 186

Text Analytics Toolbox27

Three-dimensional aircraft model173

Time history317

Titan atmosphere292–294

Titan landing control system289

aircraft model290

composite infrared image290

optimal trajectory300–307

reinforcement (see Reinforcement learning)

simulating level flight298–300

simulating the aircraft294–297

Tokamak disruption detection22

dynamical model (see Plasma dynamics model)

factors89

MHD instabilities90

numerical model (see Numerical disruptions model)

plasma control104–106

plasma stimulation101–103, 106

training and testing106–111

TokamakNeuralNet.m107

Tokamaks89

Toolbox functions26

Training function12

Training GUI87, 88, 144

training-progress54

trainNetwork23, 55, 110, 143, 166, 169, 248

Transformers285

“Truth” data49

D latent space2286

uicontrol145, 149, 150

V, W

ValidationData54

ValidationFrequency54

Variational autoencoders (VAEs)285, 287

Velocity vector173–175, 292

Viewer database generation

built-in function demo68

cell array66

CreateMovieViewers.m66

deep learning algorithm68

Gaussian/normal probability67

loop66

movies characteristics69

patternnet71–73

predictions74

probabilities66

script MovieNN.m68

test set72

turkeys66

Visualization tools25

X, Y

XORDemo.m13

XOR example28

XOR function10

Zermelo’s problem38–40

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.

Table of Contents for Back Matter

Create new playlist

Sign In

Sign Up

Table of Contents for
Back Matter