26 views

Uploaded by tunguyenlam

jj

- high voltage engineering
- 282151226 Machine Learning Coursera All Exercies
- Book
- Why Do We Overestimate Others
- POM Reviewer.docx
- 4904224.pdf
- Pol Dec and Corruption
- linear regression project
- DSS - Introduction to Regression
- Lp And RA
- Metaheuristics
- An Introduction to Gradient Descent and Linear Regression.pdf
- spss ppt
- 117544 ID Pengaruh Moralitas Individu Sistem Penge
- Scatter Diagram
- Original
- math requirement
- Kinematic Analysis of the Instep Kick in Youth Soccer Players
- 1-s2.0-S0198971598000489-main
- satellite problem 2

You are on page 1of 36

Ingmar Schuster

Patrick Jhnichen

using slides by Andrew Ng

Institut fr Informatik

Linear Regression

Hypothesis formulation,

hypthesis space

Descent

Using multiple input features

with Linear Regression

Feature Scaling

Nonlinear Regression

derivatives

Linear Regression

Institut fr Informatik

Regression Problem

Linear regression w. gradient descent

x is input (predictor) variable

features in ML-speek

y is output (response) variable

Notation

Square meters

Price in 1000

73

174

146

367

38

69

124

257

...

...

Learning procedure

Hypothesis parameters

Training data

linear regression,

one input variable (univariate)

Learning Algorithm

Size

of flat

Estimated

price

hypothesis

(mapping between

input and output)

Linear regression w. gradient descent

Optimization objective

optimization objective and cost function (often called J)

...

(h(x) close to y for (x,y) in training data)

Cost function often named J

Number

Number of

of

training

training instances

instances

Squaring

Linear regression w. gradient descent

Optimizing Cost

with Gradient Descent

Want to minimize

Keep changing

to reduce

until we end up at minimum

10

Stepwise

Stepwise

descent

descent

towards

towards

minimum

minimum

Derivatives

Derivatives

work

work only

only for

for

few

few parameters

parameters

Gradient descent

partial

partial

derivative

derivative

beware: incremental

update incorrect!

steps

steps become

become smaller

smaller

without

without changing

changing

learning

learning rate

rate

12

convergence

not lead to convergence or to

divergence

Often

13

Checking convergence

correctly if

decreases

with every step

Possible convergence

criterion: converged if

decreases by less than

constant

14

Local Minima

(e.g. J not squared error for regression with only one variable)

Random restart

with different

parameter(s)

15

16

Multiple features

Square Bedrooms Floors Age of building

meters

(years)

x1

x2

x3

x4

Price in

1000

y

200

45

460

131

40

232

142

30

315

756

36

178

Notation

17

Hypothesis representation

More compact

with definition

18

19

20

21

(scales can vary strongly)

Square meters 30 - 400

Bedrooms

1 - 10

Price

80 000

2 000 000

22

Feature Scaling

23

Feature scaling

scale

Z-score conversion

24

Z-Score conversion

Center data on 0

mean

mean // empirical

empirical

expected

expected value

value

(mu)

(mu)

empirical

empirical standard

standard

deviation

deviation (sigma)

(sigma)

25

26

Nonlinear Regression

(by cheap trickery)

27

28

29

30

31

Optimizing cost

using derivatives

32

solve

for all i

33

Gradient Descent

Need to choose

Needs many iterations,

random restarts etc.

Works well for many features

Derivation

No need to choose

No iterations

34

Linear Regression

Hypothesis formulation,

hypthesis space

Descent

Using multiple input features

with Linear Regression

Feature Scaling

Nonlinear Regression

derivatives

35

Pictures

en.wikipedia.org and

de.wikipedia.org

36

- high voltage engineeringUploaded byRadhika Priyadarshini
- 282151226 Machine Learning Coursera All ExerciesUploaded by桂翔
- BookUploaded byPalakshVismay
- Why Do We Overestimate OthersUploaded bysofiaortega
- POM Reviewer.docxUploaded byAndrea Zamora
- 4904224.pdfUploaded bynezutron
- Pol Dec and CorruptionUploaded byBabelHardy
- linear regression projectUploaded byapi-249020821
- DSS - Introduction to RegressionUploaded byBoier Zeu
- Lp And RAUploaded bySunil Wangi
- MetaheuristicsUploaded byzerbazerb
- An Introduction to Gradient Descent and Linear Regression.pdfUploaded byabolfazl
- spss pptUploaded bychandanprakash30
- 117544 ID Pengaruh Moralitas Individu Sistem PengeUploaded bydwi
- Scatter DiagramUploaded byVivek Garg
- OriginalUploaded bymiketol
- math requirementUploaded byapi-318689147
- Kinematic Analysis of the Instep Kick in Youth Soccer PlayersUploaded byBosko
- 1-s2.0-S0198971598000489-mainUploaded byJuanCarlosCarrillo
- satellite problem 2Uploaded byapi-354338441
- Fc 100 200v Statistical CalculationsUploaded byly666
- Shainin NotesUploaded bykdsophie
- 5Ed CCH Forensic Investigative Accounting Ch11Uploaded byAlysha Lil'Oprah Harvey
- 11 - Simple Linear Regression ModelsUploaded bySammy Ben Menahem
- Length-Weight Relationship and Condition Factor of an Parambasis RangaUploaded byDR. BIJAY KALI MAHAPATRA
- Brück agricultura y postconflictoUploaded byDiego Narvaez Medina
- Motivation in Foreign Language Learning- The Relationship BetweenUploaded byAinhoa Gonzalez Martinez
- Multiple Linear RegressionUploaded byngrckr
- Multiple Regression_Statistical Method for PsychologyUploaded byderafik
- Lecture 05 Demand Management and ForecastingUploaded bykpozoi

- Color SegmantationUploaded bypidaceo
- 1.MA6459_NM.pdfUploaded bymmrmathsiubd
- Applications of IntegrationUploaded byAlvin Han
- Transportation ProblemUploaded bysachin_gdgl
- Data Structure mcqsUploaded byFerroz Miland
- MA215-Fall-12-Mid1Uploaded byMohamedKeynan
- 14 (1).pdfUploaded bykulperera
- Eigensolution Technology in LS-DYNAUploaded byAmir Iskandar
- LP TutorialUploaded byIshan Rastogi
- OdeUploaded bymanoj
- Lect_5Uploaded byAshik Mahmud
- Chapter 2 OdeUploaded byclaire
- math1905_l7-8Uploaded byedmond1200
- Quiz 1Uploaded byVineet Khandelwal
- Numerical Methods and Modelling for EngineeringUploaded byNanjunda Swamy
- Alessandro Socionovo Master Thesis DraftUploaded byAnonymous VTwMuepyGO
- unit planUploaded byapi-269488866
- Machine Learning for Classification of Imbalanced Big DataUploaded byEditor IJRITCC
- Fixed Point Theorem for Neutrosophic Triplet Partial Metric SpaceUploaded byMia Amalia
- Calculus Course 2Uploaded byAbhijit Kar Gupta
- Calculus-one variable.pdfUploaded bybhulook
- e 139 ContentUploaded byHassenL
- 02 Tangent PlanesUploaded byyay
- Gauss SeidelUploaded bySulaiman Ahlaken
- School of Mathematics and Statistics Http://Www.maths.usyd.Edu.au/u/UG/JMUploaded bylhwvista
- Mathematics and AlgorithmsUploaded byjlva_lkpb
- Section_01Uploaded byNadeen Chathuranga
- formula_xm f2Uploaded byNurul Binti Yusof
- Worktext in Advanced MathematicsUploaded byCharmaine Cuyno
- Aat Solutions - Ch03Uploaded byTheodoros Maragakis