When we talk about Machine Learning and Deep Learning, terms that we frequently come across is linear and non linear functions.
Although, we would have studied this in our high school math, there will be at least a handful of them like me who would like to brush up on these mathematical terminologies.
This blog talks about two different groups of functions: linear and nonlinear. We’ll cover how they look on a graph, and how you can tell them apart when they’re written as equations.
Linear Function
Algebraic tools allow us to express functional relationships very efficiently; to find the value of one thing (such as the gas price) when we know the value of the other (say number of gallons); and display a relationship visually in a way that allows us to quickly grasp the direction, magnitude, and rate of change in one variable over a range of values of the other.
For simple problems such as determining gas prices, existing knowledge of multiplication will usually allow us to calculate the cost for a specific amount of gas, once we have the price per gallon (say $2). We can then easily infer 2 gallons cost $4, 3 gallons cost $6, 4 gallons cost $8, and so on.
While we can list each set of values, it is very efficient to say that for all values in gallons (which we call x by convention), the total cost (which we call y by convention), is equal to 2x.
Writing y = 2x is a simple way of saying a great deal. This is nothing but a linear function, where the dependent variable is proportional to the independent one.
Linear means something having to do with line. Linear function is a function where the graph is a straight line. The line can go in any direction, but it’s always a straight line.
Non Linear Function
A non-linear function on the other hand involves variables not following such a simple proportionality relation in real life.
For Example — the weight of a person changes over a period of time, a person might gain 3kg over a period of 6 months and then 1 kg. So the change in weight over time duration is not constant.
Also, non linear function has a shape that is not a straight line and, depending on the function, can have many different appearances including a U-shape or an S-shape.
Easy? But why are some functions straight lines, while other functions aren’t?
Well, the important difference between linear and non-linear is in the exponents. The linear functions do not have exponents higher than 1, i.e they are never raised to the power greater than 1.
When a linear function is written in its simplest form, it looks like y = bx, where b is constant.
For example, here are some linear functions:
y = 3 + 5x
y = 2–6x
On the other hand, non linear functions have at least one variable raised to the power of two or more.
Polynomial Function
Both these linear and non linear functions fall under a wider class of functions call Polynomial Functions.
Polynomial functions (just say “polynomials”) are functions of a single independent variable, in which that variable can appear more than once, raised to any integer power.
Below are few polynomial functions and the terms used to describe them:
Linear polynomial (Linear Function) — with a degree 1
Quadratic — with 2nd degree
Cubic — 3rd degree
Quartic — 4th degree
Quintic — with only 5th degree and a constant term
The appearance of the graph of a polynomial is largely determined by the leading term — it’s exponent and its coefficient. Because the leading term has the largest power, its size outgrows that of all other terms as the value of the independent variable grows.
For example, in f(x)=8×4−4×3+3×2−2x+22 as x grows, the term 8×4 dominates all other terms.
In Machine Learning, since the data in real world is generally not linear in nature, one common pattern is to use linear models trained on nonlinear functions of the data. This approach maintains the generally fast performance of linear methods while allowing them to fit a much wider range of data.
Polynomial regression is a special case of linear regression. A polynomial term: a quadratic (squared) or cubic (cubed) term turns a linear regression model into a curve. But because it is the data X that is squared or cubed, not the Beta coefficient, it still qualifies as a linear model.
This makes it a nice and straightforward way to model curves without having to model complicated nonlinear models.
Conclusion
In this post we discussed, two different groups of functions: linear and nonlinear. Also covered how they look on a graph, and how we can tell them apart when they’re written as equations. We also touched upon various kinds of polynomials and their anatomy.
Hope this was helpful.
Also read the blog on Slope and derivatives:
Divya's Blog 