in Education by
According to some document the weight adjustment formula will be: new weight = old weight + learning rate * delta * df(e)/de * input df(e)/de part is a derivative of activation function which acts like a sigmond function i.e tanh. Actually what is that and why are we multiplying it? Why not (learning rate * delta * input) is enough? Select the correct answer from above options

1 Answer

0 votes
by
 
Best answer
@malika, hope this answer will help you in better understanding. The derivative (df(e)/de) is used by the optimization technique for locating the minima of the loss function. A large value of derivative results in a large adjustment in the corresponding weight. Always remember the larger the derivative, the farther you are from the point of minima and if the value of the derivative is small it means you are near the point of minima. The first derivative denotes a point on the curve such that any line which is tangent to it will have a slope zero. It also tells you whether you are moving in the right direction to reach the function’s minima or not. For example- Suppose you are walking in a 3D surface which is defined by the objective function and you reach a point where the slope is equal to zero, it means that the point is minima for the function. So, if you want to minimize a function you need to follow the derivative. Wish to gain an in-depth knowledge of AI? Check out our Artificial Intelligence Tutorial and gather more insights!

Related questions

0 votes
    I read about the neural network and understood the general principle of single layer neural network. Additional ... function? Select the correct answer from above options...
asked Jan 22, 2022 in Education by JackTerrance
0 votes
    I'm learning the difference between the various machine learning algorithms. I understand that the implementations of ... for that? Select the correct answer from above options...
asked Jan 25, 2022 in Education by JackTerrance
0 votes
    I saw a few examples of a neural network but they work for a fixed set of inputs. How can I deal with ... property of neural networks? Select the correct answer from above options...
asked Jan 22, 2022 in Education by JackTerrance
0 votes
    I'm writing a game that's a variant of Gomoku. Basically a tic tac toe on a huge board. Wondering if anyone ... [self put randomly]; } Select the correct answer from above options...
asked Feb 4, 2022 in Education by JackTerrance
0 votes
    I'm trying to write a program that takes text(article) as input and outputs the polarity of this text, ... open-source implementation. Select the correct answer from above options...
asked Feb 4, 2022 in Education by JackTerrance
0 votes
    Trying to implement OCR in the bank environment but the challenge is, we don't have access to an internet connection ... in our bank. Select the correct answer from above options...
asked Jan 31, 2022 in Education by JackTerrance
0 votes
    I have learned a Machine Learning course using Matlab as a prototyping tool. Since I got addicted to F#, I ... of resources? Thanks. Select the correct answer from above options...
asked Jan 30, 2022 in Education by JackTerrance
0 votes
    Here's a puzzle... I have two databases of the same 50000+ electronic products and I want to match products ... I tackle this problem? Select the correct answer from above options...
asked Jan 29, 2022 in Education by JackTerrance
0 votes
    It is a principal question, regarding the theory of neural networks: Why do we have to normalize the input for ... is not normalized? Select the correct answer from above options...
asked Jan 27, 2022 in Education by JackTerrance
0 votes
    I know the basics of feedforward neural networks, and how to train them using the backpropagation algorithm, but I'm ... , even better. Select the correct answer from above options...
asked Feb 8, 2022 in Education by JackTerrance
0 votes
    I'm looking for some examples of robot/AI programming using Lisp. Are there any good online examples available ... in nature)? Select the correct answer from above options...
asked Feb 4, 2022 in Education by JackTerrance
0 votes
    It always amazed me how the Akinator app could guess a character by asking just several questions. So I wonder ... more about them? Select the correct answer from above options...
asked Feb 4, 2022 in Education by JackTerrance
0 votes
    I'm teaching a kid programming, and am introducing some basic artificial intelligence concepts at the moment. To begin ... and boxes)? Select the correct answer from above options...
asked Feb 4, 2022 in Education by JackTerrance
0 votes
    Nominally a good problem to have, but I'm pretty sure it is because something funny is going on... As ... labeled data vectors/instances (transformed video frames of individuals--...
asked Feb 4, 2022 in Education by JackTerrance
0 votes
    I am searching for information on algorithms to process text sentences or to follow a structure when creating sentences ... be great. Select the correct answer from above options...
asked Feb 4, 2022 in Education by JackTerrance
...