Data Science Stack Exchange
Data Science Stack Exchange

Questions tagged [convergence]

43 questions
28
votes
3 answers

local minima vs saddle points in deep learning

I heard Andrew Ng (in a video I unfortunately can't find anymore) talk about how the understanding of local minima in deep learning problems has changed in the sense that they are now regarded as less problematic because in high-dimensional spaces…
oW_
  • 6,502
  • 4
  • 29
  • 47
17
votes
5 answers

Number of epochs in Gensim Word2Vec implementation

There's an iter parameter in the gensim Word2Vec implementation class gensim.models.word2vec.Word2Vec(sentences=None, size=100, alpha=0.025, window=5, min_count=5, max_vocab_size=None, sample=0, seed=1, workers=1, min_alpha=0.0001, sg=1, hs=1,…
alvas
  • 2,510
  • 7
  • 28
  • 40
5
votes
2 answers

Logistic regression does cannot converge without poor model performance

I have a multi-class classification logistic regression model. Using a very basic sklearn pipeline I am taking in cleansed text descriptions of an object and classifying said object into a category. logreg = Pipeline([('vect', CountVectorizer()), …
jasper
  • 73
  • 1
  • 1
  • 6
4
votes
1 answer

DQN fails to find optimal policy

Based on DeepMind publication, I've recreated the environment and I am trying to make the DQN find and converge to an optimal policy. The task of an agent is to learn how to sustainably collect apples (objects), with the regrowth of the apples…
macwiatrak
  • 41
  • 4
3
votes
0 answers

Force Matching in Coarse Grained Molecular Dynamics with Jax - Forces do not match when neglecting energy loss

I am currently exploring force matching approaches for molecular dynamic simulations. As I am still in an exploration state, I'd tried investigated Force Matching Neural Network Colab Notebook corresponding to Unveiling the predictive power of…
not_converging
  • 31
  • 4
3
votes
1 answer

ElasticNet Convergence odd behavior

I am optimizing a model using ElasticNet, but am getting some odd behavior. When I set the tolerance hyperparameter with a small value, I get "ConvergenceWarning: Objective did not converge" errors. So I tried a larger tolerance value, and the…
mr_python
  • 31
  • 2
3
votes
4 answers

Weights not converging while cost function has converged in neural networks

My cost/loss function drops drastically and approaches 0, which looks a sign of convergence. But the weights are still changing in a visible way, a lot faster than the cost function. Should I ensure the weights converge too? Some details: I just…
feynman
  • 237
  • 1
  • 8
3
votes
1 answer

Convergence of vanilla or natural policy gradients (e.g. REINFORCE)

I am reading in a lot of places that policy gradients, especially vanilla and natural, are at least guaranteed to converge to a local optimum (see, e.g., pg. 2 of Policy Gradient Methods for Robotics by Peters and Schaal), though the convergence…
user650261
  • 81
  • 1
  • 5
3
votes
2 answers

Normalizing the final weights vector in the upper bound on the Perceptron's convergence

The convergence of the "simple" perceptron says that: $$k\leqslant \left ( \frac{R\left \| \bar{\theta} \right \|}{\gamma } \right )^{2}$$ where $k$ is the number of iterations (in which the weights are updated), $R$ is the maximum distance of a…
Qwerto
  • 705
  • 1
  • 8
  • 15
3
votes
1 answer

need explanation on how an equation is being converted to cvxopt logic in solver.lq

This is the equation that is given in the example: and the code to replicate it in python is >>> from cvxopt import matrix, solvers >>> A = matrix([ [-1.0, -1.0, 0.0, 1.0], [1.0, -1.0, -1.0, -2.0] ]) >>> b = matrix([ 1.0, -2.0, 0.0, 4.0 ]) >>> c =…
Sourav Roy
  • 141
  • 2
3
votes
1 answer

Is the percepetron algorithm's convergence dependent on the linearity of the data?

Does the fact that I have linearly separable data or not impact the convergence of the perceptron algorithm? Is it always gonna converge if the data is linearly separable and not if it is not ? Is there a general rule ?
astudentofmaths
  • 273
  • 1
  • 4
  • 8
2
votes
1 answer

Rate of convergence - comparison of supervised ML methods

I am working on a project with sparse labelled datasets, and am looking for references regarding the rate of convergence of different supervised ML techniques with respect to dataset size. I know that in general boosting algorithms, and other models…
Avatrin
  • 131
  • 7
2
votes
1 answer

Does convergence equal learning in Deep Q-learning?

In my current research project I'm using the Deep Q-learning algorithm. The setup is as follows: I'm training the model (using Deep Q-learning) on a static dataset made up of experiences extracted from N levels of a given game. Then, I want to use…
Aeryan
  • 81
  • 1
  • 6
2
votes
0 answers

Is there a universal convergence rate when stacking models/experts?

It's fairly common to see people stacking different models when chasing marginal gains in contexts such as Kaggle competitions or the Netflix challenge. I would like to know about the mathematics behind it. When stacking experts, under the…
Learning is a mess
  • 646
  • 1
  • 8
  • 16
2
votes
1 answer

How can we conclude that an optimization algorithm is better than another one

When we test a new optimization algorithm, what the process that we need to do?For example, do we need to run the algorithm several times, and pick a best performance,i.e., in terms of accuracy, f1 score .etc, and do the same for an old optimization…
user82620
  • 21
  • 1
1
2 3