Data Science Stack Exchange
Data Science Stack Exchange

Questions tagged [cosine-distance]

A measure of the angular distance between two vectors. Usually defined as 1-(cosine similarity).

85 questions
17
votes
4 answers

Alternatives to TF-IDF and Cosine Similarity when comparing documents of differing formats

I've been working on a small, personal project which takes a user's job skills and suggests the most ideal career for them based on those skills. I use a database of job listings to achieve this. At the moment, the code works as follows: 1) Process…
Richard Knoche
  • 171
  • 1
  • 1
  • 3
13
votes
2 answers

cosine_similarity returns matrix instead of single value

I am using below code to compute cosine similarity between the 2 vectors. It returns a matrix instead of a single value 0.8660254. [[ 1. 0.8660254] [ 0.8660254 1. ]] from sklearn.metrics.pairwise import cosine_similarity vec1 =…
Olivia Brown
  • 233
  • 1
  • 2
  • 4
13
votes
4 answers

Can I use cosine similarity as a distance metric in a KNN algorithm

Most discussions of KNN mention Euclidean,Manhattan and Hamming distances, but they dont mention cosine similarity metric. Is there a reason for this?
Victor
  • 651
  • 3
  • 8
  • 20
11
votes
5 answers

Cosine similarity vs The Levenshtein distance

I wanted to know what is the difference between them and in what situations they work best? As per my understanding: Cosine similarity is a measure of similarity between two non-zero vectors of an inner product space that measures the cosine of the…
Pluviophile
  • 4,203
  • 14
  • 32
  • 56
11
votes
1 answer

Calculate cosine similarity in Apache Spark

I have a DataFrame with IDF of certain words computed. For example (10,[0,1,2,3,4,5],[0.413734499590671,0.4244680552337798,0.4761400657781007, 1.4004620708967006,0.37876590175292424,0.48374466516332]) .... and so on Now give a query Q, I can…
Ganesh Krishnan
  • 243
  • 1
  • 2
  • 6
9
votes
1 answer

Why is the cosine distance used to measure the similatiry between word embeddings?

While computing the similarity between the words, cosine similarity or distance is computed on word vectors. Why aren't other distance metrics such as Euclidean distance suitable for this task. Let us consider 2 vectors a and b. Where, a = [-1,2,-3]…
Ashwin Geet D'Sa
  • 1,217
  • 2
  • 11
  • 20
8
votes
1 answer

Cosine Distance > 1 in scipy

I am working on a recommendation engine, and I have chosen to use SciPy's cosine distance as a way of comparing items. I have two vectors: a = [2.7654870801855078, 0.35995355443076027, 0.016221679989074141, -0.012664358453398751,…
redgem
  • 183
  • 1
  • 1
  • 4
8
votes
2 answers

What should be the value of non-rated field when finding cosine similarity

I am working on a very basic book recommender system. I want to know what to do with the fields which aren't rated by the user when finding cosine similarity, should we ignore them and calculate only with the rated fields or should we mark them…
divyum
  • 181
  • 2
6
votes
1 answer

Evaluating the performance of a machine learned recommendation system

I have a set of resumes $R=\{{r_1,...,r_n\}}$, which I've transformed to a vector space using TF-IDF. Each resume has a label, which is the name of their current employer. Each of these labels comes from the set of possible employers $E =…
Data
  • 467
  • 3
  • 12
5
votes
1 answer

Calculating cosine similarity between 3D arrays using Python

I have two matrices with multiple columns and three rows each. I calculated the cosine similarity (sklearn) but it gives the result as a matrix. How can I obtain one single value? The matrices are the embeddings of two words each, obtained from…
GAYATRI VENUGOPAL
  • 51
  • 1
  • 2
5
votes
1 answer

Cosine similarity between sentence embeddings is always positive

I have a list of documents and I am looking for a) duplicates; b) documents that are very similar. To do so, I proceed as follows: Embed the documents using paraphrase-xlm-r-multilingual-v1. Calculate the cosine similarity between the vector…
user4904
4
votes
4 answers

How to find similarity/distance matrix with mixed Continuous and Categorical data?

Say I have a dataset like this: Hotel HasPool AvgPrice 1 1 $123 2 0 $234 3 1 $200 Currently I have broken down the dataset into 2 (one containing all continuous, other all categorical). The continuous…
UD1989
  • 258
  • 2
  • 3
  • 6
4
votes
1 answer

word2vec word embeddings creates very distant vectors, closest cosine similarity is still very far, only 0.7

I started using gensim's FastText to create word embeddings on a large corpus of a specialized domain (after finding that existing open source embeddings are not performing well on this domain), although I'm not using its character level n-grams, so…
Oren Matar
  • 221
  • 1
  • 7
4
votes
1 answer

Hierarchical clustering with precomputed cosine similarity matrix using scikit learn produces error

We want to use cosine similarity with hierarchical clustering and we have cosine similarities already calculated. In the sklearn.cluster.AgglomerativeClustering documentation it says: A distance matrix (instead of a similarity matrix) is needed as…
M. Kaan
  • 43
  • 1
  • 1
  • 3
4
votes
3 answers

Cosine similarity with arrays contaning NaN

I am trying to calculate a cosine similarity using Python in order to find similar users basing on ratings they have given to movies. As it can be expected there are a lot of NaN values. I am using movie dataset from Kaggle. When I use np.dot() on…
user641597
  • 143
  • 3
  • 7
1
2 3 4 5 6