Qualitaetsmanagement

Aktuelle Stellenangebote und Stellengesuche für Qualitätsingenieure Qualitätstechniker Qualitätsmanager Qualitätsbeauftragte usw.

Eintrag vom: 16.10.2013.

Why does larger perplexity tend to produce clearer clusters in t-SNE? By reading the original paper I learned that the perplexity in t-SNE is 2 2 $2$ to the power of Shannon entropy of the conditional distribution induced by a data point.

 

I will cite the FAQ from t-SNE website. First for perplexity: How should I set the perplexity in t-SNE? The performance of t-SNE is fairly robust under different settings of the perplexity. The most appropriate value depends on the density of your data. Loosely speaking one could say that a larger / denser dataset requires a larger perplexity.

 

??????????????????????????????????????????????????????????????? \ {e_1\cdots e_ {i-1}\} ??????????? e_i ?? perplexity ????????????

 

I came across term perplexity which refers to the log-averaged inverse probability on unseen data. Wikipedia article on perplexity does not give an intuitive meaning for the same. This perplexity

 

????????Google ?deepreasearch???perplexity???????Grok?deepersearch?????perplexity????????perplexity?? ??????????grok?deeper search????2.5?????????40? ????????????perplexity????

 

????????????????????????k??????perplexity??? ????????????????????????? (????k????)??????k????80?100?150?????????????????

 

You don't since correlation does not work for categorical variables you have to do something else with those t-tests and such.

 

And this is the perplexity of the corpus to the number of words. If you feel uncomfortable with the log identities Google for a list of logarithmic identities. For further reading: NGrams (pdf).

 

Perplexity?????????????????????AI???? ??????????????????????Pro????? ???? AI?????????????? ??????????????????????????

 

?????python????? 1??gensim?log_perplexity???????????perp1?Blei????perplexity???????????????perp2? 2? ?????????5 10 15?????LDA?????list? ????perp1?perp2? 3????