Passa ai contenuti principali

Post

Visualizzazione dei post con l'etichetta text analysis with R stopwords analysing philosophical text R analysis literature

about stopwords, in machine life and in literature

  The practical part of introductions to Text Analysis with R usually starts to the sound of the hunting horn. "Tokenization" is the parole, and "elimination" the first goal. The analyst does not care about punctuation (remove_punct =TRUE)! The analyst has to reduce the burden of his word sack, and there he or she notices the "stopwords", functional words that connect and move ideas as words .. "then" and "when", "how" and "because". Any contents? No! Throw them away! It might be true that programs for Data Analysis have been developed for, and are mostly used by marketing experts who do not really care about moving ideas, and usually end up with some "sentiment analysis": Feelin good? Ok! It is also true that in AI, things like Natural Language Understanding, caution is the dominant rule. Keep the stopwords, you never know! There are good reasons for doing so. AI should include every word that touches huma...