tokenizers: word-counting – R documentation

Become an expert in R — Interactive courses, Cheat Sheets, certificates and more!

word-counting

Count words, sentences, characters

Description

Count words, sentences, and characters in input texts. These functions use the stringi package, so they handle the counting of Unicode strings (e.g., characters with diacritical marks) in a way that makes sense to people counting characters.

Usage

count_words(x)

count_characters(x)

count_sentences(x)

Arguments

`x`	A character vector or a list of character vectors. If `x` is a character vector, it can be of any length, and each element will be tokenized separately. If `x` is a list of character vectors, each element of the list should have a length of 1.

Value

An integer vector containing the counted elements. If the input vector or list has names, they will be preserved.

Examples

count_words(mobydick)
count_sentences(mobydick)
count_characters(mobydick)

tokenizers

Fast, Consistent Tokenization of Natural Language Text

v0.2.1

MIT + file LICENSE

Authors

Lincoln Mullen [aut, cre] (<https://orcid.org/0000-0001-5103-6917>), Os Keyes [ctb] (<https://orcid.org/0000-0001-5196-609X>), Dmitriy Selivanov [ctb], Jeffrey Arnold [ctb] (<https://orcid.org/0000-0001-9953-3904>), Kenneth Benoit [ctb] (<https://orcid.org/0000-0002-0797-564X>)

Initial release

2018-03-29