CLARIN taxonomy for tools and services tasks

Cerca nel vocabolario

Lingua dei contenuti

Concept information

analysis > linguistic analysis > orthographic processing > tokenisation
> annotation > linguistic annotation > orthographic processing > tokenisation

Termine preferito

tokenisation  

Definizione

  • turning a sequence of characters (a text) into a sequence of occurrences of inflected word forms and interpunction symbols (often called 'tokens')

Concetto più generale

URI

https://vocabs.sshopencloud.eu/vocabularies/csd-tool-tasks/0102

Scarica questo concetto

RDF/XML TURTLE JSON-LD Ultima modifica 02/01/24