CLARIN taxonomy for tools and services tasks

Zoek in woordenlijst

Content-taal

Conceptinformatie

analysis > linguistic analysis > orthographic processing > tokenisation
> annotation > linguistic annotation > orthographic processing > tokenisation

Voorkeursterm

tokenisation  

Definitie

  • turning a sequence of characters (a text) into a sequence of occurrences of inflected word forms and interpunction symbols (often called 'tokens')

Breder concept

URI

https://vocabs.sshopencloud.eu/vocabularies/csd-tool-tasks/0102

Download dit concept:

RDF/XML TURTLE JSON-LD Laatst gewijzigd 02-01-2024