prokbert.prokbert_tokenizer.ProkBERTTokenizer.convert_ids_to_tokens

ProkBERTTokenizer.convert_ids_to_tokens(ids: Union[int, List[int], Tensor]) → Union[str, List[str]]

Converts token IDs back to their original tokens. This function can handle a single ID or a list of IDs. It also supports handling IDs provided as a PyTorch tensor.

Parameters

ids (Union[int, List[int], torch.Tensor]) – A single token ID or a list of token IDs. Can also be a PyTorch tensor of token IDs.

Returns

The corresponding token or list of tokens. If ids is a single integer or a tensor with a single value, a single token string is returned. If ids is a list or tensor with multiple values, a list of token strings is returned.

Return type

Union[str, List[str]]

Usage Example:

>>> tokenizer = ProkBERTTokenizer()
>>> ids = [213, 3343]
>>> tokens = tokenizer.convert_ids_to_tokens(ids)
>>> print(tokens)
...