Get the possible next tokens and their log probabilities its previous context using a causal transformerSource:
Get the possible next tokens and their log probabilities based on its previous context using a causal transformer model from Hugging Face.
model = getOption("pangoling.causal.default"),
checkpoint = NULL,
add_special_tokens = NULL,
config_model = NULL,
config_tokenizer = NULL
Name of a pre-trained model or folder.
Folder of a checkpoint.
Whether to include special tokens. It has the same default as the AutoTokenizer method in Python.
List with other arguments that control how the model from Hugging Face is accessed.
List with other arguments that control how the tokenizer from Hugging Face is accessed.
A causal language model (also called GPT-like, auto-regressive, or decoder model) is a type of large language model usually used for text-generation that can predict the next word (or more accurately in fact token) based on a preceding context.
If not specified, the causal model that will be used is the one set in
specified in the global option
pangoling.causal.default, this can be
getOption("pangoling.causal.default") (by default
"gpt2"). To change the default option
options(pangoling.causal.default = "newcausalmodel").
A list of possible causal models can be found in Hugging Face website.
config_tokenizer arguments, it's possible to
control how the model and tokenizer from Hugging Face is accessed, see the
In case of errors when a new model is run, check the status of https://status.huggingface.co/
See the online article in pangoling website for more examples.