integrations.liger.models.deepseekv2

integrations.liger.models.deepseekv2

DeepseekV2 model with LigerFusedLinearCrossEntropyLoss

Functions

Name	Description
lce_forward

lce_forward

integrations.liger.models.deepseekv2.lce_forward(
    self,
    input_ids=None,
    attention_mask=None,
    position_ids=None,
    past_key_values=None,
    inputs_embeds=None,
    labels=None,
    use_cache=None,
    output_attentions=None,
    output_hidden_states=None,
    return_dict=None,
)

Parameters

Name	Type	Description	Default
labels	torch.LongTensor of shape (batch_size, sequence_length), optional	Labels for computing the masked language modeling loss. Indices should either be in [0, transformers., config.vocab_size] or -100 (see input_ids docstring). Tokens with indices set to -100 are ignored (masked), the loss is only computed for the tokens with labels in [0, transformers., config.vocab_size].	None

Returns:

Example:

>>> from transformers import AutoTokenizer, DeepseekV2ForCausalLM

>>> model = DeepseekV2ForCausalLM.from_pretrained(PATH_TO_CONVERTED_WEIGHTS)
>>> tokenizer = AutoTokenizer.from_pretrained(PATH_TO_CONVERTED_TOKENIZER)

>>> prompt = "Hey, are you conscious? Can you talk to me?"
>>> inputs = tokenizer(prompt, return_tensors="pt")

>>> # Generate
>>> generate_ids = model.generate(inputs.input_ids, max_length=30)
>>> tokenizer.batch_decode(generate_ids, skip_special_tokens=True, clean_up_tokenization_spaces=False)[0]
"Hey, are you conscious? Can you talk to me?\nI'm not conscious, but I can talk to you."