Hi, I submitted an access request for `meta-llama/Meta-Llama-3-8B-Instruct` several weeks ago and the request is still showing as pending. I also contacted Meta directly at `llamamodels@meta.com` more than a week ago, but I have not received any response there either. I understand that approval for gated Meta models is ultimately controlled by Meta rather than Hugging Face, but I would like to confirm whether my access request is correctly registered and healthy on the Hugging Face side, or whether it may be stuck in some way. Could someone from the Hugging Face team please confirm whether there is anything I should check or change on my account/request? I have not cancelled or resubmitted the request because I did not want to risk resetting its place in the review process. Model: `meta-llama/Meta-Llama-3-8B-Instruct` Status: Pending for several weeks Meta contacted directly: Yes, no response I can provide my Hugging Face username and any other account details privately if needed. Thank you.

Venice.ai provides private, unbiased AI tools emphasizing on-device data processing for complete user privacy, featuring advanced open-source models like

It's not like Zuck needs the coin despite increased infrastructure spend, headcount, losses on VR

I’m writing a custom versions of LlamaModels, and for one of those approaches I want to overwrite the attention mechanism of each layer. My code looks like this. Note that even when I define LlamaAttentionHybrid (a subclass of LlamaAttention) to be the exact same as LlamaAttention, I still get hallucination issues. This suggest I’m not correctly replacing the attention mechanism. class LlamaHybridForCausalLM(LlamaForCausalLM): def __init__(self, config: LlamaHybridConfig): super().__init__(config) if config.hybrid: for i, layer in enumerate(self.model.layers): # Need to also copy attention weights old_attn = layer.self_attn layer.self_attn = LlamaAttentionHybrid(config, i) layer.self_attn.load_state_dict(old_attn.state_dict()) However, the model works completely fine when I write this code: class LlamaHybridForCausalLM(LlamaForCausalLM): def __init__(self, config: LlamaHybridConfig): super().__init__(config) if config.hybrid: for i, layer in enumerate(self.model.layers): # Need to also copy attention weights old_attn = layer.self_attn layer.self_attn = LlamaAttention(config, i) layer.self_attn.load_state_dict(old_attn.state_dict()) Why would this happen even when in the subclass i don’t make any changes? Note, that the forward function here is defined exactly the same as the source code. class LlamaAttentionHybrid(LlamaAttention): def __init__(self, config: LlamaHybridConfig, layer_idx: int): super().__init__(config, layer_idx) def forward( self, hidden_states: torch.Tensor, position_embeddings: tuple[torch.Tensor, torch.Tensor], attention_mask: Optional[torch.Tensor], past_key_values: Optional[Cache] = None, cache_position: Optional[torch.LongTensor] = None, **kwargs: Unpack[FlashAttentionKwargs], ) -> tuple[torch.Tensor, Optional[torch.Tensor], Optional[tuple[torch.Tensor]]]: input_shape = hidden_states.shape[:-1] hidden_shape = (*input_shape, -1, self.head_dim) query_states = self.q_proj(hidden_states).view(hidden_shape).transpose(1, 2) key_states = self.k_proj(hidden_states).view(hidden_shape).transpose(1, 2) value_states = self.v_proj(hidden_states).view(hidden_shape).transpose(1, 2) cos, sin = position_embeddings query_states, key_states = apply_rotary_pos_emb(query_states, key_states, cos, sin) if past_key_values is not None: # sin and cos are specific to RoPE models; cache_position needed for the static cache cache_kwargs = {"sin": sin, "cos": cos, "cache_position": cache_position} key_states, value_states = past_key_values.update(key_states, value_states, self.layer_idx, cache_kwargs) attention_interface: Callable = eager_attention_forward if self.config._attn_implementation != "eager": attention_interface = ALL_ATTENTION_FUNCTIONS[self.config._attn_implementation] attn_output, attn_weights = attention_interface( self, query_states, key_states, value_states, attention_mask, dropout=0.0 if not self.training else self.attention_dropout, scaling=self.scaling, **kwargs, ) attn_output = attn_output.reshape(*input_shape, -1).contiguous() attn_output = self.o_proj(attn_output) return attn_output, attn_weights Thanks! EDIT: I narrowed the issue down to the redefining of the forward function. For some reason when I add the forward function into the subclass even if it’s identical, the model hallucinates dramatically.

I’m writing a custom versions of LlamaModels, and for one of those approaches I want to overwrite the attention mechanism of each layer. My code looks like this. Note that even when I define LlamaAttentionHybrid (a subclass of LlamaAttention) to be the exact same as LlamaAttention, I still get hallucination issues. This suggest I’m not correctly replacing the attention mechanism. class LlamaHybridForCausalLM(LlamaForCausalLM): def __init__(self, config: LlamaHybridConfig): super().__init__(config) if config.hybrid: for i, layer in enumerate(self.model.layers): # Need to also copy attention weights old_attn = layer.self_attn layer.self_attn = LlamaAttentionHybrid(config, i) layer.self_attn.load_state_dict(old_attn.state_dict()) However, the model works completely fine when I write this code: class LlamaHybridForCausalLM(LlamaForCausalLM): def __init__(self, config: LlamaHybridConfig): super().__init__(config) if config.hybrid: for i, layer in enumerate(self.model.layers): # Need to also copy attention weights old_attn = layer.self_attn layer.self_attn = LlamaAttention(config, i) layer.self_attn.load_state_dict(old_attn.state_dict()) Why would this happen even when in the subclass i don’t make any changes? Note, that the forward function here is defined exactly the same as the source code. class LlamaAttentionHybrid(LlamaAttention): def __init__(self, config: LlamaHybridConfig, layer_idx: int): super().__init__(config, layer_idx) def forward( self, hidden_states: torch.Tensor, position_embeddings: tuple[torch.Tensor, torch.Tensor], attention_mask: Optional[torch.Tensor], past_key_values: Optional[Cache] = None, cache_position: Optional[torch.LongTensor] = None, **kwargs: Unpack[FlashAttentionKwargs], ) -> tuple[torch.Tensor, Optional[torch.Tensor], Optional[tuple[torch.Tensor]]]: input_shape = hidden_states.shape[:-1] hidden_shape = (*input_shape, -1, self.head_dim) query_states = self.q_proj(hidden_states).view(hidden_shape).transpose(1, 2) key_states = self.k_proj(hidden_states).view(hidden_shape).transpose(1, 2) value_states = self.v_proj(hidden_states).view(hidden_shape).transpose(1, 2) cos, sin = position_embeddings query_states, key_states = apply_rotary_pos_emb(query_states, key_states, cos, sin) if past_key_values is not None: # sin and cos are specific to RoPE models; cache_position needed for the static cache cache_kwargs = {"sin": sin, "cos": cos, "cache_position": cache_position} key_states, value_states = past_key_values.update(key_states, value_states, self.layer_idx, cache_kwargs) attention_interface: Callable = eager_attention_forward if self.config._attn_implementation != "eager": attention_interface = ALL_ATTENTION_FUNCTIONS[self.config._attn_implementation] attn_output, attn_weights = attention_interface( self, query_states, key_states, value_states, attention_mask, dropout=0.0 if not self.training else self.attention_dropout, scaling=self.scaling, **kwargs, ) attn_output = attn_output.reshape(*input_shape, -1).contiguous() attn_output = self.o_proj(attn_output) return attn_output, attn_weights Thanks! EDIT: I narrowed the issue down to the redefining of the forward function. For some reason when I add the forward function into the subclass even if it’s identical, the model hallucinates dramatically.

I’m writing a custom versions of LlamaModels, and for one of those approaches I want to overwrite the attention mechanism of each layer. My code looks like this. Note that even when I define LlamaAttentionHybrid (a subclass of LlamaAttention) to be the exact same as LlamaAttention, I still get hallucination issues. This suggest I’m not correctly replacing the attention mechanism. class LlamaHybridForCausalLM(LlamaForCausalLM): def __init__(self, config: LlamaHybridConfig): super().__init__(config) if config.hybrid: for i, layer in enumerate(self.model.layers): # Need to also copy attention weights old_attn = layer.self_attn layer.self_attn = LlamaAttentionHybrid(config, i) layer.self_attn.load_state_dict(old_attn.state_dict()) However, the model works completely fine when I write this code: class LlamaHybridForCausalLM(LlamaForCausalLM): def __init__(self, config: LlamaHybridConfig): super().__init__(config) if config.hybrid: for i, layer in enumerate(self.model.layers): # Need to also copy attention weights old_attn = layer.self_attn layer.self_attn = LlamaAttention(config, i) layer.self_attn.load_state_dict(old_attn.state_dict()) Why would this happen even when in the subclass i don’t make any changes? Note, that the forward function here is defined exactly the same as the source code. class LlamaAttentionHybrid(LlamaAttention): def __init__(self, config: LlamaHybridConfig, layer_idx: int): super().__init__(config, layer_idx) def forward( self, hidden_states: torch.Tensor, position_embeddings: tuple[torch.Tensor, torch.Tensor], attention_mask: Optional[torch.Tensor], past_key_values: Optional[Cache] = None, cache_position: Optional[torch.LongTensor] = None, **kwargs: Unpack[FlashAttentionKwargs], ) -> tuple[torch.Tensor, Optional[torch.Tensor], Optional[tuple[torch.Tensor]]]: input_shape = hidden_states.shape[:-1] hidden_shape = (*input_shape, -1, self.head_dim) query_states = self.q_proj(hidden_states).view(hidden_shape).transpose(1, 2) key_states = self.k_proj(hidden_states).view(hidden_shape).transpose(1, 2) value_states = self.v_proj(hidden_states).view(hidden_shape).transpose(1, 2) cos, sin = position_embeddings query_states, key_states = apply_rotary_pos_emb(query_states, key_states, cos, sin) if past_key_values is not None: # sin and cos are specific to RoPE models; cache_position needed for the static cache cache_kwargs = {"sin": sin, "cos": cos, "cache_position": cache_position} key_states, value_states = past_key_values.update(key_states, value_states, self.layer_idx, cache_kwargs) attention_interface: Callable = eager_attention_forward if self.config._attn_implementation != "eager": attention_interface = ALL_ATTENTION_FUNCTIONS[self.config._attn_implementation] attn_output, attn_weights = attention_interface( self, query_states, key_states, value_states, attention_mask, dropout=0.0 if not self.training else self.attention_dropout, scaling=self.scaling, **kwargs, ) attn_output = attn_output.reshape(*input_shape, -1).contiguous() attn_output = self.o_proj(attn_output) return attn_output, attn_weights Thanks! EDIT: I narrowed the issue down to the redefining of the forward function. For some reason when I add the forward function into the subclass even if it’s identical, the model hallucinates dramatically.

I’m writing a custom versions of LlamaModels, and for one of those approaches I want to overwrite the attention mechanism of each layer. My code looks like this. Note that even when I define LlamaAttentionHybrid (a subclass of LlamaAttention) to be the exact same as LlamaAttention, I still get hallucination issues. This suggest I’m not correctly replacing the attention mechanism. class LlamaHybridForCausalLM(LlamaForCausalLM): def __init__(self, config: LlamaHybridConfig): super().__init__(config) if config.hybrid: for i, layer in enumerate(self.model.layers): # Need to also copy attention weights old_attn = layer.self_attn layer.self_attn = LlamaAttentionHybrid(config, i) layer.self_attn.load_state_dict(old_attn.state_dict()) However, the model works completely fine when I write this code: class LlamaHybridForCausalLM(LlamaForCausalLM): def __init__(self, config: LlamaHybridConfig): super().__init__(config) if config.hybrid: for i, layer in enumerate(self.model.layers): # Need to also copy attention weights old_attn = layer.self_attn layer.self_attn = LlamaAttention(config, i) layer.self_attn.load_state_dict(old_attn.state_dict()) Why would this happen even when in the subclass i don’t make any changes? Note, that the forward function here is defined exactly the same as the source code. class LlamaAttentionHybrid(LlamaAttention): def __init__(self, config: LlamaHybridConfig, layer_idx: int): super().__init__(config, layer_idx) def forward( self, hidden_states: torch.Tensor, position_embeddings: tuple[torch.Tensor, torch.Tensor], attention_mask: Optional[torch.Tensor], past_key_values: Optional[Cache] = None, cache_position: Optional[torch.LongTensor] = None, **kwargs: Unpack[FlashAttentionKwargs], ) -> tuple[torch.Tensor, Optional[torch.Tensor], Optional[tuple[torch.Tensor]]]: input_shape = hidden_states.shape[:-1] hidden_shape = (*input_shape, -1, self.head_dim) query_states = self.q_proj(hidden_states).view(hidden_shape).transpose(1, 2) key_states = self.k_proj(hidden_states).view(hidden_shape).transpose(1, 2) value_states = self.v_proj(hidden_states).view(hidden_shape).transpose(1, 2) cos, sin = position_embeddings query_states, key_states = apply_rotary_pos_emb(query_states, key_states, cos, sin) if past_key_values is not None: # sin and cos are specific to RoPE models; cache_position needed for the static cache cache_kwargs = {"sin": sin, "cos": cos, "cache_position": cache_position} key_states, value_states = past_key_values.update(key_states, value_states, self.layer_idx, cache_kwargs) attention_interface: Callable = eager_attention_forward if self.config._attn_implementation != "eager": attention_interface = ALL_ATTENTION_FUNCTIONS[self.config._attn_implementation] attn_output, attn_weights = attention_interface( self, query_states, key_states, value_states, attention_mask, dropout=0.0 if not self.training else self.attention_dropout, scaling=self.scaling, **kwargs, ) attn_output = attn_output.reshape(*input_shape, -1).contiguous() attn_output = self.o_proj(attn_output) return attn_output, attn_weights Thanks! EDIT: I narrowed the issue down to the redefining of the forward function. For some reason when I add the forward function into the subclass even if it’s identical, the model hallucinates dramatically.

I’m writing a custom versions of LlamaModels, and for one of those approaches I want to overwrite the attention mechanism of each layer. My code looks like this. Note that even when I define LlamaAttentionHybrid (a subclass of LlamaAttention) to be the exact same as LlamaAttention, I still get hallucination issues. This suggest I’m not correctly replacing the attention mechanism. class LlamaHybridForCausalLM(LlamaForCausalLM): def __init__(self, config: LlamaHybridConfig): super().__init__(config) if config.hybrid: for i, layer in enumerate(self.model.layers): # Need to also copy attention weights old_attn = layer.self_attn layer.self_attn = LlamaAttentionHybrid(config, i) layer.self_attn.load_state_dict(old_attn.state_dict()) However, the model works completely fine when I write this code: class LlamaHybridForCausalLM(LlamaForCausalLM): def __init__(self, config: LlamaHybridConfig): super().__init__(config) if config.hybrid: for i, layer in enumerate(self.model.layers): # Need to also copy attention weights old_attn = layer.self_attn layer.self_attn = LlamaAttention(config, i) layer.self_attn.load_state_dict(old_attn.state_dict()) Why would this happen even when in the subclass i don’t make any changes? Note, that the forward function here is defined exactly the same as the source code. class LlamaAttentionHybrid(LlamaAttention): def __init__(self, config: LlamaHybridConfig, layer_idx: int): super().__init__(config, layer_idx) def forward( self, hidden_states: torch.Tensor, position_embeddings: tuple[torch.Tensor, torch.Tensor], attention_mask: Optional[torch.Tensor], past_key_values: Optional[Cache] = None, cache_position: Optional[torch.LongTensor] = None, **kwargs: Unpack[FlashAttentionKwargs], ) -> tuple[torch.Tensor, Optional[torch.Tensor], Optional[tuple[torch.Tensor]]]: input_shape = hidden_states.shape[:-1] hidden_shape = (*input_shape, -1, self.head_dim) query_states = self.q_proj(hidden_states).view(hidden_shape).transpose(1, 2) key_states = self.k_proj(hidden_states).view(hidden_shape).transpose(1, 2) value_states = self.v_proj(hidden_states).view(hidden_shape).transpose(1, 2) cos, sin = position_embeddings query_states, key_states = apply_rotary_pos_emb(query_states, key_states, cos, sin) if past_key_values is not None: # sin and cos are specific to RoPE models; cache_position needed for the static cache cache_kwargs = {"sin": sin, "cos": cos, "cache_position": cache_position} key_states, value_states = past_key_values.update(key_states, value_states, self.layer_idx, cache_kwargs) attention_interface: Callable = eager_attention_forward if self.config._attn_implementation != "eager": attention_interface = ALL_ATTENTION_FUNCTIONS[self.config._attn_implementation] attn_output, attn_weights = attention_interface( self, query_states, key_states, value_states, attention_mask, dropout=0.0 if not self.training else self.attention_dropout, scaling=self.scaling, **kwargs, ) attn_output = attn_output.reshape(*input_shape, -1).contiguous() attn_output = self.o_proj(attn_output) return attn_output, attn_weights Thanks! EDIT: I narrowed the issue down to the redefining of the forward function. For some reason when I add the forward function into the subclass even if it’s identical, the model hallucinates dramatically.

I’m writing a custom versions of LlamaModels, and for one of those approaches I want to overwrite the attention mechanism of each layer. My code looks like this. Note that even when I define LlamaAttentionHybrid (a subclass of LlamaAttention) to be the exact same as LlamaAttention, I still get hallucination issues. This suggest I’m not correctly replacing the attention mechanism. class LlamaHybridForCausalLM(LlamaForCausalLM): def __init__(self, config: LlamaHybridConfig): super().__init__(config) if config.hybrid: for i, layer in enumerate(self.model.layers): # Need to also copy attention weights old_attn = layer.self_attn layer.self_attn = LlamaAttentionHybrid(config, i) layer.self_attn.load_state_dict(old_attn.state_dict()) However, the model works completely fine when I write this code: class LlamaHybridForCausalLM(LlamaForCausalLM): def __init__(self, config: LlamaHybridConfig): super().__init__(config) if config.hybrid: for i, layer in enumerate(self.model.layers): # Need to also copy attention weights old_attn = layer.self_attn layer.self_attn = LlamaAttention(config, i) layer.self_attn.load_state_dict(old_attn.state_dict()) Why would this happen even when in the subclass i don’t make any changes? Note, that the forward function here is defined exactly the same as the source code. class LlamaAttentionHybrid(LlamaAttention): def __init__(self, config: LlamaHybridConfig, layer_idx: int): super().__init__(config, layer_idx) def forward( self, hidden_states: torch.Tensor, position_embeddings: tuple[torch.Tensor, torch.Tensor], attention_mask: Optional[torch.Tensor], past_key_values: Optional[Cache] = None, cache_position: Optional[torch.LongTensor] = None, **kwargs: Unpack[FlashAttentionKwargs], ) -> tuple[torch.Tensor, Optional[torch.Tensor], Optional[tuple[torch.Tensor]]]: input_shape = hidden_states.shape[:-1] hidden_shape = (*input_shape, -1, self.head_dim) query_states = self.q_proj(hidden_states).view(hidden_shape).transpose(1, 2) key_states = self.k_proj(hidden_states).view(hidden_shape).transpose(1, 2) value_states = self.v_proj(hidden_states).view(hidden_shape).transpose(1, 2) cos, sin = position_embeddings query_states, key_states = apply_rotary_pos_emb(query_states, key_states, cos, sin) if past_key_values is not None: # sin and cos are specific to RoPE models; cache_position needed for the static cache cache_kwargs = {"sin": sin, "cos": cos, "cache_position": cache_position} key_states, value_states = past_key_values.update(key_states, value_states, self.layer_idx, cache_kwargs) attention_interface: Callable = eager_attention_forward if self.config._attn_implementation != "eager": attention_interface = ALL_ATTENTION_FUNCTIONS[self.config._attn_implementation] attn_output, attn_weights = attention_interface( self, query_states, key_states, value_states, attention_mask, dropout=0.0 if not self.training else self.attention_dropout, scaling=self.scaling, **kwargs, ) attn_output = attn_output.reshape(*input_shape, -1).contiguous() attn_output = self.o_proj(attn_output) return attn_output, attn_weights Thanks! EDIT: I narrowed the issue down to the redefining of the forward function. For some reason when I add the forward function into the subclass even if it’s identical, the model hallucinates dramatically.

Hi everyone, Has anyone here experienced a rejection for LLaMA 3.1 or 3.2 model access and later found a way to resubmit the Model Card? In my case, both requests were rejected—most likely because I didn’t fill out the forms properly. I’ve since been approved for LLaMA 3.3 by Meta (so I know my use case is eligible), and I reached out both via email to `llamamodels@meta.com` and in the Hugging Face discussion pages, but haven’t received any response so far. If anyone has any tips, similar experiences, or knows whether it’s even possible to unlock the form again for resubmission, I’d really appreciate your help! Thanks in advance, Paola

Hi everyone, Has anyone here experienced a rejection for LLaMA 3.1 or 3.2 model access and later found a way to resubmit the Model Card? In my case, both requests were rejected—most likely because I didn’t fill out the forms properly. I’ve since been approved for LLaMA 3.3 by Meta (so I know my use case is eligible), and I reached out both via email to `llamamodels@meta.com` and in the Hugging Face discussion pages, but haven’t received any response so far. If anyone has any tips, similar experiences, or knows whether it’s even possible to unlock the form again for resubmission, I’d really appreciate your help! Thanks in advance,