�
%�g�} � �l � d dl Z d dlmZ d dlmZmZ d dlZd dlmZ ddl m
Z
ddlmZm
Z
mZmZmZ ddlmZmZmZ ddlmZ dd lmZ dd
lmZ g d�Z G d� d
� � Z G d� de� � Zd)dej dedefd�Z d)dej fd�Z! G d� de� � Z" d*dej j dee#e$ej% f deee$e&e$ f dee&e$ dee'e#e'ej% ej( f f f
d�Z) d+dej j deej% dedee d ed!e$deee$e&e$ f dee&e$ dee'e#e'ej% ej( f f fd"�Z*dej fd#�Z+ d+dej deeej% e'e$ej% f f deee'e$ef f ded ed!e$deee$e&e$ f dee&e$ dee'e#e'ej% ej( f f fd$�Z, G d%� d&e� � Z- G d'� d(� � Z.dS ),� N)�Mapping)�Optional�Union� )�PartialState)�PrefixedDataset�find_device�named_module_tensors�send_to_device�set_module_tensor_to_device)�is_mlu_available�is_musa_available�is_npu_available)�clear_device_cache)�get_non_persistent_buffers)�recursive_getattr)�to�cuda�npu�xpu�mlu�sdaa�musac �. � e Zd ZdZdZd� Zd� Zd� Zd� ZdS )� ModelHooka
A hook that contains callbacks to be executed just before and after the forward method of a model. The difference
with PyTorch existing hooks is that they get passed along the kwargs.
Class attribute:
- **no_grad** (`bool`, *optional*, defaults to `False`) -- Whether or not to execute the actual forward pass under
the `torch.no_grad()` context manager.
Fc � � |S )z�
To be executed when the hook is attached to the module.
Args:
module (`torch.nn.Module`): The module attached to this hook.
� ��self�modules �`/home/asafur/pinokio/api/open-webui.git/app/env/lib/python3.11/site-packages/accelerate/hooks.py� init_hookzModelHook.init_hook7 � � � �
� c �
� ||fS )a�
To be executed just before the forward method of the model.
Args:
module (`torch.nn.Module`): The module whose forward pass will be executed just after this event.
args (`Tuple[Any]`): The positional arguments passed to the module.
kwargs (`Dict[Str, Any]`): The keyword arguments passed to the module.
Returns:
`Tuple[Tuple[Any], Dict[Str, Any]]`: A tuple with the treated `args` and `kwargs`.
r �r r �args�kwargss r! �pre_forwardzModelHook.pre_forward@ s � � �V�|�r$ c � � |S )a:
To be executed just after the forward method of the model.
Args:
module (`torch.nn.Module`): The module whose forward pass been executed just before this event.
output (`Any`): The output of the module.
Returns:
`Any`: The processed `output`.
r )r r �outputs r! �post_forwardzModelHook.post_forwardN s � � �
r$ c � � |S )z�
To be executed when the hook is detached from a module.
Args:
module (`torch.nn.Module`): The module detached from this hook.
r r s r! �detach_hookzModelHook.detach_hook[ r# r$ N) �__name__�
__module__�__qualname__�__doc__�no_gradr"