�
%�gQ � � � d dl Z d dlZd dlmZ d dlmZmZ d dlZd dlm Z d dl
mZmZ ddl
mZmZ ddlmZ dd lmZmZmZmZmZmZ e j e� � Z ddej j ded
eeej f deeeee eej! f f dee"e deeee ef ee ef f deeeej f de#fd�Z$ dd�Z%dd�Z& dd�Z'd� Z(d� Z)de j fd�Z*d� Z+dS )� N)�deepcopy)�Optional�Union)�is_4bit_bnb_available�is_8bit_bnb_available� )�dispatch_model�init_empty_weights� )�BnbQuantizationConfig)�find_tied_parameters�get_balanced_memory�infer_auto_device_map�load_checkpoint_in_model�offload_weight�set_module_tensor_to_deviceF�model�bnb_quantization_config�weights_location�
device_map�no_split_module_classes�
max_memory�offload_folder�offload_state_dictc
�� ��� |j }|j } | rt � � st d� � �|rt � � st d� � �g }
t
�t � � rCt �� � � � � dk rd� ��
� � D � � }
|j �t | � � |_ |r|j �
|
� � |j }|j �g |_ |j }|�
|� � || _ | | _ t# | � � }
|
j dk �rct&