�
���gY� � � � d Z ddlZddlZddlmZ ddlmZmZ ddlZddl m
Z
ddlmZm
Z
ddlmZmZ dd lmZ dd
lmZ ddlmZ ej e� � ZdAd�ZdBde
defd�Zde
fd�Zdedefd�ZdBde
dedefd�Zdededefd�Z dBd�Z!dededede"fd�Z# dCde
dedede"def
d�Z$dedededefd�Z% dDde
dedededef
d�Z&dededede"d e"d!efd"�Z' dEd%�Z(dd&�deded'ee fd(�Z) dFde
ded'ee defd)�Z*d*d+�dededede"d,e"f
d-�Z# dGde
dedede"ded.ee" d,ee" fd/�Z+deded0ed1ed2e,d3e,d4e"de"fd5�Z- dHde
ded1edee d0ee d2e,d3e,d4e"de"defd8�Z.ej/ e!ej0 e$ej1 e&ej2 e(ej3 eej4 eej5 e*ej6 eej7 e+ej8 e.i
Z9 dId9ee,ef de
dee dee d:ee: f
d;�Z; G d<� d=e
� � Z< G d>� d?e� � Z=dJd@�Z>dS )Kz$PyTorch optimization for BERT model.� N)�partial)�Optional�Union)� Optimizer)�LambdaLR�ReduceLROnPlateau� )�LayerWiseDummyOptimizer�LayerWiseDummyScheduler)�
SchedulerType)�logging)�require_versionc � � dS �Nr � )�_s �i/home/asafur/pinokio/api/open-webui.git/app/env/lib/python3.11/site-packages/transformers/optimization.py�_get_constant_lambdar "