�
i\�g� �
�X � U d dl mZ ddlmZ ddlmZ ddddddddddddd� Zded < i d
d�d d�d d�d
d�dd�dd�dd�dd�dd�dd�dd�dd�dd�dd�dd�dd�d d�i d d�d d�d d�d d�d!d�d"d�d#d�d$d�d%d�d&d�d'd�d(d)�d*d)�d+d�d,d�d-d�d.d��ddddddd/d/d0��Zded1< d9d5�Z d:d7�Z
d8S );� )� annotations� )�Encoding)� get_encoding�
o200k_base� cl100k_base) zo1-zo3-z chatgpt-4o-zgpt-4o-zgpt-4-zgpt-3.5-turbo-z
gpt-35-turbo-z ft:gpt-4ozft:gpt-4zft:gpt-3.5-turbozft:davinci-002zft:babbage-002zdict[str, str]�MODEL_PREFIX_TO_ENCODING�o1�o3zgpt-4ozgpt-4z
gpt-3.5-turbozgpt-3.5z gpt-35-turboz davinci-002z babbage-002ztext-embedding-ada-002ztext-embedding-3-smallztext-embedding-3-largeztext-davinci-003� p50k_baseztext-davinci-002ztext-davinci-001� r50k_baseztext-curie-001ztext-babbage-001z text-ada-001�davinci�curie�babbage�adazcode-davinci-002zcode-davinci-001zcode-cushman-002zcode-cushman-001z
davinci-codexz
cushman-codexztext-davinci-edit-001� p50k_editzcode-davinci-edit-001ztext-similarity-davinci-001ztext-similarity-curie-001ztext-similarity-babbage-001ztext-similarity-ada-001�gpt2)ztext-search-davinci-doc-001ztext-search-curie-doc-001ztext-search-babbage-doc-001ztext-search-ada-doc-001z code-search-babbage-code-001zcode-search-ada-code-001r zgpt-2�MODEL_TO_ENCODING�
model_name�str�returnc �� � d}| t v rt | }n8t � � � D ] \ }}| � |� � r|c S � |�t d| � d�� � d�|S )zrReturns the name of the encoding used by a model.
Raises a KeyError if the model name is not recognised.
Nz Could not automatically map z_ to a tokeniser. Please use `tiktoken.get_encoding` to explicitly get the tokeniser you expect.)r r �items�
startswith�KeyError)r �
encoding_name� model_prefix�model_encoding_names �^/home/asafur/pinokio/api/open-webui.git/app/env/lib/python3.11/site-packages/tiktoken/model.py�encoding_name_for_modelr P s� � �
�M��&�&�&�)�*�5�
�
�
2J�1O�1O�1Q�1Q� +� +� -�L�-��$�$�\�2�2�
+�*�*�*�*�
+� � ��
]�:�
]�
]�
]�
�
� � �
�� r c �: � t t | � � � � S )zfReturns the encoding used by a model.
Raises a KeyError if the model name is not recognised.
)r r )r s r �encoding_for_modelr# i s � �
�/�
�;�;� <� <�<r! N)r r r r )r r r r ) �
__future__r �corer �registryr r �__annotations__r r r# � r! r �<module>r) s� �� "� "� "� "� "� "� "� � � � � � � "� "� "� "� "� "� � � ���#�"� � �%�#�#� ,� ,� � � � � �"5%��,�5%� �,�5%�
�l� 5%� �]�
5%� �]�5%� �}�5%� �M�5%� �=�5%� �=�5%� �m� 5%� �m� 5%� �m�!5%�& � �'5%�( � �)5%�* � �+5%�, �k�-5%�. � �/5%� 5%�0 �K�15%�2 �{�35%�4 �[�55%�6 �{�75%�8
�;�95%�< � �=5%�> � �?5%�@ � �A5%�B � �C5%�D �[�E5%�F �[�G5%�J �[�K5%�L �[�M5%�P "�;�Q5%�R ��S5%�T "�;�U5%�V �{�W5%� 5%�X $/�!,�#.� *�$/� +� �
�i5%� 5%� 5%� � 5� 5� 5� 5�p� � � �2=� =� =� =� =� =r!