� 2��g�@����ddlmZddlZddlZddlmZmZddlmZddlm Z ddl m Z ddl m Z dd lmZmZmZdd lmZdd lmZd d lmZddlmZddlZddlZddlZeGd�d����ZdZdZeeed�Z dZ!dZ"e!e!e"d�Z#d�Z$d)d�Z%Gd�d��Z&Gd�d��Z'ej(��d���Z)d�Z*d*d"�Z+d+d#�Z,d$�Z-Gd%�d&��Z.Gd'�d(��Z/dS),�)� annotationsN�)�get_cache_invalidating_env_vars�ir)�backends)� GPUTarget)� __version__)�OutOfResources)�get_cache_manager�get_dump_manager�get_override_manager)�driver)� dataclass�)� ast_to_ttir)�Pathc�T�eZdZUdZded<dZded<d�Zd�Zed���Z d�Z dS) �AttrsDescriptorN�set�divisible_by_16� equal_to_1c�r�|j�t��|_|j�t��|_dSdS�N)rrr��selfs �h/home/asafur/pinokio/api/open-webui.git/app/env/lib/python3.11/site-packages/triton/compiler/compiler.py� __post_init__zAttrsDescriptor.__post_init__s6�� � � '�#&�5�5�D� � �?� "�!�e�e�D�O�O�O� #� "�c�T�t|j��t|j��d�S)N�rr)�listrrrs r�to_dictzAttrsDescriptor.to_dicts'��#'��(<�#=�#=�T�RV�Ra�Mb�Mb�c�c�crc ��tt|�dg����t|�dg�������S)Nrrr )rr�get)�datas r� from_dictzAttrsDescriptor.from_dict"sO���s�4�8�8�<M�r�3R�3R�/S�/S�*-�d�h�h�|�R�.H�.H�*I�*I�K�K�K� Krc���td�|j���D����}tj|�d�������S)Nc�,�g|]}t|����S�)�sorted)�.0�xs r� <listcomp>z(AttrsDescriptor.hash.<locals>.<listcomp>(s��=�=�=��6�!�9�9�=�=�=r�utf-8)�str�__dict__�values�hashlib�sha256�encode� hexdigest)r�keys r�hashzAttrsDescriptor.hash'sT���=�=�d�m�&:�&:�&<�&<�=�=�=�>�>���~�c�j�j��1�1�2�2�<�<�>�>�>r) �__name__� __module__� __qualname__r�__annotations__rrr"� staticmethodr&r7r)rrrrs���������O������J�����$�$�$� d�d�d��K�K��\�K�?�?�?�?�?rrz�^\s*tt\.func\s+(?:public\s+)?(@\w+)(\((?:%\w+: [\S\s]+(?: \{\S+ = \S+ : \S+\})?(?:, )?)*\))\s*(attributes \{[\S\s]+\})?\s+\{\s*$z=\.(?:visible|extern)\s+\.(?:entry|func)\s+(\w+)\s*\(([^)]*)\))�ttir�ttgir�ptxz %\w+: ((?:[^,\s<)]+|<[^>]+>)+),?z\.param\s+\.(\w+)c�~�tjd|��}|�%dt|�d����zS|S)Nz!tt\.ptr<([^,]+)�*r)�re�search�convert_type_repr�group)r,�matchs rrDrDEs?�� �I�)�1� -� -�E� ���&�u�{�{�1�~�~�6�6�6�6� �Hr�srcr/c��d}tj||��}t|��dks Jd���t|d��}|S)Nz&"triton_gpu.num-warps"\s?=\s?(\d+)\s?:rz(Expected exactly one match for num_warpsr)rB�findall�len�int)rG�ttgir_num_warps_pattern�num_warps_matches� num_warpss r�_get_num_warps_from_ir_strrONsY��G��� �#:�C�@�@�� � � !� !�Q� &� &� &�(R� &� &� &��%�a�(�)�)�I� �rc�*�eZdZdd d�Zd�Zd�Zd�ZdS) � ASTSourceN�return�Nonec�p�||_d|_|j|_||_||_||_t|jt��r6d�t|j� d����D��|_|j�t��|_|j�t��|_dSdS)Nr=c�>�i|]\}}||�����Sr))�strip�r+�k�vs r� <dictcomp>z&ASTSource.__init__.<locals>.<dictcomp>bs&��\�\�\�t�q�!�a������\�\�\r�,) �fn�extr8�name� signature� constants�attrs� isinstancer/� enumerate�split�dictr)rr\r_r`ras r�__init__zASTSource.__init__Zs����������K�� �"���"����� � �d�n�c� *� *� ]�\�\�y���AU�AU�VY�AZ�AZ�7[�7[�\�\�\�D�N� �>� !�!�V�V�D�N� �:� �(�*�*�D�J�J�J� � rc��d�t|j�����D��}td�|j���D����}|jj�d|j����d|�d|��}tj |� d����� ��S)Nc��g|]\}}|��Sr)r)rWs rr-z"ASTSource.hash.<locals>.<listcomp>is��C�C�C�D�A�q�a�C�C�Crc3�>K�|]\}}t|��|fV��dSr)r/rWs r� <genexpr>z!ASTSource.hash.<locals>.<genexpr>ls0����!Q�!Q�$�!�Q�3�q�6�6�1�+�!Q�!Q�!Q�!Q�!Q�!Qr�-r.) r*r_�itemsr`r\� cache_keyrar7r2r3r4r5)r� sorted_sig�sorted_constantsr6s rr7zASTSource.hashhs���C�C�F�4�>�+?�+?�+A�+A�$B�$B�C�C�C� �"�!Q�!Q�$�.�:N�:N�:P�:P�!Q�!Q�!Q�Q�Q����"�X�X�T�Z�_�_�%6�%6�X�X��X�X�FV�X�X���~�c�j�j��1�1�2�2�<�<�>�>�>rc�4�t|j||||���S)N)�context�options� codegen_fns)rr\)rrrrsrqs r�make_irzASTSource.make_irps���4�7�D�'�7�Xc�d�d�d�drc��t��Sr)rers r� parse_optionszASTSource.parse_optionsss ���v�v� r�NN�rRrS�r8r9r:rfr7rtrvr)rrrQrQXs]������ +� +� +� +� +�?�?�?�e�e�e�����rrQc�&�eZdZd�Zd�Zd�Zd�ZdS)�IRSourcec���||_t|��}|jdd�|_|���|_t jt|j|jt j ��}|� d��|_ |� d��}t j t|j|��}d�t|��D��|_dS)Nrrc�4�i|]\}}|t|����Sr))rD)r+rX�tys rrZz%IRSource.__init__.<locals>.<dictcomp>�s'��Q�Q�Q�u�q�"�!�.�r�2�2�Q�Q�Qr)�pathr�suffixr]� read_textrGrBrC�prototype_pattern� MULTILINErEr^rI�arg_type_patternrcr_)rrrFr_�typess rrfzIRSource.__init__ys����� ��D�z�z���;�q�r�r�?����>�>�#�#���� �+�D�H�5�t�x���N�N���K�K��N�N�� ��K�K��N�N� �� �+�D�H�5�y�A�A��Q�Q� �%�@P�@P�Q�Q�Q����rc�~�tj|j�d�������S)Nr.)r2r3rGr4r5rs rr7z IRSource.hash�s,���~�d�h�o�o�g�6�6�7�7�A�A�C�C�Crc�H�tj|j|��}||_|Sr)r�parse_mlir_modulerrq)rrrrsrq�modules rrtzIRSource.make_ir�s"���%�d�i��9�9�� ���� rc�`�|jdkrdt|j��iSt��S)Nr>rN)r]rOrGrers rrvzIRSource.parse_options�s.�� �8�w� � ��!;�D�H�!E�!E�F� F��v�v� rNryr)rrr{r{wsV������ R� R� R�D�D�D���� ����rr{c��ddl}tj�tj�tj�t ������}g}t t d��5}|tj|� ����� ��gz }ddd��n #1swxYwYtj� |d��dftj� |d��dfg}|D]�\}}|� |g|���D]�}t |j �|j��jd��5}|tj|� ����� ��gz }ddd��n #1swxYwY����tj��}t tj� |d��d��5} |� d ��} | sn|�| ���. ddd��n #1swxYwY|�|� ����tj� |d ��} |�| g��D]�}t |j �|j��jd��5}|tj|� ����� ��gz }ddd��n #1swxYwY��t(�d � |��zS) Nr�rb�compilerztriton.compiler.rztriton.backends.)�prefixz_C/libtriton.soTi�languagerk)�pkgutil�osr�dirname�abspath�__file__�openr2r3�readr5�join� walk_packages� module_finder� find_specr^�origin�update�append� iter_modulesr ) r�� TRITON_PATH�contents�f� path_prefixesrr��lib�libtriton_hash�chunk� language_paths r� triton_keyr��s����N�N�N��'�/�/�"�'�/�/�"�'�/�/�(�2K�2K�"L�"L�M�M�K��H� �h�� � �;���W�^�A�F�F�H�H�-�-�7�7�9�9�:�:��;�;�;�;�;�;�;�;�;�;�;����;�;�;�;� ����k�:� .� .�0B�C� ����k�:� .� .�0B�C��M�&�C�C� ��f��(�(�$���(�?�?� C� C�C��c�'�1�1�#�(�;�;�B�D�I�I� C�Q��W�^�A�F�F�H�H�5�5�?�?�A�A�B�B�� C� C� C� C� C� C� C� C� C� C� C���� C� C� C� C�� C� �^�%�%�N� �b�g�l�l�;�(9�:�:�D� A� A�)�Q� )��F�F�7�O�O�E�� �� � !� !�%� (� (� (�  )�� )�)�)�)�)�)�)�)�)�)�)����)�)�)�)�  �O�O�N�,�,�.�.�/�/�/��G�L�L��j�9�9�M��#�#�]�O�4�4�?�?�� �#�#�-�-�c�h�7�7�>�� E� E� ?�� ��������1�1�;�;�=�=�>� >�H� ?� ?� ?� ?� ?� ?� ?� ?� ?� ?� ?���� ?� ?� ?� ?��� �c�h�h�x�0�0� 0�0sH�:=C�C� C�%=F.�.F2 �5F2 �<0H9�9H=�H=�=L�L" �%L" c��|dks|dkrtj||��}||_|S|dks|dkr!t|�����S|dkr!t|�����SdS)Nr=r>�llirr?�cubin)rr�rqrr�� read_bytes)� full_namer]rqr�s r�parser��s��� �f�}�}��w����%�i��9�9�� ���� � �f�}�}��u� � ��I���(�(�*�*�*� �g�~�~��I���)�)�+�+�+��~r�e� BaseExceptionc���|j�t|j��|j�t|j��ddg}|j�g}��9t �fd�|D����s|�����j���9t||dd���D] \}}||_� |s d|_dSd|d_|d|_dS)z� Removes code_generator.py and related files from tracebacks. These are uninteresting to the user -- "just show me *my* code!" Nz"/triton/compiler/code_generator.pyz/ast.pyc3�d�K�|]*}�jjj�|���&|V��+dSr)�tb_frame�f_code� co_filename�endswith)r+r��tbs �rrjz#filter_traceback.<locals>.<genexpr>�s?�����V�V��2�;�+=�+I�+R�+R�ST�+U�+U�V�1�V�V�V�V�V�Vrr�����r)� __cause__�filter_traceback� __context__� __traceback__�anyr��tb_next�zip)r�� BAD_FILES�frames� cur_frame� next_framer�s @rr�r��s����  �{�����%�%�%��}� ����'�'�'� -���I� ��B� �F� �.��V�V�V�V�i�V�V�V�V�V� � �M�M�"� � � � �Z�� �.� $'�v�v�a�b�b�z�#:�#:�'�'���J�&� ��� �$������!��r� �� ��)����rc�� �|�tj���}t|t��s Jd���t |��}t|t �� }|r.t|t��s Jd���t|��}|� ��}|� t|p t��fi|����}t��}t���d|� ���d|� ���d|� ���dtt|��������� }t!j|�d�������}t)|��} t*j�dd��dk} t*j�dd��dk} | r!t1|� ����nd} | r!t3|� ����nd} |j�d �}| �|��pi}|�|��}t*j�d d��dk}|sF|�Dt9jt=|�������}tA|||��S||d �|j!�|�}t��}|�"||��tG|�$�����%|j&��}|r|d z }tOj(��}tOj)|��|�)|��|�*��} |�+|||��}n"#tX$r}t[|���d}~wwxYwt*j�d d��dk}tG|�����|d�D]�\}}|||��}|j�d|��}| �.||��||<| �| �.||��| �M| �/|��r8tad|����| �1|��}te|||��}|rB|dkr<| �1|��}|�3|��tad|����|}��| �.t9j4|tj���|d���||<| �6||��tA|||��S)Nz target must be of GPUTarget typez'source must be either AST or a filepathrkr.�TRITON_KERNEL_OVERRIDE�0�1�TRITON_KERNEL_DUMP�.json�TRITON_ALWAYS_COMPILE)r7�targetr� USE_TTGIR_LOC�.z Overriding kernel with file r>zCreate new locations for )�defaultF)�binary)7r�active�get_current_targetrbr� make_backendrQr/r{rvrerr�r7r*rlr2r3r4r5r r��environr$r r r^� get_group�json�loadsrr��CompiledKernelr0� add_stagesr!�keys�indexr]rrq� load_dialects�get_codegen_implementationrt� Exceptionr��put�has_file�print�get_filer��create_location_snapshot�dumps�vars� put_group) rGr�rr�backend� ir_source� extra_options�env_varsr6r7�fn_cache_manager�enable_override�enable_ir_dump�fn_override_manager�fn_dump_manager�metadata_filename�metadata_group� metadata_path�always_compile�metadata�stages� first_stagerqrsr�r�� use_ttgir_locr]� compile_ir� next_module� ir_filenamer��ttgir_full_names r�compiler��sB�� �~���1�1�3�3�� �f�i� (� (�L�L�*L�L�L� (��6�"�"�G��s�I�.�.�.�I����#�s�#�#�N�N�%N�N�N�#��s�m�m���%�%�'�'�M��#�#�D��):�D�F�F�$L�$L�m�$L�$L�M�M�G�.�0�0�H� �\�\� j� j�C�H�H�J�J� j� j������ j� j�'�,�,�.�.� j� j�3�v�V^�Vd�Vd�Vf�Vf�Og�Og�Kh�Kh� j� j�C� �>�#�*�*�W�-�-� .� .� 8� 8� :� :�D�(��.�.���j�n�n�%=�s�C�C�s�J�O��Z�^�^�$8�#�>�>�#�E�N�>M�W�.�s�x�x�z�z�:�:�:�SW��6D�N�&�s�x�x�z�z�2�2�2�$�O��8�*�*�*��%�/�/�0A�B�B�H�b�N�"�&�&�'8�9�9�M��Z�^�^�$;�S�A�A�S�H�N� �9�m�7��:�d�=�1�1�;�;�=�=�>�>���c�>�4�8�8�8����� � �� � �H��V�V�F� ���v�w�'�'�'��v�{�{�}�}�%�%�+�+�C�G�4�4�K����q�� ��j�l�l�G���W���� ���'�"�"�"��4�4�6�6�K�����W�k�7�;�;���� ��������� ����������J�N�N�?�C�8�8�C�?�M��� � ���/�/� � � �=�����Z� �j���2�2� ���)�)�C�)�)� �&6�&:�&:�;� �&T�&T��{�#� � &� � � � �[� 9� 9� 9� � +�0C�0L�0L�[�0Y�0Y� +� �@�;�@�@� A� A� A�+�4�4�[�A�A�I�� �3��8�8�K� � A�S�G�^�^�.�7�7� �D�D�O� � 0� 0�� A� A� A� �?�o�?�?� @� @� @����(8�(<�(<�T�Z��Z^�=_�=_�=_�ar�DI�)=�)K�)K�N�$�%����0�.�A�A�A� �#�~�t� 4� 4�4s�"N:�: O�O�Oc�����fd�tj��D��}t|��dkr*tt|���d�j�d|�d����|d���S)Nc�R��g|]#}|j�����|j��$Sr))r��supports_target)r+r,r�s �rr-z make_backend.<locals>.<listcomp>2s1���[�[�[�a�� �8R�8R�SY�8Z�8Z�[�q�z�[�[�[rrz! compatible backends for target (z) (z). There should only be one.r)rr1rJ� RuntimeErrorr�)r��activess` rr�r�1s����[�[�[�[�8�?�#4�#4�[�[�[�G� �7�|�|�q�����7�|�|� v� v�f�n� v� v�QX� v� v� v�x�x� x� �7�1�:�f� � �rc�"�eZdZd�Zdd�Zd�ZdS)�LazyDictc�"�||_g|_dSr)r%�extras)rr%s rrfzLazyDict.__init__;s���� ��� � � rrRrSc��|jD]\}}|j||�z|_�|j���|jSr)rr%�clear�r�func�argss rr$z LazyDict.get?sI���+� 0� 0�J�D�$�� �D�D�$�K�/�D�I�I� � �������y�rc�>�|j�||f��dSr)rr�rs r�addz LazyDict.addEs"�� � ���D�$�<�(�(�(�(�(rNrx)r8r9r:rfr$r r)rrrr9sF������������� )�)�)�)�)rrc�>��eZdZdZdZd�Zd�Z�fd�Zd�Zd�Z �xZ S)r�Nc �*� �ddlm}td�|���D����}t j|�����}t|d��|d<|d}t|d|d|d��|d<|d tt|� ��������}|d i|��|_ t|j j��} | �|j ��|_||_||_|j j|_d �|���D��} | j� � fd �| D��|_|j� |_d|_d|_dS) Nr)� namedtuplec3�dK�|]+\}}|�d���t|��V��,dS)r�N�r�r�r+�c�ps rrjz*CompiledKernel.__init__.<locals>.<genexpr>Rs>����`�`�$�!�Q�A�J�J�W^�L_�L_�`�d�1�g�g�`�`�`�`�`�`r� cluster_dimsr�r��arch� warp_size�KernelMetadatac�\�g|])\}}|�d���t|����*S)r�rrs rr-z+CompiledKernel.__init__.<locals>.<listcomp>`s4��[�[�[���A�q�z�z�RY�GZ�GZ�[�T�!�W�W�[�[�[rc���i|]K}|jdd�|jdd��kr|���n|�����LS)rN)r�r�r�)r+�file� binary_exts �rrZz+CompiledKernel.__init__.<locals>.<dictcomp>bse��� � � �� �K����O�$�+�a�b�b�/�Z�2O�2O�T�_�_�.�.�.�UY�Uc�Uc�Ue�Ue� � � rr))� collectionsr �nextrlr�r�r��tuplerr*r!r�r�r�r�� pack_metadata�packed_metadatarGr7r^r�asm�kernelr��function) rrGr�r7r r�r�r�rr�� asm_filesrs @rrfzCompiledKernel.__init__Ps����*�*�*�*�*�*��`�`�.�2F�2F�2H�2H�`�`�`�a�a� ��:�m�5�5�7�7�8�8��#(��.�)A�#B�#B��� ��(�#��&�v�i�'8�&��.�&�Q\�J]�^�^����#��$4�f�T�(�-�-�/�/�=R�=R�6S�6S�T�T��&��2�2��2�2�� ��t�}�3�4�4��&�4�4�T�]�C�C�������� ��M�&�� �[�[��)=�)=�)?�)?�[�[�[� ��'� � � � � �!� � � ����h�z�*�� ��� ��� � � rc��|j�dStj���}tj�|j|j��|_tjj� |��d}|jj |krt|jj |d���tjj� |j |j|jj |��\|_|_|_|_dS)N�max_shared_memz shared memory)r�rr��get_current_device� launcher_clsrGr��run�utils�get_device_properties�sharedr � load_binaryr^r!r"�n_regs�n_spills)r�device� max_shareds r� _init_handleszCompiledKernel._init_handlesms��� �;� "� �F���1�1�3�3���=�-�-�d�h�� �F�F����]�(�>�>�v�F�F�GW�X� � �=� �*� ,� ,� ���!5�z�?�S�S� S�AG��AT�A`�A`� �I�t�{�D�M�$8�&�BB�BB�>�� �T�]�D�K����rc�z��|dkr|���t���|��S)Nr()r1�super�__getattribute__)rr^� __class__s �rr4zCompiledKernel.__getattribute__{s6��� �5�=�=� � � � � � ��w�w�'�'��-�-�-rc���tj�dSt|j|j|d���}t |jt��r|jjj �|Si}d}t|jjj ��D]>\}}||jjj vr|jj |||<�.||||<|dz }�?|�|jjj ||j|f��|S)N)r^r"�streamrr)r��launch_enter_hookrr^r"rbrGrQr\�launch_metadatarc� arg_names� constexprsr`r r�) r�gridr7r�ret�arg_dict�arg_idx�i�arg_names rr9zCompiledKernel.launch_metadata�s��� � +� 3��4��� �t�}�PV�W�W�X�X���$�(�I�.�.� �$�(�+�2M�2U��J�����$�T�X�[�%:�;�;� � �K�A�x��D�H�K�*�*�*�%)�X�%7��%A���"�"�%)�'�]���"��1� ��� ����� �+�d�D�M�8�-L�M�M�M�� rc�D�������dd���fd� }|S)N)r7c �0��|�=tj���}tj�|��}�j�|g|�R�}�j�d�d�d|�j�j|tj tj g |�R�dS)Nrrr) rr�r&�get_current_streamr9r(r"rr�r8�launch_exit_hook)r7rr/r9r<rs ��r�runnerz*CompiledKernel.__getitem__.<locals>.runner�s�����~���9�9�;�;����9�9�&�A�A��2�d�2�4��G�$�G�G�G�O� �D�H�T�!�W�d�1�g�t�A�w��� �t�G[�]l�#�5�~�7V� _�Y]� _� _� _� _� _� _r)r1)rr<rFs`` r� __getitem__zCompiledKernel.__getitem__�sP���� ������!%� _� _� _� _� _� _� _� _�� r) r8r9r:r8rErfr1r4r9rG� __classcell__)r5s@rr�r�Is���������������: B� B� B�.�.�.�.�.� ���" � � � � � � rr�)rGr/)r�r�rw)0� __future__rr2r�� _C.libtritonrrr�backends.compilerr�r �runtime.autotunerr � runtime.cacher r r �runtime.driverr� dataclassesr�code_generatorr�pathlibrrB� functoolsr�r�mlir_prototype_pattern�ptx_prototype_patternr��mlir_arg_type_pattern�ptx_arg_type_patternr�rDrOrQr{� lru_cacher�r�r�r�r�rr�r)rr�<module>rYs���"�"�"�"�"�"����� � � � �>�>�>�>�>�>�>�>�������)�)�)�)�)�)�������.�.�.�.�.�.�U�U�U�U�U�U�U�U�U�U�#�#�#�#�#�#�!�!�!�!�!�!�'�'�'�'�'�'������� � � � ����� � � � � �?�?�?�?�?�?�?� ��?�>]��X�� "� #� ���� <��+�� !� "� ���� � � �������������>��������6�����1�1���1�D,�,�,�$�$�$�$�DL5�L5�L5�L5�^��� )� )� )� )� )� )� )� )� S�S�S�S�S�S�S�S�S�Sr
Memory