�
���g� � �� � d dl Z d dlmZ d dlmZ d dlZd dlmZ G d� de� � Z G d� de� � Z G d � d
e� � Z
G d � d e� � Z G d
� de� � Z G d� de� � Z
dS )� N)�partial)�Optional)�AbstractArchiveFileSystemc �� � � e Zd ZU dZdZdZeed< dZeed< dZ eed< ddede
e d e
e f� fd
�
Z e
� fd �� � Zd � Zd
efd�Z dd
edefd�Z� xZS )� BaseCompressedFileFileSystemzFRead contents of compressed file as a filesystem with one file inside.� N�protocol� compression� extension�fo�target_protocol�target_optionsc �. �� t � � j | fi |�� t |d� � r|� � � n|| _ t t j | j fd|| j ddd�|pi � di � � �d�|pi ��| _
t j �
| j � d� � d � � | _ d
| j v r'| j d | j � d
� � � n| j | _ d | _ d S ) a�
The compressed file system can be instantiated from any compressed file.
It reads the contents of compressed file as a filesystem with one file inside, as if it was an archive.
The single file inside the filesystem is named after the compresssed file,
without the compression extension at the end of the filename.
Args:
fo (:obj:``str``): Path to compressed file. Will fetch file using ``fsspec.open()``
mode (:obj:``str``): Currently, only 'rb' accepted
target_protocol(:obj:``str``, optional): To override the FS protocol inferred from a URL.
target_options (:obj:``dict``, optional): Kwargs passed when instantiating the target FS.
�
__fspath__�rbFT)�requote_redirect_url� trust_env�
client_kwargs)�moder r
r z::r �.N)�super�__init__�hasattrr r r �fsspec�openr
�pop�_open_with_fsspec�os�path�basename�split�compressed_name�rindex�uncompressed_name� dir_cache)�selfr r
r �kwargs� __class__s ��p/home/asafur/pinokio/api/open-webui.git/app/env/lib/python3.11/site-packages/datasets/filesystems/compression.pyr z%BaseCompressedFileFileSystem.__init__ s7 �� � ������(�(��(�(�(�%,�R� �%>�%>�F�"�-�-�/�/�/�B���!(� �K� �G� "
� �$� �(�(-� !�� � "�'�R�,�,�_�b�A�A��
"
� "
� �#�� "
� "
�� � "�w� /� /���
�
�d�0C�0C�A�0F� G� G�� � �d�*�*�*�
� �!C�4�#7�#>�#>�s�#C�#C�!C� D� D��%�
� �
����� c �l �� t � � � |� � � d� � S )N�/)r �_strip_protocol�lstrip)�clsr r( s �r) r- z,BaseCompressedFileFileSystem._strip_protocol; s+ �� � �w�w�&�&�t�,�,�3�3�C�8�8�8r* c � � | j �Mi | � � � j � | j � � �d| j i�}|d |i| _ d S d S )N�name)r% r �fs�infor r$ )r& �fs r) � _get_dirsz&BaseCompressedFileFileSystem._get_dirs@ s[ � � �>� !�]�4�)�)�+�+�.�3�3�D�G�<�<�]�f�d�F\�]�]�A� ��i��^�D�N�N�N� "� !r* r c � � | � � � � � � 5 }|� � � cd d d � � S # 1 swxY w Y d S )N)r r �read)r& r r4 s r) �catz BaseCompressedFileFileSystem.catE s� � �
�
#�
#�
%�
%�
*�
*�
,�
,� ���6�6�8�8� � � � � � � � � � � � ���� � � � � � s �A�A �A r Tr c � � | � |� � }|dk rt d|� d| j � d�� � �| � � � � � � S )Nr zTried to read with mode z on file z opened with mode 'rb')r- �
ValueErrorr r r )r& r r �
block_size�
autocommit�
cache_optionsr' s r) �_openz"BaseCompressedFileFileSystem._openI sb � � �#�#�D�)�)�� �4�<�<� � f�� f� f�t�w� f� f� f�g�g� g��%�%�'�'�,�,�.�.�.r* )r NN)r NTN)�__name__�
__module__� __qualname__�__doc__� root_markerr �str�__annotations__r
r r �dictr � classmethodr- r5 r8 r> �
__classcell__)r( s @r) r r s< �� � � � � � �P�P��K� �
�c� � � � �K������I�s���� ei�& � & ��& �-5�c�]�& �S[�\`�Sa�& � & � & � & � & � & �P �9� 9� 9� 9� �[�9�,� ,� ,�
�� � � � � ����
/� /�� /� � /� /� /� /� /� /� /� /r* r c � � e Zd ZdZdZdZdZdS )�
Bz2FileSystemz?Read contents of BZ2 file as a filesystem with one file inside.�bz2z.bz2N�r? r@ rA rB r r
r � r* r) rJ rJ X �# � � � � � �I�I��H��K��I�I�Ir* rJ c � � e Zd ZdZdZdZdZdS )�GzipFileSystemz@Read contents of GZIP file as a filesystem with one file inside.�gzipz.gzNrL rM r* r) rP rP ` s# � � � � � �J�J��H��K��I�I�Ir* rP c � � e Zd ZdZdZdZdZdS )�
Lz4FileSystemz?Read contents of LZ4 file as a filesystem with one file inside.�lz4z.lz4NrL rM r* r) rS rS h rN r* rS c � � e Zd ZdZdZdZdZdS )� XzFileSystemzFRead contents of .xz (LZMA) file as a filesystem with one file inside.�xzz.xzNrL rM r* r) rV rV p s# � � � � � �P�P��H��K��I�I�Ir* rV c � � e Zd ZdZdZdZdZdS )�ZstdFileSystemzK
Read contents of .zstd file as a filesystem with one file inside.
�zstdz.zstNrL rM r* r) rY rY x s) � � � � � �� � �H��K��I�I�Ir* rY )r � functoolsr �typingr r �fsspec.archiver r rJ rP rS rV rY rM r* r) �<module>r^ sb �� � � � � � � � � � � � � � � � �
�
�
�
� 4� 4� 4� 4� 4� 4�L/� L/� L/� L/� L/�#<� L/� L/� L/�^� � � � �0� � � �� � � � �1� � � �� � � � �0� � � �� � � � �/� � � �� � � � �1� � � � � r*