� ���g�*� �r�ddlZddlmZmZddlmZmZmZmZm Z m Z ddl Z ddl ZddlmZddlmZddlmZddlmZmZdd lmZer dd lmZd d lmZGd �de ��ZeGd�d����Zddde fd�Z!dddefd�Z"de j#defd�Z$ dde%dee&e%e e'e%ffde%ddfd�Z(dS)�N)� dataclass�field)� TYPE_CHECKING�Any�ClassVar�Optional� TypedDict�Union�)�config)�DownloadConfig)� array_cast)� is_local_path�xopen)�string_to_dict�� VideoReader�)� FeatureTypec�<�eZdZUeeed<eeed<dS)�Example�path�bytesN)�__name__� __module__� __qualname__r�str�__annotations__r���g/home/asafur/pinokio/api/open-webui.git/app/env/lib/python3.11/site-packages/datasets/features/video.pyrrs2������� �3�-���� �E�?�����r rc ���eZdZUdZdZeed<dZee ed<dZ e e ed<e j e j��e j��d���Ze eed <edd d � ��Ze ed <d �Zdee eeejdfdefd�Z ddee efdeee eee ffddfd�Zdedee dfffd�Zdee je je j fde jfd�Z!dS)�Videoa% **Experimental.** Video [`Feature`] to read video data from a video file. Input: The Video feature accepts as input: - A `str`: Absolute path to the video file (i.e. random access is allowed). - A `dict` with the keys: - `path`: String with relative path of the video file in a dataset repository. - `bytes`: Bytes of the video file. This is useful for archived files with sequential access. - A `torchvision.io.VideoReader`: torchvision video reader object. Args: mode (`str`, *optional*): The mode to convert the video to. If `None`, the native mode of the video is used. decode (`bool`, defaults to `True`): Whether to decode the video data. If `False`, returns the underlying dictionary in the format `{"path": video_path, "bytes": video_bytes}`. Examples: ```py >>> from datasets import Dataset, Video >>> ds = Dataset.from_dict({"video":["path/to/Screen Recording.mov"]}).cast_column("video", Video()) >>> ds.features["video"] Video(decode=True, id=None) >>> ds[0]["video"] <torchvision.io.video_reader.VideoReader object at 0x325b1aae0> >>> ds = ds.cast_column('video', Video(decode=False)) {'bytes': None, 'path': 'path/to/Screen Recording.mov'} ``` T�decodeN�idztorchvision.io.VideoReader�dtype�rr�pa_typeF)�default�init�repr�_typec��|jS�N)r()�selfs r!�__call__zVideo.__call__Gs ���|�r �valuer�returnc��tjrddlm}nd}t |t ��rt j|��}t |t��r|dd�St |t��rd|d�St |t j ��rt|��S|�t ||��rt|��St |t��rl|�d��|�d��}}|�$tj�|��rd|d�S|�|�||d�St%d|�d ����t'd t)|�������) z�Encode example into a format for Arrow. Args: value (`str`, `np.ndarray`, `VideoReader` or `dict`): Data passed as input to Video feature. Returns: `dict` with "path" and "bytes" fields rrN�rrrrr'zTA video sample should have one of 'path' or 'bytes' but they are missing or None in �.z!Unsupported encode_example type: )r �TORCHVISION_AVAILABLE�torchvision.ior� isinstance�list�np�arrayrr�ndarray�encode_np_array�encode_torchvision_video�dict�get�osr�isfile� ValueError� TypeError�type)r/r1rr�bytes_s r!�encode_examplezVideo.encode_exampleJs��� � '� � 2� 2� 2� 2� 2� 2� 2��K� �e�T� "� "� $��H�U�O�O�E� �e�S� !� !� O�!�D�1�1� 1� ��u� %� %� O� �5�1�1� 1� ��r�z� *� *� O�"�5�)�)� )� � $��E�;�)G�)G� $�+�E�2�2� 2� ��t� $� $� O� �9�9�V�,�,�e�i�i��.@�.@�&�D���B�G�N�N�4�$8�$8��!%�t�4�4�4��#�t�'7�!'��6�6�6� �s�kp�s�s�s�����M��U� � �M�M�N�N� Nr �token_per_repo_idc��|jstd���tjrddlm}nt d���|�i}t|t��r|d}}n|d|d}}|�B|�td|�d ����t|��r ||��}nt||� ��}n ||��}||d �|_ |S) a|Decode example video file into video data. Args: value (`str` or `dict`): A string with the absolute video file path, a dictionary with keys: - `path`: String with absolute or relative video file path. - `bytes`: The bytes of the video file. token_per_repo_id (`dict`, *optional*): To access and decode video files from private repositories on the Hub, you can pass a dictionary repo_id (`str`) -> token (`bool` or `str`). Returns: `torchvision.io.VideoReader` zMDecoding is disabled for this feature. Please use Video(decode=True) instead.rrz9To support decoding videos, please install 'torchvision'.NrrzBA video should have one of 'path' or 'bytes' but both are None in r5)rHr4) r$� RuntimeErrorr r6r7r� ImportErrorr8rrCr�hf_video_reader� _hf_encoded)r/r1rHrrrF�videos r!�decode_examplezVideo.decode_examplevs ��,�{� p��n�o�o� o� � '� [� 2� 2� 2� 2� 2� 2� 2��Y�Z�Z� Z� � $� "� � �e�S� !� !� 9� �$�&�D�D� ��=�%��.�&�D� �>��|� �!n�fk�!n�!n�!n�o�o�o��t�$�$� S�#� �D�)�)���'��@Q�R�R�R����K��'�'�E�%)�F�;�;���� r rc�N�ddlm}|jr|n|d��|d��d�S)zfIf in the decodable state, return the feature itself, otherwise flatten the feature into a dictionary.r)�Value�binary�stringr')�featuresrQr$)r/rQs r!�flattenz Video.flatten�sK��#�#�#�#�#�#��{� �D�D���x�����h����� r �storagec��tj�|j��rrtjdgt |��ztj�����}tj�||gddg|� �����}�n�tj� |j��rrtjdgt |��ztj �����}tj�||gddg|� �����}�n tj� |j���r|j� d��dkr|�d��}n8tjdgt |��ztj�����}|j� d��dkr|�d��}n8tjdgt |��ztj �����}tj�||gddg|� �����}n�tj�|j��r�tjd�|���D��tj�����}tjdgt |��ztj �����}tj�||gddg|� �����}t#||j��S)a'Cast an Arrow array to the Video arrow storage type. The Arrow types that can be converted to the Video pyarrow storage type are: - `pa.string()` - it must contain the "path" data - `pa.binary()` - it must contain the video bytes - `pa.struct({"bytes": pa.binary()})` - `pa.struct({"path": pa.string()})` - `pa.struct({"bytes": pa.binary(), "path": pa.string()})` - order doesn't matter - `pa.list(*)` - it must contain the video array data Args: storage (`Union[pa.StringArray, pa.StructArray, pa.ListArray]`): PyArrow array to cast. Returns: `pa.StructArray`: Array in the Video arrow storage type, that is `pa.struct({"bytes": pa.binary(), "path": pa.string()})`. N)rErr)�maskrc�d�g|]-}|�'ttj|����dnd��.S)Nr)r=r:r;)�.0�arrs r!� <listcomp>z&Video.cast_storage.<locals>.<listcomp>�s8��u�u�u�Z]�C�O����#���/�/��8�8�QU�u�u�ur )�pa�types� is_stringrEr;�lenrR� StructArray� from_arrays�is_null� is_binaryrS� is_struct�get_field_indexr�is_list� to_pylistrr()r/rV� bytes_array� path_arrays r!� cast_storagezVideo.cast_storage�s���& �8� � �g�l� +� +� ��(�D�6�C��L�L�#8�r�y�{�{�K�K�K�K��n�0�0�+�w�1G�'�SY�IZ�ah�ap�ap�ar�ar�0�s�s�G�G� �X� � �� � -� -� ���4�&�3�w�<�<�"7�b�i�k�k�J�J�J�J��n�0�0�'�:�1F��RX�HY�`g�`o�`o�`q�`q�0�r�r�G�G� �X� � �� � -� -� ��|�+�+�G�4�4��9�9�%�m�m�G�4�4� � � �h��v��G� � �'<�2�9�;�;�O�O�O� ��|�+�+�F�3�3�q�8�8�$�]�]�6�2�2� � ��X�t�f�s�7�|�|�&;�"�)�+�+�N�N�N� ��n�0�0�+�z�1J�W�V\�L]�dk�ds�ds�du�du�0�v�v�G�G� �X� � �g�l� +� +� ��(�u�u�ah�ar�ar�at�at�u�u�u��Y�[�[����K���4�&�3�w�<�<�"7�b�i�k�k�J�J�J�J��n�0�0��j�)�G�V�+<�;�CV�CV�CX�CX�1���G��'�4�<�0�0�0r r.)"rrr�__doc__r$�boolrr%rrr&rr]�structrRrSr(rrr,r0r rrr:r<rGr?rOrU� StringArrayra� ListArrayrkrr r!r#r#s��������"�"�H�F�D�����B��� ����7�E�8�C�=�7�7�7�&�R�Y������i�b�i�k�k�'R�'R�S�S�G�X�c�]�S�S�S���w�U��?�?�?�E�3�?�?�?����*O�E�#�u�g�r�z�=�*X�$Y�*O�^e�*O�*O�*O�*O�^DH�1�1��S�'�\�"�1�$�D��e�D�#�I�.>�)>�$?�@�1� � 1�1�1�1�f  ��}�d�3� �3E�.F�F�G�  �  �  �  �,1�E�"�.�"�.�"�,�*V�$W�,1�\^�\j�,1�,1�,1�,1�,1�,1r r#rNrr2c��t���)zPConvert a torchvision Video object to bytes using native compression if possible��NotImplementedError�rNs r!�video_to_bytesru�s�� � � �r c�N�t|d��r|jStd���)NrMzXEncoding a VideoReader that doesn't come from datasets.Video.decode() is not implemented)�hasattrrMrsrts r!r>r>�s2���u�m�$�$� �� � �!� f� � � r r;c��t���r.rr)r;s r!r=r=�s�� � � �r rrH�streamc�b�ddl}ddlm}ddlm}|�i}|�d��d}|�tj��r tj n tj }t||��}|�|� |d��nd} t| ���} t|d| � ��} t�|��} |��| _| jd krt%d | j�����|�| d � ��| _|�d��d} t+|�d����dkrdn't-|�d��d��}| |i| _| jjdi| j��| _| S)Nr)�get_video_backendrz::������repo_id)�token�rb)�download_config�pyavz9Unsupported video backend for VideoReader from HF files: �ignore)�metadata_errors�:rr)�av� torchvisionr{r7r�split� startswithr � HF_ENDPOINT�HUB_DATASETS_URL�HUB_DATASETS_HFFS_URLrr@r r�object�__new__�backendrJ�open� containerr`�int� pyav_streamr$�_c)rrHryr�r{r� source_url�pattern�source_url_fieldsr~r��f�vr� stream_type� stream_ids r!rLrL�s����I�I�I�-�-�-�-�-�-�*�*�*�*�*�*�� ������D�!�!�"�%�J�)3�)>�)>�v�?Q�)R�)R�t�f�%�%�X^�Xt�G�&�z�7�;�;��CT�C`� � !� !�"3�I�">� ?� ?� ?�fj�E�$�5�1�1�1�O� �d�D�/�:�:�:�A� ��� � $� $�B�"�"�$�$�B�J� �z�V����c�WY�Wa�c�c�d�d�d��7�7�1�h�7�7�7�B�L��,�,�s�#�#�A�&�K�����c�*�*�+�+�q�0�0���c�&�,�,�s�:K�:K�A�:N�6O�6O�I�!�9�-�B�N� �B�L� � 1� 1�"�.� 1� 1�B�E� �Ir )NrN))rA� dataclassesrr�typingrrrrr r �numpyr:�pyarrowr]�r �download.download_configr �tabler�utils.file_utilsrr�utils.py_utilsrr7rrTrrr#rrur>r<r=rr?rmrLrr r!�<module>r�s-�� � � � �(�(�(�(�(�(�(�(�K�K�K�K�K�K�K�K�K�K�K�K�K�K�K�K���������������5�5�5�5�5�5�������3�3�3�3�3�3�3�3�+�+�+�+�+�+��&�*�*�*�*�*�*�%�%�%�%�%�%������i����  �G1�G1�G1�G1�G1�G1�G1� ��G1�T �-� �E� � � � �  �M� �g� � � � � �2�:� �'� � � � �_f��� ��"*�4��U�4��9�5E�0E�+F�"G��X[��������r
Memory