�
K�g2 � � � d Z ddlmZmZmZmZmZ ddlmZ ddl m
Z
ddlmZ ddl
mZ ddlmZ d d
lmZ d dlmZ erddlmZmZ G d
� de� � ZdS )a� This module implements an Earley parser with a dynamic lexer
The core Earley algorithm used here is based on Elizabeth Scott's implementation, here:
https://www.sciencedirect.com/science/article/pii/S1571066108001497
That is probably the best reference for understanding the algorithm here.
The Earley parser outputs an SPPF-tree as per that document. The SPPF tree format
is better documented here:
http://www.bramvandersanden.com/post/2014/06/shared-packed-parse-forest/
Instead of running a lexer beforehand, or using a costy char-by-char method, this parser
uses regular expressions by necessity, achieving high-performance while maintaining all of
Earley's power in parsing any CFG.
� )�
TYPE_CHECKING�Callable�Optional�List�Any)�defaultdict� )�Tree)�UnexpectedCharacters)�Token��Terminal� )�Parser)� TokenNode)� LexerConf�
ParserConfc �j � e Zd Zdddedfdddddeded ed
edeeeege f defd
�Z
dd�ZdS )r TF�
lexer_confr �parser_confr �term_matcher�resolve_ambiguity�complete_lex�debug�
tree_class�ordered_setsc
�v � t j | |||||||� � d� |j D � � | _ || _ d S )Nc �, � g | ]}t |� � ��S � r
)�.0�ts �d/home/asafur/pinokio/api/open-webui.git/app/env/lib/python3.11/site-packages/lark/parsers/xearley.py�
<listcomp>z#Parser.__init__.<locals>.<listcomp>$ s � �>�>�>�q�x��{�{�>�>�>� )�
BaseParser�__init__�ignorer ) �selfr r r r r r r r s r"