Parser Tree¶
The parser tree is returned by calling parso.Grammar.parse()
.
Note
Note that parso positions are always 1 based for lines and zero based for columns. This means the first position in a file is (1, 0).
Parser Tree Base Classes¶
Generally there are two types of classes you will deal with:
parso.tree.Leaf
and parso.tree.BaseNode
.
-
class
parso.tree.
BaseNode
(children: List[parso.tree.NodeOrLeaf])[source]¶ Bases:
parso.tree.NodeOrLeaf
The super class for all nodes. A node has children, a type and possibly a parent node.
-
children
¶ A list of
NodeOrLeaf
child nodes.
-
start_pos
¶ Returns the starting position of the prefix as a tuple, e.g. (3, 4).
Return tuple of int: (line, column)
-
get_start_pos_of_prefix
()[source]¶ Returns the start_pos of the prefix. This means basically it returns the end_pos of the last prefix. The get_start_pos_of_prefix() of the prefix + in 2 + 1 would be (1, 1), while the start_pos is (1, 2).
Return tuple of int: (line, column)
-
end_pos
¶ Returns the end position of the prefix as a tuple, e.g. (3, 4).
Return tuple of int: (line, column)
-
get_code
(include_prefix=True)[source]¶ Returns the code that was the input for the parser for this node.
Parameters: include_prefix – Removes the prefix (whitespace and comments) of e.g. a statement.
-
get_leaf_for_position
(position, include_prefixes=False)[source]¶ Get the
parso.tree.Leaf
atposition
Parameters: Returns: parso.tree.Leaf
atposition
, orNone
-
-
class
parso.tree.
Leaf
(value: str, start_pos: Tuple[int, int], prefix: str = '')[source]¶ Bases:
parso.tree.NodeOrLeaf
Leafs are basically tokens with a better API. Leafs exactly know where they were defined and what text preceeds them.
-
value
¶ str()
The value of the current token.
-
prefix
¶ str()
Typically a mixture of whitespace and comments. Stuff that is syntactically irrelevant for the syntax tree.
-
start_pos
¶ Returns the starting position of the prefix as a tuple, e.g. (3, 4).
Return tuple of int: (line, column)
-
get_start_pos_of_prefix
()[source]¶ Returns the start_pos of the prefix. This means basically it returns the end_pos of the last prefix. The get_start_pos_of_prefix() of the prefix + in 2 + 1 would be (1, 1), while the start_pos is (1, 2).
Return tuple of int: (line, column)
-
get_code
(include_prefix=True)[source]¶ Returns the code that was the input for the parser for this node.
Parameters: include_prefix – Removes the prefix (whitespace and comments) of e.g. a statement.
-
end_pos
¶ Returns the end position of the prefix as a tuple, e.g. (3, 4).
Return tuple of int: (line, column)
-
All nodes and leaves have these methods/properties:
-
class
parso.tree.
NodeOrLeaf
[source]¶ Bases:
object
The base class for nodes and leaves.
-
type
= None¶ The type is a string that typically matches the types of the grammar file.
-
get_root_node
()[source]¶ Returns the root node of a parser tree. The returned node doesn’t have a parent node like all the other nodes/leaves.
-
get_next_sibling
()[source]¶ Returns the node immediately following this node in this parent’s children list. If this node does not have a next sibling, it is None
-
get_previous_sibling
()[source]¶ Returns the node immediately preceding this node in this parent’s children list. If this node does not have a previous sibling, it is None.
-
get_previous_leaf
()[source]¶ Returns the previous leaf in the parser tree. Returns None if this is the first element in the parser tree.
-
get_next_leaf
()[source]¶ Returns the next leaf in the parser tree. Returns None if this is the last element in the parser tree.
-
start_pos
¶ Returns the starting position of the prefix as a tuple, e.g. (3, 4).
Return tuple of int: (line, column)
-
end_pos
¶ Returns the end position of the prefix as a tuple, e.g. (3, 4).
Return tuple of int: (line, column)
-
get_start_pos_of_prefix
()[source]¶ Returns the start_pos of the prefix. This means basically it returns the end_pos of the last prefix. The get_start_pos_of_prefix() of the prefix + in 2 + 1 would be (1, 1), while the start_pos is (1, 2).
Return tuple of int: (line, column)
-
get_code
(include_prefix=True)[source]¶ Returns the code that was the input for the parser for this node.
Parameters: include_prefix – Removes the prefix (whitespace and comments) of e.g. a statement.
-
search_ancestor
(*node_types) → Optional[parso.tree.BaseNode][source]¶ Recursively looks at the parents of this node or leaf and returns the first found node that matches
node_types
. ReturnsNone
if no matching node is found.Parameters: node_types – type names that are searched for.
-
dump
(*, indent: Union[str, int, None] = 4) → str[source]¶ Returns a formatted dump of the parser tree rooted at this node or leaf. This is mainly useful for debugging purposes.
The
indent
parameter is interpreted in a similar way asast.dump()
. Ifindent
is a non-negative integer or string, then the tree will be pretty-printed with that indent level. An indent level of 0, negative, or""
will only insert newlines.None
selects the single line representation. Using a positive integer indent indents that many spaces per level. Ifindent
is a string (such as"\t"
), that string is used to indent each level.Parameters: indent – Indentation style as described above. The default indentation is 4 spaces, which yields a pretty-printed dump. >>> import parso >>> print(parso.parse("lambda x, y: x + y").dump()) Module([ Lambda([ Keyword('lambda', (1, 0)), Param([ Name('x', (1, 7), prefix=' '), Operator(',', (1, 8)), ]), Param([ Name('y', (1, 10), prefix=' '), ]), Operator(':', (1, 11)), PythonNode('arith_expr', [ Name('x', (1, 13), prefix=' '), Operator('+', (1, 15), prefix=' '), Name('y', (1, 17), prefix=' '), ]), ]), EndMarker('', (1, 18)), ])
-
Python Parser Tree¶
This is the syntax tree for Python 3 syntaxes. The classes represent syntax elements like functions and imports.
All of the nodes can be traced back to the Python grammar file. If you want to know how a tree is structured, just analyse that file (for each Python version it’s a bit different).
There’s a lot of logic here that makes it easier for Jedi (and other libraries) to deal with a Python syntax tree.
By using parso.tree.NodeOrLeaf.get_code()
on a module, you can get
back the 1-to-1 representation of the input given to the parser. This is
important if you want to refactor a parser tree.
>>> from parso import parse
>>> parser = parse('import os')
>>> module = parser.get_root_node()
>>> module
<Module: @1-1>
Any subclasses of Scope
, including Module
has an attribute
iter_imports
:
>>> list(module.iter_imports())
[<ImportName: import os@1,0>]
Changes to the Python Grammar¶
A few things have changed when looking at Python grammar files:
Param
does not exist in Python grammar files. It is essentially a part of aparameters
node. parso splits it up to make it easier to analyse parameters. However this just makes it easier to deal with the syntax tree, it doesn’t actually change the valid syntax.- A few nodes like lambdef and lambdef_nocond have been merged in the syntax tree to make it easier to do deal with them.
Parser Tree Classes¶
-
class
parso.python.tree.
PythonLeaf
(value: str, start_pos: Tuple[int, int], prefix: str = '')[source]¶ Bases:
parso.python.tree.PythonMixin
,parso.tree.Leaf
-
get_start_pos_of_prefix
()[source]¶ Basically calls
parso.tree.NodeOrLeaf.get_start_pos_of_prefix()
.
-
-
class
parso.python.tree.
PythonNode
(type, children)[source]¶ Bases:
parso.python.tree.PythonMixin
,parso.tree.Node
-
class
parso.python.tree.
PythonErrorNode
(children: List[parso.tree.NodeOrLeaf])[source]¶ Bases:
parso.python.tree.PythonMixin
,parso.tree.ErrorNode
-
class
parso.python.tree.
PythonErrorLeaf
(token_type, value, start_pos, prefix='')[source]¶ Bases:
parso.tree.ErrorLeaf
,parso.python.tree.PythonLeaf
-
class
parso.python.tree.
EndMarker
(value: str, start_pos: Tuple[int, int], prefix: str = '')[source]¶ Bases:
parso.python.tree._LeafWithoutNewlines
-
type
= 'endmarker'¶
-
-
class
parso.python.tree.
Newline
(value: str, start_pos: Tuple[int, int], prefix: str = '')[source]¶ Bases:
parso.python.tree.PythonLeaf
Contains NEWLINE and ENDMARKER tokens.
-
type
= 'newline'¶
-
-
class
parso.python.tree.
Name
(value: str, start_pos: Tuple[int, int], prefix: str = '')[source]¶ Bases:
parso.python.tree._LeafWithoutNewlines
A string. Sometimes it is important to know if the string belongs to a name or not.
-
type
= 'name'¶
-
-
class
parso.python.tree.
Literal
(value: str, start_pos: Tuple[int, int], prefix: str = '')[source]¶ Bases:
parso.python.tree.PythonLeaf
-
class
parso.python.tree.
Number
(value: str, start_pos: Tuple[int, int], prefix: str = '')[source]¶ Bases:
parso.python.tree.Literal
-
type
= 'number'¶
-
-
class
parso.python.tree.
String
(value: str, start_pos: Tuple[int, int], prefix: str = '')[source]¶ Bases:
parso.python.tree.Literal
-
type
= 'string'¶
-
string_prefix
¶
-
-
class
parso.python.tree.
FStringString
(value: str, start_pos: Tuple[int, int], prefix: str = '')[source]¶ Bases:
parso.python.tree.PythonLeaf
f-strings contain f-string expressions and normal python strings. These are the string parts of f-strings.
-
type
= 'fstring_string'¶
-
-
class
parso.python.tree.
FStringStart
(value: str, start_pos: Tuple[int, int], prefix: str = '')[source]¶ Bases:
parso.python.tree.PythonLeaf
f-strings contain f-string expressions and normal python strings. These are the string parts of f-strings.
-
type
= 'fstring_start'¶
-
-
class
parso.python.tree.
FStringEnd
(value: str, start_pos: Tuple[int, int], prefix: str = '')[source]¶ Bases:
parso.python.tree.PythonLeaf
f-strings contain f-string expressions and normal python strings. These are the string parts of f-strings.
-
type
= 'fstring_end'¶
-
-
class
parso.python.tree.
Operator
(value: str, start_pos: Tuple[int, int], prefix: str = '')[source]¶ Bases:
parso.python.tree._LeafWithoutNewlines
,parso.python.tree._StringComparisonMixin
-
type
= 'operator'¶
-
-
class
parso.python.tree.
Keyword
(value: str, start_pos: Tuple[int, int], prefix: str = '')[source]¶ Bases:
parso.python.tree._LeafWithoutNewlines
,parso.python.tree._StringComparisonMixin
-
type
= 'keyword'¶
-
-
class
parso.python.tree.
Scope
(children)[source]¶ Bases:
parso.python.tree.PythonBaseNode
,parso.python.tree.DocstringMixin
Super class for the parser tree, which represents the state of a python text file. A Scope is either a function, class or lambda.
-
class
parso.python.tree.
Module
(children)[source]¶ Bases:
parso.python.tree.Scope
The top scope, which is always a module. Depending on the underlying parser this may be a full module or just a part of a module.
-
type
= 'file_input'¶
-
-
class
parso.python.tree.
Decorator
(children: List[parso.tree.NodeOrLeaf])[source]¶ Bases:
parso.python.tree.PythonBaseNode
-
type
= 'decorator'¶
-
-
class
parso.python.tree.
ClassOrFunc
(children)[source]¶ Bases:
parso.python.tree.Scope
-
name
¶ Returns the Name leaf that defines the function or class name.
-
-
class
parso.python.tree.
Class
(children)[source]¶ Bases:
parso.python.tree.ClassOrFunc
Used to store the parsed contents of a python class.
-
type
= 'classdef'¶
-
-
class
parso.python.tree.
Function
(children)[source]¶ Bases:
parso.python.tree.ClassOrFunc
Used to store the parsed contents of a python function.
Children:
0. <Keyword: def> 1. <Name> 2. parameter list (including open-paren and close-paren <Operator>s) 3. or 5. <Operator: :> 4. or 6. Node() representing function body 3. -> (if annotation is also present) 4. annotation (if present)
-
type
= 'funcdef'¶
-
name
¶ Returns the Name leaf that defines the function or class name.
-
iter_raise_stmts
()[source]¶ Returns a generator of raise_stmt. Includes raise statements inside try-except blocks
-
annotation
¶ Returns the test node after -> or None if there is no annotation.
-
-
class
parso.python.tree.
Lambda
(children)[source]¶ Bases:
parso.python.tree.Function
Lambdas are basically trimmed functions, so give it the same interface.
Children:
0. <Keyword: lambda> *. <Param x> for each argument x -2. <Operator: :> -1. Node() representing body
-
type
= 'lambdef'¶
-
name
¶ Raises an AttributeError. Lambdas don’t have a defined name.
-
annotation
¶ Returns None, lambdas don’t have annotations.
-
-
class
parso.python.tree.
IfStmt
(children: List[parso.tree.NodeOrLeaf])[source]¶ Bases:
parso.python.tree.Flow
-
type
= 'if_stmt'¶
-
get_test_nodes
()[source]¶ E.g. returns all the test nodes that are named as x, below:
- if x:
- pass
- elif x:
- pass
-
-
class
parso.python.tree.
WhileStmt
(children: List[parso.tree.NodeOrLeaf])[source]¶ Bases:
parso.python.tree.Flow
-
type
= 'while_stmt'¶
-
-
class
parso.python.tree.
ForStmt
(children: List[parso.tree.NodeOrLeaf])[source]¶ Bases:
parso.python.tree.Flow
-
type
= 'for_stmt'¶
-
-
class
parso.python.tree.
TryStmt
(children: List[parso.tree.NodeOrLeaf])[source]¶ Bases:
parso.python.tree.Flow
-
type
= 'try_stmt'¶
-
-
class
parso.python.tree.
WithStmt
(children: List[parso.tree.NodeOrLeaf])[source]¶ Bases:
parso.python.tree.Flow
-
type
= 'with_stmt'¶
-
-
class
parso.python.tree.
Import
(children: List[parso.tree.NodeOrLeaf])[source]¶
-
class
parso.python.tree.
ImportFrom
(children: List[parso.tree.NodeOrLeaf])[source]¶ Bases:
parso.python.tree.Import
-
type
= 'import_from'¶
-
get_defined_names
(include_setitem=False)[source]¶ Returns the a list of Name that the import defines. The defined names are set after import or in case an alias - as - is present that name is returned.
-
level
¶ The level parameter of
__import__
.
-
-
class
parso.python.tree.
ImportName
(children: List[parso.tree.NodeOrLeaf])[source]¶ Bases:
parso.python.tree.Import
For
import_name
nodes. Covers normal imports withoutfrom
.-
type
= 'import_name'¶
-
get_defined_names
(include_setitem=False)[source]¶ Returns the a list of Name that the import defines. The defined names is always the first name after import or in case an alias - as - is present that name is returned.
-
level
¶ The level parameter of
__import__
.
-
-
class
parso.python.tree.
KeywordStatement
(children: List[parso.tree.NodeOrLeaf])[source]¶ Bases:
parso.python.tree.PythonBaseNode
For the following statements: assert, del, global, nonlocal, raise, return, yield.
pass, continue and break are not in there, because they are just simple keywords and the parser reduces it to a keyword.
-
type
¶ Keyword statements start with the keyword and end with _stmt. You can crosscheck this with the Python grammar.
-
keyword
¶
-
-
class
parso.python.tree.
AssertStmt
(children: List[parso.tree.NodeOrLeaf])[source]¶ Bases:
parso.python.tree.KeywordStatement
-
assertion
¶
-
-
class
parso.python.tree.
YieldExpr
(children: List[parso.tree.NodeOrLeaf])[source]¶ Bases:
parso.python.tree.PythonBaseNode
-
type
= 'yield_expr'¶
-
-
class
parso.python.tree.
ExprStmt
(children: List[parso.tree.NodeOrLeaf])[source]¶ Bases:
parso.python.tree.PythonBaseNode
,parso.python.tree.DocstringMixin
-
type
= 'expr_stmt'¶
-
-
class
parso.python.tree.
NamedExpr
(children: List[parso.tree.NodeOrLeaf])[source]¶ Bases:
parso.python.tree.PythonBaseNode
-
type
= 'namedexpr_test'¶
-
-
class
parso.python.tree.
Param
(children, parent=None)[source]¶ Bases:
parso.python.tree.PythonBaseNode
It’s a helper class that makes business logic with params much easier. The Python grammar defines no
param
node. It defines it in a different way that is not really suited to working with parameters.-
type
= 'param'¶
-
star_count
¶ Is 0 in case of foo, 1 in case of *foo or 2 in case of **foo.
-
default
¶ The default is the test node that appears after the =. Is None in case no default is present.
-
annotation
¶ The default is the test node that appears after :. Is None in case no annotation is present.
-
name
¶ The Name leaf of the param.
-
position_index
¶ Property for the positional index of a paramter.
-
-
class
parso.python.tree.
SyncCompFor
(children: List[parso.tree.NodeOrLeaf])[source]¶ Bases:
parso.python.tree.PythonBaseNode
-
type
= 'sync_comp_for'¶
-
-
parso.python.tree.
CompFor
¶ alias of
parso.python.tree.SyncCompFor
-
class
parso.python.tree.
UsedNamesMapping
(dct)[source]¶ Bases:
collections.abc.Mapping
This class exists for the sole purpose of creating an immutable dict.
Utility¶
-
parso.tree.
search_ancestor
(node: parso.tree.NodeOrLeaf, *node_types) → Optional[parso.tree.BaseNode][source]¶ Recursively looks at the parents of a node and returns the first found node that matches
node_types
. ReturnsNone
if no matching node is found.This function is deprecated, use
NodeOrLeaf.search_ancestor()
instead.Parameters: - node – The ancestors of this node will be checked.
- node_types – type names that are searched for.