- Implementing Stack Oriented Languages
- Implementing Stack Oriented Languages — Part 2
- Implementing Stack Oriented Languages – Part 3
- Implementing Stack Oriented Languages – Part 4
Post Stastics
- This post has 5444 words.
- Estimated read time is 25.92 minute(s).
Parsing Keywords
Most programming languages include special words used by the language to perform operations on data. These may include I/O operations, data management, etc. Our language is no different. We have included many keywords to perform operations on the stack, perform I/O, and provide flow control. In this installment, we will add keyword handling to our language.
Before we move on to adding more features to our language, I think it is high time we combine our disparate parts into a single entity. This will turn our pieces into a real program! As a real program, we need to consider how the end-user will interact with our program. For example, right now we have test programs hard-coded into the parser. Our users will not want to hard code their programs into the parser. However, they need a method to pass their source code to our language for interpretation. To allow this interaction we will create a file named sol.py. SOL is our stack-oriented-language. This file will pull all the pieces together and allow the end-user to pass in their source file on the command line.
#! usr/bin/env/python3
from lexer import Lexer
from parser import Parser
import argparse
def main(src):
_lexer = Lexer(src)
_parser = Parser(_lexer)
_parser.parse()
if __name__ == '__main__':
argparse = argparse.ArgumentParser()
argparse.add_argument("--infile",
help="Input source file", required=True)
argparse.add_argument("-v", "--verbose", help="Echo source file", action="store_true")
args = argparse.parse_args()
infile = args.infile
sorce = ''
with open(infile, 'r') as ifh:
source = ifh.read()
if args.verbose:
print(source)
main(source)
#! usr/bin/env/python3
from lexer import Lexer
from parser import Parser
import argparse
def main(src):
_lexer = Lexer(src)
_parser = Parser(_lexer)
_parser.parse()
if __name__ == '__main__':
argparse = argparse.ArgumentParser()
argparse.add_argument("--infile",
help="Input source file", required=True)
argparse.add_argument("-v", "--verbose", help="Echo source file", action="store_true")
args = argparse.parse_args()
infile = args.infile
sorce = ''
with open(infile, 'r') as ifh:
source = ifh.read()
if args.verbose:
print(source)
main(source)
#! usr/bin/env/python3
from lexer import Lexer
from parser import Parser
import argparse
def main(src):
_lexer = Lexer(src)
_parser = Parser(_lexer)
_parser.parse()
if __name__ == '__main__':
argparse = argparse.ArgumentParser()
argparse.add_argument("--infile",
help="Input source file", required=True)
argparse.add_argument("-v", "--verbose", help="Echo source file", action="store_true")
args = argparse.parse_args()
infile = args.infile
sorce = ''
with open(infile, 'r') as ifh:
source = ifh.read()
if args.verbose:
print(source)
main(source)
Looking at the new file sol.py, we can see that we have the typical python if dunder name, dunder ‘main’ statement used to execute the main routine. First however, we use the argparser to handle command-line parameters. The program supports two possible parameters, “-i”/”–infile’ and “-v” / “–verbose”. The “–infile” parameter is used to pass the input source file to our lexer/parser. The “–verbose” parameter is used to display the input file to the user.
We can easily call the approgram like so:
python3 sol.py --infile test_arith.sol --verbose
python3 sol.py --infile test_arith.sol --verbose
python3 sol.py --infile test_arith.sol --verbose
Or
python3 sol.py -i test_arith.sol -v
python3 sol.py -i test_arith.sol -v
python3 sol.py -i test_arith.sol -v
Next, create a directory named “prog” as a child directory of your code. We will place our test programs in this folder.
Now create a file in the new “prog” folder named “test_arith.sol”. This file will hold a small program to test the arithmetic operations.
\ test_arith.sol
\
\ +, -, *, /, **
\
\ Arithmetic operators take the top two items
\ from the stack and performs the operations
\ on them, placing the results back on the
\ stack.
2 3 +
dump
5 *
dump
5 /
dump
3 -
dump
8 **
dump
\ test_arith.sol
\
\ +, -, *, /, **
\
\ Arithmetic operators take the top two items
\ from the stack and performs the operations
\ on them, placing the results back on the
\ stack.
2 3 +
dump
5 *
dump
5 /
dump
3 -
dump
8 **
dump
\ test_arith.sol \ \ +, -, *, /, ** \ \ Arithmetic operators take the top two items \ from the stack and performs the operations \ on them, placing the results back on the \ stack. 2 3 + dump 5 * dump 5 / dump 3 - dump 8 ** dump
In this program we place the values 2 and 3 on the stack. Then we use the “+” operator to pull the two values off the stack and add them together. The operator places the result back on the stack. We then add 5 to the stack and use the the “*” operator to multiply the two values on the stack together and replace them with the result of the multiplication. We then divide this value by 5 and continue on to subtract 3 and raise the result (2), to the power of 8.
Before we run this program, we need to add the “power” operator and “dump” command to the parser. First, at the bottom of the Arithmetic operator handlers in the parser’s parse method, we will add the code to handle the raising a number to a power.
# Parser.py
...
def parse(self):
# get token and decide what to do with it
tok = self.current_tok
...
# Arithmetic
elif tok.type == TT_PLUS:
right = self.data.pop()
...
elif tok.type == TT_POW:
right = self.data.pop()
left = self.data.pop()
result = left ** right
self.data.push(result)
# Logical Operations
...
# Parser.py
...
def parse(self):
# get token and decide what to do with it
tok = self.current_tok
...
# Arithmetic
elif tok.type == TT_PLUS:
right = self.data.pop()
...
elif tok.type == TT_POW:
right = self.data.pop()
left = self.data.pop()
result = left ** right
self.data.push(result)
# Logical Operations
...
# Parser.py
...
def parse(self):
# get token and decide what to do with it
tok = self.current_tok
...
# Arithmetic
elif tok.type == TT_PLUS:
right = self.data.pop()
...
elif tok.type == TT_POW:
right = self.data.pop()
left = self.data.pop()
result = left ** right
self.data.push(result)
# Logical Operations
...
That’s all we need to handle the power operation. However, we still need to add the “dump” command. Dump is a keyword in our language and the keywords are handled a bit differently than we handle the operators. With the operators, we match against the token type. However, all our keywords will have a token type of TT_KEYWORD. To tell which keyword we are dealing with we have to first ensure we are dealing with a keyword token and then match against the token value. All our keywords are handled this way.
# Parser.py
...
def parse(self):
...
# Handle KEYWORDS
elif tok.type == TT_KEYWORD:
# Process keyword
if tok.value == 'dump':
"""dump
Displays the data stack contents in the console.
"""
print(str(self.data))
# Parser.py
...
def parse(self):
...
# Handle KEYWORDS
elif tok.type == TT_KEYWORD:
# Process keyword
if tok.value == 'dump':
"""dump
Displays the data stack contents in the console.
"""
print(str(self.data))
# Parser.py
...
def parse(self):
...
# Handle KEYWORDS
elif tok.type == TT_KEYWORD:
# Process keyword
if tok.value == 'dump':
"""dump
Displays the data stack contents in the console.
"""
print(str(self.data))
As you can see this code simply nests the handling of all keywords under the outer “elif” statement in the parser’s parse method.
If you run this code, passing it our test_arith.sol file as input, you should receive an out put similar to:
\ +, -, *, /, **
\
\ Arithmetic operators take the top two items
\ from the stack and performs the operations
\ on them, placing the results back on the
\ stack.
2 3 +
dump
5 *
dump
5 /
dump
3 -
dump
8 **
dump
[ '5' ]
[ '25' ]
[ '5' ]
[ '2' ]
[ '256' ]
\ +, -, *, /, **
\
\ Arithmetic operators take the top two items
\ from the stack and performs the operations
\ on them, placing the results back on the
\ stack.
2 3 +
dump
5 *
dump
5 /
dump
3 -
dump
8 **
dump
[ '5' ]
[ '25' ]
[ '5' ]
[ '2' ]
[ '256' ]
\ +, -, *, /, ** \ \ Arithmetic operators take the top two items \ from the stack and performs the operations \ on them, placing the results back on the \ stack. 2 3 + dump 5 * dump 5 / dump 3 - dump 8 ** dump [ '5' ] [ '25' ] [ '5' ] [ '2' ] [ '256' ]
Dump is a great tool to use to see what is on the stack at any point in your program. However, it is meant for the developer’s use and not as an output method for the program. We will work on that later. For now, let’s work on the methods we need to manage and manipulate the stack.
Let’s start with the DROP keyword. Drop simply pops the first value off the stack and discards it. The value will be lost forever. Now let’s see how to implement the Drop operation.
# Handle Keywords
elif tok.type == TT_KEYWORD:
# Process keyword
if tok.value == 'dump':
"""dump
Display the data stack contents in the console
"""
print(str(self.data))
elif tok.value == 'drop':
"""drop
Remove the top value from the stack and discard it.
"""
self.data.pop()
# Handle Keywords
elif tok.type == TT_KEYWORD:
# Process keyword
if tok.value == 'dump':
"""dump
Display the data stack contents in the console
"""
print(str(self.data))
elif tok.value == 'drop':
"""drop
Remove the top value from the stack and discard it.
"""
self.data.pop()
# Handle Keywords
elif tok.type == TT_KEYWORD:
# Process keyword
if tok.value == 'dump':
"""dump
Display the data stack contents in the console
"""
print(str(self.data))
elif tok.value == 'drop':
"""drop
Remove the top value from the stack and discard it.
"""
self.data.pop()
I think that’s got to be the simplest operation we have performed to date! Now let’s move on. The next operation we might like to complete on the stack would be to empty the stack. This is done using the CLEAR keyword. If you recall, the stack implementation we created includes a clear method for doing just this. So the code is as simple as it was for the pop keyword:
# Handle Keywords
elif tok.type == TT_KEYWORD:
# Process keyword
if tok.value == 'dump':
"""dump
Display the data stack contents in the console
"""
print(str(self.data))
elif tok.value == 'drop':
"""pop
Remove the top value from the stack and discard it.
"""
self.data.pop()
elif tok.value == 'clear':
"""clear
Empty the stack.
"""
self.data.clear()
# Handle Keywords
elif tok.type == TT_KEYWORD:
# Process keyword
if tok.value == 'dump':
"""dump
Display the data stack contents in the console
"""
print(str(self.data))
elif tok.value == 'drop':
"""pop
Remove the top value from the stack and discard it.
"""
self.data.pop()
elif tok.value == 'clear':
"""clear
Empty the stack.
"""
self.data.clear()
# Handle Keywords
elif tok.type == TT_KEYWORD:
# Process keyword
if tok.value == 'dump':
"""dump
Display the data stack contents in the console
"""
print(str(self.data))
elif tok.value == 'drop':
"""pop
Remove the top value from the stack and discard it.
"""
self.data.pop()
elif tok.value == 'clear':
"""clear
Empty the stack.
"""
self.data.clear()
One operation we may want is the SWAP operation. The Swap operation takes the top two values on the stack and swaps their order. This can be very helpful when values are in the wrong order for the next operation.
# Handle Keywords
elif tok.type == TT_KEYWORD:
# Process keyword
if tok.value == 'dump':
"""dump
Display the data stack contents in the console
"""
print(str(self.data))
elif tok.value == 'drop':
"""pop
Remove the top value from the stack and discard it.
"""
self.data.pop()
elif tok.value == 'clear':
"""clear
Empty the stack.
"""
self.data.clear()
elif tok.value.lower() == 'swap':
"""swap
Swaps the position of the top two items on the stack.
Ex: x y z --> x z y
"""
if self.data.count() < 2:
raise IndexError(
f'Attempt to SWAP on stack of size {self.data.count()}!\n\t\t\tStack must contain at least 2 elements to SWAP.')
n1 = self.data.pop()
n2 = self.data.pop()
self.data.push(n1)
self.data.push(n2)
# Handle Keywords
elif tok.type == TT_KEYWORD:
# Process keyword
if tok.value == 'dump':
"""dump
Display the data stack contents in the console
"""
print(str(self.data))
elif tok.value == 'drop':
"""pop
Remove the top value from the stack and discard it.
"""
self.data.pop()
elif tok.value == 'clear':
"""clear
Empty the stack.
"""
self.data.clear()
elif tok.value.lower() == 'swap':
"""swap
Swaps the position of the top two items on the stack.
Ex: x y z --> x z y
"""
if self.data.count() < 2:
raise IndexError(
f'Attempt to SWAP on stack of size {self.data.count()}!\n\t\t\tStack must contain at least 2 elements to SWAP.')
n1 = self.data.pop()
n2 = self.data.pop()
self.data.push(n1)
self.data.push(n2)
# Handle Keywords
elif tok.type == TT_KEYWORD:
# Process keyword
if tok.value == 'dump':
"""dump
Display the data stack contents in the console
"""
print(str(self.data))
elif tok.value == 'drop':
"""pop
Remove the top value from the stack and discard it.
"""
self.data.pop()
elif tok.value == 'clear':
"""clear
Empty the stack.
"""
self.data.clear()
elif tok.value.lower() == 'swap':
"""swap
Swaps the position of the top two items on the stack.
Ex: x y z --> x z y
"""
if self.data.count() < 2:
raise IndexError(
f'Attempt to SWAP on stack of size {self.data.count()}!\n\t\t\tStack must contain at least 2 elements to SWAP.')
n1 = self.data.pop()
n2 = self.data.pop()
self.data.push(n1)
self.data.push(n2)
All we do to swap the values on the stack is to pop the value off and then push them back on in the opposite order.
Sometimes we need to swap the top two pairs of values on the stack. We can use Swap2 for this. The code is as one would expect, we simply pop off the top four values and push them back onto the stack in the new order.
...
elif tok.value == 'swap2':
"""swap2
Swaps the position of the top two pairs of items on the stack.
Ex: w x y z --> y z w x
"""
if self.data.count() < 4:
raise IndexError(
f'Attempt to SWAP on stack of size {self.data.count()}!\n\t\t\tStack must contain at least 4 elements to SWAP2.')
n3 = self.data.pop()
n2 = self.data.pop()
n1 = self.data.pop()
n0 = self.data.pop()
self.data.push(n2)
self.data.push(n3)
self.data.push(n0)
self.data.push(n1)
...
elif tok.value == 'swap2':
"""swap2
Swaps the position of the top two pairs of items on the stack.
Ex: w x y z --> y z w x
"""
if self.data.count() < 4:
raise IndexError(
f'Attempt to SWAP on stack of size {self.data.count()}!\n\t\t\tStack must contain at least 4 elements to SWAP2.')
n3 = self.data.pop()
n2 = self.data.pop()
n1 = self.data.pop()
n0 = self.data.pop()
self.data.push(n2)
self.data.push(n3)
self.data.push(n0)
self.data.push(n1)
...
elif tok.value == 'swap2':
"""swap2
Swaps the position of the top two pairs of items on the stack.
Ex: w x y z --> y z w x
"""
if self.data.count() < 4:
raise IndexError(
f'Attempt to SWAP on stack of size {self.data.count()}!\n\t\t\tStack must contain at least 4 elements to SWAP2.')
n3 = self.data.pop()
n2 = self.data.pop()
n1 = self.data.pop()
n0 = self.data.pop()
self.data.push(n2)
self.data.push(n3)
self.data.push(n0)
self.data.push(n1)
All this swapping is good but sometimes we simply need to copy an item to the top of the stack. For this we use the OVER keyword. Again, we simply pop the value off the stack and push them back on in the order we desire:
elif tok.value == 'over':
"""over
Copies the second item on the stack to the head position.
Ex: x y z --> x y z y
"""
top = self.data.pop()
next = self.data.pop()
self.data.push(next)
self.data.push(top)
self.data.push(next)
elif tok.value == 'over':
"""over
Copies the second item on the stack to the head position.
Ex: x y z --> x y z y
"""
top = self.data.pop()
next = self.data.pop()
self.data.push(next)
self.data.push(top)
self.data.push(next)
elif tok.value == 'over':
"""over
Copies the second item on the stack to the head position.
Ex: x y z --> x y z y
"""
top = self.data.pop()
next = self.data.pop()
self.data.push(next)
self.data.push(top)
self.data.push(next)
Sometimes we will need to duplicate the value on the top of the stack. This is helpful because many of the operations we use will consume the value at the top of the stack and replace it with the result of the operation. So if we need to keep the original value for further processing, we need to create a copy. As expected we simply pop the value on the top of the stack and then push it back on the stack two times. Here’s the code:
...
elif tok.value.lower() == 'dup':
"""dup
Duplicates the item on the top of the stack.
Ex: x y z --> x y z z
"""
n1 = self.data.pop()
self.data.push(n1)
self.data.push(n1)
...
elif tok.value.lower() == 'dup':
"""dup
Duplicates the item on the top of the stack.
Ex: x y z --> x y z z
"""
n1 = self.data.pop()
self.data.push(n1)
self.data.push(n1)
...
elif tok.value.lower() == 'dup':
"""dup
Duplicates the item on the top of the stack.
Ex: x y z --> x y z z
"""
n1 = self.data.pop()
self.data.push(n1)
self.data.push(n1)
Duplicating items is so common that it is often helpful to have difference versions of dup to duplicate different numbers of items on the stack. We will provide three additional versions Dup2, Dup3, and Dup4:
elif tok.value == 'dup2':
"""dup2
Duplicates the top two items on the stack.
Ex: x y z --> x y z y z
"""
n1 = self.data.pop()
n2 = self.data.pop()
self.data.push(n2)
self.data.push(n1)
self.data.push(n2)
self.data.push(n1)
elif tok.value == 'dup3':
"""dup3
Duplicates the top three items on the stack.
Ex: x y z --> x y z x y z
"""
n1 = self.data.pop()
n2 = self.data.pop()
n3 = self.data.pop()
self.data.push(n3)
self.data.push(n2)
self.data.push(n1)
self.data.push(n3)
self.data.push(n2)
self.data.push(n1)
elif tok.value == 'dup4':
"""dup3
Duplicates the top three items on the stack.
Ex: w x y z --> w x y z w x y z
"""
n1 = self.data.pop()
n2 = self.data.pop()
n3 = self.data.pop()
n4 = self.data.pop()
self.data.push(n4)
self.data.push(n3)
self.data.push(n2)
self.data.push(n1)
self.data.push(n4)
self.data.push(n3)
self.data.push(n2)
self.data.push(n1)
elif tok.value == 'dup2':
"""dup2
Duplicates the top two items on the stack.
Ex: x y z --> x y z y z
"""
n1 = self.data.pop()
n2 = self.data.pop()
self.data.push(n2)
self.data.push(n1)
self.data.push(n2)
self.data.push(n1)
elif tok.value == 'dup3':
"""dup3
Duplicates the top three items on the stack.
Ex: x y z --> x y z x y z
"""
n1 = self.data.pop()
n2 = self.data.pop()
n3 = self.data.pop()
self.data.push(n3)
self.data.push(n2)
self.data.push(n1)
self.data.push(n3)
self.data.push(n2)
self.data.push(n1)
elif tok.value == 'dup4':
"""dup3
Duplicates the top three items on the stack.
Ex: w x y z --> w x y z w x y z
"""
n1 = self.data.pop()
n2 = self.data.pop()
n3 = self.data.pop()
n4 = self.data.pop()
self.data.push(n4)
self.data.push(n3)
self.data.push(n2)
self.data.push(n1)
self.data.push(n4)
self.data.push(n3)
self.data.push(n2)
self.data.push(n1)
elif tok.value == 'dup2':
"""dup2
Duplicates the top two items on the stack.
Ex: x y z --> x y z y z
"""
n1 = self.data.pop()
n2 = self.data.pop()
self.data.push(n2)
self.data.push(n1)
self.data.push(n2)
self.data.push(n1)
elif tok.value == 'dup3':
"""dup3
Duplicates the top three items on the stack.
Ex: x y z --> x y z x y z
"""
n1 = self.data.pop()
n2 = self.data.pop()
n3 = self.data.pop()
self.data.push(n3)
self.data.push(n2)
self.data.push(n1)
self.data.push(n3)
self.data.push(n2)
self.data.push(n1)
elif tok.value == 'dup4':
"""dup3
Duplicates the top three items on the stack.
Ex: w x y z --> w x y z w x y z
"""
n1 = self.data.pop()
n2 = self.data.pop()
n3 = self.data.pop()
n4 = self.data.pop()
self.data.push(n4)
self.data.push(n3)
self.data.push(n2)
self.data.push(n1)
self.data.push(n4)
self.data.push(n3)
self.data.push(n2)
self.data.push(n1)
Another operation that can be helpful is the ability to rotate the stack. The ROL keyword rotates the values in the stack by moving the top element to the bottom and shifting every other element up one position. The ROR keyword takes the bottom element and moves it to the top of the stack shifting every other element down one position. Again, these operations are performed using simple calls to the stack methods.
...
elif tok.value == 'rol':
"""ror
Rotates the values in the stack by moving the top
element to the bottom and shifting everything else up
one position.
Ex: rot x y z -> y z x
"""
item = self.data.pop()
last = self.data.que(item)
elif tok.value == 'ror':
"""rot
Rotates the values in the stack by moving the bottom
element to the top and pushing everything else down
one position.
Ex: rot x y z -> y z x
"""
last = self.data.tail()
self.data.push(last)
...
elif tok.value == 'rol':
"""ror
Rotates the values in the stack by moving the top
element to the bottom and shifting everything else up
one position.
Ex: rot x y z -> y z x
"""
item = self.data.pop()
last = self.data.que(item)
elif tok.value == 'ror':
"""rot
Rotates the values in the stack by moving the bottom
element to the top and pushing everything else down
one position.
Ex: rot x y z -> y z x
"""
last = self.data.tail()
self.data.push(last)
...
elif tok.value == 'rol':
"""ror
Rotates the values in the stack by moving the top
element to the bottom and shifting everything else up
one position.
Ex: rot x y z -> y z x
"""
item = self.data.pop()
last = self.data.que(item)
elif tok.value == 'ror':
"""rot
Rotates the values in the stack by moving the bottom
element to the top and pushing everything else down
one position.
Ex: rot x y z -> y z x
"""
last = self.data.tail()
self.data.push(last)
Ok, that’s about it for the stack manipulation operations. It’s hard to believe you can accomplish much with these primitive operations on such a simple data structure. But you’ll be surprised what can be achieved. Next, let’s tackle input/output methods.
I/O Operations
A program isn’t much use if you can’t get data into it or result out of it. This is the job of I/O routines. Our language will use a couple very simple I/O routines to display information in the console and read keyboard input.
The first routine we will implement is the EMIT keyword. Emit takes the value at the top of the stack and outputs it to the console as if it is a single ASCII character, using the lower order byte only. It is the programmer’s responsibility to ensure the value is printable as an ASCII character.
...
elif tok.value == 'emit':
"""emit
The word EMIT takes a single ASCII representation on the
stack, using the low-order byte only, and sends the
character to stdout. For example, in decimal:
65 EMIT↵ A ok
66 EMIT↵ B ok
67 EMIT↵ C ok
"""
ch = self.data.pop()
if ch > 0x110000 or ch < 0:
raise ValueError(f'[Error] character popped by emit: {ch} is out of printable range!')
char = chr(ch)
print(char, end='')
...
elif tok.value == 'emit':
"""emit
The word EMIT takes a single ASCII representation on the
stack, using the low-order byte only, and sends the
character to stdout. For example, in decimal:
65 EMIT↵ A ok
66 EMIT↵ B ok
67 EMIT↵ C ok
"""
ch = self.data.pop()
if ch > 0x110000 or ch < 0:
raise ValueError(f'[Error] character popped by emit: {ch} is out of printable range!')
char = chr(ch)
print(char, end='')
...
elif tok.value == 'emit':
"""emit
The word EMIT takes a single ASCII representation on the
stack, using the low-order byte only, and sends the
character to stdout. For example, in decimal:
65 EMIT↵ A ok
66 EMIT↵ B ok
67 EMIT↵ C ok
"""
ch = self.data.pop()
if ch > 0x110000 or ch < 0:
raise ValueError(f'[Error] character popped by emit: {ch} is out of printable range!')
char = chr(ch)
print(char, end='')
Next, we need a way to get data into the program. We will use the KEY keyword to enter data. The KEY command simply pauses program execution and waits for a key to be pressed. Once a key is pressed, the key value is pushed onto the stack for use by the program. This is a very primitive input method. We will work on more complex input methods later in this series. For this to work you will need to import the sys module. I place my imports grouped together near the top of the file to keep things organized.
...
import sys
from lexer import Lexer
...
class Parser:
...
def parse(self):
...
# Handle Keywords
elif tok.type == TT_KEYWORD:
# Process keyword
if tok.value == 'dump':
...
elif tok.value == 'key':
"""key
Pauses execution until a key is pressed. Once a key
is pressed, the key code is pushed onto the stack.
Ex: <A> --> 65
"""
getch = sys.stdin.read
key = None
while not key:
key = getch(1)
self.data.push(ord(key))
...
import sys
from lexer import Lexer
...
class Parser:
...
def parse(self):
...
# Handle Keywords
elif tok.type == TT_KEYWORD:
# Process keyword
if tok.value == 'dump':
...
elif tok.value == 'key':
"""key
Pauses execution until a key is pressed. Once a key
is pressed, the key code is pushed onto the stack.
Ex: <A> --> 65
"""
getch = sys.stdin.read
key = None
while not key:
key = getch(1)
self.data.push(ord(key))
...
import sys
from lexer import Lexer
...
class Parser:
...
def parse(self):
...
# Handle Keywords
elif tok.type == TT_KEYWORD:
# Process keyword
if tok.value == 'dump':
...
elif tok.value == 'key':
"""key
Pauses execution until a key is pressed. Once a key
is pressed, the key code is pushed onto the stack.
Ex: <A> --> 65
"""
getch = sys.stdin.read
key = None
while not key:
key = getch(1)
self.data.push(ord(key))
The final keywords we are going to implement this week are constant values used for printing. These values include CR for ASCII 13 which is the carriage return, LF: ASCII 10, line feed. SP ASCII 32 which is the space character. NULL, ASCII 0 which is the ASCII null character. BELL, ASCII 7 which is the bell character. On teletypes and older computers this character used to cause the system to emit a sound. On newer systems that no-longer occurs. BS, ASCII 8 is the backspace character. Further constants are the TAB, VT (Vertical tab), FF (Form feed), and ESC (Escape) characters. Each of these simple push their numeric value onto the stack.
...
# Helpful Constants
elif tok.value == 'cr': # Carriage Return
self.data.push(13)
elif tok.value == 'lf': # Line Feed
self.data.push(10)
elif tok.value == 'sp':
self.data.push(32)
elif tok.value == 'null': # NUll
self.data.push(0)
elif tok.value == 'bell': # System Bell
self.data.push(7)
elif tok.value == 'bs': # Backspace
self.data.push(8)
elif tok.value == 'tab': # TAB
self.data.push(9)
elif tok.value == 'vt': # Vertical TAB
self.data.push(11)
elif tok.value == 'ff': # Form Feed
self.data.push(12)
elif tok.value == 'esc': # Escape
self.data.push(27)
...
# Helpful Constants
elif tok.value == 'cr': # Carriage Return
self.data.push(13)
elif tok.value == 'lf': # Line Feed
self.data.push(10)
elif tok.value == 'sp':
self.data.push(32)
elif tok.value == 'null': # NUll
self.data.push(0)
elif tok.value == 'bell': # System Bell
self.data.push(7)
elif tok.value == 'bs': # Backspace
self.data.push(8)
elif tok.value == 'tab': # TAB
self.data.push(9)
elif tok.value == 'vt': # Vertical TAB
self.data.push(11)
elif tok.value == 'ff': # Form Feed
self.data.push(12)
elif tok.value == 'esc': # Escape
self.data.push(27)
...
# Helpful Constants
elif tok.value == 'cr': # Carriage Return
self.data.push(13)
elif tok.value == 'lf': # Line Feed
self.data.push(10)
elif tok.value == 'sp':
self.data.push(32)
elif tok.value == 'null': # NUll
self.data.push(0)
elif tok.value == 'bell': # System Bell
self.data.push(7)
elif tok.value == 'bs': # Backspace
self.data.push(8)
elif tok.value == 'tab': # TAB
self.data.push(9)
elif tok.value == 'vt': # Vertical TAB
self.data.push(11)
elif tok.value == 'ff': # Form Feed
self.data.push(12)
elif tok.value == 'esc': # Escape
self.data.push(27)
Ok, try writing some sample programs using the keywords and operators we have implemented. Run these programs using the new command-line interface and be sure to dump the stack contents so you can see the results of each operation. Verify that each operation does what you expect.
Ok, that’s it’s for this installment. Next time we will continue to implement features of the language. As always, I have included a complete listing below.
# keywords.py
KEYWORDS = [
# Stack Operators
'SWAP', 'SWAP2', 'DUMP', 'DROP', 'ROL', 'ROR', 'OVER',
'DUP', 'DUP2', 'DUP3', 'DUP4', 'CLEAR', 'DUMP',
# Conditional Operators
'IF', 'ELSE', 'THEN', 'NOT',
# Iterative
'DO', 'LOOP',
'WHILE', 'WEND',
# I/O Operators
'TYPE', # Outputs a string to stdout
'EMIT', # Outputs a character to stdout
'KEY', # Takes input character from stdin
# Special Character Values
# See ASCII Character Table for values
'CR', 'LF', 'NULL', 'BELL', 'BS', 'TAB',
'VT', 'FF', 'ESC',
# Logical Operators
'NEQ', 'AND', 'OR',
]
# keywords.py
KEYWORDS = [
# Stack Operators
'SWAP', 'SWAP2', 'DUMP', 'DROP', 'ROL', 'ROR', 'OVER',
'DUP', 'DUP2', 'DUP3', 'DUP4', 'CLEAR', 'DUMP',
# Conditional Operators
'IF', 'ELSE', 'THEN', 'NOT',
# Iterative
'DO', 'LOOP',
'WHILE', 'WEND',
# I/O Operators
'TYPE', # Outputs a string to stdout
'EMIT', # Outputs a character to stdout
'KEY', # Takes input character from stdin
# Special Character Values
# See ASCII Character Table for values
'CR', 'LF', 'NULL', 'BELL', 'BS', 'TAB',
'VT', 'FF', 'ESC',
# Logical Operators
'NEQ', 'AND', 'OR',
]
# keywords.py
KEYWORDS = [
# Stack Operators
'SWAP', 'SWAP2', 'DUMP', 'DROP', 'ROL', 'ROR', 'OVER',
'DUP', 'DUP2', 'DUP3', 'DUP4', 'CLEAR', 'DUMP',
# Conditional Operators
'IF', 'ELSE', 'THEN', 'NOT',
# Iterative
'DO', 'LOOP',
'WHILE', 'WEND',
# I/O Operators
'TYPE', # Outputs a string to stdout
'EMIT', # Outputs a character to stdout
'KEY', # Takes input character from stdin
# Special Character Values
# See ASCII Character Table for values
'CR', 'LF', 'NULL', 'BELL', 'BS', 'TAB',
'VT', 'FF', 'ESC',
# Logical Operators
'NEQ', 'AND', 'OR',
]
# lexer.py
from keywords import KEYWORDS
from tokens import *
class Lexer:
def __init__(self, text, line=None, col=None):
self.text = text
self.pos = 0
self.line = line if line else 1
self.col = col if col else 1
self.current_char = self.text[0]
#############################################
# Utility Methods #
#-------------------------------------------#
# These methods help manage the internal #
# workings of the lexer. #
#############################################
def advance(self):
"""advance():
Advances the current position in the character stream.
"""
if not self.is_eoi():
self.pos += 1
self.current_char = self.text[self.pos]
self.update_position()
else:
self.current_char = ''
def update_position(self):
"""update_position
Maintains the line and column positions for the current token scan.
"""
if self.current_char == '\n':
self.line += 1
self.col = 0
else:
self.col += 1
def is_eoi(self):
"""is_eoi():
:return:
True if current position is End Of Input stream.
Returns False otherwise.
"""
return self.pos >= (len(self.text) - 1)
def expected(self, expected, found):
"""expected
Raises error is expected and found are different.
"""
if expected == found:
return
else:
raise ValueError(f'[Error] Expected {expected} but found {found} in line: {self.line} position: {self.col}')
def peek(self):
"""peek
Returns next character in input stream after the current character.
Note: If we are at EOI (End Of Input) then a space character is returned.
"""
if not self.is_eoi():
return self.text[self.pos + 1]
else:
return ' ' # space
#############################################
# Recognizer Methods #
#-------------------------------------------#
# These methods collect the characters that #
# belong to a single multi-character token #
# into a single unit, and return that unit #
# to the caller. #
#############################################
def char(self):
if self.current_char != "'":
self.expected("'", self.current_char)
self.advance() # Consume opening single quote
char = self.current_char
self.advance() # Consume character
# Now we must have a closing single quote mark
if self.current_char != "'":
self.expected("'", self.current_char)
self.advance() # Consume closing single quote
return char
def string(self):
if self.current_char != '"':
self.expected('"', self.peek())
self.advance() # Consume leading '"'
string = ''
while self.current_char != '"':
string += self.current_char
self.advance()
self.advance() # Consume trailing '"'
return string
def number(self):
number = ''
while self.current_char.isdigit():
number += self.current_char
self.advance()
return number
def is_ident(self):
return self.current_char.isalpha() or self.current_char == '_'
def _ident(self):
_id = ''
# identifiers begin with alpha or underscores
if self.current_char.isalpha() or self.current_char == '_':
while self.current_char.isalnum() or self.current_char == '_':
_id += self.current_char
self.advance()
return _id
def is_keyword(self, kw):
return kw.upper() in KEYWORDS
def eat_comment(self):
if self.current_char == '(':
while self.current_char != ')' and not self.is_eoi():
self.advance()
self.advance() # skip trailing )
elif self.current_char == '\\':
while self.current_char != '\n' and not self.is_eoi():
self.advance()
self.advance() # skip new line character
#############################################
# Tokenization #
#-------------------------------------------#
# The method next_token() is called by the #
# client to obtain the next token in the #
# input stream. #
#############################################
def next_token(self) -> Token:
if not self.is_eoi():
if self.current_char.isspace():
while self.current_char.isspace():
self.advance()
if self.current_char.isdigit():
return Token(TT_NUMBER, self.number(), self.line, self.col)
elif self.current_char == '+':
val = self.current_char
self.advance()
return Token(TT_PLUS, val, self.line, self.col)
elif self.current_char == '-':
val = self.current_char
self.advance()
return Token(TT_MINUS, val, self.line, self.col)
elif self.current_char == '*':
val = self.current_char
if self.peek() == '*':
val += '*'
self.advance()
self.advance()
return Token(TT_POW, val, self.line, self.col)
self.advance()
return Token(TT_MUL, val, self.line, self.col)
elif self.current_char == '/':
val = self.current_char
self.advance()
return Token(TT_DIV, val, self.line, self.col)
elif self.current_char == '&':
val = self.current_char
self.advance()
return Token(TT_AND, val, self.line, self.col)
elif self.current_char == '|':
val = self.current_char
self.advance()
return Token(TT_OR, val, self.line, self.col)
elif self.current_char == '^':
val = self.current_char
self.advance()
return Token(TT_XOR, val, self.line, self.col)
elif self.current_char == '~':
val = self.current_char
self.advance()
return Token(TT_INV, val, self.line, self.col)
elif self.is_ident():
_id = self._ident()
if self.is_keyword(_id):
return Token(TT_KEYWORD, _id, self.line, self.col)
else:
return Token(TT_ID, _id, self.line, self.col)
elif self.current_char == '<':
val = self.current_char
if self.peek() == '<':
val += '<'
self.advance()
self.advance()
return Token(TT_LSL, val, self.line, self.col)
self.advance()
return Token(TT_LESS, val, self.line, self.col)
elif self.current_char == '>':
val = self.current_char
if self.peek() == '>':
val += '>'
self.advance()
self.advance()
return Token(TT_LSR, val, self.line, self.col)
self.advance()
return Token(TT_GREATER, val, self.line, self.col)
elif self.current_char == '=':
val = self.current_char
if self.peek() == '=':
val += '='
self.advance()
self.advance()
return Token(TT_EQ, val, self.line, self.col)
self.advance()
return Token(TT_ASSIGN, val, self.line, self.col)
elif self.current_char == ':':
val = self.current_char
self.advance()
return Token(TT_STARTDEF, val, self.line, self.col)
elif self.current_char == ';':
val = self.current_char
self.advance()
return Token(TT_ENDDEF, val, self.line, self.col)
elif self.current_char == '(' or self.current_char == '\\':
self.eat_comment()
return self.next_token()
elif self.current_char == "'":
return Token(TT_CHAR, self.char(), self.line, self.col)
elif self.current_char == '"':
string = self.string()
return Token(TT_STRING, string, self.line, self.col)
elif self.is_eoi():
return Token(TT_EOF, None, self.line, self.col)
else:
raise ValueError(f'Unexpected character: {self.current_char} in input stream at position: {str(self.pos)}.')
else:
return Token(TT_EOF, None, self.line, self.col)
def main(text: str):
lexer = Lexer(text)
tok = lexer.next_token()
while tok.type != TT_EOF:
print(tok)
tok = lexer.next_token()
if __name__ == "__main__":
text = """
( this is a multi-line
comment. )
\ This ia a line comment."
_myIdent """
main(text)
# lexer.py
from keywords import KEYWORDS
from tokens import *
class Lexer:
def __init__(self, text, line=None, col=None):
self.text = text
self.pos = 0
self.line = line if line else 1
self.col = col if col else 1
self.current_char = self.text[0]
#############################################
# Utility Methods #
#-------------------------------------------#
# These methods help manage the internal #
# workings of the lexer. #
#############################################
def advance(self):
"""advance():
Advances the current position in the character stream.
"""
if not self.is_eoi():
self.pos += 1
self.current_char = self.text[self.pos]
self.update_position()
else:
self.current_char = ''
def update_position(self):
"""update_position
Maintains the line and column positions for the current token scan.
"""
if self.current_char == '\n':
self.line += 1
self.col = 0
else:
self.col += 1
def is_eoi(self):
"""is_eoi():
:return:
True if current position is End Of Input stream.
Returns False otherwise.
"""
return self.pos >= (len(self.text) - 1)
def expected(self, expected, found):
"""expected
Raises error is expected and found are different.
"""
if expected == found:
return
else:
raise ValueError(f'[Error] Expected {expected} but found {found} in line: {self.line} position: {self.col}')
def peek(self):
"""peek
Returns next character in input stream after the current character.
Note: If we are at EOI (End Of Input) then a space character is returned.
"""
if not self.is_eoi():
return self.text[self.pos + 1]
else:
return ' ' # space
#############################################
# Recognizer Methods #
#-------------------------------------------#
# These methods collect the characters that #
# belong to a single multi-character token #
# into a single unit, and return that unit #
# to the caller. #
#############################################
def char(self):
if self.current_char != "'":
self.expected("'", self.current_char)
self.advance() # Consume opening single quote
char = self.current_char
self.advance() # Consume character
# Now we must have a closing single quote mark
if self.current_char != "'":
self.expected("'", self.current_char)
self.advance() # Consume closing single quote
return char
def string(self):
if self.current_char != '"':
self.expected('"', self.peek())
self.advance() # Consume leading '"'
string = ''
while self.current_char != '"':
string += self.current_char
self.advance()
self.advance() # Consume trailing '"'
return string
def number(self):
number = ''
while self.current_char.isdigit():
number += self.current_char
self.advance()
return number
def is_ident(self):
return self.current_char.isalpha() or self.current_char == '_'
def _ident(self):
_id = ''
# identifiers begin with alpha or underscores
if self.current_char.isalpha() or self.current_char == '_':
while self.current_char.isalnum() or self.current_char == '_':
_id += self.current_char
self.advance()
return _id
def is_keyword(self, kw):
return kw.upper() in KEYWORDS
def eat_comment(self):
if self.current_char == '(':
while self.current_char != ')' and not self.is_eoi():
self.advance()
self.advance() # skip trailing )
elif self.current_char == '\\':
while self.current_char != '\n' and not self.is_eoi():
self.advance()
self.advance() # skip new line character
#############################################
# Tokenization #
#-------------------------------------------#
# The method next_token() is called by the #
# client to obtain the next token in the #
# input stream. #
#############################################
def next_token(self) -> Token:
if not self.is_eoi():
if self.current_char.isspace():
while self.current_char.isspace():
self.advance()
if self.current_char.isdigit():
return Token(TT_NUMBER, self.number(), self.line, self.col)
elif self.current_char == '+':
val = self.current_char
self.advance()
return Token(TT_PLUS, val, self.line, self.col)
elif self.current_char == '-':
val = self.current_char
self.advance()
return Token(TT_MINUS, val, self.line, self.col)
elif self.current_char == '*':
val = self.current_char
if self.peek() == '*':
val += '*'
self.advance()
self.advance()
return Token(TT_POW, val, self.line, self.col)
self.advance()
return Token(TT_MUL, val, self.line, self.col)
elif self.current_char == '/':
val = self.current_char
self.advance()
return Token(TT_DIV, val, self.line, self.col)
elif self.current_char == '&':
val = self.current_char
self.advance()
return Token(TT_AND, val, self.line, self.col)
elif self.current_char == '|':
val = self.current_char
self.advance()
return Token(TT_OR, val, self.line, self.col)
elif self.current_char == '^':
val = self.current_char
self.advance()
return Token(TT_XOR, val, self.line, self.col)
elif self.current_char == '~':
val = self.current_char
self.advance()
return Token(TT_INV, val, self.line, self.col)
elif self.is_ident():
_id = self._ident()
if self.is_keyword(_id):
return Token(TT_KEYWORD, _id, self.line, self.col)
else:
return Token(TT_ID, _id, self.line, self.col)
elif self.current_char == '<':
val = self.current_char
if self.peek() == '<':
val += '<'
self.advance()
self.advance()
return Token(TT_LSL, val, self.line, self.col)
self.advance()
return Token(TT_LESS, val, self.line, self.col)
elif self.current_char == '>':
val = self.current_char
if self.peek() == '>':
val += '>'
self.advance()
self.advance()
return Token(TT_LSR, val, self.line, self.col)
self.advance()
return Token(TT_GREATER, val, self.line, self.col)
elif self.current_char == '=':
val = self.current_char
if self.peek() == '=':
val += '='
self.advance()
self.advance()
return Token(TT_EQ, val, self.line, self.col)
self.advance()
return Token(TT_ASSIGN, val, self.line, self.col)
elif self.current_char == ':':
val = self.current_char
self.advance()
return Token(TT_STARTDEF, val, self.line, self.col)
elif self.current_char == ';':
val = self.current_char
self.advance()
return Token(TT_ENDDEF, val, self.line, self.col)
elif self.current_char == '(' or self.current_char == '\\':
self.eat_comment()
return self.next_token()
elif self.current_char == "'":
return Token(TT_CHAR, self.char(), self.line, self.col)
elif self.current_char == '"':
string = self.string()
return Token(TT_STRING, string, self.line, self.col)
elif self.is_eoi():
return Token(TT_EOF, None, self.line, self.col)
else:
raise ValueError(f'Unexpected character: {self.current_char} in input stream at position: {str(self.pos)}.')
else:
return Token(TT_EOF, None, self.line, self.col)
def main(text: str):
lexer = Lexer(text)
tok = lexer.next_token()
while tok.type != TT_EOF:
print(tok)
tok = lexer.next_token()
if __name__ == "__main__":
text = """
( this is a multi-line
comment. )
\ This ia a line comment."
_myIdent """
main(text)
# lexer.py
from keywords import KEYWORDS
from tokens import *
class Lexer:
def __init__(self, text, line=None, col=None):
self.text = text
self.pos = 0
self.line = line if line else 1
self.col = col if col else 1
self.current_char = self.text[0]
#############################################
# Utility Methods #
#-------------------------------------------#
# These methods help manage the internal #
# workings of the lexer. #
#############################################
def advance(self):
"""advance():
Advances the current position in the character stream.
"""
if not self.is_eoi():
self.pos += 1
self.current_char = self.text[self.pos]
self.update_position()
else:
self.current_char = ''
def update_position(self):
"""update_position
Maintains the line and column positions for the current token scan.
"""
if self.current_char == '\n':
self.line += 1
self.col = 0
else:
self.col += 1
def is_eoi(self):
"""is_eoi():
:return:
True if current position is End Of Input stream.
Returns False otherwise.
"""
return self.pos >= (len(self.text) - 1)
def expected(self, expected, found):
"""expected
Raises error is expected and found are different.
"""
if expected == found:
return
else:
raise ValueError(f'[Error] Expected {expected} but found {found} in line: {self.line} position: {self.col}')
def peek(self):
"""peek
Returns next character in input stream after the current character.
Note: If we are at EOI (End Of Input) then a space character is returned.
"""
if not self.is_eoi():
return self.text[self.pos + 1]
else:
return ' ' # space
#############################################
# Recognizer Methods #
#-------------------------------------------#
# These methods collect the characters that #
# belong to a single multi-character token #
# into a single unit, and return that unit #
# to the caller. #
#############################################
def char(self):
if self.current_char != "'":
self.expected("'", self.current_char)
self.advance() # Consume opening single quote
char = self.current_char
self.advance() # Consume character
# Now we must have a closing single quote mark
if self.current_char != "'":
self.expected("'", self.current_char)
self.advance() # Consume closing single quote
return char
def string(self):
if self.current_char != '"':
self.expected('"', self.peek())
self.advance() # Consume leading '"'
string = ''
while self.current_char != '"':
string += self.current_char
self.advance()
self.advance() # Consume trailing '"'
return string
def number(self):
number = ''
while self.current_char.isdigit():
number += self.current_char
self.advance()
return number
def is_ident(self):
return self.current_char.isalpha() or self.current_char == '_'
def _ident(self):
_id = ''
# identifiers begin with alpha or underscores
if self.current_char.isalpha() or self.current_char == '_':
while self.current_char.isalnum() or self.current_char == '_':
_id += self.current_char
self.advance()
return _id
def is_keyword(self, kw):
return kw.upper() in KEYWORDS
def eat_comment(self):
if self.current_char == '(':
while self.current_char != ')' and not self.is_eoi():
self.advance()
self.advance() # skip trailing )
elif self.current_char == '\\':
while self.current_char != '\n' and not self.is_eoi():
self.advance()
self.advance() # skip new line character
#############################################
# Tokenization #
#-------------------------------------------#
# The method next_token() is called by the #
# client to obtain the next token in the #
# input stream. #
#############################################
def next_token(self) -> Token:
if not self.is_eoi():
if self.current_char.isspace():
while self.current_char.isspace():
self.advance()
if self.current_char.isdigit():
return Token(TT_NUMBER, self.number(), self.line, self.col)
elif self.current_char == '+':
val = self.current_char
self.advance()
return Token(TT_PLUS, val, self.line, self.col)
elif self.current_char == '-':
val = self.current_char
self.advance()
return Token(TT_MINUS, val, self.line, self.col)
elif self.current_char == '*':
val = self.current_char
if self.peek() == '*':
val += '*'
self.advance()
self.advance()
return Token(TT_POW, val, self.line, self.col)
self.advance()
return Token(TT_MUL, val, self.line, self.col)
elif self.current_char == '/':
val = self.current_char
self.advance()
return Token(TT_DIV, val, self.line, self.col)
elif self.current_char == '&':
val = self.current_char
self.advance()
return Token(TT_AND, val, self.line, self.col)
elif self.current_char == '|':
val = self.current_char
self.advance()
return Token(TT_OR, val, self.line, self.col)
elif self.current_char == '^':
val = self.current_char
self.advance()
return Token(TT_XOR, val, self.line, self.col)
elif self.current_char == '~':
val = self.current_char
self.advance()
return Token(TT_INV, val, self.line, self.col)
elif self.is_ident():
_id = self._ident()
if self.is_keyword(_id):
return Token(TT_KEYWORD, _id, self.line, self.col)
else:
return Token(TT_ID, _id, self.line, self.col)
elif self.current_char == '<':
val = self.current_char
if self.peek() == '<':
val += '<'
self.advance()
self.advance()
return Token(TT_LSL, val, self.line, self.col)
self.advance()
return Token(TT_LESS, val, self.line, self.col)
elif self.current_char == '>':
val = self.current_char
if self.peek() == '>':
val += '>'
self.advance()
self.advance()
return Token(TT_LSR, val, self.line, self.col)
self.advance()
return Token(TT_GREATER, val, self.line, self.col)
elif self.current_char == '=':
val = self.current_char
if self.peek() == '=':
val += '='
self.advance()
self.advance()
return Token(TT_EQ, val, self.line, self.col)
self.advance()
return Token(TT_ASSIGN, val, self.line, self.col)
elif self.current_char == ':':
val = self.current_char
self.advance()
return Token(TT_STARTDEF, val, self.line, self.col)
elif self.current_char == ';':
val = self.current_char
self.advance()
return Token(TT_ENDDEF, val, self.line, self.col)
elif self.current_char == '(' or self.current_char == '\\':
self.eat_comment()
return self.next_token()
elif self.current_char == "'":
return Token(TT_CHAR, self.char(), self.line, self.col)
elif self.current_char == '"':
string = self.string()
return Token(TT_STRING, string, self.line, self.col)
elif self.is_eoi():
return Token(TT_EOF, None, self.line, self.col)
else:
raise ValueError(f'Unexpected character: {self.current_char} in input stream at position: {str(self.pos)}.')
else:
return Token(TT_EOF, None, self.line, self.col)
def main(text: str):
lexer = Lexer(text)
tok = lexer.next_token()
while tok.type != TT_EOF:
print(tok)
tok = lexer.next_token()
if __name__ == "__main__":
text = """
( this is a multi-line
comment. )
\ This ia a line comment."
_myIdent """
main(text)
# parser.py
import sys
from lexer import Lexer
from stack import Stack
from tokens import *
#######################################
# Parser #
#######################################
class Parser:
def __init__(self, lexer: Lexer, data_stack: Stack = None):
self.lexer = lexer
# Setup data stack
if data_stack:
self.data = data_stack
else:
self.data = Stack()
self.current_tok = Token(TT_START, None)
def next_token(self):
if self.current_tok != TT_EOF:
self.current_tok = self.lexer.next_token()
def parse(self):
# get token and decide what to do with it
tok = self.current_tok
while tok.type != TT_EOF:
self.next_token()
tok = self.current_tok
#print(tok)
#print(self.data)
if tok.type == TT_START:
continue
if tok.type == TT_NUMBER:
self.data.push(int(tok.value))
continue
# Arithmetic
elif tok.type == TT_PLUS:
right = self.data.pop()
left = self.data.pop()
result = left + right
self.data.push(result)
elif tok.type == TT_MINUS:
right = self.data.pop()
left = self.data.pop()
result = left - right
self.data.push(result)
elif tok.type == TT_MUL:
right = self.data.pop()
left = self.data.pop()
result = left * right
self.data.push(result)
elif tok.type == TT_DIV:
right = self.data.pop()
left = self.data.pop()
result = left // right
self.data.push(result)
elif tok.type == TT_POW:
right = self.data.pop()
left = self.data.pop()
result = left ** right
self.data.push(result)
# Logical Operations
elif tok.type == TT_INV:
left = self.data.pop()
left = ~left
self.data.push(left)
elif tok.type == TT_XOR:
right = self.data.pop()
left = self.data.pop()
self.data.push((left ^ right))
elif tok.type == TT_AND:
right = self.data.pop()
left = self.data.pop()
self.data.push((left & right))
elif tok.type == TT_OR:
right = self.data.pop()
left = self.data.pop()
self.data.push((left | right))
elif tok.type == TT_LSL:
right = self.data.pop()
left = self.data.pop()
self.data.push(int(left) << int(right))
elif tok.type == TT_LSR:
right = self.data.pop()
left = self.data.pop()
self.data.push((left >> right))
# Handle Keywords
elif tok.type == TT_KEYWORD:
# Process keyword
if tok.value == 'dump':
"""dump
Display the data stack contents in the console
"""
print(str(self.data))
elif tok.value == 'drop':
"""pop
Remove the top value from the stack and discard it.
"""
self.data.pop()
elif tok.value == 'clear':
"""clear
Empty the stack.
"""
self.data.clear()
elif tok.value.lower() == 'swap':
"""swap
Swaps the position of the top two items on the stack.
Ex: x y z --> x z y
"""
if self.data.count() < 2:
raise IndexError(
f'Attempt to SWAP on stack of size {self.data.count()}!\n\t\t\tStack must contain at least 2 elements to SWAP.')
n1 = self.data.pop()
n2 = self.data.pop()
self.data.push(n1)
self.data.push(n2)
elif tok.value == 'swap2':
"""swap2
Swaps the position of the top two pairs of items on the stack.
Ex: w x y z --> y z w x
"""
if self.data.count() < 4:
raise IndexError(
f'Attempt to SWAP on stack of size {self.data.count()}!\n\t\t\tStack must contain at least 4 elements to SWAP2.')
n3 = self.data.pop()
n2 = self.data.pop()
n1 = self.data.pop()
n0 = self.data.pop()
self.data.push(n2)
self.data.push(n3)
self.data.push(n0)
self.data.push(n1)
elif tok.value == 'over':
"""over
Copies the second item on the stack to the head position.
Ex: x y z --> x y z y
"""
top = self.data.pop()
next = self.data.pop()
self.data.push(next)
self.data.push(top)
self.data.push(next)
elif tok.value == 'over2':
"""over
Copies the second item on the stack to the head position.
Ex: x y z --> x y z y
"""
top = self.data.pop()
sec = self.data.pop()
next = self.data.pop()
self.data.push(next)
self.data.push(top)
self.data.push(next)
elif tok.value == 'dup2':
"""dup2
Duplicates the top two items on the stack.
Ex: x y z --> x y z y z
"""
n1 = self.data.pop()
n2 = self.data.pop()
self.data.push(n2)
self.data.push(n1)
self.data.push(n2)
self.data.push(n1)
elif tok.value == 'dup3':
"""dup3
Duplicates the top three items on the stack.
Ex: x y z --> x y z x y z
"""
n1 = self.data.pop()
n2 = self.data.pop()
n3 = self.data.pop()
self.data.push(n3)
self.data.push(n2)
self.data.push(n1)
self.data.push(n3)
self.data.push(n2)
self.data.push(n1)
elif tok.value == 'dup4':
"""dup3
Duplicates the top three items on the stack.
Ex: w x y z --> w x y z w x y z
"""
n1 = self.data.pop()
n2 = self.data.pop()
n3 = self.data.pop()
n4 = self.data.pop()
self.data.push(n4)
self.data.push(n3)
self.data.push(n2)
self.data.push(n1)
self.data.push(n4)
self.data.push(n3)
self.data.push(n2)
self.data.push(n1)
elif tok.value == 'rol':
"""ror
Rotates the values in the stack by moving the top
element to the bottom and shifting everything else up
one position.
Ex: rot x y z -> y z x
"""
item = self.data.pop()
last = self.data.que(item)
elif tok.value == 'ror':
"""rot
Rotates the values in the stack by moving the bottom
element to the top and pushing everything else down
one position.
Ex: rot x y z -> y z x
"""
last = self.data.tail()
self.data.push(last)
# Handle I/O Routines
elif tok.value == 'emit':
"""emit
The word EMIT takes a single ASCII representation on the
stack, using the low-order byte only, and sends the
character to stdout. For example, in decimal:
65 EMIT↵ A ok
66 EMIT↵ B ok
67 EMIT↵ C ok
"""
ch = self.data.pop()
if ch > 0x110000 or ch < 0:
raise ValueError(f'[Error] character popped by emit: {ch} is out of printable range!')
char = chr(ch)
print(char, end='')
elif tok.value == 'key':
"""key
Pauses execution until a key is pressed. Once a key
is pressed, the key code is pushed onto the stack.
Ex: <A> --> 65
"""
getch = sys.stdin.read
key = None
while not key:
key = getch(1)
self.data.push(ord(key))
# Helpful Constants
elif tok.value == 'cr': # Carriage Return
self.data.push(13)
elif tok.value == 'lf': # Line Feed
self.data.push(10)
elif tok.value == 'sp':
self.data.push(32)
elif tok.value == 'null': # NUll
self.data.push(0)
elif tok.value == 'bell': # System Bell
self.data.push(7)
elif tok.value == 'bs': # Backspace
self.data.push(8)
elif tok.value == 'tab': # TAB
self.data.push(9)
elif tok.value == 'vt': # Vertical TAB
self.data.push(11)
elif tok.value == 'ff': # Form Feed
self.data.push(12)
elif tok.value == 'esc': # Escape
self.data.push(27)
if __name__ == '__main__':
source = """ 16 2 >> """
lexer = Lexer(source)
parser = Parser(lexer)
parser.parse()
print(parser.data)
# parser.py
import sys
from lexer import Lexer
from stack import Stack
from tokens import *
#######################################
# Parser #
#######################################
class Parser:
def __init__(self, lexer: Lexer, data_stack: Stack = None):
self.lexer = lexer
# Setup data stack
if data_stack:
self.data = data_stack
else:
self.data = Stack()
self.current_tok = Token(TT_START, None)
def next_token(self):
if self.current_tok != TT_EOF:
self.current_tok = self.lexer.next_token()
def parse(self):
# get token and decide what to do with it
tok = self.current_tok
while tok.type != TT_EOF:
self.next_token()
tok = self.current_tok
#print(tok)
#print(self.data)
if tok.type == TT_START:
continue
if tok.type == TT_NUMBER:
self.data.push(int(tok.value))
continue
# Arithmetic
elif tok.type == TT_PLUS:
right = self.data.pop()
left = self.data.pop()
result = left + right
self.data.push(result)
elif tok.type == TT_MINUS:
right = self.data.pop()
left = self.data.pop()
result = left - right
self.data.push(result)
elif tok.type == TT_MUL:
right = self.data.pop()
left = self.data.pop()
result = left * right
self.data.push(result)
elif tok.type == TT_DIV:
right = self.data.pop()
left = self.data.pop()
result = left // right
self.data.push(result)
elif tok.type == TT_POW:
right = self.data.pop()
left = self.data.pop()
result = left ** right
self.data.push(result)
# Logical Operations
elif tok.type == TT_INV:
left = self.data.pop()
left = ~left
self.data.push(left)
elif tok.type == TT_XOR:
right = self.data.pop()
left = self.data.pop()
self.data.push((left ^ right))
elif tok.type == TT_AND:
right = self.data.pop()
left = self.data.pop()
self.data.push((left & right))
elif tok.type == TT_OR:
right = self.data.pop()
left = self.data.pop()
self.data.push((left | right))
elif tok.type == TT_LSL:
right = self.data.pop()
left = self.data.pop()
self.data.push(int(left) << int(right))
elif tok.type == TT_LSR:
right = self.data.pop()
left = self.data.pop()
self.data.push((left >> right))
# Handle Keywords
elif tok.type == TT_KEYWORD:
# Process keyword
if tok.value == 'dump':
"""dump
Display the data stack contents in the console
"""
print(str(self.data))
elif tok.value == 'drop':
"""pop
Remove the top value from the stack and discard it.
"""
self.data.pop()
elif tok.value == 'clear':
"""clear
Empty the stack.
"""
self.data.clear()
elif tok.value.lower() == 'swap':
"""swap
Swaps the position of the top two items on the stack.
Ex: x y z --> x z y
"""
if self.data.count() < 2:
raise IndexError(
f'Attempt to SWAP on stack of size {self.data.count()}!\n\t\t\tStack must contain at least 2 elements to SWAP.')
n1 = self.data.pop()
n2 = self.data.pop()
self.data.push(n1)
self.data.push(n2)
elif tok.value == 'swap2':
"""swap2
Swaps the position of the top two pairs of items on the stack.
Ex: w x y z --> y z w x
"""
if self.data.count() < 4:
raise IndexError(
f'Attempt to SWAP on stack of size {self.data.count()}!\n\t\t\tStack must contain at least 4 elements to SWAP2.')
n3 = self.data.pop()
n2 = self.data.pop()
n1 = self.data.pop()
n0 = self.data.pop()
self.data.push(n2)
self.data.push(n3)
self.data.push(n0)
self.data.push(n1)
elif tok.value == 'over':
"""over
Copies the second item on the stack to the head position.
Ex: x y z --> x y z y
"""
top = self.data.pop()
next = self.data.pop()
self.data.push(next)
self.data.push(top)
self.data.push(next)
elif tok.value == 'over2':
"""over
Copies the second item on the stack to the head position.
Ex: x y z --> x y z y
"""
top = self.data.pop()
sec = self.data.pop()
next = self.data.pop()
self.data.push(next)
self.data.push(top)
self.data.push(next)
elif tok.value == 'dup2':
"""dup2
Duplicates the top two items on the stack.
Ex: x y z --> x y z y z
"""
n1 = self.data.pop()
n2 = self.data.pop()
self.data.push(n2)
self.data.push(n1)
self.data.push(n2)
self.data.push(n1)
elif tok.value == 'dup3':
"""dup3
Duplicates the top three items on the stack.
Ex: x y z --> x y z x y z
"""
n1 = self.data.pop()
n2 = self.data.pop()
n3 = self.data.pop()
self.data.push(n3)
self.data.push(n2)
self.data.push(n1)
self.data.push(n3)
self.data.push(n2)
self.data.push(n1)
elif tok.value == 'dup4':
"""dup3
Duplicates the top three items on the stack.
Ex: w x y z --> w x y z w x y z
"""
n1 = self.data.pop()
n2 = self.data.pop()
n3 = self.data.pop()
n4 = self.data.pop()
self.data.push(n4)
self.data.push(n3)
self.data.push(n2)
self.data.push(n1)
self.data.push(n4)
self.data.push(n3)
self.data.push(n2)
self.data.push(n1)
elif tok.value == 'rol':
"""ror
Rotates the values in the stack by moving the top
element to the bottom and shifting everything else up
one position.
Ex: rot x y z -> y z x
"""
item = self.data.pop()
last = self.data.que(item)
elif tok.value == 'ror':
"""rot
Rotates the values in the stack by moving the bottom
element to the top and pushing everything else down
one position.
Ex: rot x y z -> y z x
"""
last = self.data.tail()
self.data.push(last)
# Handle I/O Routines
elif tok.value == 'emit':
"""emit
The word EMIT takes a single ASCII representation on the
stack, using the low-order byte only, and sends the
character to stdout. For example, in decimal:
65 EMIT↵ A ok
66 EMIT↵ B ok
67 EMIT↵ C ok
"""
ch = self.data.pop()
if ch > 0x110000 or ch < 0:
raise ValueError(f'[Error] character popped by emit: {ch} is out of printable range!')
char = chr(ch)
print(char, end='')
elif tok.value == 'key':
"""key
Pauses execution until a key is pressed. Once a key
is pressed, the key code is pushed onto the stack.
Ex: <A> --> 65
"""
getch = sys.stdin.read
key = None
while not key:
key = getch(1)
self.data.push(ord(key))
# Helpful Constants
elif tok.value == 'cr': # Carriage Return
self.data.push(13)
elif tok.value == 'lf': # Line Feed
self.data.push(10)
elif tok.value == 'sp':
self.data.push(32)
elif tok.value == 'null': # NUll
self.data.push(0)
elif tok.value == 'bell': # System Bell
self.data.push(7)
elif tok.value == 'bs': # Backspace
self.data.push(8)
elif tok.value == 'tab': # TAB
self.data.push(9)
elif tok.value == 'vt': # Vertical TAB
self.data.push(11)
elif tok.value == 'ff': # Form Feed
self.data.push(12)
elif tok.value == 'esc': # Escape
self.data.push(27)
if __name__ == '__main__':
source = """ 16 2 >> """
lexer = Lexer(source)
parser = Parser(lexer)
parser.parse()
print(parser.data)
# parser.py
import sys
from lexer import Lexer
from stack import Stack
from tokens import *
#######################################
# Parser #
#######################################
class Parser:
def __init__(self, lexer: Lexer, data_stack: Stack = None):
self.lexer = lexer
# Setup data stack
if data_stack:
self.data = data_stack
else:
self.data = Stack()
self.current_tok = Token(TT_START, None)
def next_token(self):
if self.current_tok != TT_EOF:
self.current_tok = self.lexer.next_token()
def parse(self):
# get token and decide what to do with it
tok = self.current_tok
while tok.type != TT_EOF:
self.next_token()
tok = self.current_tok
#print(tok)
#print(self.data)
if tok.type == TT_START:
continue
if tok.type == TT_NUMBER:
self.data.push(int(tok.value))
continue
# Arithmetic
elif tok.type == TT_PLUS:
right = self.data.pop()
left = self.data.pop()
result = left + right
self.data.push(result)
elif tok.type == TT_MINUS:
right = self.data.pop()
left = self.data.pop()
result = left - right
self.data.push(result)
elif tok.type == TT_MUL:
right = self.data.pop()
left = self.data.pop()
result = left * right
self.data.push(result)
elif tok.type == TT_DIV:
right = self.data.pop()
left = self.data.pop()
result = left // right
self.data.push(result)
elif tok.type == TT_POW:
right = self.data.pop()
left = self.data.pop()
result = left ** right
self.data.push(result)
# Logical Operations
elif tok.type == TT_INV:
left = self.data.pop()
left = ~left
self.data.push(left)
elif tok.type == TT_XOR:
right = self.data.pop()
left = self.data.pop()
self.data.push((left ^ right))
elif tok.type == TT_AND:
right = self.data.pop()
left = self.data.pop()
self.data.push((left & right))
elif tok.type == TT_OR:
right = self.data.pop()
left = self.data.pop()
self.data.push((left | right))
elif tok.type == TT_LSL:
right = self.data.pop()
left = self.data.pop()
self.data.push(int(left) << int(right))
elif tok.type == TT_LSR:
right = self.data.pop()
left = self.data.pop()
self.data.push((left >> right))
# Handle Keywords
elif tok.type == TT_KEYWORD:
# Process keyword
if tok.value == 'dump':
"""dump
Display the data stack contents in the console
"""
print(str(self.data))
elif tok.value == 'drop':
"""pop
Remove the top value from the stack and discard it.
"""
self.data.pop()
elif tok.value == 'clear':
"""clear
Empty the stack.
"""
self.data.clear()
elif tok.value.lower() == 'swap':
"""swap
Swaps the position of the top two items on the stack.
Ex: x y z --> x z y
"""
if self.data.count() < 2:
raise IndexError(
f'Attempt to SWAP on stack of size {self.data.count()}!\n\t\t\tStack must contain at least 2 elements to SWAP.')
n1 = self.data.pop()
n2 = self.data.pop()
self.data.push(n1)
self.data.push(n2)
elif tok.value == 'swap2':
"""swap2
Swaps the position of the top two pairs of items on the stack.
Ex: w x y z --> y z w x
"""
if self.data.count() < 4:
raise IndexError(
f'Attempt to SWAP on stack of size {self.data.count()}!\n\t\t\tStack must contain at least 4 elements to SWAP2.')
n3 = self.data.pop()
n2 = self.data.pop()
n1 = self.data.pop()
n0 = self.data.pop()
self.data.push(n2)
self.data.push(n3)
self.data.push(n0)
self.data.push(n1)
elif tok.value == 'over':
"""over
Copies the second item on the stack to the head position.
Ex: x y z --> x y z y
"""
top = self.data.pop()
next = self.data.pop()
self.data.push(next)
self.data.push(top)
self.data.push(next)
elif tok.value == 'over2':
"""over
Copies the second item on the stack to the head position.
Ex: x y z --> x y z y
"""
top = self.data.pop()
sec = self.data.pop()
next = self.data.pop()
self.data.push(next)
self.data.push(top)
self.data.push(next)
elif tok.value == 'dup2':
"""dup2
Duplicates the top two items on the stack.
Ex: x y z --> x y z y z
"""
n1 = self.data.pop()
n2 = self.data.pop()
self.data.push(n2)
self.data.push(n1)
self.data.push(n2)
self.data.push(n1)
elif tok.value == 'dup3':
"""dup3
Duplicates the top three items on the stack.
Ex: x y z --> x y z x y z
"""
n1 = self.data.pop()
n2 = self.data.pop()
n3 = self.data.pop()
self.data.push(n3)
self.data.push(n2)
self.data.push(n1)
self.data.push(n3)
self.data.push(n2)
self.data.push(n1)
elif tok.value == 'dup4':
"""dup3
Duplicates the top three items on the stack.
Ex: w x y z --> w x y z w x y z
"""
n1 = self.data.pop()
n2 = self.data.pop()
n3 = self.data.pop()
n4 = self.data.pop()
self.data.push(n4)
self.data.push(n3)
self.data.push(n2)
self.data.push(n1)
self.data.push(n4)
self.data.push(n3)
self.data.push(n2)
self.data.push(n1)
elif tok.value == 'rol':
"""ror
Rotates the values in the stack by moving the top
element to the bottom and shifting everything else up
one position.
Ex: rot x y z -> y z x
"""
item = self.data.pop()
last = self.data.que(item)
elif tok.value == 'ror':
"""rot
Rotates the values in the stack by moving the bottom
element to the top and pushing everything else down
one position.
Ex: rot x y z -> y z x
"""
last = self.data.tail()
self.data.push(last)
# Handle I/O Routines
elif tok.value == 'emit':
"""emit
The word EMIT takes a single ASCII representation on the
stack, using the low-order byte only, and sends the
character to stdout. For example, in decimal:
65 EMIT↵ A ok
66 EMIT↵ B ok
67 EMIT↵ C ok
"""
ch = self.data.pop()
if ch > 0x110000 or ch < 0:
raise ValueError(f'[Error] character popped by emit: {ch} is out of printable range!')
char = chr(ch)
print(char, end='')
elif tok.value == 'key':
"""key
Pauses execution until a key is pressed. Once a key
is pressed, the key code is pushed onto the stack.
Ex: <A> --> 65
"""
getch = sys.stdin.read
key = None
while not key:
key = getch(1)
self.data.push(ord(key))
# Helpful Constants
elif tok.value == 'cr': # Carriage Return
self.data.push(13)
elif tok.value == 'lf': # Line Feed
self.data.push(10)
elif tok.value == 'sp':
self.data.push(32)
elif tok.value == 'null': # NUll
self.data.push(0)
elif tok.value == 'bell': # System Bell
self.data.push(7)
elif tok.value == 'bs': # Backspace
self.data.push(8)
elif tok.value == 'tab': # TAB
self.data.push(9)
elif tok.value == 'vt': # Vertical TAB
self.data.push(11)
elif tok.value == 'ff': # Form Feed
self.data.push(12)
elif tok.value == 'esc': # Escape
self.data.push(27)
if __name__ == '__main__':
source = """ 16 2 >> """
lexer = Lexer(source)
parser = Parser(lexer)
parser.parse()
print(parser.data)
#! usr/bin/env/python3
# sol.py
from lexer import Lexer
from parser import Parser
import argparse
def main(src):
_lexer = Lexer(src)
_parser = Parser(_lexer)
_parser.parse()
if __name__ == '__main__':
argparse = argparse.ArgumentParser()
argparse.add_argument("-i", "--infile",
help="Input source file", required=True)
argparse.add_argument("-v", "--verbose", help="Echo source file", action="store_true")
args = argparse.parse_args()
infile = args.infile
sorce = ''
with open(infile, 'r') as ifh:
source = ifh.read()
if args.verbose:
print(source)
main(source)
#! usr/bin/env/python3
# sol.py
from lexer import Lexer
from parser import Parser
import argparse
def main(src):
_lexer = Lexer(src)
_parser = Parser(_lexer)
_parser.parse()
if __name__ == '__main__':
argparse = argparse.ArgumentParser()
argparse.add_argument("-i", "--infile",
help="Input source file", required=True)
argparse.add_argument("-v", "--verbose", help="Echo source file", action="store_true")
args = argparse.parse_args()
infile = args.infile
sorce = ''
with open(infile, 'r') as ifh:
source = ifh.read()
if args.verbose:
print(source)
main(source)
#! usr/bin/env/python3
# sol.py
from lexer import Lexer
from parser import Parser
import argparse
def main(src):
_lexer = Lexer(src)
_parser = Parser(_lexer)
_parser.parse()
if __name__ == '__main__':
argparse = argparse.ArgumentParser()
argparse.add_argument("-i", "--infile",
help="Input source file", required=True)
argparse.add_argument("-v", "--verbose", help="Echo source file", action="store_true")
args = argparse.parse_args()
infile = args.infile
sorce = ''
with open(infile, 'r') as ifh:
source = ifh.read()
if args.verbose:
print(source)
main(source)
# stack.py
class Stack:
def __init__(self, report_errors=False):
self.store = []
self.report_error = report_errors
def push(self, item):
self.store.append(item)
def pop(self):
if self.store:
return self.store.pop()
else:
if self.report_error:
raise IndexError('Attempt to pop a value from empty stack!')
else:
return None
def tail(self):
if self.store:
return self.store.pop(0)
else:
return None
def clear(self):
self.store.clear()
def count(self):
return len(self.store)
def is_empty(self):
return not bool(self.store)
def __str__(self):
s = '[ '
for v in self.store:
s += "'" + str(v) + "', "
s = s[:-2]
s += ' ]'
return s
def __repr__(self):
return self.__str__()
# stack.py
class Stack:
def __init__(self, report_errors=False):
self.store = []
self.report_error = report_errors
def push(self, item):
self.store.append(item)
def pop(self):
if self.store:
return self.store.pop()
else:
if self.report_error:
raise IndexError('Attempt to pop a value from empty stack!')
else:
return None
def tail(self):
if self.store:
return self.store.pop(0)
else:
return None
def clear(self):
self.store.clear()
def count(self):
return len(self.store)
def is_empty(self):
return not bool(self.store)
def __str__(self):
s = '[ '
for v in self.store:
s += "'" + str(v) + "', "
s = s[:-2]
s += ' ]'
return s
def __repr__(self):
return self.__str__()
# stack.py
class Stack:
def __init__(self, report_errors=False):
self.store = []
self.report_error = report_errors
def push(self, item):
self.store.append(item)
def pop(self):
if self.store:
return self.store.pop()
else:
if self.report_error:
raise IndexError('Attempt to pop a value from empty stack!')
else:
return None
def tail(self):
if self.store:
return self.store.pop(0)
else:
return None
def clear(self):
self.store.clear()
def count(self):
return len(self.store)
def is_empty(self):
return not bool(self.store)
def __str__(self):
s = '[ '
for v in self.store:
s += "'" + str(v) + "', "
s = s[:-2]
s += ' ]'
return s
def __repr__(self):
return self.__str__()
# tokens.py
# Data
TT_CHAR = 'CHAR' # 'c'
TT_STRING = 'STRING' # "string"
TT_NUMBER = 'NUMBER' # 1234 (integers only
TT_ID = 'ID'
# Arithmatic Operators
TT_ASSIGN = 'ASSIGN' # '='
TT_PLUS = 'PLUS' # '+'
TT_MINUS = 'MINUS' # '-'
TT_MUL = 'MUL' # '*'
TT_DIV = 'DIV' # '/'
TT_POW = 'POW' # '**"
# Bitwise Operators
TT_LSR = 'LSR' # '>>'
TT_LSL = 'LSL' # '<<'
TT_AND = 'AND' # '&' Bitwise AND
TT_OR = 'OR' # '|' Bitwise OR
TT_XOR = 'XOR' # '^' Bitwise XOR
TT_INV = 'INV' # '~' Bitwise NOT/Invert
# Relational Operators
TT_EQ = 'EQ' # '==' Is Equal?
TT_LESS = 'LESS' # '<' Is Less?
TT_GREATER = 'GREATER' # '>' Is greater?
# Logical Operators
TT_LAND = 'LAND' # AND Logical
TT_LOR = 'LOR' # OR Logical
TT_NEQ = 'NEQ' # NEQ Logical
# DEFINE Functions
TT_STARTDEF = 'STARTDEFINE' # ':'
TT_ENDDEF = 'ENDDEFINE' # ';'
# INTERNAL USE
TT_KEYWORD = 'KEYWORD'
TT_START = "START"
TT_EOF = 'EOF'
class Token:
def __init__(self, _type, _value, line=None, col=None):
self.type = _type
self.value = _value
self.col = col
self.line = line
def __str__(self):
return f'''type: {self.type}, value: {self.value},
at line {self.line} in column {self.col}'''
def __repr__(self):
return self.__str__()
# tokens.py
# Data
TT_CHAR = 'CHAR' # 'c'
TT_STRING = 'STRING' # "string"
TT_NUMBER = 'NUMBER' # 1234 (integers only
TT_ID = 'ID'
# Arithmatic Operators
TT_ASSIGN = 'ASSIGN' # '='
TT_PLUS = 'PLUS' # '+'
TT_MINUS = 'MINUS' # '-'
TT_MUL = 'MUL' # '*'
TT_DIV = 'DIV' # '/'
TT_POW = 'POW' # '**"
# Bitwise Operators
TT_LSR = 'LSR' # '>>'
TT_LSL = 'LSL' # '<<'
TT_AND = 'AND' # '&' Bitwise AND
TT_OR = 'OR' # '|' Bitwise OR
TT_XOR = 'XOR' # '^' Bitwise XOR
TT_INV = 'INV' # '~' Bitwise NOT/Invert
# Relational Operators
TT_EQ = 'EQ' # '==' Is Equal?
TT_LESS = 'LESS' # '<' Is Less?
TT_GREATER = 'GREATER' # '>' Is greater?
# Logical Operators
TT_LAND = 'LAND' # AND Logical
TT_LOR = 'LOR' # OR Logical
TT_NEQ = 'NEQ' # NEQ Logical
# DEFINE Functions
TT_STARTDEF = 'STARTDEFINE' # ':'
TT_ENDDEF = 'ENDDEFINE' # ';'
# INTERNAL USE
TT_KEYWORD = 'KEYWORD'
TT_START = "START"
TT_EOF = 'EOF'
class Token:
def __init__(self, _type, _value, line=None, col=None):
self.type = _type
self.value = _value
self.col = col
self.line = line
def __str__(self):
return f'''type: {self.type}, value: {self.value},
at line {self.line} in column {self.col}'''
def __repr__(self):
return self.__str__()
# tokens.py
# Data
TT_CHAR = 'CHAR' # 'c'
TT_STRING = 'STRING' # "string"
TT_NUMBER = 'NUMBER' # 1234 (integers only
TT_ID = 'ID'
# Arithmatic Operators
TT_ASSIGN = 'ASSIGN' # '='
TT_PLUS = 'PLUS' # '+'
TT_MINUS = 'MINUS' # '-'
TT_MUL = 'MUL' # '*'
TT_DIV = 'DIV' # '/'
TT_POW = 'POW' # '**"
# Bitwise Operators
TT_LSR = 'LSR' # '>>'
TT_LSL = 'LSL' # '<<'
TT_AND = 'AND' # '&' Bitwise AND
TT_OR = 'OR' # '|' Bitwise OR
TT_XOR = 'XOR' # '^' Bitwise XOR
TT_INV = 'INV' # '~' Bitwise NOT/Invert
# Relational Operators
TT_EQ = 'EQ' # '==' Is Equal?
TT_LESS = 'LESS' # '<' Is Less?
TT_GREATER = 'GREATER' # '>' Is greater?
# Logical Operators
TT_LAND = 'LAND' # AND Logical
TT_LOR = 'LOR' # OR Logical
TT_NEQ = 'NEQ' # NEQ Logical
# DEFINE Functions
TT_STARTDEF = 'STARTDEFINE' # ':'
TT_ENDDEF = 'ENDDEFINE' # ';'
# INTERNAL USE
TT_KEYWORD = 'KEYWORD'
TT_START = "START"
TT_EOF = 'EOF'
class Token:
def __init__(self, _type, _value, line=None, col=None):
self.type = _type
self.value = _value
self.col = col
self.line = line
def __str__(self):
return f'''type: {self.type}, value: {self.value},
at line {self.line} in column {self.col}'''
def __repr__(self):
return self.__str__()
\ file: test.sol
\
\ Expected Output
\ -----------------------
\ [ '2', '3' ]
\ [ '5' ]
\ [ '5', '5' ]
\ [ '10' ]
\ [ '10', '6' ]
\
2 3 dump + dump
5 dump + dump
6 dump * dump
\ file: test.sol
\
\ Expected Output
\ -----------------------
\ [ '2', '3' ]
\ [ '5' ]
\ [ '5', '5' ]
\ [ '10' ]
\ [ '10', '6' ]
\
2 3 dump + dump
5 dump + dump
6 dump * dump
\ file: test.sol \ \ Expected Output \ ----------------------- \ [ '2', '3' ] \ [ '5' ] \ [ '5', '5' ] \ [ '10' ] \ [ '10', '6' ] \ 2 3 dump + dump 5 dump + dump 6 dump * dump
\ tests/test_arith.sol
\
\ +, -, *, /, **
\
\ Arithmetic operators take the top two items
\ from the stack and performs the operations
\ on them, placing the results back on the
\ stack.
2 3 +
dump
5 *
dump
5 /
dump
3 -
dump
8 **
dump
\ tests/test_arith.sol
\
\ +, -, *, /, **
\
\ Arithmetic operators take the top two items
\ from the stack and performs the operations
\ on them, placing the results back on the
\ stack.
2 3 +
dump
5 *
dump
5 /
dump
3 -
dump
8 **
dump
\ tests/test_arith.sol \ \ +, -, *, /, ** \ \ Arithmetic operators take the top two items \ from the stack and performs the operations \ on them, placing the results back on the \ stack. 2 3 + dump 5 * dump 5 / dump 3 - dump 8 ** dump