AUTHORS:
- William Stein (2006-02-19)
- Fixed bug when loading .py files.
- William Stein (2006-03-09)
- Fixed crash in parsing exponentials.
- Precision of real literals now determined by digits of input (like Mathematica).
- Joe Wetherell (2006-04-14)
- Added MAGMA-style constructor preparsing.
- Bobby Moretti (2007-01-25)
- Added preliminary function assignment notation.
- Robert Bradshaw (2007-09-19)
- Added strip_string_literals, containing_block utility functions. Arrr!
- Added [1,2,..,n] notation.
- Robert Bradshaw (2008-01-04)
- Implicit multiplication (off by default).
- Robert Bradshaw (2008-09-23)
- Factor out constants.
- Robert Bradshaw (2000-01)
- Simplify preparser by making it modular and using regular expressions.
- Bug fixes, complex numbers, and binary input.
EXAMPLES:
Preparsing:
sage: preparse('2/3')
'Integer(2)/Integer(3)'
sage: preparse('2.5')
"RealNumber('2.5')"
sage: preparse('2^3')
'Integer(2)**Integer(3)'
sage: preparse('a^b') # exponent
'a**b'
sage: preparse('a**b')
'a**b'
sage: preparse('G.0') # generator
'G.gen(0)'
sage: preparse('a = 939393R') # raw
'a = 939393'
sage: implicit_multiplication(True)
sage: preparse('a b c in L') # implicit multiplication
'a*b*c in L'
sage: preparse('2e3x + 3exp(y)')
"RealNumber('2e3')*x + Integer(3)*exp(y)"
A string with escaped quotes in it (the point here is that the preparser doesn’t get confused by the internal quotes):
sage: ""Yes," he said."
'"Yes," he said.'
sage: s = "\"; s
'\'
A hex literal:
sage: preparse('0x2e3')
'Integer(0x2e3)'
sage: 0xA
10
sage: 0xe
14
Raw and hex work correctly:
sage: type(0xa1)
<type 'sage.rings.integer.Integer'>
sage: type(0xa1r)
<type 'int'>
sage: type(0Xa1R)
<type 'int'>
In Sage, methods can also be called on integer and real literals (note that in pure Python this would be a syntax error):
sage: 16.sqrt()
4
sage: 87.factor()
3 * 29
sage: 15.10.sqrt()
3.88587184554509
sage: preparse('87.sqrt()')
'Integer(87).sqrt()'
sage: preparse('15.10.sqrt()')
"RealNumber('15.10').sqrt()"
Note that calling methods on int literals in pure Python is a syntax error, but Sage allows this for Sage integers and reals, because users frequently request it:
sage: eval('4.__add__(3)')
...
SyntaxError: invalid syntax
Symbolic functional notation:
sage: a=10; f(theta, beta) = theta + beta; b = x^2 + theta
sage: f
(theta, beta) |--> beta + theta
sage: a
10
sage: b
x^2 + theta
sage: f(theta,theta)
2*theta
sage: a = 5; f(x,y) = x*y*sqrt(a)
sage: f
(x, y) |--> sqrt(5)*x*y
This involves an =-, but should still be turned into a symbolic expression:
sage: preparse('a(x) =- 5')
'__tmp__=var("x"); a = symbolic_expression(- Integer(5)).function(x)'
sage: f(x)=-x
sage: f(10)
-10
This involves -=, which should not be turned into a symbolic expression (of course a(x) isn’t an identifier, so this will never be valid):
sage: preparse('a(x) -= 5')
'a(x) -= Integer(5)'
Raw literals:
Raw literals are not preparsed, which can be useful from an efficiency point of view. Just like Python ints are denoted by an L, in Sage raw integer and floating literals are followed by an”r” (or “R”) for raw, meaning not preparsed.
We create a raw integer:
sage: a = 393939r
sage: a
393939
sage: type(a)
<type 'int'>
We create a raw float:
sage: z = 1.5949r
sage: z
1.5949
sage: type(z)
<type 'float'>
You can also use an upper case letter:
sage: z = 3.1415R
sage: z
3.1415000000000002
sage: type(z)
<type 'float'>
This next example illustrates how raw literals can be very useful in certain cases. We make a list of even integers up to 10000:
sage: v = [ 2*i for i in range(10000)]
This takes a noticeable fraction of a second (e.g., 0.25 seconds). After preparsing, what Python is really executing is the following:
sage: preparse('v = [ 2*i for i in range(10000)]')
'v = [ Integer(2)*i for i in range(Integer(10000))]'
If instead we use a raw 2 we get execution that is instant (0.00 seconds):
sage: v = [ 2r * i for i in range(10000r)]
Behind the scenes what happens is the following:
sage: preparse('v = [ 2r * i for i in range(10000r)]')
'v = [ 2 * i for i in range(10000)]'
Implements Matlab-style backslash operator for solving systems:
A / b
EXAMPLES:
sage: preparse("A \ matrix(QQ,2,1,[1/3,'2/3'])")
"A * BackslashOperator() * matrix(QQ,Integer(2),Integer(1),[Integer(1)/Integer(3),'2/3'])"
sage: preparse("A \ matrix(QQ,2,1,[1/3,2*3])")
'A * BackslashOperator() * matrix(QQ,Integer(2),Integer(1),[Integer(1)/Integer(3),Integer(2)*Integer(3)])'
sage: preparse("A \ B + C")
'A * BackslashOperator() * B + C'
sage: preparse("A \ eval('C+D')")
"A * BackslashOperator() * eval('C+D')"
sage: preparse("A \ x / 5")
'A * BackslashOperator() * x / Integer(5)'
sage: preparse("A^3 \ b")
'A**Integer(3) * BackslashOperator() * b'
Returns a list of all files attached to the current session.
OUTPUT:
EXAMPLES:
sage: sage.misc.reset.reset_attached()
sage: t=tmp_filename()+'.py'; open(t,'w').write("print 'hello world'")
sage: attach(t)
hello world
sage: attached_files() == [t]
True
Returns the smallest range (start,end) such that code[start,end] is delimited by balanced delimiters (e.g., parentheses, brackets, and braces).
INPUT:
OUTPUT:
EXAMPLES:
sage: from sage.misc.preparser import containing_block
sage: s = "factor(next_prime(L[5]+1))"
sage: s[22]
'+'
sage: start, end = containing_block(s, 22); print start, end
17 25
sage: s[start:end]
'(L[5]+1)'
sage: s[20]
'5'
sage: start, end = containing_block(s, 20); s[start:end]
'[5]'
sage: start, end = containing_block(s, 20, delimiters=['()']); s[start:end]
'(L[5]+1)'
sage: start, end = containing_block(s, 10); s[start:end]
'(next_prime(L[5]+1))'
Detaches a file, if it was attached with the attach command.
INPUT:
EXAMPLES:
sage: sage.misc.reset.reset_attached()
sage: t=tmp_filename()+'.py'; open(t,'w').write("print 'hello world'")
sage: attach(t)
hello world
sage: attached_files() == [t]
True
sage: detach(t)
sage: attached_files()
[]
Pulls out numeric literals and assigns them to global variables. This eliminates the need to re-parse and create the literals, e.g., during every iteration of a loop.
INPUT:
OUTPUT:
EXAMPLES:
sage: from sage.misc.preparser import extract_numeric_literals
sage: code, nums = extract_numeric_literals("1.2 + 5")
sage: print code
_sage_const_1p2 + _sage_const_5
sage: print nums
{'_sage_const_1p2': "RealNumber('1.2')", '_sage_const_5': 'Integer(5)'}
sage: extract_numeric_literals("[1, 1.1, 1e1, -1e-1, 1.]")[0]
'[_sage_const_1 , _sage_const_1p1 , _sage_const_1e1 , -_sage_const_1en1 , _sage_const_1p ]'
sage: extract_numeric_literals("[1.sqrt(), 1.2.sqrt(), 1r, 1.2r, R.1, R0.1, (1..5)]")[0]
'[_sage_const_1 .sqrt(), _sage_const_1p2 .sqrt(), 1 , 1.2 , R.1, R0.1, (_sage_const_1 .._sage_const_5 )]'
Inserts *’s to make implicit multiplication explicit.
INPUT:
OUTPUT:
EXAMPLES:
sage: from sage.misc.preparser import implicit_mul
sage: implicit_mul('(2x^2-4x+3)a0')
'(2*x^2-4*x+3)*a0'
sage: implicit_mul('a b c in L')
'a*b*c in L'
sage: implicit_mul('1r + 1e3 + 5exp(2)')
'1r + 1e3 + 5*exp(2)'
sage: implicit_mul('f(a)(b)', level=10)
'f(a)*(b)'
Turns implicit multiplication on or off, optionally setting a specific level. Returns the current level if no argument is given.
INPUT:
EXAMPLES:
sage: implicit_multiplication(True)
sage: implicit_multiplication()
5
sage: preparse('2x')
'Integer(2)*x'
sage: implicit_multiplication(False)
sage: preparse('2x')
'2x'
Returns whether a file can be loaded into Sage. This checks only whether its name ends in one of the supported extensions .py, .pyx, .sage, and .spyx.
INPUT:
OUTPUT:
EXAMPLES:
sage: sage.misc.preparser.is_loadable_filename('foo.bar')
False
sage: sage.misc.preparser.is_loadable_filename('foo.c')
False
sage: sage.misc.preparser.is_loadable_filename('foo.sage')
True
Executes a file in the scope given by globals. The filename itself is also evaluated in the scope. If the name starts with http://, it is treated as a URL and downloaded.
Note
For Cython files, the situation is more complicated – the module is first compiled to a temporary module t and executed via:
from t import *
INPUT:
EXAMPLES:
Note that .py files are not preparsed:
sage: t=tmp_filename()+'.py'; open(t,'w').write("print 'hi',2/3; z=-2/9")
sage: sage.misc.preparser.load(t,globals())
hi 0
sage: z
-1
A .sage file is preparsed:
sage: t=tmp_filename()+'.sage'; open(t,'w').write("print 'hi',2/3; s=-2/7")
sage: sage.misc.preparser.load(t,globals())
hi 2/3
sage: s
-2/7
Cython files are not preparsed:
sage: t=tmp_filename()+'.pyx'; open(t,'w').write("print 'hi',2/3; z=-2/9")
sage: z=0; sage.misc.preparser.load(t,globals())
Compiling ....pyx...
hi 0
sage: z
-1
If the file isn’t a Cython, Python, or a Sage file, a ValueError is raised:
sage: sage.misc.preparser.load('a.foo',globals())
...
ValueError: argument (='a.foo') to load or attach must have extension py, sage, or pyx
A filename given as an expression get evaluated. This ensures that load DATA+'foo.sage' works in the Notebook, say:
sage: t=tmp_filename()+'.py'; open(t,'w').write("print 'hello world'")
sage: sage.misc.preparser.load(t, globals())
hello world
We load a file given at a remote URL:
sage: sage.misc.preparser.load('http://wstein.org/loadtest.py', globals()) # optional - internet
hi from the net
We attach a file:
sage: t=tmp_filename()+'.py'; open(t,'w').write("print 'hello world'")
sage: sage.misc.preparser.load(t, globals(), attach=True)
hello world
sage: t in sage.misc.preparser.attached
True
You can’t attach remote URLs (yet):
sage: sage.misc.preparser.load('http://wstein.org/loadtest.py', globals(), attach=True) # optional - internet
...
NotImplementedError: you can't attach a URL
Encodes a load or attach command as valid Python code.
INPUT:
OUTPUT:
EXAMPLES:
sage: sage.misc.preparser.load_wrap('foo.py', True)
'sage.misc.preparser.load(sage.misc.preparser.base64.b64decode("Zm9vLnB5"),globals(),True)'
sage: sage.misc.preparser.load_wrap('foo.sage')
'sage.misc.preparser.load(sage.misc.preparser.base64.b64decode("Zm9vLnNhZ2U="),globals(),False)'
sage: sage.misc.preparser.base64.b64decode("Zm9vLnNhZ2U=")
'foo.sage'
Returns an iterator over the names of the attached files that have changed since last time this function was called.
OUTPUT:
EXAMPLES:
sage: sage.misc.reset.reset_attached()
sage: t=tmp_filename()+'.py';
sage: a = 0
sage: f = open(t,'w'); f.write("a = 5"); f.close()
sage: attach(t)
sage: a
5
sage: len(list(sage.misc.preparser.modified_attached_files()))
0
sage: import time; time.sleep(2)
sage: f = open(t,'w'); f.write("a = 10"); f.close()
sage: len(list(sage.misc.preparser.modified_attached_files()))
1
sage: len(list(sage.misc.preparser.modified_attached_files()))
0
Preparses [0,2,..,n] notation.
INPUT:
OUTPUT:
EXAMPLES:
sage: from sage.misc.preparser import parse_ellipsis
sage: parse_ellipsis("[1,2,..,n]")
'(ellipsis_range(1,2,Ellipsis,n))'
sage: parse_ellipsis("for i in (f(x) .. L[10]):")
'for i in (ellipsis_iter(f(x) ,Ellipsis, L[10])):'
sage: [1.0..2.0]
[1.00000000000000, 2.00000000000000]
Preparses a line of input.
INPUT:
OUTPUT:
EXAMPLES:
sage: preparse("ZZ.<x> = ZZ['x']")
"ZZ = ZZ['x']; (x,) = ZZ._first_ngens(1)"
sage: preparse("ZZ.<x> = ZZ['y']")
"ZZ = ZZ['y']; (x,) = ZZ._first_ngens(1)"
sage: preparse("ZZ.<x,y> = ZZ[]")
"ZZ = ZZ['x, y']; (x, y,) = ZZ._first_ngens(2)"
sage: preparse("ZZ.<x,y> = ZZ['u,v']")
"ZZ = ZZ['u,v']; (x, y,) = ZZ._first_ngens(2)"
sage: preparse("ZZ.<x> = QQ[2^(1/3)]")
'ZZ = QQ[Integer(2)**(Integer(1)/Integer(3))]; (x,) = ZZ._first_ngens(1)'
sage: QQ[2^(1/3)]
Number Field in a with defining polynomial x^3 - 2
sage: preparse("a^b")
'a**b'
sage: preparse("a^^b")
'a^b'
sage: 8^1
8
sage: 8^^1
9
sage: 9^^1
8
sage: preparse("A \ B")
'A * BackslashOperator() * B'
sage: preparse("A^2 \ B + C")
'A**Integer(2) * BackslashOperator() * B + C'
sage: preparse("a \\ b \\") # There is really only one backslash here, it's just being escaped.
'a * BackslashOperator() * b \\'
sage: preparse("time R.<x> = ZZ[]", do_time=True)
'__time__=misc.cputime(); __wall__=misc.walltime(); R = ZZ[\'x\']; print "Time: CPU %.2f s, Wall: %.2f s"%(misc.cputime(__time__), misc.walltime(__wall__)); (x,) = R._first_ngens(1)'
Supports calculus-like function assignment, e.g., transforms:
f(x,y,z) = sin(x^3 - 4*y) + y^x
into:
__tmp__=var("x,y,z")
f = symbolic_expression(sin(x**3 - 4*y) + y**x).function(x,y,z)
AUTHORS:
EXAMPLES:
sage: preparse("f(x) = x^3-x")
'__tmp__=var("x"); f = symbolic_expression(x**Integer(3)-x).function(x)'
sage: preparse("f(u,v) = u - v")
'__tmp__=var("u,v"); f = symbolic_expression(u - v).function(u,v)'
sage: preparse("f(x) =-5")
'__tmp__=var("x"); f = symbolic_expression(-Integer(5)).function(x)'
sage: preparse("f(x) -= 5")
'f(x) -= Integer(5)'
sage: preparse("f(x_1, x_2) = x_1^2 - x_2^2")
'__tmp__=var("x_1,x_2"); f = symbolic_expression(x_1**Integer(2) - x_2**Integer(2)).function(x_1,x_2)'
For simplicity, this function assumes all statements begin and end with a semicolon:
sage: from sage.misc.preparser import preparse_calculus
sage: preparse_calculus(";f(t,s)=t^2;")
';__tmp__=var("t,s"); f = symbolic_expression(t^2).function(t,s);'
sage: preparse_calculus(";f( t , s ) = t^2;")
';__tmp__=var("t,s"); f = symbolic_expression(t^2).function(t,s);'
Preparses input, attending to numeric literals and load/attach file directives.
Note
Temporarily, if @parallel is in the input, then numeric_literals is always set to False.
INPUT:
OUTPUT:
TESTS:
sage: from sage.misc.preparser import preparse_file
sage: lots_of_numbers = "[%s]" % ", ".join(str(i) for i in range(3000))
sage: _ = preparse_file(lots_of_numbers)
sage: print preparse_file("type(100r), type(100)")
_sage_const_100 = Integer(100)
type(100 ), type(_sage_const_100 )
Parses generator syntax, converting:
obj.<gen0,gen1,...,genN> = objConstructor(...)
into:
obj = objConstructor(..., names=("gen0", "gen1", ..., "genN"))
(gen0, gen1, ..., genN,) = obj.gens()
and:
obj.<gen0,gen1,...,genN> = R[interior]
into:
obj = R[interior]; (gen0, gen1, ..., genN,) = obj.gens()
INPUT:
OUTPUT:
LIMITATIONS:
- The entire constructor must be on one line.
AUTHORS:
TESTS:
sage: from sage.misc.preparser import preparse, preparse_generators
Vanilla:
sage: preparse("R.<x> = ZZ['x']")
"R = ZZ['x']; (x,) = R._first_ngens(1)"
sage: preparse("R.<x,y> = ZZ['x,y']")
"R = ZZ['x,y']; (x, y,) = R._first_ngens(2)"
No square brackets:
sage: preparse("R.<x> = PolynomialRing(ZZ, 'x')")
"R = PolynomialRing(ZZ, 'x', names=('x',)); (x,) = R._first_ngens(1)"
sage: preparse("R.<x,y> = PolynomialRing(ZZ, 'x,y')")
"R = PolynomialRing(ZZ, 'x,y', names=('x', 'y',)); (x, y,) = R._first_ngens(2)"
Names filled in:
sage: preparse("R.<x> = ZZ[]")
"R = ZZ['x']; (x,) = R._first_ngens(1)"
sage: preparse("R.<x,y> = ZZ[]")
"R = ZZ['x, y']; (x, y,) = R._first_ngens(2)"
Names given not the same as generator names:
sage: preparse("R.<x> = ZZ['y']")
"R = ZZ['y']; (x,) = R._first_ngens(1)"
sage: preparse("R.<x,y> = ZZ['u,v']")
"R = ZZ['u,v']; (x, y,) = R._first_ngens(2)"
Number fields:
sage: preparse("K.<a> = QQ[2^(1/3)]")
'K = QQ[Integer(2)**(Integer(1)/Integer(3))]; (a,) = K._first_ngens(1)'
sage: preparse("K.<a, b> = QQ[2^(1/3), 2^(1/2)]")
'K = QQ[Integer(2)**(Integer(1)/Integer(3)), Integer(2)**(Integer(1)/Integer(2))]; (a, b,) = K._first_ngens(2)'
Just the .<> notation:
sage: preparse("R.<x> = ZZx")
'R = ZZx; (x,) = R._first_ngens(1)'
sage: preparse("R.<x, y> = a+b")
'R = a+b; (x, y,) = R._first_ngens(2)'
sage: preparse("A.<x,y,z>=FreeAlgebra(ZZ,3)")
"A = FreeAlgebra(ZZ,Integer(3), names=('x', 'y', 'z',)); (x, y, z,) = A._first_ngens(3)"
Ensure we don’t eat too much:
sage: preparse("R.<x, y> = ZZ;2")
'R = ZZ; (x, y,) = R._first_ngens(2);Integer(2)'
sage: preparse("R.<x, y> = ZZ['x,y'];2")
"R = ZZ['x,y']; (x, y,) = R._first_ngens(2);Integer(2)"
sage: preparse("F.<b>, f, g = S.field_extension()")
"F, f, g = S.field_extension(names=('b',)); (b,) = F._first_ngens(1)"
For simplicity, this function assumes all statements begin and end with a semicolon:
sage: preparse_generators("; R.<x>=ZZ[];")
"; R = ZZ['x']; (x,) = R._first_ngens(1);"
This preparses numerical literals into their Sage counterparts, e.g. Integer, RealNumber, and ComplexNumber.
INPUT:
OUTPUT:
EXAMPLES:
sage: from sage.misc.preparser import preparse_numeric_literals
sage: preparse_numeric_literals("5")
'Integer(5)'
sage: preparse_numeric_literals("5j")
"ComplexNumber(0, '5')"
sage: preparse_numeric_literals("5jr")
'5J'
sage: preparse_numeric_literals("5l")
'5l'
sage: preparse_numeric_literals("5L")
'5L'
sage: preparse_numeric_literals("1.5")
"RealNumber('1.5')"
sage: preparse_numeric_literals("1.5j")
"ComplexNumber(0, '1.5')"
sage: preparse_numeric_literals(".5j")
"ComplexNumber(0, '.5')"
sage: preparse_numeric_literals("5e9j")
"ComplexNumber(0, '5e9')"
sage: preparse_numeric_literals("5.")
"RealNumber('5.')"
sage: preparse_numeric_literals("5.j")
"ComplexNumber(0, '5.')"
sage: preparse_numeric_literals("5.foo()")
'Integer(5).foo()'
sage: preparse_numeric_literals("5.5.foo()")
"RealNumber('5.5').foo()"
sage: preparse_numeric_literals("5.5j.foo()")
"ComplexNumber(0, '5.5').foo()"
sage: preparse_numeric_literals("5j.foo()")
"ComplexNumber(0, '5').foo()"
sage: preparse_numeric_literals("1.exp()")
'Integer(1).exp()'
sage: preparse_numeric_literals("1e+10")
"RealNumber('1e+10')"
sage: preparse_numeric_literals("0x0af")
'Integer(0x0af)'
sage: preparse_numeric_literals("0x10.sqrt()")
'Integer(0x10).sqrt()'
sage: preparse_numeric_literals('0o100')
"Integer('100', 8)"
sage: preparse_numeric_literals('0b111001')
"Integer('111001', 2)"
sage: preparse_numeric_literals('0xe')
'Integer(0xe)'
sage: preparse_numeric_literals('0xEAR')
'0xEA'
sage: preparse_numeric_literals('0x1012Fae')
'Integer(0x1012Fae)'
Removes leading sage: and >>> prompts so that pasting of examples from the documentation works.
INPUT:
OUTPUT:
EXAMPLES:
sage: from sage.misc.preparser import strip_prompts
sage: strip_prompts("sage: 2 + 2")
'2 + 2'
sage: strip_prompts("sage: 3 + 2")
'3 + 2'
sage: strip_prompts(" 2 + 4")
' 2 + 4'
Returns a string with all literal quotes replaced with labels and a dictionary of labels for re-substitution. This makes parsing easier.
INPUT:
OUTPUT:
EXAMPLES:
sage: from sage.misc.preparser import strip_string_literals
sage: s, literals, state = strip_string_literals(r'''['a', "b", 'c', "d\""]''')
sage: s
'[%(L1)s, %(L2)s, %(L3)s, %(L4)s]'
sage: literals
{'L4': '"d\\""', 'L2': '"b"', 'L3': "'c'", 'L1': "'a'"}
sage: print s % literals
['a', "b", 'c', "d\""]
sage: print strip_string_literals(r'-"\\\""-"\\"-')[0]
-%(L1)s-%(L2)s-
Triple-quotes are handled as well:
sage: s, literals, state = strip_string_literals("[a, '''b''', c, '']")
sage: s
'[a, %(L1)s, c, %(L2)s]'
sage: print s % literals
[a, '''b''', c, '']
Comments are substitute too:
sage: s, literals, state = strip_string_literals("code '#' # ccc 't'"); s
'code %(L1)s #%(L2)s'
sage: s % literals
"code '#' # ccc 't'"
A state is returned so one can break strings across multiple calls to this function:
sage: s, literals, state = strip_string_literals('s = "some'); s
's = %(L1)s'
sage: s, literals, state = strip_string_literals('thing" * 5', state); s
'%(L1)s * 5'
TESTS:
Even for raw strings, a backslash can escape a following quote:
sage: s, literals, state = strip_string_literals(r"r'somethin\' funny'"); s
'r%(L1)s'
sage: dep_regex = r'^ *(?:(?:cimport +([\w\. ,]+))|(?:from +(\w+) +cimport)|(?:include *[\'"]([^\'"]+)[\'"])|(?:cdef *extern *from *[\'"]([^\'"]+)[\'"]))' # Ticket 5821