syntax -- an overview

syntax -- an overview

A newline ends a statement if it can, otherwise it acts like any white space.

Parsing is determined by a triple of numbers attached to each token. The following table (produced by the command seeParsing), displays each of these numbers.

i2 : seeParsing()



o2 =  parsing     binary    unary                                          

      precedence  binding   binding                 operators

                  strength  strength



           0                                         )  ]  }               



           1          1                                 ;                  



           2          2         2                       ,                  



           3                    3             do  else  list  then         



           4          3                      ->  :=  <-  =  ==>  =>        



           5                    5              from  of  to  when          



           6          6         6                      <<                  



           7          6                                >>                  



           8          8                                or                  



           9          9                                and                 



          10                   10                      not                 



          11         11                        !=  =!=  ==  ===  ?         



          11         11        11                 <  <=  >  >=             



          12         12                                ||                  



          13         13                                &&                  



          14         13                                 :                  



          15         15                                 |                  



          16         16                                ^^                  



          17         17                                 &                  



          18         18                                ..                  



          19         19                               +  ++                



          19         19        19                       -                  



          20         20                                **                  



          21                    0                       [                  



          22         21                                \\                  



          23         22                                 \                  



          23         23                             %  /  //               



          23         23        23                       *                  



          24         23                               ##  @                



          25                                           (*)                 



          25                    0                     (  {                 



          25                    3     if  shield  time  timing  try  while 



          25                    5            break  for  new  return       



          25                   30             global  local  symbol        



          25         24                            <ADJACENCY>             



          25         25        25                   <SYMBOLS>              



          26         26                                @@                  



          27                                            ~                  



          28         28                     #?  .  .?  /^  ^  ^**  _       



          28         28        24                       #                  



          29                                            !                  



o2 : Table

Here is the way these numbers work. The parser maintains a number which we will call the current parsing level, or simply, the level. The parser builds up an expression until it encounters an input token whose parsing precedence is less than or equal to the current level. The tokens preceding the offending token are bundled into an expression appropriately and incorporated into the containing expression.

When an operator or token is encountered, its binding strength serves as the level for parsing the subsequent expression, unless the current level is higher, in which case it is used.

Consider a binary operator such as *. The relationship between its binary binding strength and its parsing precedence turns out to determine whether a*b*c is parsed as (a*b)*c or as a*(b*c). When the parser encounters the second *, the current parsing level is equal to the binding strength of the first *. If the binding strength is less than the precedence, then the second * becomes part of the right hand operand of the first *, and the expression is parsed as a*(b*c). Otherwise, the expression is parsed as (a*b)*c.

For unary operators, the unary binding strength is used instead of the binary binding strength to reset the current level. The reason for having both numbers is that some operators can be either unary or binary, depending on the context. A good example is # which binds as tightly as . when used as an infix operator, and binds as loosely as adjacency or function application when used as a prefix operator.

To handle expressions like b c d, where there are no tokens present which can serve as a binary multiplication operator, after parsing b, the level will be set to 1 less than the precedence of an identifier, so that b c d will be parsed as b (c d).

The comma and semicolon get special treatment; the empty expression can occur to the right of the comma or semicolon or to the left of the comma.