doc/ref/spec: alternative syntax for definitions

This change is prompted by the need to have separate namespaces
for definitions and regular fields. The solution is inspired by
Go's exporting rules and introduces a lexical distinction
between the identifiers of the two field types.

Advantages:

- puts regular fields and definitions are in a different namespace
- this makes much nicer mappings possible for JSON schema
  and even OpenAPI
- makes exporting rules sensible and workable
- clarity what a reference refers to (similar to casing rules in Go)
- removes confusing "::"
- simplifies spec a little bit
- already partly specifies the syntax for associative lists for free
- reintroduces hidden fields as a special kind of definitions.
  This simplifies the transition away from hidden fields, which
  would be quite cumbersome, as it is a frequently used feature.

```
   #C: 3
}

a: #MyDef.#C
```

Issue #339

Change-Id: I98e187219f720ccec6af6fc9808d5545a9595c21
Reviewed-on: https://cue-review.googlesource.com/c/cue/+/5480
Reviewed-by: Marcel van Lohuizen <mpvl@golang.org>
diff --git a/doc/ref/spec.md b/doc/ref/spec.md
index 7daf5e1..519cfab 100644
--- a/doc/ref/spec.md
+++ b/doc/ref/spec.md
@@ -178,9 +178,10 @@
 
 Identifiers name entities such as fields and aliases.
 An identifier is a sequence of one or more letters (which includes `_` and `$`)
-and digits.
-It may not be `_` or `$`.
-The first character in an identifier must be a letter.
+and digits, optionally preceded by a `#`.
+It may not be `_`, `$`, or `#`.
+The first character in an identifier must be a letter or `#`.
+Identifiers starting with a `#` or `_` are reserved for definitions.
 
 <!--
 TODO: allow identifiers as defined in Unicode UAX #31
@@ -190,7 +191,7 @@
 -->
 
 ```
-identifier  = letter { letter | unicode_digit } .
+identifier  = [ "#" ] letter { letter | unicode_digit } .
 ```
 
 ```
@@ -273,14 +274,12 @@
 
 ```
 +     div   &&    ==    <     =     (     )
--     mod   ||    !=    >     ::    {     }
-*     quo   &     =~    <=    :     [     ]
-/     rem   |     !~    >=    .     ...   ,
-            _|_   !
+-     mod   ||    !=    >     :     {     }
+*     quo   &     =~    <=    ?     [     ]     ,
+/     rem   |     !~    >=    !     _|_   ...   .
 ```
 <!--
-Free tokens: # ; ~ $ ^
-
+Free tokens:  ; ~ ^
 // To be used:
   @   at: associative lists.
 
@@ -1123,9 +1122,8 @@
 ```
 StructLit       = "{" { Declaration "," } [ "..." ] "}" .
 Declaration     = Field | Comprehension | AliasExpr | attribute .
-Field           = LabelSpec { LabelSpec } Expression { attribute } .
-LabelSpec       = Label ( ":" | "::" ) .
-Label           = LabelName [ "?" ] | "[" AliasExpr "]".
+Field           = Label ":" { Label ":" } Expression { attribute } .
+Label           = LabelName [ "?" ] | "[" AliasExpr "]" .
 LabelName       = identifier | simple_string_lit  .
 
 attribute       = "@" identifier "(" attr_tokens ")" .
@@ -1303,13 +1301,11 @@
 
 #### Definitions
 
-A field of a struct may be declared as a regular field (using `:`)
-or as a _definition_ (using `::`).
+A field is a _definition_ if its identifier starts with `#` or `_`.
 Definitions are not emitted as part of the model and are never required
 to be concrete when emitting data.
-It is illegal to have a regular field and a definition with the same name
-within the same struct.
-Literal structs that are part of a definition's value are implicitly closed,
+For definitions with identifiers starting with `#`,
+literal structs that are part of a definition's value are implicitly closed,
 but may unify unrestricted with other structs within the field's declaration.
 This excludes literals structs in embeddings and aliases.
 
@@ -1337,7 +1333,7 @@
 Finally, excluding embeddings from recursive closing allows for
 a mechanism to not recursively close, without needing an additional language
 construct, such as a triple colon or something else:
-foo :: {
+#foo: {
     {
         // not recursively closed
     }
@@ -1350,30 +1346,30 @@
 -->
 
 ```
-MyStruct :: {
-    sub field:    string
+#MyStruct: {
+    sub: field:    string
 }
 
-MyStruct :: {
-    sub enabled?: bool
+#MyStruct: {
+    sub: enabled?: bool
 }
 
-myValue: MyStruct & {
-    sub feild:   2     // error, feild not defined in MyStruct
-    sub enabled: true  // okay
+myValue: #MyStruct & {
+    sub: feild:   2     // error, feild not defined in #MyStruct
+    sub: enabled: true  // okay
 }
 
-D :: {
-    OneOf
+#D: {
+    #OneOf
 
     c: int // adds this field.
 }
 
-OneOf :: { a: int } | { b: int }
+#OneOf: { a: int } | { b: int }
 
 
-D1: D & { a: 12, c: 22 }  // { a: 12, c: 22 }
-D2: D & { a: 12, b: 33 }  // _|_ // cannot define both `a` and `b`
+D1: #D & { a: 12, c: 22 }  // { a: 12, c: 22 }
+D2: #D & { a: 12, b: 33 }  // _|_ // cannot define both `a` and `b`
 ```
 
 
@@ -1683,35 +1679,30 @@
 
 ### Exported identifiers
 
+<!-- move to a more logical spot -->
+
 An identifier of a package may be exported to permit access to it
 from another package.
-An identifier is exported if
-the first character of the identifier's name is a Unicode upper case letter
-(Unicode class "Lu"); and
-the identifier is declared in the file block.
-All other top-level identifiers used for fields not exported.
-
-In addition, any definition declared anywhere within a package of which
-the first character of the identifier's name is a Unicode upper case letter
-(Unicode class "Lu") is visible outside this package.
+All identifiers of regular fields (those not starting with a `#` or `_`)
+are exported.
+A definition identifier is exported if it does not start with `_` or `#_`.
 Any other defintion is not visible outside the package and resides
 in a separate namespace than namesake identifiers of other packages.
-This is in contrast to ordinary field declarations that do not begin with
-an upper-case letter, which are visible outside the package.
 
 ```
 package mypackage
 
-foo: string  // not visible outside mypackage
+foo:   string  // visible outside mypackage
+"bar": string  // visible outside mypackage
 
-Foo :: {     // visible outside mypackage
-    a: 1     // visible outside mypackage
-    B: 2     // visible outside mypackage
+#Foo: {      // visible outside mypackage
+    a:  1    // visible outside mypackage
+    _b: 2    // not visible outside mypackage
 
-    C :: {   // visible outside mypackage
+    #C: {    // visible outside mypackage
         d: 4 // visible outside mypackage
     }
-    e :: foo // not visible outside mypackage
+    #_E: foo // not visible outside mypackage
 }
 ```
 
@@ -2873,7 +2864,7 @@
 PackageName    = identifier .
 ```
 
-The PackageName must not be the blank identifier.
+The PackageName must not be the blank identifier or a definition identifier.
 
 ```
 package math