Update changelog and upgrading guide

Author: nikic

CHANGELOG.md

@@ -1,9 +1,41 @@
 Version 3.0.0-dev
 -----------------
 
+This release primarily improves our support for error recovery.
+
 ### Added
 
 * Added `Node::setDocComment()` method.
+* Added `Error::getMessageWithColumnInfo()` method.
+* Added support for recovery from lexer errors.
+* Added support for recovering from "special" errors (i.e. non-syntax parse errors).
+* Added precise ___location information for lexer errors.
+* Added `ErrorHandler` interface, and `ErrorHandler\Throwing` and `ErrorHandler\Collecting` as
+  specific implementations. These provide a general mechanism for handling error recovery.
+* Added optional `ErrorHandler` argument to `Parser::parse()`, `Lexer::startLexing()` and
+  `NameResolver::__construct()`.
+* The `NameResolver` now adds a `namespacedName` attribute on name nodes that cannot be statically
+  resolved (unqualified unaliased function or constant names in namespaces).
+  
+### Fixed
+
+* Fixed attribute assignment for `GroupUse` prefix and variables in interpolated strings.
+
+### Changed
+
+* The constants on `NameTraverserInterface` have been moved into the `NameTraverser` class.
+* Due to the error handling changes, the `Parser` interface and `Lexer` API have changed.
+* The emulative lexer now directly postprocesses tokens, instead of using `~__EMU__~` sequences.
+  This changes the protected API of the lexer.
+
+### Removed
+
+* Removed `Error::getRawLine()` and `Error::setRawLine()`. These methods have been superseded by
+  `Error::getStartLine()` and `Error::setStartLine()`.
+* Removed support for node cloning in the `NodeTraverser`.
+* Removed `$separator` argument from `Name::toString()`.
+* Removed `throw_on_error` parser option and `Parser::getErrors()` method. Use the `ErrorHandler`
+  mechanism instead.
 
 Version 3.0.0-beta1 (2016-09-16)
 --------------------------------

UPGRADE-3.0.md

@@ -1,8 +1,13 @@
 Upgrading from PHP-Parser 2.x to 3.0
 ====================================
 
-This version does not include any major API changes. Only specific details of the node
-representation have changed in some cases.
+The backwards-incompatible changes in this release may be summarized as follows:
+
+ * The specific details of the node representation have changed in some cases, primarily to
+   accomodate new PHP 7.1 features.
+ * There have been significant changes to the error recovery implementation. This may affect you,
+   if you used the error recovery mode or have a custom lexer implementation.
+ * A number of deprecated methods were removed.
 
 ### PHP version requirements
 
@@ -31,18 +36,119 @@ The following changes are unlikely to require code changes:
  * `void` and `iterable` types are now stored as strings if the PHP 7 parser is used. Previously
    these would have been represented as `Name` instances.
 
-### Removed methods
+### Changes to error recovery mode
+
+Previously, error recovery mode was enabled by setting the `throwOnError` option to `false` when
+creating the parser, while collected errors were retrieved using the `getErrors()` method:
+
+```php
+$lexer = ...;
+$parser = (new ParserFactory)->create(ParserFactor::ONLY_PHP7, $lexer, [
+    'throwOnError' => true,
+]);
+
+$stmts = $parser->parse($code);
+$errors = $parser->getErrors();
+if ($errors) {
+    handleErrors($errors);
+}
+processAst($stmts);
+```
+
+Both the `throwOnError` option and the `getErrors()` method have been removed in PHP-Parser 3.0.
+Instead an instance of `ErrorHandler\Collecting` should be passed to the `parse()` method:
+
+```php
+$lexer = ...;
+$parser = (new ParserFactory)->create(ParserFactor::ONLY_PHP7, $lexer);
+
+$errorHandler = new ErrorHandler\Collecting;
+$stmts = $parser->parse($code, $errorHandler);
+if ($errorHandler->hasErrors()) {
+    handleErrors($errorHandler->getErrors());
+}
+processAst($stmts);
+```
+
+#### Multiple parser fallback in error recovery mode
+
+As a result of this change, if a `Multiple` parser is used (e.g. through the `ParserFactory` using
+`PREFER_PHP7` or `PREFER_PHP5`), it will now return the result of the first *non-throwing* parse. As
+parsing never throws in error recovery mode, the result from the first parser will always be
+returned.
+
+The PHP 7 parser is a superset of the PHP 5 parser, with the exceptions that `=& new` and
+`global $$foo->bar` are not supported (other differences are in representation only). The PHP 7
+parser will be able to recover from the error in both cases. For this reason, this change will
+likely pass unnoticed if you do not specifically test for this syntax.
+
+It is possible to restore the precise previous behavior with the following code:
+
+```php
+$lexer = ...;
+$parser7 = new Parser\Php7($lexer);
+$parser5 = new Parser\Php5($lexer);
+
+$errors7 = new ErrorHandler\Collecting();
+$stmts7 = $parser7->parse($code, $errors7);
+if ($errors7->hasErrors()) {
+    $errors5 = new ErrorHandler\Collecting();
+    $stmts5 = $parser5->parse($code, $errors5);
+    if (!$errors5->hasErrors()) {
+        // If PHP 7 parse has errors but PHP 5 parse has no errors, use PHP 5 result
+        return [$stmts5, $errors5];
+    }
+}
+// If PHP 7 succeeds or both fail use PHP 7 result
+return [$stmts7, $errors7];
+```
+
+#### Error handling in the lexer
+
+In order to support recovery from lexer errors, the signature of the `startLexing()` method changed
+to optionally accept an `ErrorHandler`:
+
+```php
+// OLD
+public function startLexing($code);
+// NEW
+public function startLexing($code, ErrorHandler $errorHandler = null);
+```
+
+If you use a custom lexer with overriden `startLexing()` method, it needs to be changed to accept
+the extra parameter. The value should be passed on to the parent method.
+
+#### Error checks in node constructors
+
+The constructors of certain nodes used to contain additional checks for semantic errors, such as
+creating a try block without either catch or finally. These checks have been moved from the node
+constructors into the parser. This allows recovery from such errors, as well as representing the
+resulting (invalid) AST.
+
+This means that certain error conditions are no longer checked for manually constructed nodes.
+
+### Removed methods, arguments, options
 
-The following methods have been removed:
+The following methods, arguments or options have been removed:
 
  * `Comment::setLine()`, `Comment::setText()`: Create new `Comment` instances instead.
  * `Name::set()`, `Name::setFirst()`, `Name::setLast()`: Create new `Name` instances instead. For
    the latter two a combination of `Name::concat()` and `Name::slice()` can be used.
+ * `Error::getRawLine()`, `Error::setRawLine()`. Use `Error::getStartLine()` and
+   `Error::setStartLine()` instead.
+ * `Parser::getErrors()`. Use `ErrorHandler\Collecting` instead.
+ * `$separator` argument of `Name::toString()`. Use `strtr()` instead, if you really need it.
+ * `$cloneNodes` argument of `NodeTraverser::__construct()`. Explicitly clone nodes in the visitor
+   instead.
+ * `throwOnError` parser option. Use `ErrorHandler\Collecting` instead.
 
 ### Miscellaneous
 
  * All methods on `PrettyPrinter\Standard` are now protected. Previoulsy most of them were public.
    The pretty printer should only be invoked using the `prettyPrint()`, `prettyPrintFile()` and
    `prettyPrintExpr()` methods.
  * The node dumper now prints numeric values that act as enums/flags in a string representation.
-   If node dumper results are used in tests, updates may be needed to account for this.
+   If node dumper results are used in tests, updates may be needed to account for this.
+ * The constants on `NameTraverserInterface` have been moved into the `NameTraverser` class.
+ * The emulative lexer now directly postprocesses tokens, instead of using `~__EMU__~` sequences.
+   This changes the protected API of the emulative lexer.