Saturday, May 30, 2009

Synpl broken code uploaded on GitHub

Yes, I know, broken code is not very useful to other people. I just wanted to have some kind of backup :-)

Today I worked pretty hard at finding a way to store text changes between the last successful parse and the current moment. Turns out that storing information about white space and comments in the parsed information is not very useful. I've given up on moving comments along with the item they're referring to. For now.

The most interesting result of today is that a very neat way of storing the text changes is a kind of a diff from the last successful parse. This can also be stored on disk very efficiently (before that I was thinking about storing the parse tree representation, which is much larger).

The only file that works right now is TextWithChanges.py in the root of the repository. The unit tests show what it can do. I'm pretty happy with the abstraction (I'm certain I'm rediscovering something classic here, just not realizing it clearly yet).

Basically, a TextWithChanges (TWC) stores the characters that were added or deleted since the last successful (local) parse. It can provide the 'old' version (that parses) and the current version, using current coordinates (which are actual cursor positions from the editor).

An old version of a slice of a TWC can look quite weird, as deleted characters share the same position (in terms of actual cursor positions).

"An example looks like this."

The red characters were deleted, the green ones added. The old version for a slice will not include the green characters, the current version will not include the red ones. Once a slice parses successfully, the changes are forgotten (the characters become black again).

Storing the version with changes on disk allows the editor to have access to a previous valid parse. Since it's possible to store only the changes and refer the file content with a hash, this can be very space-effective.

No comments: