Friday, January 4, 2013

Runtime bytecode generation comes to Sapphire

Sapphire developers define the model by writing Java interfaces annotated with data semantics. A simple element definition might look like this:

@GenerateImpl

public interface Person extends IModelElement
{
    ModelElementType TYPE = new ModelElementType( Person.class );
    
    // *** Name ***
    
    @Required
    
    ValueProperty PROP_NAME = new ValueProperty( TYPE, "Name" );
    
    Value<String> getName();
    void setName( String value );
    
    // *** Age ***
    
    @Type( base = Integer.class )

    ValueProperty PROP_AGE = new ValueProperty( TYPE, "Age" );
    
    Value<Integer> getAge();
    void setAge( String value );
    void setAge( Integer value );
}

To instantiate a Person object, we need a concrete class that implements this interface. For the last few years, Sapphire developers relied on an annotation processor that is part of Sapphire SDK and is triggered by the @GenerateImpl annotation. The annotation processor would generate an implementation class like this:

public final class PersonImpl extends ModelElement implements Person
{
    public PersonImpl( final IModelParticle parent, final ModelProperty parentProperty, final Resource resource )
    {
        super( TYPE, parent, parentProperty, resource );
    }
    
    public Value<String> getName()
    {
        return (Value) read( PROP_NAME );
    }
    
    public void setName( final String value )
    {
        write( PROP_NAME, value );
    }

    public Value<Integer> getAge()
    {
        return (Value) read( PROP_AGE );
    }
    
    public void setAge( final String value )
    {
        write( PROP_AGE, value );
    }
    
    public void setAge( final Integer value )
    {
        write( PROP_AGE, value );
    }
}

The generated class is trivial as all the heavy lifting is done by the code in the ModelElement base class. Nevertheless, generating these implementation classes is important. No one wants to write any significant amount of code with a model that is accessible only via the generic read and write methods.

The annotation processor has been working well enough, but I have been wanting to see if on-demand runtime bytecode generation would be a better solution. Deferring generation of implementation classes until runtime removes the burden of incorporating Sapphire compilation into the application build.

Let me preface the next part by saying that I know next to nothing about Java bytecode, so I have been putting off this project for a while. Bytecode generation is difficult, I thought. I would have to learn a lot of new concepts and it would take a long time to re-implement the compiler. Boy was I wrong! I started this project two days ago and today I was able to remove the old annotation processor and push the changes. I haven’t kept track of how long it took to implement the original compiler, but it wasn’t two days!

Another surprising aspect is that the new compiler is significantly simpler than the original one. Purely in numerical terms:

  • Old Compiler: 17 classes, 3219 lines of code
  • New Compiler: 3 classes, 808 lines of code

I attribute the size disparity primarily to two factors:

  1. Java reflection API is far easier to use than the equivalent Java mirror API that you must use to build an annotation processor.
  2. Generating readable Java source code requires managing formatting and imports. Neither factors into bytecode generation.

The fast progress on the new compiler was further made possible by ASM, a Java bytecode manipulation framework. Leveraging ASM, a framework completely new to me, was made particularly easy by the Bytecode Outline plugin for Eclipse and its ASMifier mode. With the Bytecode Outline view open, you just select a method and you see either the Java bytecode or an ASM code snippet. An incredibly effective way to use ASM without taking the time to learn new API.

ByteCodeOutline-1

ByteCodeOutline-2

Major kudos to those behind ASM and Bytecode Outline. Secondary kudos to Java Decompiler Project. I used the standalone version (JD-GUI) to check the bytecode that I was generating.

The new compiler referenced here will ship as part of the upcoming Sapphire 0.7 release.

Monday, November 26, 2012

Announcing Sapphire 0.6 Release

On behalf of all who contributed, I am very proud to announce availability of Sapphire 0.6 release. This release includes major enhancements, such as better support for versions and version constraints, minimal generated code, improved entry points, and much more.

Enhancements
Migration Guide
Developer Guide
Downloads

Tuesday, November 13, 2012

Calling all Sapphire adopters

There is no better way to evaluate a framework than seeing how others have used it in real world projects. With that in mind, I have created a wiki page to highlight adopters of Sapphire and how they are using the framework.

http://wiki.eclipse.org/Sapphire/Adopters

I have seeded the wiki with some initial content describing how Oracle is using Sapphire.

If you are a current adopter, please consider contributing a section. You can promote your product or project a bit while benefiting others in the process.

Here are some other ways that you can show your support for Sapphire:

  1. Click on Ohloh's "I Use This!" button, rate Sapphire, write a review, etc. Make sure that you've entered your locale in your profile, so that you can be placed on the world map of Sapphire users.
  2. Favorite Sapphire on Eclipse Marketplace.
  3. Like Sapphire on Facebook.
  4. Blog about Sapphire and post a link on Sapphire forum.

Friday, September 28, 2012

Announcing Sapphire 0.5.3 Release

On behalf of all who contributed, I am very proud to announce availability of Sapphire 0.5.3 release. This maintenance release includes bug fixes for issues raised by adopters. Adopters of previous 0.5.x releases can expect full backwards compatibility. No migration guide is being published.

This release is also available in the Juno SR1 repository. Look for Sapphire under Application Development Frameworks category.

Developer Guide
Downloads

Monday, September 10, 2012

The Safe Road

A lot has been said about Eclipse 4.2 performance and stability in the past few weeks. Unfortunately, the community feedback is not positive. This is undoubtedly very difficult to hear for developers who worked so hard for several years to deliver the new platform. This feedback is also difficult to hear for those of us working on layered technology and products. No one in the Eclipse community likes to see what is happening to the reputation of their favorite IDE.

So where do we go from here?

Some have chosen the path of blaming the community. If only there were more people contributing to the platform over the last few years, we wouldn’t be in the current predicament. Now that we are here, if only more contributors would step forward and “give back” to the platform, we would get to a better place quicker. There is no doubt, that more active contribution to the platform would yield measurably better platform, but have calls for more contribution ever yielded the desired result? Why should we expect a different result now?

Calling on the community to “give back” to the platform is frankly insulting and makes about as much sense as calling on the platform team to give back to WTP or EMF or GEF. After all, where would Eclipse be without those toolsets and frameworks. Eclipse achieved its current place in developer mind share not by having the best platform than the competition, but through the breadth of solutions. I don’t know of a single Eclipse.org project or a third-party solution that couldn’t benefit from more resources, so lets stop with guilt casting. The community contributes to success of Eclipse in a variety of ways.

Is 4.2 SR1 or SR2 or 4.3 going to be better? I don’t doubt it, but is it reasonable to ask users to to put up with performance, stability and polish issues for however long it takes for 4.x to match 3.x in these areas? The technology landscape is littered with once-successful projects that have undertaken lengthy re-writes while neglecting the user need for stability.

On the opposite side of the spectrum, some have called for Juno to switch to the more stable 3.8 stream. This isn’t a good solution either. Cut off from the distributions and the common repository, the 4.x platform will have a hard time gaining user base necessary to stabilize. Absent users, the stability would not meaningfully improve for Kepler or the year after that. Effectively, this road abandons the investment made in the 4.x stream. Some may say they don’t care if 4.x stream dies, but no technology remains relevant by stagnating, so this road is just as risky in the long term as the current one.

The community cannot afford the reputation hit of stabilizing 4.x stream by forced adoption and the community cannot afford to abandon the investment made in the 4.x stream. So what is there left to do? I propose that until 4.x stream reaches the stability and polish of 3.x stream, the simultaneous release process should produce both 4.x and 3.x distributions along with the corresponding repositories. It is human nature to seek that what is new and shiny, so there will be developers eager to use 4.x stream. As long as there is a reasonable fallback, we can make progress without antagonizing the user base we all have spend so much time and money building.

Let’s take a deep breath and choose the safe road. We have all invested far too much in Eclipse to risk it by driving too fast on the switchbacks with no guard rails.

If you have an opinion on this proposal, leave a comment in the bugzilla entry that I opened.

Tuesday, July 24, 2012

Announcing Sapphire 0.5.2 Release

On behalf of all who contributed, I am very proud to announce availability of Sapphire 0.5.2 release. This maintenance release includes bug fixes for issues raised by adopters. Adopters of 0.5 and 0.5.1 releases can expect full backwards compatibility. No migration guide is being published.

This is the first release of Sapphire built from source hosted in the new Git Repository. The old CVS repository will continue to be available for some time as a read-only snapshot at the time of Git migration. New changes will not be visible in the CVS repository.

Developer Guide
Downloads

Tuesday, July 3, 2012

Versions and Version Constraints

The following is a preview of functionality in the upcoming Sapphire 0.6 release. If you are new to Sapphire, get introduced, download the latest release and ask questions.

In many complex Sapphire models, it is useful to be able to constrain functionality based on a version. To simplify these scenarios, Sapphire now has native constructs for dealing with versions and version constraints.

Version - Represents a version as a sequence of long integers. In string format, it is represented as a dot-separated list of numeric segments, such as "1.2.3" or "5.7.3.2012070310003".

VersionConstraint - A boolean expression that can check versions for applicability. In string format, it is represented as a comma-separated list of specific versions, closed ranges (expressed using "[1.2.3-4.5)" syntax and open ranges (expressed using "[1.2.3" or "4.5)" syntax). The square brackets indicate that the range includes the specified version. The parenthesis indicate that the range goes up to, but does not actually include the specified version.

Sapphire.version() - Determines the version of Sapphire.

Both Version and VersionConstraint classes can be used as a type of a value property.

Example

// *** Version ***
    
@Type( base = Version.class )

ValueProperty PROP_VERSION = new ValueProperty( TYPE, "Version" );
    
Value<Version> getVersion();
void setVersion( String value );
void setVersion( Version value );
    
// *** VersionConstraint ***
    
@Type( base = VersionConstraint.class )

ValueProperty PROP_VERSION_CONSTRAINT = new ValueProperty( TYPE, "VersionConstraint" );
    
Value<VersionConstraint> getVersionConstraint();
void setVersionConstraint( String value );
void setVersionConstraint( VersionConstraint value );

Further, version constraints can be evaluated in an expression via a pair of new functions. The VersionMatches function takes a version as the first parameter, a version constraint as a second parameter and returns a boolean. The SapphireVersionMatches function takes a version constraint as the sole parameter, evaluates it against Sapphire version and returns a boolean.

Example

In this example, the VersionMatches function is used to control property enablement.

// *** Provider ***
    
@Label( standard = "provider" )
@Enablement( expr = "${ VersionMatches( Root().Version, '[1.1' ) }" )
@XmlBinding( path = "provider" )

ValueProperty PROP_PROVIDER = new ValueProperty( TYPE, "Provider" );

Value<String> getProvider();
void setProvider( String value );

Example

In this example, the VersionMatches function is used in sdef to control visibility of a properties view page.

<properties-view>
    <page>
        <label>provider</label>
        <visible-when>${ VersionMatches( Root().Version, '[1.1' ) }</visible-when>
        <content>
            <property-editor>Provider</property-editor>
            <property-editor>
                <property>Copyright</property>
                <scale-vertically>true</scale-vertically>
            </property-editor>
        </content>
    </page>
</properties-view>