There are a couple differences (most likely deficiencies) with my implementation. First, I use a list instead of a set when constructing the candidate word list. Second, I created a separate occurrence function in order to provide the smoothing capability for our occurrence distribution. Third, I didn’t really care much for a low line count. It’s not the LOC that matter in the end, it’s how easily you can comprehend the code!

public class SpellingCorrector {
  def wordoccur = [:]
 
  def words(File file) {
    Scanner scanner = new Scanner(file)
    def words = scanner.findAll{ x -> x.toLowerCase() ==~ ~/[a-z]+/  }
  }
 
  def train(List words) {
    words.each {
      wordoccur[it] = wordoccur.containsKey(it) ? wordoccur[it] + 1 : 1
    }
  }
 
  def edits1(String word) {
    def results = []
    int n = word.length()
    //Deletion. Remove a character.
    for (i in 0..<n)
      results << word[0..<i] + word[i+1..<n]
 
    //Transposition. Swap adjacent characters.
    for (i in 0..<n-1)
      results << word[0..<i] + word[i+1] + word[i] + word[i+2..<n]
 
    //Alteration. Change one character for another letter.
    for (i in 0..<n)
      for (c in 'a'..'z')
        results << word[0..<i] + c + word[i+1..<n]
 
    //Insertion. Add a letter in between the others.
    for (i in 0..<n)
      for (c in 'a'..'z')
        results << word[0..<i] + c + word[i+1..<n]
 
    return results
  }
 
  def knownedits2(String word) {
    def candidates = []
    edits1(word).each {
      candidates.addAll( edits1(it).findAll { wordoccur.containsKey(it) } )
    }
    return candidates
  }
 
  /**
   * Smoothing distribution. If the word hasn't been encountered (novel words),
   *  we give it an occurence value of 1.
   */
  def int occurrence(String word) {
    return wordoccur[word] == null ? 1 : wordoccur[word];  
  }
 
  def List known(List words) {
    return words.findAll { wordoccur.containsKey(it.toLowerCase()) }
  }
 
  def correct(String word) {
    def candidates = [word] + known([word]) + known(edits1(word)) + knownedits2(word)
    return candidates.max {  occurrence(it) }
  }
}

First, we don’t attempt to split words into two sub-words. For example, a common typo may be “Ihave” rather than “I have”. Second, the training and known function can definitely be improved to with support for proper nouns, stemming, and more. I think it would be a fun exercise to try and to create a simple implementation of these features, much like the SpellingCorrector.

So, Groovy has great support for regular expressions, list construction and compositions and best of all, closures! I also had a chance to play with the Groovy NodeBuilder (on a separate program), which is a great way for constructing tree structures. All said and done, I would hate to implement this in Java.

Update: The slides for my presentation have been uploaded. Download them here: SPLC 2008 Slides (20).

Software systems have grown larger and more complex in recent years. Generative software development strives to automate software development from a systems family by generating implementations using domain-specific languages. In current practice, specifying domain-specific languages is a manual task requiring expert analysis of multiple information sources. Furthermore, the concepts and relations represented in a language are grown through its usage. Keeping the language consistent with its usage is a time-consuming process requiring manual comparison between the language instances and its language specification. Feature model mining addresses these issues by synthesizing a representative model bottom-up from a sample set of instances called configurations.

This thesis presents a mining algorithm that reverse-engineers a probabilistic feature model from a set of individual configurations. A configuration consists of a list of features that are defined as system properties that a stakeholder is interested in. Probabilistic expressions are retrieved from the sample configurations through the use of conjunctive and disjunctive association rule mining. These expressions are used to construct a probabilistic feature model.

The mined feature model consists of a hierarchy of features, a set of additional hard constraints and soft constraints. The hierarchy describes the dependencies and alternative relations exhibited among the features. The additional hard constraints are a set of propositional formulas which must be satisfied in a legal configuration. Soft constraints describe likely defaults or common patterns.

Systems families are often realized using object-oriented frameworks that provide reusable designs for constructing a family of applications. The mining algorithm is evaluated on a set of applications to retrieve a metamodel of the Java Applet framework. The feature model is then applied to the development of framework-specific modeling languages (FSMLs). FSMLs are domain-specific languages that model the framework-provided concepts and their rules for development.

The work presented in this thesis provides the foundation for further research in feature model mining. The strengths and weaknesses of the algorithm are analyzed and the thesis concludes with a discussion of possible extensions.
Software systems have grown larger and more complex in recent years. Generative software development strives to automate software development from a systems family by generating implementations using domain-specific languages. In current practice, specifying domain-specific languages is a manual task requiring expert analysis of multiple information sources. Furthermore, the concepts and relations represented in a language are grown through its usage. Keeping the language consistent with its usage is a time-consuming process requiring manual comparison between the language instances and its language specification. Feature model mining addresses these issues by synthesizing a representative model bottom-up from a sample set of instances called configurations.

This thesis presents a mining algorithm that reverse-engineers a probabilistic feature model from a set of individual configurations. A configuration consists of a list of features that are defined as system properties that a stakeholder is interested in. Probabilistic expressions are retrieved from the sample configurations through the use of conjunctive and disjunctive association rule mining. These expressions are used to construct a probabilistic feature model.

The mined feature model consists of a hierarchy of features, a set of additional hard constraints and soft constraints. The hierarchy describes the dependencies and alternative relations exhibited among the features. The additional hard constraints are a set of propositional formulas which must be satisfied in a legal configuration. Soft constraints describe likely defaults or common patterns.

Systems families are often realized using object-oriented frameworks that provide reusable designs for constructing a family of applications. The mining algorithm is evaluated on a set of applications to retrieve a metamodel of the Java Applet framework. The feature model is then applied to the development of framework-specific modeling languages (FSMLs). FSMLs are domain-specific languages that model the framework-provided concepts and their rules for development.

The work presented in this thesis provides the foundation for further research in feature model mining. The strengths and weaknesses of the algorithm are analyzed and the thesis concludes with a discussion of possible extensions.

Download via UWSpace

Update: Here are the slides that I’ve used for my presentation.

Abstract

Software systems have grown larger and more complex in recent years. Generative software development strives to automate software development from a systems family by generating implementations using domain-specific languages. In current practice, such languages are built using a top-down approach. In addition, keeping the language specifications consistent with its usage is a difficult and manual task.

An algorithm for reverse-engineering a probabilistic feature model from a sample set of configurations is presented in this thesis. The expressions needed to construct a feature model are discovered by mining for so called association rules. The mined feature model consists of two components: a hierarchy of features that represent feature dependencies and alternative choices, in addition to a set of soft constraints that describe likely defaults or patterns exhibited in the sample set. Consequently, the mined model represents a language that describes the dependencies between features and its exhibited variability in a given sample set.

The mining algorithm is evaluated on a set of Java Applets to retrieve a model representing its framework usage. The retrieved feature model is further applied towards the development of framework-specific modeling languages (FSMLs), which are domain-specific languages that model framework-provided concepts and their rules for development.

Terms of Use / Privacy Policy:

By filling out this form, you authorize [spam company], Inc to spread the word about this 100% real and upcomming Messenger Community Site.  You will receive your share of the credit in helping us spread the word.  This is a harmless Community site which is offering users a platform to meet each other for free.

Right, real and “upcomming” Community site, sounds encouraging to me! In fact, they offer to share the credit in spreading the word. What exactly are they they going to share, their revenue? They even state that the site is harmless. I feel safer already.

We do not share your private information with any third parties.  By using our service/website  you hereby fully authorize [spam company], Inc to send messages of a commercial nature via Instant Messages and E-Mails on behalf of third parties via the information you provide us. This is not a "phishing" site that attempts to "trick" you into revealing personal information. Everything we do with your information is disclosed here. If you are under eighteen (18), you MUST obtain permission from a parent or guardian before using our website/service.

Great! They state that they will send “messages of a commercial nature”, ie. SPAM, via IM and E-Mail, on behalf of third parties. The quotes around “phishing” and “trick” definitely make me feel safe. Afterall, the meaning of phishing and trick must be ambiguous, which necessitates the quotes… right?

This page is not affiliated with or operated by Microsoft(tm) or MSN Network(tm).

...

We may temporarily access your MSN account to do a combination of the following:
1.  Send Instant Messages to your friends promoting this site.
2.  Introduce new entertaining sites to your friends via Instant Messages.

Sign me up!

This is a free service. You will not be asked to pay at any time.  You will not be subscribed to anything asking for payment.  This service is made possible by many hours of human effort.

Wow, all these features… for free. Who woulde have thought? A project of such caliber, that was made possible by many hours of human effort, all for free!

Source

So distributed source control systems seemed to fit the bill. I looked at two systems in particular, Mercurial and git. Mercurial caught my eye because of its simplicity, and similarity with the traditional, centralized SCMs such as CVS and Subversion. However, I actually started using git first. The reason was that many open source projects had switched to git and I needed to compile several bleeding edge packages. So, I had no choice but to learn to use git. However, I couldn’t really wrap my head around it. While git is no doubt, a very powerful SCM, it was also a very complicated SCM. I took me a good hour or so before I understood how to track branches.

So, I settled for Mercurial. While I was worried that Mercurial was too immature, the fact that the Mozilla projects are also using Mercurial was very comforting.

Usage, Tips and Tricks

I will not describe the setup and typical usage of Mercurial, since there are plenty of good tutorials out there. What I will do, is highlight some of the features that have really caught my eye and I find indispensible now.

The first great feature of Mercurial is the addremove command, which removes deleted files and adds newly created files to the repository. The command also has an optional parameter which you can specify the directory to process. I typically use ., such that my next commit would only contain the changes that I’ve made in the current directory.

hg addremove   #For the entire repository
hg addremove . #For the current directory

Another nice feature of Mercurial is its ability to ignore certain files and file patterns specified using a single .hgignore file. This is very convenient, for excluding temporary files, such as the intermediary files that are left behind after compiling LaTeX source code. Mercurial supports two syntax types when specifying patterns in .hgignore, glob and regex. The glob syntax allows you to specify patterns using wildcards (*). I primarily use regex, since regular expressions are very powerful.

Here are the contents of my current .hgignore:

syntax: regexp
.*.swp
.*.swo
^results/.*
^projects/.*/bin/
^lab/[0-9][0-9](winter|spring|fall)/.+\.(log|glo|glg|gls|dvi|aux|bm|bbl|blg|brf|ist|lof|lot|out|toc|vrb|ps)

After creating the .hgignore file, don’t forget to add it to the repository for tracking using:

hg add .hgignore

Tracking Branches

For those interested in how to track branches using git, here is the command that I used. the -b modifer creates a branch in the local repository. The checkout command further require the origin/ prefix for branches. In addition, the --track modifier tracks the remote branch such that a future pull will retrieve updates from the branch.

Using Mercurial:

hg co somebranch

Using git:

git checkout --track -b localbranch origin/somebranch

Update on git
Thanks to the comment by Jakub, tracking a remote branch is actually rather easy using the git remote command. Here is an example I had to do with retrieving a xf86-video-ati git repository.

git remote add agd5f git://cgit.freedesktop.org/~agd5f/xf86-video-ati/

To restrict the remote repository to a particular branch, use -t like the following:

git remote add agd5f -t agd-powerplay git://cgit.freedesktop.org/~agd5f/xf86-video-ati/

We can now fetch data from the remote repository using the command:

git fetch agd5f

We can list the branches that are present in our repository by executing the command. Note that the remote repository shows up just like a separate branch to our local repository:

git branch -r

Now, we can pull from this remote repository, and merge the changes from the branch agd-powreplay:

git pull agd5f agd-powerplay

Restoring a Single File to a Previous Revision

It seems that Mercurial is currently not able to checkout a single file or subdirectory. It is one of the “todo” items in their Google Summer of Code project this year, so perhaps we’ll have this feature soon enough. However, this is rather inconvenient at the moment. A workaround for this issue is to use the hg cat command, to view a previous revision of a file. In my case, I had overwritten a file applet.svg with changes that I did not want to keep. To revert the file, I executed:

hg cat applet.svg > applet.svg

Tags

In Mercurial, tags are simply aliases for changesets.

hg checkout -r tagname

git treats tags, branches and trunk/master as one and the same, in the sense that they are all refspecs. This is similar to Subversion’s model, however, tags and branches aren’t simply another directory in the repository. You have to add the tags/ prefix when checking out a tag:

Using git:

git checkout -b localtag tags/remotetag

Conclusions

I am very happy with Mercurial. I ran into a very annoying issue with Mercurial early on, when I executed hg fetch, and it used my repository URI, along with my username and password as a commit message. However, luckily this issue was fixed in the Mercurial 1.0 release. Admittedly, I’ve only been using a single person Mercurial repository for the past couple months, so I haven’t really experienced a situation where I needed the full power of a SCM. However, Mercurial has been so far, been smooth sailing.

Both PDFTeX and XeTeX create a PDF directly from the LaTeX source. XeTeX is built on PDFTeX, and is of particular interest since it has added support for TrueType and OpenType fonts. For beamer presentations, this was great, since it opens up a huge selection of fonts for use in presentations. To change the default font in the document with XeTeX, use the fontspec package. The xunicode package provides additional mapping between LaTeX accents and the selected font. A third package, xltxtra provides some fixes relating to fonts.

\documentclass[xetex,mathserif,serif]{beamer}
 
\usepackage{fontspec}
\usepackage{xunicode} %Unicode extras!
\usepackage{xltxtra}  %Fixes
\setmainfont{Calibri}
\setmonofont[Scale=0.86]{Andale Mono}

Of course, you should replace Calibri and Andale Mono with a font of your choice.

Another nice package to use with PDFTeX, is the microtype package, which provides better font output. Enable the package with this line:

\usepackage[final,expansion=true,protrusion=true,spacing=true,kerning=true]{microtype}

XeTeX and PGF / TiKZ

PGF / TiKZ is a TeX library for drawing graphics using the PDFTeX and XeTeX drivers. However, you may encounter the following error message when attempting to compile a presentation with PGF / TiKZ pictures in your Beamer slides:

Package pgf Warning: Your graphic driver pgfsys-dvipdfm.def does not supported marking the current position.

Unfortunately, the included TiKZ library in the TeXLive 2007 distribution does not support XeTeX. This causes cross-picture coordinates to break, which can be used to draw arrows between various TiKZ pictures in a Beamer frame.

While we wait for TeXLive 2008, you can install the new version of PGF from the CTAN which adds support for the XeTeX driver. Simply download the package, and copy the files to your local ~/texmf/tex/ directory and execute texhash to update the TeX listings.

XeTeX and Wide Pages

Although I haven’t had much time to investigate the issue, but it seems that the pgfpages package that is used with beamer, is not entirely compatible with XeTeX. In particular, the commands:

\setbeameroption{show notes on second screen}        %beamer
\pgfpagesuselayout{two screens with optional second} %pgfpages

does not have any effect on the page dimensions when compiling with xelatex. Compiling with pdflatex does generate a double wide page that is suitable for displaying on two screens.

I’ve been doing some development on the feature modeling plug-in during the past week and have implemented several new features and bug fixes (shown below).

I’m releasing the plug-in as a development release, for now. I have started rewriting the configuration backend, but my thesis deadline is fast approaching and I will not have enough time to complete the changes in fmp. In any case, please let me know of any bugs you find, or if you have a feature request. The source code is also included in the plug-in, so feel free to hack away at it yourself if you are inclined. When the plug-in is sufficiently tested, I will merge this branch into the trunk of the CVS repository on SourceForge. Give it a try!

New Features

• New, more robust and featureful constraint view.
• Constraints are shown for the feature hierarchy in addition to additional constraints.
• When a configuration is selected in the feature model editor, the constraints are evaluated and the status of each constraint is shown (ie. satisfied or not satisfied).
• Support for arbitrary propositional formulas when writing additional constraints. NOTE: constraints are written using node Id instead of an XPath expression. However, feature models created using fmp 0.6.6 are compatible, but will require re-writing the constraints using the new grammar. See below for examples.
• Ability to view Node Ids next to feature names in the feature model.
• Constraint input validation.
• Constraint resolution. An unsatisfied constraint can be resolved in a configuration by right-clicking and selecting ‘Resolve Constraint’.

Installation

• Download ca.uwaterloo.gp.fmp_0.7.0.jar
• Compiled for Java 5 (Java 6 compatible), Eclipse 3.2

Project Homepage: http://gsd.uwaterloo.ca/projects/fmp-plugin/fmp-070/

public static EList open(File file) throws IOException {
	ResourceSet resourceSet = new ResourceSetImpl();
 
	//Initialize the FSML Package information (ie. URI)
	MyPackageImpl.init();
 
	//Set OPTION_RECORD_UNKNOWN_FEATURE prior to calling getResource
	Resource.Factory.Registry.INSTANCE.getExtensionToFactoryMap().put(
		"*", new EcoreResourceFactoryImpl() {
			@Override
			public Resource createResource(URI uri) {
			XMIResourceImpl resource = (XMIResourceImpl) super.createResource(uri);
			resource.getDefaultLoadOptions().put(XMLResource.OPTION_RECORD_UNKNOWN_FEATURE, Boolean.TRUE);
			return resource;
		}
	});
 
	XMIResource resource = (XMIResource) resourceSet.getResource(
	URI.createFileURI(file.toString()),
	true);
 
	//Unknown elements will appear in this map
	System.out.println(resource.getEObjectToExtensionMap());
 
	resource.load(Collections.EMPTY_MAP);
	return resource.getContents();
}

Be aware that any unrecognized elements will be null in the retrieved Ecore model.