Klangism

Things that were important enough at the time when they were written.

Viktor Klang is a legendary programmer, known from places like the Internet. Consider following him on Twitter.

Posts tagged dependency

Dec 8

Hardcore Pom

A fairly nice technique to speed up update- times in SBT is by using ModuleConfigurations. A ModuleConfiguration can be viewed as a filter on top of a repository, saying: "Only look in the repository if the artifact you’re looking for has the following group id."

Pattern:

class MyProject(info: ProjectInfo) extends DefaultProject(info) {

object Repositories {

  //Add repositories here, they won’t be used for artifact lookup since they’re inside the object

  lazy val ScalaToolsRepo = MavenRepository(“Scala Tools Repository”, “http://nexus.scala-tools.org/content/repositories/hosted”)

  //… add more repos here

}

//Here we add our ModuleConfigurations

lazy val scalaTestModuleConfig  = ModuleConfiguration(“org.scalatest”, ScalaToolsRepo) //equiv of saying: “When you look for something with group id “org.scalatest”, look n this repo, but not otherwise

//Here we add our dependencies

val scalatest = “org.scalatest” % “scalatest” % “1.2” % “test”

}

That’s the entire pattern, this leads to extremely fast “sbt update"s and you are in more control of what gets downloaded from where.

However, I noticed that there is a big caveat.

Unfortunately SBT doesn’t take ModuleConfigurations into consideration when generating poms during publish-local and publish, this leads to poms that are missing a lot if not all of the repositories needed for its dependencies if you use the technique above.

With a big list of dependencies and ModuleConfigurations this is what I got in my pom:

<repositories>
        <repository>
            <id>ScalaToolsReleasesRepo</id>
            <name>Scala Tools Releases Repo</name>
            <url>http://scala-tools.org/repo-releases/</url>
        </repository>

</repositories>

Not what I had expected, there should be quite a few repos in there…

So I hacked together a small chunk of Scala that will post-process the poms and add all repos in your ModuleConfigurations, and I’ve named it McPom (ModuleConfiguration into Pom).

And this is the same artifact after McPom is thrown into the game:

    <repositories>
        <repository>
            <id>ScalaToolsReleasesRepo</id>
            <name>Scala Tools Releases Repo</name>
            <url>http://scala-tools.org/repo-releases/</url>
        </repository>
        <repository>
            <id>javanetRepo</id>
            <name>java.net Repo</name>
            <url>http://download.java.net/maven/2/</url>
        </repository>
        <repository>
            <id>JBossRepo</id>
            <name>JBoss Repo</name>
            <url>http://repository.jboss.org/nexus/content/groups/public/</url>
        </repository>
        <repository>
            <id>public</id>
            <name>public</name>
            <url>http://repo1.maven.org/maven2/</url>
        </repository>
        <repository>
            <id>CodehausRepo</id>
            <name>Codehaus Repo</name>
            <url>http://repository.codehaus.org/</url>
        </repository>
        <repository>
            <id>GuiceyFruitRepo</id>
            <name>GuiceyFruit Repo</name>
            <url>http://guiceyfruit.googlecode.com/svn/repo/releases/</url>
        </repository>
        <repository>
            <id>DatabinderRepo</id>
            <name>Databinder Repo</name>
            <url>http://databinder.net/repo/</url>
        </repository>
        <repository>
            <id>GlassfishRepo</id>
            <name>Glassfish Repo</name>
            <url>http://download.java.net/maven/glassfish/</url>
        </repository>
    </repositories>

Woot! McPom saves the day!

The only thing you need to do is to add the McPom trait to your project file, and mix it into the projects that needs it, and add the following override to them:

override def pomPostProcess(node: Node): Node = mcPom(moduleConfigurations)(super.pomPostProcess(node))
That’s it!
McPom can be found here
Enjoy!

Nov 28

Dependency Hell

I’ve been meaning to write something very thoughtful about dependency management over the past year, but haven’t found time to do it.

So now I’ll just blurt it out and see what you think of it.

Dependency management in Java land usually means keeping track of .jar-files - this is stupid. “What?” you say?

Well packaging a .jar file means that you feel you’re qualified to make the decision on what parts of your code other people will need and use. In other words, you’re trying to see into the future and create the optimal jar for your users.

Well, wake up and smell the snow. You ain’t going to do that well.

That’s why tools like ProGuard exist - to remove superfluous code from your jars, code that isn’t needed for the program to work.

Now, wouldn’t it be nicer if things were the other way around? So instead of _removing_ what isn’t needed, you only get the classes/code you need instead?

Think about it for a second.

Then we have the current situation with Maven repositories. I hate it. It means that you have to manually keep track of tons of servers and if one is down when you need it, you either have to make sure your build works without it, or start looking for the lint in your bellybutton for a while, waiting for it to come online again.

We’ve had p2p-networks, BitTorrent and tons of different distributed storage solutions, so _why_ Dear God do we need to have manual repository management and SPOFs?

Think about it for a second.

I don’t have the technical solutions, but something where the needed classes/code would be loaded at build time and a class loader would find runtime dependencies on a distributed network at runtime and cache them locally.

Make it happen, I dare you