Does git live up to the hype if we actually measure it?

Alias on 2009-08-12T05:15:28

Clearly, with an inflammatory subject like that you're probably expecting a false question, and I quickly in my journal post say "Yes!" or "No!".

I am, clearly, not pro-git. But I'm actually asking a real question :)

Git still has problems being Windows friendly (most solutions still make you act in a very unix-like manner) and so it's still not on my radar to move any time soon.

But I keep seeing one particular pro-git (and pro-distributed in general) argument come up over and over again in git discussions, the argument that git helps encourage contribution.

And THIS question is one I'm very interested in, it's something that might actually push me towards switching.

I much preferred CVS for it's better branch logic, but because svn makes it so much easier to contribute switching to SVN resulted in improved contribution rates despite the shitty branching.

Has anyone actually measured rate of change and rate of contribution across a SVN to Git conversion?

For my repository, I have quite a good understanding of how many people are contributing.

http://www.ohloh.net/p/3299/contributors

Here's a better one showing contributors over time, compared to the (epic) contributor growth rate of Padre.

http://www.ohloh.net/p/compare?metric=Contributors&project_0=Adam+K+%28et+al%29+CPAN+Modules&project_1=Padre

Is it possible to establish via actual metrics that the diversity of contributors and rate of change flowing into the final production CPAN releases is increased relative to the rate and diversity of change prior to the move?

Have you done it? Can I see the comparative results?

Does the theoretical improvement to contribution result in a real world improvement in contribution? Or does it suffer from the "You are not Linux" scaling problem, that our projects mostly just aren't big enough to benefit?


VCS *does* matter

sartak on 2009-08-12T06:27:10

I used to version all of my modules with darcs. Other than patches from my friends who were already sold on darcs, I received only diffs against CPAN HEAD. Once I moved to git (and perhaps more importantly, github), I did begin receiving patches (many for Any::Moose, a few for App::Nopaste, etc). darcs was cited as a major barrier to entry because it required ghc (which has a notoriously long compilation time).

So in my experience, choice of VCS definitely impacts user contributions. Whether it matters significantly for git versus Subversion, I don't know. Both are pretty easy to install. Both have a lot of mindshare (and thus, documentation and support communities).

git has two advantages over Subversion for me. Its distributed nature is totally a win. Would you grant me a commit bit to your repository if you knew I was malicious? With a distributed VCS, that problem vanishes for you. But I'm still a first-class VCS citizen. I still get to use the VCS to commit atomically, revert commits, and anything else.

The other advantage is that git's tools totally beat Subversion's. Until recently, Subversion didn't even have good tools (except SVK) for merging. But git has all sorts of useful things like bisect, amend-commit, rebase -i, apply, cherry-pick, and (as alluded to earlier) github.

Re:VCS *does* matter

sartak on 2009-08-12T06:38:27

Another quick point I forgot to make: I have had comaint on Log::Dispatch for a month now, but I haven't touched the repository. I don't have any inclination to deal with a fifth version control system (Mercurial) for a single module. I'm sure I'm not alone in this laziness. :)

The Git Advantage

chromatic on 2009-08-12T06:39:28

Git's primary advantage, as I see it, is that forks become first-class citizens. Forking is cheap in all good DVCSs, but Git seems particularly strong about not enforcing one central repository with strict commit access.

I can use git-svn to get some of the benefits of a local Git repo against a centralized SVN repo, but that's a workaround. It's much easier to pull from someone else's Git repo than to apply a series of patches against SVN.

Re:The Git Advantage

Alias on 2009-08-12T07:29:46

I completely agree. For untrusted and non-maintainer forking, it's a world of improvement.

What I'm curious about is whether this actually results in more changes at the bottom line, or is the flexibility of all this forking largely just something that's nice to have (but won't impact much on the overall progress).

Windows

davegaramond on 2009-08-12T09:02:41

Git still has problems being Windows friendly

Not necessarily a problem for everyone :-) You can't totally blame the original designer if he exploits characteristics of a certain OS to gain speed. After all, being cross-platform is not the top design goal for git.

Re:Windows

Alias on 2009-08-12T12:16:18

Raising a barrier to contribution is always a problem. It means less people in the world you don't know about yet are able to effectively contribute.

I've yet to contribute to any git project, because the time investment to learn all the concepts and find a toolchain that doesn't suck just isn't worth it for the sort of small improvements I might want to add.

And as for cross-platform, it was just fine right up until the point people start promoting it for more than just the kernel.

Re:Windows

tgape on 2009-08-14T11:52:22

The problem is that the vast majority of programmers use Windows. Sad to say, given that skill doesn't necessarily choose platform, this also means that most of the really good programmers use Windows - and you're shutting them out.

It is not necessarily a problem that is too difficult to overcome, once one determines one wants to fix it. If all else fails, it should be possible to write a svn-git module, to be able to commit to a git repo as easily as to a svn repo (while, of course, losing the git easy forking advantages for the people who use it). Similarly, there could be an hg-git module, as it seems some claim Mercurial also has a fairly low barrier to submit.

Of course, given the platform of the primary maintainer of Padre, an svn-git module may not be as useful as it might otherwise be...

Disclaimer: I know nothing practical about git. It's been on my list of tools to learn since about a week after it was written, but I've not even read all of the theory, despite having read more of the theory than I have of the practical. I suspect this speaks to the tool's complexity, because I find it to be very exciting in theory - very close to my 'dream VCS'.

I can only offer anecdotal evidence, but:

Aristotle on 2009-08-12T09:24:14

Despite intents, I end contributing to other projects only very very rarely, for whatever reason. Github in particular, however, has had me make impulse contributions of various small patches on several occasions (two of which ended up ballooning into not so small patches when all was said and done).

With appropriate cautions and qualifiers applied, that would suggest that the effect being talked about is, at the very least, plausible.

Re:I can only offer anecdotal evidence, but:

Alias on 2009-08-12T12:17:53

It's certainly plausible, but while it was easy for you to make the patches, there's the question of whether those changes make it back to the production release.

And then there's the question of whether the increase in contributions from people like you is balanced by the decrease in contributions from people that aren't you :)

Re:I can only offer anecdotal evidence, but:

Aristotle on 2009-08-12T17:07:58

The patches did get pulled in upstream – else I wouldn’t mention this at all.

But yeah, the latter is a good point.

Github makes git useful to everyone

brian_d_foy on 2009-08-12T10:53:23

Github has made it really easy for people to send me patches, even for the most trivial things. They fork, make their change, then send me a pull request. Github manages that so I can look at a queue of all my pull requests and apply them in any order I like, and ignore the ones I don't want.

This has more to say about Github than git though. Github is what I always wanted Sourceforge to be and they could never deliver. And, it's a lot better looking :)

Re:Github makes git useful to everyone

Alias on 2009-08-12T12:19:22

So has the number of external contributions applied by you and subsequently released to the CPAN on a month-by-month basis gone up or down?

Re:Github makes git useful to everyone

brian_d_foy on 2009-08-12T14:05:32

It's definitely gone up, from almost nothing over all time to about 50 in the past month. Not all of that was for stuff showing up on CPAN. Once people realize how easy it is to create a patch, send it, and for my not to lose track of it, they send more.

I forgot to mention that Github also lets you edit files in the web browser and commit from there, so minor things like doc patches aren't a pain in the ass. You fix the error in the browser, commit, and send a pull request all with a couple of clicks in Github. You don't even see a patch.

Just more anecdotal evidence

cowens on 2009-08-12T14:38:29

Before GitHub I mostly lurked. I would grab code out of cvs, svn, etc., compile it, fix anything that went wrong for me, and rarely send a patch back. Doing a proper diff, tracking down the right email address to send it to, writing the email, etc. was too much work and I always assumed that if I had the problem then someone else would also have the problem and would jump through the hoops. Now that I have GitHub I fork projects I am compiling and if I have to make a change it is as easy as git add file, git commit -m "foo", git push, and then hitting GitHub and sending out a pull request.

The great benefit of Git is that I don't have to ask for a commit bit, and my changes are local to me only and the maintainer can decide if my fix is good enough or necessary.

Worth using either way

leedo on 2009-08-12T15:17:57

I don't think git is going to magically draw more contributors to a project, but it certainly does make the lives of everyone involved with the project easier.

Even if you end up with the same number of contributions when compared to svn, I still think git is worth the switch. Being able to add a remote repository and cherry-pick a specific patch, or merge a branch in a matter of minutes is great. I don't maintain any popular projects, but for the few that I do get contributions, it has made my job much quicker and simpler.

Re:Worth using either way

nicholas on 2009-08-13T10:12:55

I don't think git is going to magically draw more contributors to a project, but it certainly does make the lives of everyone involved with the project easier.

That's my view so far, qualitative and as best I can quantitative*, based this project that converted to git.

So I think that Adam's hypothesis that a conversion to git doesn't materially change the contributor profile seems to remain unchallenged.

* I'm not a statistician, and the data that I extract seems to have a lot of variation, so I'm not comfortable that I can reliably draw any conclusions about commit rate and contributor change, apart from "it's noisy". Certainly, there has been no massive increase in either. Despite a lot of noise about "perforce being the barrier that prevents me contributing".

Re:Worth using either way

Alias on 2009-08-13T13:14:51

To clarify, my main annoyance with git is just general usability stuff on the platform I work on.

I don't have a hypothesis on contribution rates yet, I'm just looking for data and open to either rates going up or down.

Re:Worth using either way

Aristotle on 2009-08-13T22:57:37

Despite a lot of noise about “perforce being the barrier that prevents me contributing”.

We have learned from the discipline of program optimisation that removing the major bottleneck just reveals the next, and what these people are actually saying is “Perforce was the first thing I ran into that discouraged me beyond my motivation threshold”. Progress exists, though, regardless of whether silver bullets do.

I use(d) CVS and I use Git, and Git is much easier

jnareb on 2009-08-12T18:30:13

I used both CVS and Git, and with possible exception of ad-hoc version control of loosely coupled files, Git is much, much easier to use. Branching and especially merging branches in CVS is deep, dark magic; in Git it is extremly easy.

See also my answer to Difference between GIT and CVS question on StackOverflow (community Q&A site).

Some of those transfer to comparison between Subversion and Git: see For home projects, can Git (or other DVCS) provide advantage over Subversion? also on StackOverflow.

Re:I use(d) CVS and I use Git, and Git is much eas

Alias on 2009-08-13T13:29:22

Git is, unfortunately, a lot harder for me.

But that's not the issue, it's whether or not it makes better for other people, and if that subsequently means they commit more code without me needing to be involved.

Regular vs. One off

nothingmuch on 2009-08-12T19:45:04

Putting aside the other technical superiorities, the entry barrier that Git removes is that of one off contributions.

Once you have an account it's easy to regularly contribute using SVN.

Git makes it easy to contribute to a project even if you've never contributed before, without knowing what the framework for contribution is, and without needing to be authorized.

It fills in the gap between emailing patches to someone (which is has technical annoyances but no administrative overhead) and giving commit access (which is technically streamlined but required administration) using its distributed nature to allow sending of patch files that have all the proper metadata to be cleanly applied.

This is taken further by hosting sites like github which let you create your own private repository from anyone else's repository, where you get full write access, so that people can pull changes from you.

Because of this getting one off fixes from non regular contributers is much lower overhead for both parties, and usually leads to more regular contributions later.

With subversion, even with a streamlined invite mechanism things would never be as smooth and safe.

Re:Regular vs. One off

nothingmuch on 2009-08-12T19:45:56

(and yes, i've received many many more contributions since I've switched, even if most of them did not become collaborators afterwords)

As far as I'm concerned...

mauzo on 2009-08-13T02:12:20

the most important advantages of git over other systems I've used (CVS, svn, svk) are
  • it's fast: I can download the complete history of perl in a few minutes and checkout any commit in seconds;
  • it doesn't (ever, in my experience) lose its mind, which all of the others have done for me on several occasions.

I (now) like and use some of the other features, like cheap branches, easy rebasing and convenient publishing on github, but if I were looking for a reason to persuade someone to switch those would be my choice.

Re:As far as I'm concerned...

Alias on 2009-08-13T08:46:19

The thing is, I don't particularly care about those things.

svn is slightly annoyingly slow, but not so much it's a problem.
svn has never gone bad for me since I moved to FSFS.

In exchange I get a number of other useful things I'd need to give up if I moved away.

But all of that is not what this post is about. The bit I'm really curious about is the real world volumes of external contributions.